Server 4.7.11 (and Beta 4.8.21) crashes during basic setup in Windows

[Tarlock 5]

An new setup from scratch on a Windows 11 Pro machine (running virtualized in VMWare, but I assume that this does not change anything) of the server in version 4.7.11 did not work because the server stopped working almost immediately (1st configuration page for select main language showed up, but than everything died). In event viewer I got:

.Net Runtime Error EventID 1023
Application: EmbyServer.exe
CoreCLR Version: 6.0.1022.47605
.NET Version: 6.0.10
Description: The process was terminated due to an internal error in the .NET Runtime at IP 00007FF88661224B (00007FF886440000) with exit code c0000005.

It was the same with latest beta 4.8.21

Find some addtional information inside emby-event.txt and see also the the emby logs inside the attachments.

I have Visual C++ Runtimes installed (2015 - 2019 redistributable x86 and x64).

Besides that the machine has nothing installed besides the Windows Updates and latest drivers (an Intel Arc 380 GFX card runs dedicated for the guest VM and QuickSync runs in Handbrake w/o problems).

I briefly removed the portmapper plugin because it throwed some exceptions -> it did not change anything (same event but nothing from portmapper in the emby logs... ).

Some advice woud be really appreciated.

Regards,

Lothar

emby-event.txt embyserver.txt embyserver-beta-4821.txt

[Abobader 3464]

Hello Tarlock,

** This is an auto reply **

Please wait for someone from staff support or our members to reply to you.

It's recommended to provide more info, as it explain in this thread:

Thank you.

Emby Team

[Luke 42079]

Hi, that's strange. Nothing obvious there. Is windows up to date?

[Happy2Play 9780]

Is this a new install of Windows?

Just seems odd as I have 5 Windows server that are installs, portables, and Hyper-V systems without issue.

Wonder how this is something new but may be something in a clean install of Windows 11?

[Tarlock 5]

Hi everybody,

installation of windows is new and almost untouched besides updates and drivers, when I started the Emby setup there was even no Visual C++ runtime package on the machine and I was confident that the installation would fix the issue, but I encountered always same behavior machine did never fail so far ( like strange bluescreen or similar during setup ). Some other similar .Net problems mentioned something about „garbage collection“ under heavy load or misinterpretation of CPU load states (and VMware ESX does not handle P- and E-cores of underlying 12700 Intel CPU perfectly and it is maybe some sort of timing problem? )

Anything more I can bring up for some dedicated debugging?

Thanks so far,

Lothar

 

[Tarlock 5]

Hi everybody,

I re-installed .Net 3.5 and 4.8 today (and other potentially relevant component like the vc++ runtimes) and there was absolutely no change! Always the same fault with CoreCLR. I even gave the system only P-cores to avoid timing issues if threads would be handover from P- to E-core... And it was always the same problem!?!!!

Any options left?

Regards,
Lothar

[Luke 42079]

So it's only a problem within the vm, right?

[Tarlock 5]

When I tested  the Emby release in a bare-metal Windows 11 installation in early December 2022, it startet w/o any problem. The installation inside the VM is one month later... a lot of things can happen in a MS .Net world ( i.e. https://www.techadvisor.com/article/746378/windows-11-security-update-is-causing-app-crashes-but-there-is-a-fix.html ). I doubt that the problem depends on virtualization (even if I agree that my environment is somewhat special). But I am open for some reasonable argument why the virtualization can brake the .Net runtime environment...

Greets,
Lothar

 

[Luke 42079]

Some helpful info on that exit code:

https://stackoverflow.com/questions/17168982/exception-error-c0000005-in-vc

 

[Tarlock 5]

Well, basically I knew, what the error code stands for. But generally it means that some code tries to read / write to memory which is not in the right area / space „owned“ by the thread. And usually that has to do with code in a c++ based library… but where is the approach to fix it?

[Tarlock 5]

I just installed a fresh Windows 11 Pro VM and Emby runs w/o any problem (22H2 with all updates). The only difference between the machine(s) is now the dedicated GFX-card (Intel ARC A380 based for the transcoding). The VM with the GFX card did not show anything else as a problem so far (could use it even for gaming on a dedicated monitor): could use Handbrake 1.6 for re-encoding as h265 UHD movie to AV1 with the full hw acceleration (round about 60 fps at 30% GPU load). I will add things to the new VM step-by-step to encounter at least the sw / hw which corrupts the runtime env for Emby...

Thanks so far, I will let you at least know which component causes the trouble... sorry for any inconvenience so far  

Greets,
Lothar

 

[Luke 42079]

Thanks for the info. Let us know how you get on. Thanks !

[Tarlock 5]

Well, here are the sad news:

it is a problem with the Intel ARC graphics driver:

I check it twice also on a bare metal installation of Windows 11 Pro with almost the same hardware configuration as the Vmware ESX box. The driver version 31.0.101.4032 causes the trouble. Both other drivers (Version 31.0.101.3959 and 31.0.101.3802) do not have any problem at all (besides the fact that they are somewhat slower in transcoding jobs; but the 2 - 8 fps less can be taken easily).

I will try to give a bug report to Intel's driver team in the next days and if I will encounter that newer drivers are not suffering from the same problem I will announce it here as well...

Thanks,
Lothar

 

[Mnejing 23]

Was there a particular reason you went with ARC in the first place? It's completely unproven tech, and has had massive driver issues from the start. Surely you could have found an AMD or nVidia solution that cost the same (or less)....

[Tarlock 5]

@Mnejing

Well, generally you are right, but things are a little bit different here in Germany: the prices of comparable consumer / business GFX cards from NVidia are reasonably higher (paid 160 € for the ARC 380); NVidia 1650 (and better) would be 230 € and higher by the end of last year. And most of this NV cards are limiting encode / decode threads to 3 (I know there are driver hacks available to come over that limit, but I'd like to stay with pure official relases ).

The ARC family can also handle AV1 encoding in HW, which is only available in latest RTX 4xxx family. AMD cards show often strange behaviour at en- / decoding tasks (at least in my personal experience). Usually the GFX drivers (and multimedia stack) from Intel are extremely good in 2D tasks (not in 3D etc. but that is not relevant) and nowadays it is possible to make even the internal XeGraphics of th i7 12700 available for passthrough...

So I can run 2 machines with a dedicated GFX card on ESX and if I switch I do not have to handly several installation because currently the driver installer comes combined for ARC / XeGraphics and UHD...

But if new architecture based gfx cards from NVidia become available for a reasonable price I would not deny that I will switch to it in the future

Greets,
Lothar

[rbjtech 5284]

33 minutes ago, Tarlock said:

@Mnejing

Well, generally you are right, but things are a little bit different here in Germany: the prices of comparable consumer / business GFX cards from NVidia are reasonably higher (paid 160 € for the ARC 380); NVidia 1650 (and better) would be 230 € and higher by the end of last year. And most of this NV cards are limiting encode / decode threads to 3 (I know there are driver hacks available to come over that limit, but I'd like to stay with pure official relases ).

The ARC family can also handle AV1 encoding in HW, which is only available in latest RTX 4xxx family. AMD cards show often strange behaviour at en- / decoding tasks (at least in my personal experience). Usually the GFX drivers (and multimedia stack) from Intel are extremely good in 2D tasks (not in 3D etc. but that is not relevant) and nowadays it is possible to make even the internal XeGraphics of th i7 12700 available for passthrough...

So I can run 2 machines with a dedicated GFX card on ESX and if I switch I do not have to handly several installation because currently the driver installer comes combined for ARC / XeGraphics and UHD...

But if new architecture based gfx cards from NVidia become available for a reasonable price I would not deny that I will switch to it in the future

Greets,
Lothar

The Intel ARC discreet GPU's are a far better choice for AV encoding/decoding than Nvidia or AMD for the reasons you stated above.  You also have an intel 12th Gen CPU/iGPU - so technically this also allows parallel decode across iGPU/dGPU.

Yes drivers still have some issues but intel are aware and are improving things rapidly.

I will be purchasing an ARC dGPU to compliment my 12700 also (for the AV1 encode), when the prices drop a bit. 

 

[Tarlock 5]

The best thing you can do for any Intel 12xxx is to add an ARC card, if you do not want to get 100 fps in current games (and want to pay 1-2k € for an AMD / NVidia). And you are right: the driver improvements in almost all aspects are extremely good... 

[Luke 42079]

Have your questions been answered?

[Tarlock 5]

Regarding the questions / topic / statements I posted here everything is fine... Only Intel did not answer so far regarding my bug report...

[Luke 42079]

HI, we found a solution and this will be resolved in Emby Server 4.8. If you need immediate relief we'll have a new 4.8 beta server with the fix sometime in the next few days.

[Tarlock 5]

6 hours ago, Luke said:

HI, we found a solution and this will be resolved in Emby Server 4.8. If you need immediate relief we'll have a new 4.8 beta server with the fix sometime in the next few days.

Hi Luke,

that sounds really great. Thanks for investigating the issue further even if it was already discovered as a problem with the driver of the gfx card...

I was quite patient and relentless to get a case at Intel as well (#: 05735724) and they will try to find a solution (as far as I know there is a similar problem with Luminar).

Finally I will give the 4.8 beta a chance as well

 

[Happy2Play 9780]

 

[Tarlock 5]

Well, the latest Beta of Emby works, but as I got reply from Intel that they fixed the problem in general I can confirm that driver version 31.0.101.4314 and up should work even with Emby  4.7.11.0 w/o problems... 

[Luke 42079]

9 hours ago, Tarlock said:

Well, the latest Beta of Emby works, but as I got reply from Intel that they fixed the problem in general I can confirm that driver version 31.0.101.4314 and up should work even with Emby  4.7.11.0 w/o problems... 

thanks for the info!

[Tarlock 5]

Sorry,I have to re-open the problem:

Since Intel driver higher than 31.0.101.4502 with all Emby Server versions until 4.7.11 to 4.7.14 are suffering again from the same problem:
as soon as I start the server app I will get an CLR privileg access violation and the process stops working...

I contacted Intel already a couple of times about the regression of the issue and under Case #: 05978352 they confirmed once that they would look after the issue, but since 6 - 8 weeks nothing happened (an I tried all new driver releases since then)

A few Emby beta version removed the problem also but since a couple of month the problem remains the same...