Jump to content


Photo

4.2.0.18 / NVENC sometimes randomly fails with CUDA_ERROR_OUT_OF_MEMORY: out of memory

linux nvenc

  • Please log in to reply
9 replies to this topic

#1 ProperlyFormattedDataFile OFFLINE  

ProperlyFormattedDataFile

    Advanced Member

  • Members
  • 35 posts
  • Local time: 05:22 AM

Posted 22 June 2019 - 08:33 PM

I'm not sure if this is beta related or not, but since I'm running the beta server I'll start here.

 

I'm running Emby on a Ubuntu Linux server with an Geforce GTX 1050 2GB card.

 

Maybe one out of 5 or 10 times I try to transcode through Emby, the transcoding fails will CUDA_ERROR_OUT_OF_MEMORY: out of memory.  Emby will then fall back on software encoding, even when there's too much other load on the box for software encoding to keep up.

 

I've checked nvidia-smi, and there is nothing else using the card, all 2GB of memory is free.  If I restart the transcode, it will usually work.  During a successful recording, the ffmpeg process is only using about 200MB out of the 2GB video memory available.

 

I've tried with both the 418 and 430 drivers.

 

What could be causing this issue?



#2 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 131774 posts
  • Local time: 05:22 AM

Posted 22 June 2019 - 08:41 PM

Hi there, can we please look at an example ffmpeg log? thanks.



#3 ProperlyFormattedDataFile OFFLINE  

ProperlyFormattedDataFile

    Advanced Member

  • Members
  • 35 posts
  • Local time: 05:22 AM

Posted 22 June 2019 - 09:30 PM

Oh, sorry, meant to include this with my original post.

 

Hi there, can we please look at an example ffmpeg log? thanks.

 

Attached Files



#4 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 131774 posts
  • Local time: 05:22 AM

Posted 22 June 2019 - 09:41 PM

Thanks. @softworkz will take a look.



#5 ProperlyFormattedDataFile OFFLINE  

ProperlyFormattedDataFile

    Advanced Member

  • Members
  • 35 posts
  • Local time: 05:22 AM

Posted 07 July 2019 - 03:12 PM

Thanks for looking at this.  Any thoughts on what could be causing this?  This error is currently present in 6 of 22 ffmpeg transcode logs on my server, with a fallback to software encoding in each case.



#6 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 131774 posts
  • Local time: 05:22 AM

Posted 07 July 2019 - 03:26 PM

We'll see what @softworkz thinks. Thanks.



#7 softworkz OFFLINE  

softworkz

    Advanced Member

  • Developers
  • 1533 posts
  • Local time: 11:22 AM

Posted 07 July 2019 - 07:24 PM

Thanks for looking at this.  Any thoughts on what could be causing this?  This error is currently present in 6 of 22 ffmpeg transcode logs on my server, with a fallback to software encoding in each case.

 

Have you - by chance - installed one of those patches to circumvent the encoding/decoding session limit for GeForce cards?

 

Otherwise I don't have any useful ideas at this time. From searching around I could see that there are just few reports about a problem like that.

In those cases that I found, there wasn't any common cause nor any reliable reproduction scenario.

It seems as if it would occur only with certain combinations of hardware, OS/kernal and drivers.

 

As such I don't have any smart ideas for troubleshooting this, except some radical ones like: Re-installing the OS, maybe a different OS version, replace motherboard, try a different Nvidia GPU with your current setup, etc. etc.

 

What I can say for sure is that this problem is not specific to Emby. Also it's not a frequently repeating and prominent problem, so it should be possible to get it working in some way...



#8 ProperlyFormattedDataFile OFFLINE  

ProperlyFormattedDataFile

    Advanced Member

  • Members
  • 35 posts
  • Local time: 05:22 AM

Posted 08 July 2019 - 09:27 PM

Have you - by chance - installed one of those patches to circumvent the encoding/decoding session limit for GeForce cards?

 

Otherwise I don't have any useful ideas at this time. From searching around I could see that there are just few reports about a problem like that.

In those cases that I found, there wasn't any common cause nor any reliable reproduction scenario.

It seems as if it would occur only with certain combinations of hardware, OS/kernal and drivers.

 

As such I don't have any smart ideas for troubleshooting this, except some radical ones like: Re-installing the OS, maybe a different OS version, replace motherboard, try a different Nvidia GPU with your current setup, etc. etc.

 

What I can say for sure is that this problem is not specific to Emby. Also it's not a frequently repeating and prominent problem, so it should be possible to get it working in some way...

 

Thanks, this is Ubuntu 19.04 running the 5.0.0-19 kernel.  The drivers are not patched. I tried the official canonical packaged drivers (418) and the official 430 drivers packaged here: https://launchpad.ne...hive/ubuntu/ppa.

 

Is there a known good configuration that I could use here?  I'd like to avoid Windows if at all possible, but I'm open to other linux distros, an LTS version of Ubuntu, or older drivers if they're known to work.  I'm pretty much only using the machine for emby and as a file server (I even disabled X to make sure nothing else was using graphics memory).

 

If I get to the point where I need to start swapping hardware, nvidia is getting the boot entirely :)



#9 softworkz OFFLINE  

softworkz

    Advanced Member

  • Developers
  • 1533 posts
  • Local time: 11:22 AM

Posted 09 July 2019 - 09:54 PM

I would try Ubuntu 18 LTS that's where we're doing most of the tests.

 

Some new drivers have been released yesterday: https://www.nvidia.c...px/148589/en-us

 

Also I would perform the full driver installation including X and leave all OS configuration and settings at their defaults as far as possible.

 

PS: running a desktop GUI shouldn't affect transcoding performance, it will rather operate the card like expected (it's a consumer card)



#10 ProperlyFormattedDataFile OFFLINE  

ProperlyFormattedDataFile

    Advanced Member

  • Members
  • 35 posts
  • Local time: 05:22 AM

Posted 10 July 2019 - 12:19 AM

I would try Ubuntu 18 LTS that's where we're doing most of the tests.

 

Some new drivers have been released yesterday: https://www.nvidia.c...px/148589/en-us

 

Also I would perform the full driver installation including X and leave all OS configuration and settings at their defaults as far as possible.

 

PS: running a desktop GUI shouldn't affect transcoding performance, it will rather operate the card like expected (it's a consumer card)

 

Running X in Ubuntu uses about 70mb of graphics memory for gnome-shell, which strangely enough seems to matter.  Despite none of these processes using anywhere near the 2gb limit, the cuda memory error seems to happen less without X running (though I don't actually have data to back that up, just a general sense).  I'll give the new drivers a shot, and install 18 this weekend if that doesn't help. Thanks for the tips.







Also tagged with one or more of these keywords: linux, nvenc

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users