Jump to content

Tips to get the most out of your NVIDIA GPU


Ikario
 Share

Recommended Posts

Ikario

If you are using a GPU for transcoding, you might have realized that there's a limit to how many transcoding streams you can have going on at the same time.  This limit is hard coded by NVIDIA on their drivers to push heavy users into buying the more expensive Quadro cards if they want to do heavy transcoding.  With a consumer card, no matter the model, you get 3 (or 2 if you are running older drivers) transcoding sessions max.  With Quadro cards, you get Unrestricted transcoding streams for everything including and above the P2000/P2200.

There are patches for both Linux and Windows that remove that limitation, however due to several factors (ugly file structure of the drivers in QNAP NAS, something weird going on with the sed command), these do not work.

However, once I opened the script and see what it actually does, I realized it would be extremely easy to do manually if you have some minimal know-how. This is not a tutorial or a guide.  You should not do this if you don't know what you are doing and if you do, it wouldn't hurt to do a backup (with permissions, acl, everything) of your GPU driver folder.  All you have to do is modify 6 files or rather, one file that is repeated 6 times in different directories.

This is a summary of what needs to be done:

1)You need to locate a specific file.
2)You need to modify a few bytes (in most cases, just ONE) of that file using a Hex Editor (I'm using HxD on Windows) of that file.
3)You need to copy that file back to where you found it.
4)???
5)Profit

That's literally it.

Let's get started.  First, you need to find where your drivers are located and what driver version you have (not the QNAP version, but the actual nvidia driver version).  If you need help doing this, you shouldn't be doing it, fair warning.  

find /share/ -name "NVIDIA_GPU_DRV"

That should help you find it, searching for nvidia might yield some results too.  There's several ways to find the driver version.  nvidia-smi is not working on QNAP NAS UNLESS you have managed to pass your GPU to a docker container (you can see my other post if you need help with this).  An easier but less trustworthy way is to navigate .../NVIDIA_GPU_DRV/usr/lib where you'll find a ton of files and you'll see a pattern.  You'll find a file like, for example libnvidia-encode.so and right next to it a libnvidia-encode.so.XXX.XXX (in my case, 460.39).  That last number would be your driver version.

 

Now that you have your driver version, you should go to the linux patch github, check that there's a patch for your driver and if so, download the linux patch.

https://github.com/Snawoot/nvidia-patch

Let's take 460.39 as a driver version as an example. You now need to open the patch.sh file in a text editor.  90% of that script deals with finding the driver version, checking if you have drivers installed and nvidia-smi is working, making a backup, etc.  None of that works on the QNAP so if you are doing this, I assume you are responsible enough of your hardware/software to make proper backups of the files you are going to edit and deal with the fallout if something goes wrong.  There's only three things on that whole script that matter.

1)The patch_list array
2)The object_list array
3)

sed "$patch" "$backup_path/$object.$driver_version$backup_suffix" > \
      "${PATCH_OUTPUT_DIR-$driver_dir}/$object.$driver_version"

That line is the only line responsible of patching your drivers. 

Let's take a look at the object_list array.  You can see that it lists driver versions and a filename.  That's the filename you are going to edit more than once in different locations. You need to locate that file in every location.  There's a trick to this whole thing though.  Let's continue with 460.39 as driver version, the object_list shows that the file to edit is called "libnvidia-encode.so".  Remember how in the usr/lib folder there were two copies of every file, one regular one and another one with the driver version as the extension?  You need to find every copy of both, so in this case, you'd need to find every copy of:

libnvidia-encode.so
libnvidia-encode.so.460.39

Those are the files you are going to edit.  In my case, there were 3 copies of each.  In most driver versions (if not all) these copies (six total) are identical, but this might not be always the case.  you can use the diff command to check if they are the same, this would save you time editing as you'd just edit one file and copy it wherever you need rather than editing every file one at a time.

 

Once you located which files you need to edit, now you need to know what and how to edit them. If you see the sed command I posted above, it has the following structure (minus the new line of course)

sed "patch" orig_file > dest_file

This will modify orig_file and save the modified copy as dest_file.  What is patch, anyway?  Well, if we check the patch_list and go to our driver (again, 460.39 in this case), we'll find this:

's/\x22\xff\xff\x85\xc0\x41\x89\xc4\x0f\x85/\x22\xff\xff\x31\xc0\x41\x89\xc4\x0f\x85/g'

I won't go into detail into the syntax of that line but it's basically something like this:

"s/whattosearchfor/replacewiththis/g"

In this case, it will be searching for the following:

\x22\xff\xff\x85\xc0\x41\x89\xc4\x0f\x85

and replacing it with this:

\x22\xff\xff\x31\xc0\x41\x89\xc4\x0f\x85

Those are hex values and you can see there's only one byte difference between them. 

Now comes the actual patching.  You are going to open the files you found before with a Hex Editor and you are going to search for every occurrence of that string (make sure to search as hex values and not integer or char values, otherwise you won't find the string). Also, if you are doing this you should already know that "\x22" means hex value 22 or 0x22 or whatever way you like to put it. You need to replace the first string with the second. Note that only one byte is different and that's the same for most if not all of the drivers.  Most of the time, the string appears just once per file.  Also, most of the time, all six files are identical.  So you only end up editing one byte once.

Now that you have the file/files edited.  Replace the original files with the edited version. In the 460.39 version there's six appearances of them:

NVIDIA_GPU_DRV/usr/lib/libnvidia-encode.so
NVIDIA_GPU_DRV/usr/lib/libnvidia-encode.so.460.39
NVIDIA_GPU_DRV/usr/nvidia.u18.04/libnvidia-encode.so
NVIDIA_GPU_DRV/usr/nvidia.u18.04/libnvidia-encode.so.460.39
NVIDIA_GPU_DRV/usr/nvidia/libnvidia-encode.so
NVIDIA_GPU_DRV/usr/nvidia/libnvidia-encode.so.460.39

Reboot the NAS for good measure and that's pretty much it! The drivers are hacked and there's no longer a software limitation on concurrent streams/sessions.

  • Thanks 1
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

×
×
  • Create New...