NVENC working for mkv/mp4, but not full blurays/m2ts

[vlix 1]

Hi,

 

I use Emby Server 3.3.0.0 on a Windows Server 2016 server with an nVidia GT 710 graphics card. I'm trying to get NVENC to work properly in order to take some load off my CPU. I have enabled Hardware Encoding with NVENC in the server dashboard. It seems to work alright for .mkv and .mp4 containers (snippet from transcode log):

 

C:\Users\vlix\AppData\Roaming\Emby-Server\system\ffmpeg.exe -c:v h264_cuvid  -i file:"\\APU\HD Rips\2010 - The Year We Make Contact\2010.The.Year.We.Make.Contact.1984.mkv" -threads 7 -map 0:0 -map 0:1 -map -0:s -codec:v:0 h264_nvenc -vf "scale=trunc(min(max(iw\,ih*dar)\,1920)/2)*2:trunc(ow/dar/2)*2" -pix_fmt yuv420p -preset default -b:v 5616001 -maxrate 5616001 -bufsize 11232002 -profile:v high -force_key_frames "expr:if(isnan(prev_forced_t),eq(t,t),gte(t,prev_forced_t+2))" -copyts -vsync -1 -codec:a:0 aac -strict experimental -ac 2 -ab 384000 -af "volume=2" -f segment -max_delay 5000000 -avoid_negative_ts disabled -map_metadata -1 -map_chapters -1 -start_at_zero -segment_time 3  -individual_header_trailer 0 -segment_format mpegts -segment_list_type m3u8 -segment_start_number 0 -segment_list "C:\Users\vlix\AppData\Roaming\Emby-Server\transcoding-temp\66b9e79f7487bb115c8298ea2776ba75.m3u8" -y "C:\Users\vlix\AppData\Roaming\Emby-Server\transcoding-temp\66b9e79f7487bb115c8298ea2776ba75%d.ts"

 

and another snippet:

 

Stream mapping:  Stream #0:0 -> #0:0 (h264 (h264_cuvid) -> h264 (h264_nvenc))  Stream #0:1 -> #0:1 (ac3 (native) -> aac (native))Press [q] to stop, [?] for help[segment @ 000002be5035c140] Opening 'C:\Users\vlix\AppData\Roaming\Emby-Server\transcoding-temp\8229849f3f9a797b217ee0e7b67014dd0.ts' for writingOutput #0, segment, to 'C:\Users\vlix\AppData\Roaming\Emby-Server\transcoding-temp\8229849f3f9a797b217ee0e7b67014dd%d.ts':  Metadata:    encoder         : Lavf58.3.100    Stream #0:0: Video: h264 (h264_nvenc) (High), yuv420p, 1280x536 [SAR 1:1 DAR 160:67], q=-1--1, 5616 kb/s, 23.98 fps, 90k tbn, 23.98 tbc (default)    Metadata:      encoder         : Lavc58.9.100 h264_nvenc    Side data:      cpb: bitrate max/min/avg: 5616001/0/5616001 buffer size: 11232002 vbv_delay: -1    Stream #0:1: Audio: aac (LC), 48000 Hz, stereo, fltp, 384 kb/s (default)    Metadata:      encoder         : Lavc58.9.100 aacframe=    6 fps=0.0 q=14.0 size=N/A time=00:00:00.68 bitrate=N/A speed=1.29x    

 

... But when trying to transcode an untouched Bluray file (.m2ts extension) Emby doesn't even pass a command to ffmpeg to use NVENC, instead it specifies x264 (snip from transcode log):

 

C:\Users\vlix\AppData\Roaming\Emby-Server\system\ffmpeg.exe -i file:"\\APU\Blu-ray 2\2001 - A Space Odyssey\BDMV\STREAM\00000.m2ts" -threads 7 -map 0:0 -map 0:1 -map -0:s -codec:v:0 libx264 -vf "scale=trunc(min(max(iw\,ih*dar)\,1920)/2)*2:trunc(ow/dar/2)*2" -pix_fmt yuv420p -preset veryfast -crf 23 -maxrate 5616001 -bufsize 11232002 -profile:v high -level 4.1 -x264opts:0 subme=0:me_range=4:rc_lookahead=10:me=dia:no_chroma_me:8x8dct=0:partitions=none -force_key_frames "expr:if(isnan(prev_forced_t),eq(t,t),gte(t,prev_forced_t+3))" -copyts -vsync -1 -codec:a:0 aac -strict experimental -ac 2 -ab 384000 -af "volume=2" -f segment -max_delay 5000000 -avoid_negative_ts disabled -map_metadata -1 -map_chapters -1 -start_at_zero -segment_time 3  -individual_header_trailer 0 -segment_format mpegts -segment_list_type m3u8 -segment_start_number 0 -segment_list "C:\Users\vlix\AppData\Roaming\Emby-Server\transcoding-temp\3ff858573d756aa5751104fe26b88d9e.m3u8" -y "C:\Users\vlix\AppData\Roaming\Emby-Server\transcoding-temp\3ff858573d756aa5751104fe26b88d9e%d.ts"

 

 

To me it seems like for some reason Emby is passing the "wrong" command to ffmpeg when transcoding a full Bluray, but I'm sure the truth is more complicated  . Any ideas why this is happening?

 

Thanks very much!!

[Luke 42079]

We've disabled it as realistically we are not going to be able to support and troubleshoot it. For the best experience possible I would suggest converting to mkv files.

[vlix 1]

I see... thank you for your answer, Luke. It's too bad though, because I like to use full quality sources in Emby, including H265 encoded UHD Blurays. That's actually where my real problem is. I like to be able to remote stream these UHD Blurays, but software decoding a H265 4K Bluray, then software encoding it to H264 is so processor intensive that even my fairly powerful Ryzen 1700 can only manage 23fps or so.

 

How about hardware DEcoding full Blurays (both normal ones and UHD ones)? Is that supported? I mean as part of a transcoding process, of course. Would buying a Geforce GT 1030 (which can hardware decode 4K H265) solve my problem?

Edited March 2, 2018 by vlix

[ebr 16184]

Changing the container does not change the quality.

[vlix 1]

Changing the container does not change the quality.

 

Hi ebr,

 

Not sure what you're trying to suggest with that. Sure, I could remux the main movie in my full Bluray images to an .mkv file without quality loss, but that would mean a GIGANTIC waste of storage space, since I also want to keep the full images with all the extras and what not. Is that what you mean though?

 

In my experience the experimental playback of Bluray folders in Emby actually works fairly well, except for a few Blurays where the movie is divided across several .m2ts files. Now I just need some kind of hardware acceleration for transcoding those Blurays...

[ebr 16184]

Hi ebr,

 

Not sure what you're trying to suggest with that. Sure, I could remux the main movie in my full Bluray images to an .mkv file without quality loss, but that would mean a GIGANTIC waste of storage space, since I also want to keep the full images with all the extras and what not. Is that what you mean though?

 

Why would changing the items container equate to a waste of space?  The space should be exactly the same...

[Guest asrequested]

Software support for disc image playback is quickly becoming obsolete, throughout the industry. Stored movies are extensively file driven. Speaking globally, very few people store disc images, digitally. More and more discs are providing digital copies, because of this. Even when they can be played, most of time they just play the movie. It's advised that you convert to files. If there are trailers and other videos on the disc you still want, convert those and add a specials folder. Limiting yourself to only images has no future.

[vlix 1]

Why would changing the items container equate to a waste of space?  The space should be exactly the same...

 

Because I would like to keep the original Blu-ray files, and avoid having to remux every extra on every Bluray in my collection...

 

 

Software support for disc image playback is quickly becoming obsolete, throughout the industry. Stored movies are extensively file driven. Speaking globally, very few people store disc images, digitally. More and more discs are providing digital copies, because of this. Even when they can be played, most of time they just play the movie. It's advised that you convert to files. If there are trailers and other videos on the disc you still want, convert those and add a specials folder. Limiting yourself to only images has no future.

 

Fair enough, I appreciate that. At the same time though, we're not talking complex images here. For most Blurays, we're talking about a single .m2ts file containing the entire movie, which is not so different from a single .mkv file, is it? I certainly don't expect Emby to present me with a fully functional Bluray menu but it would be so nice if it could simply remote stream that single .m2ts without any limitations.

[Guest asrequested]

Well, to ebr's point, if it's just the movie, you can just remux without transcoding, and you'll be able to achieve your goal. And it's more about folder structure than file container. If you were to rename the file to the name of the movie, move it to the top folder, rename the folder, correctly. I suspect it would play, as you wish it.

Edited March 2, 2018 by Doofus

[vlix 1]

Well, to ebr's point, if it's just the movie, you can just remux without transcoding, and you'll be able to achieve your goal. And it's more about folder structure than file container. If you were to rename the file to the name of the movie, move it to the top folder, rename the folder, correctly. I suspect it would play, as you wish it.

 

Thanks Doofus. Sure I get your (and ebr's) point, but I would like to be able to watch the movie with Emby, while still retaining the option of watching the extras with a software Blu-ray player like PowerDVD.

 

I think I'm going to mess around with NTFS symbolic links, perhaps that way I'll be able to do what you suggest without actually touching the original files. Or what also could work is if Emby were able to read Avisynth scripts...?

[Luke 42079]

We don't do anything with avisynth, sorry.

[Jdiesel 1431]

I'd recommend keeping two versions. The original BluRay m2ts and a second ripped version with 2-channel AAC audio in a MP4 container. Rip it to a reasonable bitrate depending on your network connection and the rips should play direct play on any device outside your home network. Compared to the 50GB of a 4k BluRay an additional 3Mbps MP4 rip should be about an additional 3GB per movie.

 

There are issues with transcoding HDR material right now anyways which result in inaccurate colors on SDR displays.

Edited March 2, 2018 by Jdiesel

[vlix 1]

A quick update on my progress, just in case someone else is interested in the same kind of goal I'm trying to achieve: hardware accelerated transcoding of .m2ts files straight from Blu-ray folder structures.

 

I've found two ways to play the main title from a Blu-ray folder structure, while allowing Emby to deal with just a regular single video file, and while leaving the original files and folder structure untouched.

 

1) Create a .strm file in a separate directory named after the film, and simply add the path to the .m2ts file containing the film, to the .strm file. This only half works, in my experience: when transcoding, Emby would software decode the file, but then use NVENC to encode it.

 

2) Create an NTFS Hard Link to the .m2ts file containing the film, and keep that in a separate directory. This way you basically have a completely new and separate file, except that it won't take up any extra disk space. This seems to work fine: when transcoding, Emby uses CUVID to decode and NVENC to encode.

 

Now it should even be possible to play films spread across several .m2ts files. I can create NTFS Hard Links to each .m2ts file, and name these hard links film-part1.m2ts, film-part2.m2ts, etc. I will experiment with this. However, if I understand correctly, Emby still does not support playing split video files one after the other automatically, which is too bad...

[Luke 42079]

That's getting really complicated.

[vlix 1]

That's getting really complicated.

 

:D You telling me?!? ... of course I'd much rather have it "just work". For me, the "experimental" Blu-ray folder playback works OK in most cases, so I guess all I'd need to get this to work the way I want is for you to enable hardware acceleration. But if you can't do that (I'm sure there are some good reasons) then I'm willing to jump through some hoops...

[Carlo 4561]

It sounds like you're still "fighting" the point of a media server designed to serve up media "files" not discs.

 

No disrespect meant but that's old school these days.  Unless you like looking at disc menus there is nothing gained by storing the actual disc rip on disc.  You are far better off creating one file for the movie and one file per extra (if needed).  Any of the online media centers like Emby, Plex, Kodi, Infuse will use this perfectly.  You will also grow to like it much better as well.  You can navigate the HDD and just play any of these files in VLC, Power DVD (whatever) and know exactly what you are playing since you gave them proper names.

 

Remuxing the files is just about changing the container and not the streams inside it.  So doing this doesn't change the quality one bit.  Matter of fact you can augment this to remove audio tracks you will never use and be able to create 2 channel stereo tracks if needed in addition to the original tracks.

 

MakeMKV is a great tool for extracting this type of thing.  Combine that with Filebot and it's super easy to rename the base movie names then just setup the extra names if needed for each rip.

 

Once you do this and find out how much easier your life is being able to use Emby EASILY and get full HW support I doubt you'll go back.

 

Carlo

[vlix 1]

It sounds like you're still "fighting" the point of a media server designed to serve up media "files" not discs.

 

No and Yes. No, because I think I do "get" the point of a media server. As I wrote before, I don't expect Emby to give me a working Blu-ray menu, I just want it to play the main title properly, which most of the time is basically just a file. Yes, because certainly I am a bit of a purist and a completist. I know I won't lose any quality when remuxing video content from a Blu-ray, but I will lose menus, interactive content, slide shows, text, etc. Besides, remuxing all the video content for every Blu-ray in my collection sounds like a hell of a job.

 

For better or worse, I want to keep the "original". And when I say "original", I don't mean "original quality" because yes, then I could just remux. I mean the "original presentation", i.e. the actual Blu-ray title.

 

Of course, I could also just remux the main title from each Blu-ray in my collection to a separate full quality .mkv, then store the original Blu-ray folder somewhere else for archival purposes. But like I wrote before, that would be a monumental waste of disk space. Terabytes and terabytes. How much easier would it be if Emby were juuust smart enough to know how to read the correct .m2ts file, and then just treat it as a standard video file? Kodi can do this without a sweat. Even Emby already does it fairly well, except that it can't deal with films spread across several .m2ts files. All I need is for hardware acceleration during transcoding to not be artificially disabled .

[Guest asrequested]

A lot of that is down to the player. Kodi has just one player. If you want to only use one player, theater desktop will do that. MPV will play the largest file in the folder. And it supports almost everything. It'll take all the load off the server and has a multitude of options.

[Carlo 4561]

Of course, I could also just remux the main title from each Blu-ray in my collection to a separate full quality .mkv, then store the original Blu-ray folder somewhere else for archival purposes. But like I wrote before, that would be a monumental waste of disk space. Terabytes and terabytes. How much easier would it be if Emby were juuust smart enough to know how to read the correct .m2ts file, and then just treat it as a standard video file? Kodi can do this without a sweat. Even Emby already does it fairly well, except that it can't deal with films spread across several .m2ts files. All I need is for hardware acceleration during transcoding to not be artificially disabled .

 

I guess what it comes down to is why you need/want to keep a digital copy on HDD of something you have on DISC and could rip again if needed?  The whole point of software like Emby is to get you away from DVD/BluRay menus and to just make your media super easy to use/navigate and playback on a whole host of devices both inside your house and over the internet.

 

When ever I have to touch/use a DVD or Bluray in an actual player it just drives me crazy now. It's just feels so antiquated and 1990s.   Hell I've got a pile of BluRay and DVD players on a shelf in my basement and don't even have one hooked up anymore anywhere in the house.  No need to have DVD/Bluray players installed on any computers now or have to deal with trying to figure out how to play them on a tablet or other device.

 

Basically what you're asking for is not likely to happen.  As the days go by it will be less and less likely to happen.  The system is built around files having a specific name that matches the movie/show in specific folder structures.  It's not only the reading of this information but also the storage of the pics/banners/nfo/meta info, Roku Bif files, etc that Emby relies on.  So to accommodate this type of thing would require breaking functionality of how the system currently works.

 

Plus on top of that you won't find many people who would request this.  It's just a very "old-school" way of hording media that by and large people have rapidly moved away from.

 

But with that said open a feature request in the proper section of the site and make a request for it.  Maybe you could get enough votes that Emby devs will entertain the idea and add it.  You never know.

[vlix 1]

A lot of that is down to the player. Kodi has just one player. If you want to only use one player, theater desktop will do that. MPV will play the largest file in the folder. And it supports almost everything. It'll take all the load off the server and has a multitude of options.

 

Thanks for the tip Doofus, but that won't help when transcoding, will it? Because then it's the back-end that will handle the file first, before passing it on to the client?

 

 

I guess what it comes down to is why you need/want to keep a digital copy on HDD of something you have on DISC and could rip again if needed?  The whole point of software like Emby is to get you away from DVD/BluRay menus and to just make your media super easy to use/navigate and playback on a whole host of devices both inside your house and over the internet.

 

...

 

Thanks for your input cayars. Everything you say makes perfect sense. For my particular use case I guess I'm just square out of luck, and perhaps for good reason when you look at how Emby is "supposed" to be used. HOWEVER, since playback of Blu-ray folders is experimental anyway, would it actually hurt to enable hardware acceleration? The argument that it will be a b!tch to support doesn't apply, because it's an experimental feature. And unless I'm mistaken, it's not so much adding a feature as removing an artificially imposed limitation. I guess I'll do a feature request, who knows . In the mean time, my trick with NTFS hard links actually kind of works.

Edited March 7, 2018 by vlix

[Guest asrequested]

Thanks for the tip Doofus, but that won't help when transcoding, will it? Because then it's the back-end that will handle the file first, before passing it on to the client?

The server won't be involved. It will be played directly, no transcoding, the player will handle everything and do what you want.

 

https://emby.media/emby-theater.html

Edited March 7, 2018 by Doofus

[ebr 16184]

The argument that it will be a b!tch to support doesn't apply, because it's an experimental feature. 

 

Ah... if only that were true .  Take look around out here at the number of playback problem reports that we have to troubleshoot before discovering the issue is due to one of these "experimental" features.

[vlix 1]

The server won't be involved. It will be played directly, no transcoding, the player will handle everything and do what you want.

 

https://emby.media/emby-theater.html

 

Ah yes, but that's for when you use it on a local network, isn't it? I already got that working alright using Emby for Kodi with Direct Paths. But I also need to be able to transcode and serve a movie to remote clients...

 

 

Ah... if only that were true .  Take look around out here at the number of playback problem reports that we have to troubleshoot before discovering the issue is due to one of these "experimental" features.

 

Oh well, I'll just keep messing about with my nasty workarounds then

[Guest asrequested]

Well, if you're transcoding due to bandwidth limitations, then you have no other option.

[Luke 42079]

The next release of Emby Server for Linux x64 will include NVENC support out of the box so all you need to do is enable it. 

 

Nvidia Cuda and OpenCL drivers will need to be installed first. These cannot be freely distributed by us so you will need to refer to your respective distro to learn how to install them.

 

Thanks !