GPU Transcoding (Intel QuickSync and nVidia NVENC)

[Luke 42077]

Hi @@bxsteez, perhaps if you can provide a sample file for testing it will be easier to understand what your issue is. thanks.

[bxsteez 3]

Hi @@bxsteez, perhaps if you can provide a sample file for testing it will be easier to understand what your issue is. thanks.

Let me do some searching around for a sample file.  The files I have are super huge.

[Andy777 21]

I have been trying to get hardware acceleration working on a specific file for a while now and I've given up until I can build ffmpeg myself.  I'm pretty sure that is the issue.   I think it's my pixel format that is throwing the error.  I don't believe yuv420p10le is supported. 

 

Has anyone gotten this to work? It does play fine on the source computer but just not with ffmpeg.

 

The chances of getting hw-acceleration for 10bit h264 (even decode) are close to zero. The format/profile is not used anywhere else except the anime community so there is no interest to any companies implementing it in hardware. 

[jscoys 147]

The chances of getting hw-acceleration for 10bit h264 (even decode) are close to zero. The format/profile is not used anywhere else except the anime community so there is no interest to any companies implementing it in hardware.

It seems that GeForce 980 is able to. I don't know for the 1050 or 1080 but I think they should be able to too.

 

 

Sent from my iPad using Tapatalk

[lorac 118]

When I last tried it with my 1050 the CPU usage was over 70% as compared to about 20% w/o hardware decoding.

 

Sent from my STV100-3 using Tapatalk

[Waldonnis 148]

Hi @@bxsteez, perhaps if you can provide a sample file for testing it will be easier to understand what your issue is. thanks.

 

I do have a 30sec clip that I generated to test things out with, if that'll help.  It's just basic colour bars, so it's nothing fancy and is quite tiny (~430k).  I'd need for bxsteez to verify that it too fails to transcode on his end, since my 970 can't do HEVC Main10 encoding.

 

I'd love to see the output from a recent build of ffmpeg with -loglevel 56 on this using a Pascal card. I'm trying to figure out if it's just dying because the driver isn't reporting that it's a 10bit capable card to ffmpeg.  After a little browsing of the source and some more recent commits, I have a few new questions as well and am hoping the log output answers some questions.  I can supply a command line for this if needed.

yuv420p10le.zip

Edited January 12, 2017 by Waldonnis

[mediacowboy 438]

Okay this may sound noobish but I have read this whole thread and wondering does quicksync work realilby now or is it still experimental? I am working on a new build hoping to take advantage of this feature but wasn't sure.

[esseki 12]

Okay this may sound noobish but I have read this whole thread and wondering does quicksync work realilby now or is it still experimental? I am working on a new build hoping to take advantage of this feature but wasn't sure.

 

 

I can confirm that i'm using Quicksync running from an ASRock Beebox with an Intel Celeron N3150 (Braswell) and Win10.

Everything is working like a charm for months.

In my opinion you can go ahead

[Waldonnis 148]

While playing with nvenc a bit to try to find some workable/reliable settings for decent quality, I noticed something that I must've missed before: the encoding output only has one reference frame by default (h264) and no b-frames at all (this totally surprised me).  There are options to set/increase each (-refs and -bf), but they're not listed in the encoder help text for some reason (there are also ways to define the GOP order, if that's of any interest).  Given that we've seen issues with some playback devices and videos with only 1 reference frame, I'm wondering if anyone's run across playback issues related to this and if forcing more ref frames during nvenc hardware transcoding may be a good idea if that limitation is still an issue on those devices (after proper testing, of course).  B-frames would be nice to have with h.264 transcodes, though (the nvenc hevc implementations don't seem to support b-frames, but I haven't verified that against the newer cards' documented feature sets yet).

 

Additionally, the -crf isn't obeyed at all by the X_nvenc ffmpeg encoders (wtb decent docs), but it is achievable by setting -rc to constqp and using -global_quality to define the value (I think the range is 0-51 with a default of 23, but I'd have to verify that).

 

Sample command line combining the above (no audio considered):

ffmpeg -i foo.mkv -c:v "h264_nvenc" -rc constqp -refs 4 -bf 4 -global_quality 23 bar.mkv

There are also ways to set min/max quantizer values for a specified target/max bitrate.  I tested this before, but would have to dig out my notes on that.  This would probably be of interest, since Emby may need to transcode due to bitrate limitations of a connection or device, so I'll poke around for where I noted that info and follow up on that.  I still have to check into a few more options but I've played with most of them at this point and found a few that aren't documented properly (if at all).

 

Not sure how helpful this is on an Emby front, but figured I'd post it in case Luke/ebr weren't aware and/or if there may be an issue with the reference frame default.  Also, it may be helpful to others if they're thinking of using hardware encoders for other encoding tasks.

[bxsteez 3]

I do have a 30sec clip that I generated to test things out with, if that'll help.  It's just basic colour bars, so it's nothing fancy and is quite tiny (~430k).  I'd need for bxsteez to verify that it too fails to transcode on his end, since my 970 can't do HEVC Main10 encoding.

 

I'd love to see the output from a recent build of ffmpeg with -loglevel 56 on this using a Pascal card. I'm trying to figure out if it's just dying because the driver isn't reporting that it's a 10bit capable card to ffmpeg.  After a little browsing of the source and some more recent commits, I have a few new questions as well and am hoping the log output answers some questions.  I can supply a command line for this if needed.

Sorry.  It's been a while for me since my laptop broke.  Shoot me over the command and I'll run it on my server that holds my GTX1070.

[bxsteez 3]

The chances of getting hw-acceleration for 10bit h264 (even decode) are close to zero. The format/profile is not used anywhere else except the anime community so there is no interest to any companies implementing it in hardware. 

I did have this working on my 1070 for a min.  It had CPU utilization of about 30% while decoding a 10bit 4k movie.  Of course have 10bit hevc(h265) video file. 

[Waldonnis 148]

Sorry.  It's been a while for me since my laptop broke.  Shoot me over the command and I'll run it on my server that holds my GTX1070.

 

Try this:

ffmpeg -i yuv420p10le.mkv -c:v "hevc_nvenc" -loglevel 56 -y -report test.mkv

It will generate a report file that you can PM to me (should be named ffmpeg-date-number.log) that contains the entire output.  The sample clip itself has no audio and shouldn't take much time to process if it does encode.

 

Edit: Corrected my typo, thanks to bxsteez pointing out my finger failure 

Edited January 22, 2017 by Waldonnis

[bxsteez 3]

Try this:

ffmpeg -i yuv420p10le.mkv -c:v "hevc_nvnenc" -loglevel 56 -y -report test.mkv

It will generate a report file that you can PM to me (should be named ffmpeg-date-number.log) that contains the entire output.  The sample clip itself has no audio and shouldn't take much time to process if it does encode.

I PMed you the log. 

[KingDaveRa 2]

My Emby server is running on Ubuntu Server 16.10. It's an AMD A8-based system with a quad core and 6 GPU cores on the processor. I believe it's capable of doing video transcoding based on what I've read, and I have a feeling Emby supports it through OpenMax possibly. However, I'm having little luck fully joining the dots in documentation (most of what I've found is about the APIs, rather than how to set the thing up). So I'm a bit confused about what I need to do to get it running, or even if it's possible. My Emby server is my NAS, and runs headless - there's no xserver or anything on it, so I'm pretty sure I've got a bit of setup to do first, but I'm honestly not sure.

 

Any help most gratefully received.

[Waldonnis 148]

My Emby server is running on Ubuntu Server 16.10. It's an AMD A8-based system with a quad core and 6 GPU cores on the processor. I believe it's capable of doing video transcoding based on what I've read, and I have a feeling Emby supports it through OpenMax possibly. However, I'm having little luck fully joining the dots in documentation (most of what I've found is about the APIs, rather than how to set the thing up). So I'm a bit confused about what I need to do to get it running, or even if it's possible. My Emby server is my NAS, and runs headless - there's no xserver or anything on it, so I'm pretty sure I've got a bit of setup to do first, but I'm honestly not sure.

 

Any help most gratefully received.

 

The ffmpeg docs claim AMD VCE should work by using VA-API on Linux (no Windows support, though, if others are interested), but I can't personally test it to verify that.  Here's the relevant snippet from the ffmpeg site:

 

AMD VCE is exposed through VA-API on linux. For windows there have been port attempts but nothing official yet.

 

I know OpenMax support was added to ffmpeg for the RPi platform, but no clue about others (or even if it would be applicable).  Best bet seems to be trying VA-API and see how it works out.

[KingDaveRa 2]

The ffmpeg docs claim AMD VCE should work by using VA-API on Linux (no Windows support, though, if others are interested), but I can't personally test it to verify that.  Here's the relevant snippet from the ffmpeg site:

 

 

I know OpenMax support was added to ffmpeg for the RPi platform, but no clue about others (or even if it would be applicable).  Best bet seems to be trying VA-API and see how it works out.

 

Aaah, excellent, thanks. I've been looking at the wrong API all this time!

 

I'll do some digging into VA-API. Many thanks.

[gbone8106 25]

I have 1050 card, has anyone gotten this to work?  I am on windows using zeranoe ffmpeg static build. I am getting about 60fps and the cpu is around 90%, which doesn't seem right. Any help would be appreciated?  below is the command from the transcode log.

 

 

Batman v Superman Dawn of Justice (2016)\Batman v Superman Dawn of Justice (2016).mkv" -map 0:1 -map 0:2 -map -0:s -codec:v:0 h264_nvenc -force_key_frames "expr:gte(t,n_forced*5)" -pix_fmt yuv420p -preset default -maxrate 3465143 -bufsize 6930286 -vsync -1 -profile:v high -level 4.1 -map_metadata -1 -map_chapters -1 -threads 0 -codec:a:0 libmp3lame -ac 2 -ab 192000 -af "aresample=async=1,volume=2" -y "C:\Users\gary\AppData\Roaming\MediaBrowser-Server\transcoding-temp\30e82d130117623ea75f346b88c5e70c.mkv"

[lorac 118]

I don't believe it's working with 'Pascal' cards right now. My testing has shown it to be worse. Others have reported it working with older NVIDIA cards though.

 

Sent from my STV100-3 using Tapatalk

[Waldonnis 148]

I have 1050 card, has anyone gotten this to work?  I am on windows using zeranoe ffmpeg static build. I am getting about 60fps and the cpu is around 90%, which doesn't seem right. Any help would be appreciated?  below is the command from the transcode log.

 

Can you attach the transcode log from that sesssion as well as the original media info on the file?  Please attach if you can rather than pasting, since it's a lot to scroll through in a forum and it makes it easier for me to search for specific things.  Thanks!

 

The command looks okay at a glance, but without seeing the ffmpeg output and what you're transcoding from, it'll be hard to tell what may be going on.  Since I think Emby's ffmpeg command lines use software decoding, that may account for the CPU use as well (especially if the original file is HEVC or VP9).  I'll know more when I see the source's media info and the log output, but it's very possible that it's actually using the hardware encoder while still requiring a lot of CPU power as well for related operations.

[gbone8106 25]

hey yea go

 

 

Media Info

Split Versions Apart

AudioLanguageeng
CodecDCA
ProfileDTS-HD MA
Layout7.1
Channels8 ch
Sample rate48000 Hz
Bit depth24 bit
DefaultYes
TitleEng Batman.v.Superman.Dawn.of.Justice.2016.EXTENDED.1080p.BluRay.x264.DTS-HD.MA.7.1-RARBG

VideoCodecH264
AVCYes
ProfileHigh
Level41
Resolution1920x1080
Aspect ratio16:9
AnamorphicNo
InterlacedNo
Framerate23.97602
Bitrate33731 kbps
Bit depth8 bit
Pixel formatyuv420p
Ref frames2
NAL4
TitleEng Batman.v.Superman.Dawn.of.Justice.2016.EXTENED.Ultimate.Edition.1080p.Blu-ray.REMUX.AVC.TrueHD.Atmos.7.1-HDArea

 

ffmpeg-transcode-33144aab-bf2e-402d-aadc-adee52c83d2a.txt

[Waldonnis 148]

 

hey yea go

 

...

 

Odd that it's taking that much CPU time for ffmpeg, unless the processor is relatively weak or somehow unable to keep the card fed with data to encode at higher speeds.  ffmpeg is still doing some stuff on the CPU (audio transcoding, muxing, video decoding, handing the data off to the card, etc) so there will always be some overhead involved.  Muxing overhead was low, though, and the fps values were on the low side, but still within what I would expect for hardware video encoding early on.  I'm concerned about it dipping into the 60s that early, though, since the first few chapters of the movie aren't that difficult to encode.  What processor is in your server?

 

I actually keep a full copy of BvS that I use for testing certain things and my fps is often in the mid-to-high 100s using a 970 - and that's with forcing refframe/b-frame creation and doing constant quality encodes (worth noting that I rarely transcode the audio during testing, so that may account for some of it).  I'll run a few tests locally on BvS with a similar command line to see what I can come up with.

 

Update: Just ran a test with the same command line that encoded the first 10mins of BvS and the 970's chewing through it at around 160-180fps throughout.  Processor use was under 25% the entire time, and I was transcoding the DTS-HD audio track as well.  I'm definitely curious now where the bottleneck may be with your system.

Edited January 27, 2017 by Waldonnis

[brando56894 6]

I have 1050 card, has anyone gotten this to work?  I am on windows using zeranoe ffmpeg static build. I am getting about 60fps and the cpu is around 90%, which doesn't seem right. Any help would be appreciated?  below is the command from the transcode log.

 

 

Batman v Superman Dawn of Justice (2016)\Batman v Superman Dawn of Justice (2016).mkv" -map 0:1 -map 0:2 -map -0:s -codec:v:0 h264_nvenc -force_key_frames "expr:gte(t,n_forced*5)" -pix_fmt yuv420p -preset default -maxrate 3465143 -bufsize 6930286 -vsync -1 -profile:v high -level 4.1 -map_metadata -1 -map_chapters -1 -threads 0 -codec:a:0 libmp3lame -ac 2 -ab 192000 -af "aresample=async=1,volume=2" -y "C:\Users\gary\AppData\Roaming\MediaBrowser-Server\transcoding-temp\30e82d130117623ea75f346b88c5e70c.mkv"

 

I usually use my FreeBSD-based home server to do my encoding but since it doesn't have a videocard, I decided to give it a try on Windows 10 since I just got a GTX 1070 and wanted to see how the performance differed. After the initial setup I had it transcode a video, but I saw it was using the CPU and not the GPU, so I grabbed the static build you mentioned, but it seems it fails before it even starts to transcode.

http://192.168.1.37:8096/videos/665d599e1e33f670931bc49cb5e071cc/stream.mkv?DeviceId=4f2a3aa8e67908606455faf811a09dc1811ebdee&MediaSourceId=665d599e1e33f670931bc49cb5e071cc&VideoCodec=h264&AudioCodec=ac3,mp3,aac&AudioStreamIndex=1&VideoBitrate=10808000&AudioBitrate=192000&Level=51&Profile=high&PlaySessionId=b03f823172934706bb7346739f7e1d50&api_key=5ff1d50ef0d14e74a1be57f9b9370ffd&CopyTimestamps=true&TranscodingMaxAudioChannels=6&EnableSubtitlesInManifest=false&Tag=287d4db39bb8906186841557318a2e22&RequireAvc=true

{"Protocol":"File","Id":"665d599e1e33f670931bc49cb5e071cc","Path":"\\\\FREENAS\\multimedia\\movies\\Deadpool (2016)\\Deadpool (2160p).mkv","Type":"Default","Container":"mkv","Name":"4K/H264/DTS-HD MA","IsRemote":false,"ETag":"287d4db39bb8906186841557318a2e22","RunTimeTicks":64862400512,"ReadAtNativeFramerate":false,"SupportsTranscoding":true,"SupportsDirectStream":true,"SupportsDirectPlay":true,"IsInfiniteStream":false,"RequiresOpening":false,"RequiresClosing":false,"SupportsProbing":true,"VideoType":"VideoFile","MediaStreams":[{"Codec":"h264","Language":"eng","TimeBase":"1/1000","CodecTimeBase":"1001/48000","Title":"UHD 4K -DDR","DisplayTitle":"Eng UHD 4K -DDR","NalLengthSize":"4","IsInterlaced":false,"IsAVC":true,"BitRate":44707955,"BitDepth":8,"RefFrames":1,"IsDefault":true,"IsForced":false,"Height":2160,"Width":3840,"AverageFrameRate":23.97602,"RealFrameRate":23.97602,"Profile":"Constrained Baseline","Type":"Video","AspectRatio":"16:9","Index":0,"IsExternal":false,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"PixelFormat":"yuv420p","Level":13,"IsAnamorphic":false},{"Codec":"dca","Language":"eng","TimeBase":"1/1000","CodecTimeBase":"1/48000","Title":"DTS-HDMA 7.1 -DDR","DisplayTitle":"Eng DTS-HDMA 7.1 -DDR","IsInterlaced":false,"ChannelLayout":"7.1","BitDepth":24,"Channels":8,"SampleRate":48000,"IsDefault":true,"IsForced":false,"Profile":"DTS-HD MA","Type":"Audio","Index":1,"IsExternal":false,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Level":0,"IsAnamorphic":false}],"PlayableStreamFileNames":[],"Formats":[],"Bitrate":44707955,"RequiredHttpHeaders":{}}

C:\Users\bran\Downloads\ffmpeg-20170125-2080bc3-win64-static\ffmpeg-20170125-2080bc3-win64-static\bin\ffmpeg.exe -fflags +genpts -i file:"\\FREENAS\multimedia\movies\Deadpool (2016)\Deadpool (2160p).mkv" -map 0:0 -map 0:1 -map -0:s -codec:v:0 h264_nvenc -force_key_frames "expr:gte(t,n_forced*5)" -copyts -avoid_negative_ts disabled -start_at_zero -pix_fmt yuv420p -preset default -b:v 10808000 -maxrate 10808000 -bufsize 21616000 -vsync -1 -profile:v high -level 4.1 -map_metadata -1 -map_chapters -1 -threads 0 -codec:a:0 ac3 -ac 6 -ab 192000 -af "aresample=async=1" -y "C:\Users\bran\AppData\Roaming\Emby-Server\transcoding-temp\171d6aafb302d3722b10311c2d12d213.mkv"


ffmpeg version N-83243-g2080bc3 Copyright (c) 2000-2017 the FFmpeg developers
built with gcc 5.4.0 (GCC)
configuration: --enable-gpl --enable-version3 --enable-cuda --enable-cuvid --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-decklink --enable-zlib
libavutil 55. 45.100 / 55. 45.100
libavcodec 57. 75.100 / 57. 75.100
libavformat 57. 63.100 / 57. 63.100
libavdevice 57. 2.100 / 57. 2.100
libavfilter 6. 70.100 / 6. 70.100
libswscale 4. 3.101 / 4. 3.101
libswresample 2. 4.100 / 2. 4.100
libpostproc 54. 2.100 / 54. 2.100
Input #0, matroska,webm, from 'file:\\FREENAS\multimedia\movies\Deadpool (2016)\Deadpool (2160p).mkv':
Metadata:
title : Deadpool 2016 VFQ 2160p UHD 4K AC3 x264-SubZero
encoder : libebml v1.3.3 + libmatroska v1.4.4
creation_time : 2016-05-18T21:17:41.000000Z
Duration: 01:48:06.24, start: 0.000000, bitrate: 44707 kb/s
Stream #0:0(eng): Video: h264 (Constrained Baseline), yuv420p(progressive), 3840x2160 [SAR 1:1 DAR 16:9], 23.98 fps, 23.98 tbr, 1k tbn, 47.95 tbc (default)
Metadata:
title : UHD 4K -DDR
BPS : 39812682
BPS-eng : 39812682
DURATION : 01:48:06.105000000
DURATION-eng : 01:48:06.105000000
NUMBER_OF_FRAMES: 155511
NUMBER_OF_FRAMES-eng: 155511
NUMBER_OF_BYTES : 32278655118
NUMBER_OF_BYTES-eng: 32278655118
_STATISTICS_WRITING_APP: mkvmerge v9.0.1 ('Obstacles') 64bit
_STATISTICS_WRITING_APP-eng: mkvmerge v9.0.1 ('Obstacles') 64bit
_STATISTICS_WRITING_DATE_UTC: 2016-05-18 21:17:41
_STATISTICS_WRITING_DATE_UTC-eng: 2016-05-18 21:17:41
_STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream #0:1(eng): Audio: dts (DTS-HD MA), 48000 Hz, 7.1, s32p (24 bit) (default)
Metadata:
title : DTS-HDMA 7.1 -DDR
BPS : 4891980
BPS-eng : 4891980
DURATION : 01:48:06.240000000
DURATION-eng : 01:48:06.240000000
NUMBER_OF_FRAMES: 608085
NUMBER_OF_FRAMES-eng: 608085
NUMBER_OF_BYTES : 3966320176
NUMBER_OF_BYTES-eng: 3966320176
_STATISTICS_WRITING_APP: mkvmerge v9.0.1 ('Obstacles') 64bit
_STATISTICS_WRITING_APP-eng: mkvmerge v9.0.1 ('Obstacles') 64bit
_STATISTICS_WRITING_DATE_UTC: 2016-05-18 21:17:41
_STATISTICS_WRITING_DATE_UTC-eng: 2016-05-18 21:17:41
_STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
[h264_nvenc @ 0000000000e6d480] InitializeEncoder failed: invalid param (8)
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_nvenc))
Stream #0:1 -> #0:1 (dts (dca) -> ac3 (native))
Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height

Edited January 29, 2017 by brando56894

[Waldonnis 148]

I usually use my FreeBSD-based home server to do my encoding but since it doesn't have a videocard, I decided to give it a try on Windows 10 since I just got a GTX 1070 and wanted to see how the performance differed. After the initial setup I had it transcode a video, but I saw it was using the CPU and not the GPU, so I grabbed the static build you mentioned, but it seems it fails before it even starts to transcode.

C:\Users\bran\Downloads\ffmpeg-20170125-2080bc3-win64-static\ffmpeg-20170125-2080bc3-win64-static\bin\ffmpeg.exe -fflags +genpts -i file:"\\FREENAS\multimedia\movies\Deadpool (2016)\Deadpool (2160p).mkv" -map 0:0 -map 0:1 -map -0:s -codec:v:0 h264_nvenc -force_key_frames "expr:gte(t,n_forced*5)" -copyts -avoid_negative_ts disabled -start_at_zero -pix_fmt yuv420p -preset default -b:v 10808000 -maxrate 10808000 -bufsize 21616000 -vsync -1 -profile:v high -level 4.1 -map_metadata -1 -map_chapters -1 -threads 0 -codec:a:0 ac3 -ac 6 -ab 192000 -af "aresample=async=1" -y "C:\Users\bran\AppData\Roaming\Emby-Server\transcoding-temp\171d6aafb302d3722b10311c2d12d213.mkv"

 

Transcoding 2160p h.264 is probably going to be iffy no matter what (ffmpeg isn't being told to downscale for some reason either; browser playback attempt?).  The encoder on a 1070 should be able to handle the resolution, but not necessarily all of the parameters of that particular file.  Playback of 2160p h.264 can also be spotty as many devices don't support it with hardware decoders (or at all).  You can try manually transcoding it with an increased loglevel to find out what's really choking it, but I doubt you'll find a workaround for this short of re-encoding the file and fixing whatever the issue might be.  I can see a few potential causes, but don't want to speculate without more info.  Granted, it's not a 10bit HDR file, but HEVC would still probably be a better and more compatible option for decoding/transcoding at that resolution.

 

I kinda like 2160p h.264 personally, but it's sadly not supported by much of anything since the advent of HEVC as it's just not as efficient comparatively, especially at higher bit depths.  I've managed to coax my 970 to encode >1080p h.264, but mostly just to see if it could.

[brando56894 6]

Thanks, this was just a quick test. I know my roommate's first generation chromecast can't plat 4K and Luke was telling me before that my CPU wasn't fast enough to transcode it, so I figured I'd give my GPU a try. 

[lorac 118]

I'm finding NVENC working properly now using my 1050 video card. I'm using the latest driver that came out this month from Nvidia and ffmpeg built Jan. 25th from Zeranoe. I have three transcoding sessions in progress using less than 20% CPU load.