NVEnc leads to wrong segments playing?

[jhoff80 87]

I'm trying out hardware encode and decode using a Quadro P2000.  If I turn on hardware decode and encode, I've been seeing a weird problem where Android TV clients play a few minutes of the video (with sound) correctly.  But then after a few minutes, the picture freezes and the audio jumps backwards to the beginning of where the video started playing from.  However, if I leave hardware decode on and disable hardware encode, it seems to play the entire video correctly.

 

This happened on 3.4.0.0, 3.4.1.0, and I also tried 3.4.1.1 beta to see if that fixed it.  Anyone else seeing anything remotely similar?

server-log.txt

[Luke 37287]

Interesting, we have seen this with quicksync. Can you please attach an ffmpeg sample log? thanks.

[jhoff80 87]

Sure, this is a video that was playing that had this issue right before I saved that log to upload.  (This specific instance was resumed around 7 minutes in).

encode-log.txt

[Luke 37287]

@@Waldonnis, what do you think about this? 

[Waldonnis 148]

Hmm, was the jumping happening not long after the 7min mark and only occur if you started playback somewhere other than at the beginning?

 

I just tried to reproduce this to look at the segments/playlist and saw a very strange value in the playlist:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-ALLOW-CACHE:YES
#EXT-X-TARGETDURATION:452
#EXTINF:451.409300,
seg_147.ts
#EXTINF:10.427089,
seg_148.ts

 

That first segment's duration is definitely not 451secs long, but does match the segment duration added to the initial seek time.  This might indicated an issue with the segment muxer.  I tried using the hls muxer instead and the playlist looked more like what I would expect (note the difference in the EXT-X-MEDIA-SEQUENCE as well):

 

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:10
#EXT-X-MEDIA-SEQUENCE:147
#EXTINF:10.427778,
seg_147.ts
#EXTINF:10.427778,
seg_148.ts

Even stranger, taking hardware de/encoding completely out of the picture changed nothing - segment muxer's output is virtually unchanged.  Timestamps of the generated segments themselves seem fine, so I'm not sure why hardware encoding would loop and software encoding wouldn't unless the command line changes appreciably when using libx264. Edited May 10, 2018 by Waldonnis

[Luke 37287]

Interesting thanks. I'm guessing you're not seeing that with all videos though right? I have seen that before once or twice.

[simpman 6]

This seems similar to a problem I reported a while back, where when it reached the end of the m3u8, the client requests a ts out of range and emby relaunches ffmpeg ~50% into the video instead of ending playback and would repeat indefinitely until it doesn't have enough video left to give you 50% of then it quits. Affects all my hardware transcoding using NVEnc. I'm currently using Emby 3.4.1.2 and ffmpeg 4 and remains an issue. You can check my logs in my original bug report https://emby.media/community/index.php?/topic/51845-video-loops-when-transcoding/

 

 

 

Using Windows 10 build 1803 Emby 3.4.1.2 beta Geforce GTX 960

[Waldonnis 148]

I've actually tried to reproduce this behaviour several times, but can't seem to reliably do it.  I've seen it even with remuxes in the past, which is why I'm wondering why software encoding is so special so as not to trigger it.  The only thing I can think of relates to keyframe generation (specifically whether or not the hardware encoders obey forcing keyframes the way Emby does it), but I really want to reproduce it reliably first so I have some known-repeating conditions to look at and work with.  Sadly, this is just one of those issues that logs don't really help with since they aren't detailed enough to tell what's going on with the output and within ffmpeg.

[simpman 6]

I have done more investigating on this matter and have found a solution, I believe. The problem is that when I use the exact same ffmpeg command for both software and hardware encoding, the segments on the hardware encoded end up being at least twice as long as the ones from the software encoder. When I use the "-forced-idr 1" option in ffmpeg command line for the hardware encode is produces segments equal to that of the software encoder without that option. I have not tested that this fixes the issue of the video playing wrong segments but I suspect it may. How can I test this within Emby?

 

I used this command line, based on the one found in my emby logs,  but I changed the video codec and added -forced-idr 1 on the various tests

C:\Users\simpm\AppData\Roaming\Emby-Server\system\ffmpeg.exe -resize 1280x720 -i file:"M:\test.mkv" -threads 0 -map 0:0 -map 0:1 -map -0:s -codec:v:0 h264_nvenc -pix_fmt yuv420p -preset medium -b:v 6530560 -maxrate 6530560 -bufsize 13061120 -force_key_frames "expr:if(isnan(prev_forced_t),eq(t,t),gte(t,prev_forced_t+2))" -copyts -vsync -1 -codec:a:0 ac3 -ac 2 -ab 192000  -f segment -max_delay 5000000 -avoid_negative_ts disabled -map_metadata -1 -map_chapters -1 -start_at_zero -segment_time 3  -individual_header_trailer 0 -segment_format mpegts -segment_list_type m3u8 -segment_start_number 0 -segment_list "L:\Buffer\transcoding-temp\hardware.m3u8" -y "L:\Buffer\transcoding-temp\hardware%d.ts" 

 

I've included the m3u8 files from my test, hopefully this helps to sort this issue out finally.

m3u8.zip

[Luke 37287]

@@Waldonnis, what do you think about this change?

[Waldonnis 148]

-forced-idr is something I keep meaning to get back to, but my tests before looked good.  Basically, the nvenc encoders seem to ignore -force_key_frames...well, it doesn't ignore it but rather doesn't mark them as IDR, which may as well make them irrelevent when the segment muxer tries to do its thing and you get segments of various lengths.

 

I'd say that it's definitely worth adding to a beta server, as the -force_key_frames option does nothing without it when using h264_nvenc or hevc_nvenc. I can't think of a drawback offhand, but it would be good to put through its paces just in case and should allow for predictable (and intended) segmentation.  I'd have to go back and look at my notes about hardware encoding and the segment muxer to remember if QuickSync suffers from something similar, but I think it behaved itself without any extra options.  I didn't have time or hardware to test AMF for this effect at the time, so that's still a big unknown.

[simpman 6]

I could never test QuickSync properly as it always crashes most likely due to this bug in ffmpeg. If anyone knows how to use quicksync in ffmpeg working in Windows 10 while having NVidia as primary graphics let me know thanks!

 

Edit: tried it with most recent nightly, still the same 

>ffmpeg -init_hw_device qsv

ffmpeg version N-91398-gd08d4a8c73 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 7.3.0 (GCC)
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth
  libavutil      56. 18.102 / 56. 18.102
  libavcodec     58. 20.104 / 58. 20.104
  libavformat    58. 17.101 / 58. 17.101
  libavdevice    58.  4.101 / 58.  4.101
  libavfilter     7. 25.100 /  7. 25.100
  libswscale      5.  2.100 /  5.  2.100
  libswresample   3.  2.100 /  3.  2.100
  libpostproc    55.  2.100 / 55.  2.100
[AVHWDeviceContext @ 000001ec156b9dc0] Error setting child device handle: -6

Edited June 30, 2018 by simpman

[Luke 37287]

Your sample command line doesn't seem to have it. Can you provide a sample? thanks.

[simpman 6]

Here is a log that is generated when I try to use QSV, but ends up defaulting to libx264. I have a i7-6700 with Intel HD Graphics 530 and NVidia GTX 960. Both are enabled.

ffmpeg_qsv.log

[Luke 37287]

Do you have a sample command line that you tested with forced-idr?

[simpman 6]

Sure, here is what I used, I just put the forced-idr 1 before the forced keyframes and kept the identical ffmpeg command line that Emby generated.

ffmpeg.exe -i file:"M:\test.mkv" -threads 0 -map 0:0 -map 0:1 -map -0:s -codec:v:0 h264_nvenc -vf "scale=trunc(min(max(iw\,ih*dar)\,min(1280\,720*dar))/2)*2:trunc(min(max(iw/dar\,ih)\,min(1280/dar\,720))/2)*2" -pix_fmt yuv420p -preset medium -b:v 10387518 -maxrate 10387518 -bufsize 20775036 -forced-idr 1 -force_key_frames "expr:if(isnan(prev_forced_t),eq(t,t),gte(t,prev_forced_t+2))" -copyts -vsync -1 -codec:a:0 ac3 -ac 2 -ab 192000  -f segment -max_delay 5000000 -avoid_negative_ts disabled -map_metadata -1 -map_chapters -1 -start_at_zero -segment_time 3  -individual_header_trailer 0 -segment_format mpegts -segment_list_type m3u8 -segment_start_number 0 -segment_list "L:\Buffer\transcoding-temp\29b49cf38da6a54a3a66c9dfb95875a9.m3u8" -y "L:\Buffer\transcoding-temp\29b49cf38da6a54a3a66c9dfb95875a9%d.ts"

[Luke 37287]

Thanks.

[simpman 6]

Can confirm that using beta version 3.4.1.24 seems to have fixed the looping videos/wrong segment problem for me. I will report if I notice any other changes, thanks!

[Luke 37287]

Thanks for the feedback.

[jhoff80 87]

Looks promising here as well.  Haven't seen this issue in the .24 beta.