ffmpeg error: DTS out of order

[adamcureton 0]

Hi,

I cannot playback or transcode in emby certain livestreams that I record because, it seems, the streams are corrupted somehow.  VLC, however, does play the ts files emby produces and Handbrake can convert them to mkv.  In an effort to reproduce the problem, I recorded in emby this ts file without any conversion and then issued the same ffmpeg command that emby uses (replacing the remote stream with the local ts file emby produced):

$ /usr/local/bin/ffmpeg -async 1 -vsync -1 -analyzeduration 5000000 -i "CBS Evening News [New] 2017-07-11.ts" -t 00:29:53.379 -codec:v:0 copy -fflags +genpts -map_metadata -1 -threads 0 -codec:a:0 copy -sn -y "/Volumes/Emby/Emby/livetv/recordings/CBS Evening News [New]/CBS Evening News [New] 2017-07-12.mkv"

This produced an error:

ffmpeg version 3.3.2 Copyright (c) 2000-2017 the FFmpeg developers
  built with Apple LLVM version 8.1.0 (clang-802.0.42)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/3.3.2 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-libmp3lame --enable-libx264 --enable-libxvid --enable-opencl --disable-lzma --enable-vda
  libavutil      55. 58.100 / 55. 58.100
  libavcodec     57. 89.100 / 57. 89.100
  libavformat    57. 71.100 / 57. 71.100
  libavdevice    57.  6.100 / 57.  6.100
  libavfilter     6. 82.100 /  6. 82.100
  libavresample   3.  5.  0 /  3.  5.  0
  libswscale      4.  6.100 /  4.  6.100
  libswresample   2.  7.100 /  2.  7.100
  libpostproc    54.  5.100 / 54.  5.100
Input #0, mpegts, from 'CBS Evening News [New] 2017-07-11.ts':
  Duration: 00:01:04.98, start: 1680.373000, bitrate: 50277 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 (Baseline) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1280x720, 29.97 fps, 29.97 tbr, 90k tbn, 180k tbc
    Stream #0:1[0x101]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 130 kb/s
Output #0, matroska, to '/Volumes/Emby/Emby/livetv/recordings/CBS Evening News [New]/CBS Evening News [New] 2017-07-12.mkv':
  Metadata:
    encoder         : Lavf57.71.100
    Stream #0:0: Video: h264 (Baseline) (H264 / 0x34363248), yuv420p(progressive), 1280x720, q=2-31, 29.97 fps, 29.97 tbr, 1k tbn, 90k tbc
    Stream #0:1: Audio: aac (LC) ([255][0][0][0] / 0x00FF), 48000 Hz, stereo, fltp, 130 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
[mpegts @ 0x7f9f89802600] DTS 8589934592 < 8591730542 out of order
    Last message repeated 1 times
frame= 6414 fps=0.0 q=-1.0 size=   43658kB time=00:03:34.12 bitrate=1670.3kbits/frame=12976 fps=12976 q=-1.0 size=   88299kB time=00:07:13.05 bitrate=1670.3kbitframe=19523 fps=13015 q=-1.0 size=  132770kB time=00:10:51.51 bitrate=1669.4kbitframe=22711 fps=11354 q=-1.0 size=  154553kB time=00:12:37.89 bitrate=1670.5kbitframe=27959 fps=11180 q=-1.0 size=  190055kB time=00:15:43.02 bitrate=1651.0kbitframe=33029 fps=10795 q=-1.0 size=  224424kB time=00:18:42.21 bitrate=1638.3kbitframe=35909 fps=10058 q=-1.0 size=  243853kB time=00:20:18.30 bitrate=1639.7kbitframe=40858 fps=9921 q=-1.0 size=  277523kB time=00:23:03.47 bitrate=1643.3kbitsframe=47411 fps=10266 q=-1.0 size=  321865kB time=00:26:42.10 bitrate=1645.8kbit[mpegts @ 0x7f9f89802600] DTS 145835190 < 8735766812 out of order
frame=53142 fps=10491 q=-1.0 Lsize=  360729kB time=00:29:53.37 bitrate=1647.8kbits/s speed= 354x    
video:332016kB audio:28270kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.123082%

Perhaps the issue is related to https://trac.ffmpeg.org/ticket/3553, which has something to do with when the source stream is switched.

 

Attached are logs from attempting to record the same show on a different day, with transcoding turned on.  Here's the mkv file that was produced.

 

Any help would be greatly appreciated.

server log 7-10.txt

record-transcode log - 7-10.txt

[Luke 42079]

Hi, as a test, can you try unchecking the recording option to preserve original video? See how it compares when video is converted during the recording. thanks !

[adamcureton 0]

I'm fairly sure I already tried it both ways, with that option checked and without it checked.  The same results occur.  I can try again this evening.

 

It's strange because it only seems to be certain shows that cause this problem, and the ts file I sent you does seem to be corrupted somehow.

 

Thanks.

[adamcureton 0]

Same problem.  Here's the file that was produced:  

[Luke 42079]

Are you able to play this file from Emby apps?

[adamcureton 0]

Hi Luke,

 

Sorry to be bothering you about this.  Yes, the file plays, but it's only 30 seconds long (10.8 MB) for what should be a 30 minute program.  Strangely, ffmpeg seems to have problems when the station I'm streaming switches from local news to the national broadcast.  As you can tell from the ts file I sent earlier, emby seems to record all of the data, but there is some sort of corruption that results in ffmpeg exiting early.  The FireTV emby app also has trouble playing the ts file.  It's just very strange.

 

Again, thanks for your help!

Adam

[Waldonnis 148]

I've seen DTS issues with several streams myself, but not this particular source...makes sense why it happens, but I don't have live TV, so my experience is a tad different.  I quickly downloaded the ts file you posted and saw the same thing in playback with MPC-HC and seeking (probably libav-related, since both the filters and ffmpeg use it).  No matter where I seek, it just plays the local segment at the beginning of the file ~65secs and shows that as the duration by the scrub bar).  Hilariously, if I just let it play without seeking, it continues on and plays the full stream, but can't seek beyond that first 65secs since the progress indicates it's at the end...

 

I tried a quick remux with -fflags +igndts and the resulting file seemed to play well, shows the proper duration, and seeking was fine.  May be worth a test locally if you can manage it manually.  I didn't do a frame dump to confirm that it fixed the DTS sequence, but a quick basic remux of the seemingly-fixed copy (no flags other than -c copy) didn't show any DTS warnings so it probably fixed them.

[Luke 42079]

@@Waldonnis, yes we're doing that with HDHR streams, but he's using IPTV, so the input is a little less predictable for us.

 

@@adamcureton are you able to play live tv normally?

[adamcureton 0]

This is all so weird, isn't it.

 

@@Waldonnis - I converted the file with Handbrake, which worked like you said, so now I'll figure out how to do it with ffmpeg and report back.

 

@@Luke - Yes, I can watch the stream live, but I haven't tried watching it live when the local segment switches to the national segment, so I'll try that.

 

I'm just glad you all are seeing the same problem, which was driving me nuts!

[adamcureton 0]

Just to report, the +igndts flag does seem to work for me.

 

When I do:

ffmpeg -y -fflags +igndts -f mpegts -i "/Volumes/Emby/Emby/livetv/recordings/CBS Evening News [New]/CBS Evening News [New] 2017-07-11.ts" -acodec copy -vcodec copy test.mp4

This file is produced, which still has some repetitive bits at the beginning but plays in emby.

 

I suppose for now I'll just remux the suspect files manually and hope you all come up with a more effective solution.

 

Thanks!

[Waldonnis 148]

@@Waldonnis, yes we're doing that with HDHR streams, but he's using IPTV, so the input is a little less predictable for us.

 

@@adamcureton are you able to play live tv normally?

 

Ah, then it's exactly the same situation that I've run into     I've seen this a lot with IPTV streams when multiple independently-encoded sources are concatenated...which is quite frequent with some material.  Drives me nuts when it breaks playback or causes audio sync issues, forcing me to fix the DTS values.

 

Just to report, the +igndts flag does seem to work for me.

 

When I do:

ffmpeg -y -fflags +igndts -f mpegts -i "/Volumes/Emby/Emby/livetv/recordings/CBS Evening News [New]/CBS Evening News [New] 2017-07-11.ts" -acodec copy -vcodec copy test.mp4

This file is produced, which still has some repetitive bits at the beginning but plays in emby.

 

I suppose for now I'll just remux the suspect files manually and hope you all come up with a more effective solution.

 

Thanks!

 

Glad I could provide a workaround for now.  I've run across this type of thing quite a bit in the past...so much so that I added igndts to a few of my scripts just so I didn't have to think about it for specific types of sources.  There are many reasons for wacky DTS jumps like that, though, so it may not fix all of them (as Luke implied).

[adamcureton 0]

Great, thanks @@Waldonnis.  @@Luke, let us know about any updates on your end.

 

Again, thanks to you both.

[Luke 42079]

Thanks for the feedback.