[solved] Question about my custom ffmpeg build

[mueslo 16]

I recently upgraded Emby from 4.1.0.x to 4.2.0.24. At the same time, I also upgraded my FreeNAS from 11.1-U7 to 11.2-U5, since the base system had diverged too much that 11.1 jails would no longer run with the 11.2 packages required by Emby>=4.2 (missing libdl.so.1 for ffmpeg/x265 packages)

 

I of course upgraded the jail base system ("iocage upgrade -r 11.2-RELEASE emby"), and updated the jail packages "pkg update && pkg upgrade". Now ffmpeg runs fine again, in theory.

 

However, I'm getting the "no compatible streams" error message. The relevant section of the emby log is pasted below. Exit code 139 (128+11) suggests a SIGSEGV/Segmentation fault is the culprit. However, when I run the command listed in the firstline with "sudo -u emby <...>", it runs just fine.

2019-07-03 04:49:09.680 Info App: ProcessRun 'StreamTranscode 365ab0' Execute: /usr/local/bin/ffmpeg -f matroska -noaccurate_seek -i file:"/path/file.mkv" -threads 1 -map 0:0 -map 0:1 -sn -c:v:0 copy -bsf:v h264_mp4toannexb -copyts -vsync -1 -codec:a:0 aac -metadata:s:a:0 language=eng -disposition:a:0 default -ac:a:0 2 -ab:a:0 192000 -af:a:0 "volume=2" -f segment -max_delay 5000000 -avoid_negative_ts disabled -map_metadata -1 -map_chapters -1 -start_at_zero -segment_time 6  -individual_header_trailer 0 -break_non_keyframes 1 -segment_format mpegts -segment_list_type m3u8 -segment_start_number 0 -segment_list "/var/db/emby-server/transcoding-temp/3b9665b343cee13ec513598dffd1f528.m3u8" -y "/var/db/emby-server/transcoding-temp/3b9665b343cee13ec513598dffd1f528%d.ts" -map 0:2 -codec:0 webvtt -sc_threshold 0 -copyts -vsync -1 -f segment -segment_time 6 -segment_format webvtt -max_delay 5000000 -avoid_negative_ts disabled -start_at_zero -write_empty_segments 1 -segment_list_type m3u8 -segment_start_number 0 -break_non_keyframes 1 -segment_list "/var/db/emby-server/transcoding-temp/3b9665b343cee13ec513598dffd1f528_s2.m3u8" -y "/var/db/emby-server/transcoding-temp/3b9665b343cee13ec513598dffd1f528_s2%d.vtt" -map 0:3 -codec:0 webvtt -sc_threshold 0 -copyts -vsync -1 -f segment -segment_time 6 -segment_format webvtt -max_delay 5000000 -avoid_negative_ts disabled -start_at_zero -write_empty_segments 1 -segment_list_type m3u8 -segment_start_number 0 -break_non_keyframes 1 -segment_list "/var/db/emby-server/transcoding-temp/3b9665b343cee13ec513598dffd1f528_s3.m3u8" -y "/var/db/emby-server/transcoding-temp/3b9665b343cee13ec513598dffd1f528_s3%d.vtt"
2019-07-03 04:49:09.690 Info App: ProcessRun 'StreamTranscode 365ab0' Started.
2019-07-03 04:49:09.930 Info App: ProcessRun 'StreamTranscode 365ab0' Process exited with code 139
2019-07-03 04:49:09.931 Error HttpServer: Error processing request
        *** Error Report ***
        Version: 4.2.0.24
        Command line: /usr/local/lib/emby-server/system/EmbyServer.exe -os freebsd -ffdetect /usr/local/lib/emby-server/bin/ffdetect -ffmpeg /usr/local/bin/ffmpeg -ffprobe /usr/local/bin/ffprobe -programdata /var/db/emby-server -updatepackage emby-server-freebsd11_{version}_amd64.txz
        Operating system: Unix 11.2.0.0
        64-Bit OS: True
        64-Bit Process: True
        User Interactive: False
        Mono: 5.10.1.57 (5.10.1.57 Tue Jun 25 02:11:04 UTC 2019)
        Runtime: file:///usr/local/lib/mono/4.5/mscorlib.dll
        Processor count: 4
        Program data path: /var/db/emby-server
        Application directory: /usr/local/lib/emby-server/system
        System.Exception: System.Exception: Error starting ffmpeg
          at Emby.Server.MediaEncoding.Api.BaseStreamingService+<StartFfMpeg>d__72.MoveNext () [0x00403] in <5a7882dc58e84f3daae16c712139f47a>:0
        --- End of stack trace from previous location where exception was thrown ---
          at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <0e06830de9a44394a7e366951eabca52>:0
          at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0003e] in <0e06830de9a44394a7e366951eabca52>:0
          at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x00028] in <0e06830de9a44394a7e366951eabca52>:0
          at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x00008] in <0e06830de9a44394a7e366951eabca52>:0
          at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter[TResult].GetResult () [0x00000] in <0e06830de9a44394a7e366951eabca52>:0
          at Emby.Server.MediaEncoding.Api.Hls.DynamicHlsService+<GetDynamicSegment>d__14.MoveNext () [0x00739] in <5a7882dc58e84f3daae16c712139f47a>:0
        --- End of stack trace from previous location where exception was thrown ---
          at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <0e06830de9a44394a7e366951eabca52>:0
          at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0003e] in <0e06830de9a44394a7e366951eabca52>:0
          at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x00028] in <0e06830de9a44394a7e366951eabca52>:0
          at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x00008] in <0e06830de9a44394a7e366951eabca52>:0
          at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter[TResult].GetResult () [0x00000] in <0e06830de9a44394a7e366951eabca52>:0
          at Emby.Server.Implementations.Services.ServiceExecGeneral+<GetTaskResult>d__3.MoveNext () [0x00081] in <0792edfa455d43edb306e0b4cefc6049>:0
        --- End of stack trace from previous location where exception was thrown ---
          at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <0e06830de9a44394a7e366951eabca52>:0
          at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0003e] in <0e06830de9a44394a7e366951eabca52>:0
          at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x00028] in <0e06830de9a44394a7e366951eabca52>:0
          at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x00008] in <0e06830de9a44394a7e366951eabca52>:0
          at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter[TResult].GetResult () [0x00000] in <0e06830de9a44394a7e366951eabca52>:0
          at Emby.Server.Implementations.Services.ServiceHandler+<ProcessRequestAsync>d__14.MoveNext () [0x00253] in <0792edfa455d43edb306e0b4cefc6049>:0
        --- End of stack trace from previous location where exception was thrown ---
          at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <0e06830de9a44394a7e366951eabca52>:0
          at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0003e] in <0e06830de9a44394a7e366951eabca52>:0
          at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x00028] in <0e06830de9a44394a7e366951eabca52>:0
          at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x00008] in <0e06830de9a44394a7e366951eabca52>:0
          at System.Runtime.CompilerServices.ConfiguredTaskAwaitable+ConfiguredTaskAwaiter.GetResult () [0x00000] in <0e06830de9a44394a7e366951eabca52>:0
          at Emby.Server.Implementations.HttpServer.HttpListenerHost+<RequestHandler>d__50.MoveNext () [0x00ba1] in <0792edfa455d43edb306e0b4cefc6049>:0
        Source: mscorlib
        TargetSite: Void Throw()

Any ideas what the cause might be? There are no errors in the transcoding logs as far as I can see. It just seems like the transcoding never starts (see attached transcoding log). Per attempted stream, 2 transcoding logs and 1 remux log is created (they all look similar to the one attached).

translog.txt

Edited July 3, 2019 by mueslo

[Luke 37008]

Hi there, please attach the complete emby server log. Also did you follow the beta instructions for freebsd on our website?

 

https://emby.media/download.html

 

thanks.

[mueslo 16]

Yes, all dependencies are installed, and the 11.2 instructions here are precisely how I installed it.

 

Here's the full embyserver.txt log

embyserver.txt

[Luke 37008]

Did you tamper with the ffmpeg build at all?

[mueslo 16]

Nope, it's straight from the binary repos. And as I wrote in the first post, I can run the ffmpeg command from the command line just fine.

 

Is there a way to get the stderr of ffmpeg in the logs? That might clarify what the issue is. The log above is with emby debug logging enabled, but it doesn't seem to show (or stderr is empty?)

 

I've attached a log of manually running the same ffmpeg command from the shell (stopped by ctrl-c).

ffmpeg_shell.txt

Edited July 3, 2019 by mueslo

[Luke 37008]

Right, but we don't do that, we ship our own ffmpeg. Your startup command line looks like this:

Command line: /usr/local/lib/emby-server/system/EmbyServer.exe -os freebsd -ffdetect /usr/local/lib/emby-server/bin/ffdetect -ffmpeg /usr/local/bin/ffmpeg -ffprobe /usr/local/bin/ffprobe

When I believe it should look like this:

Command line: /usr/local/lib/emby-server/system/EmbyServer.exe -os freebsd -ffdetect /usr/local/lib/emby-server/bin/ffdetect -ffmpeg /usr/local/lib/emby-server/bin/ffmpeg -ffprobe /usr/local/lib/emby-server/bin/ffprobe

I believe we have actually exposed this for configuration so that you can edit the path to the ffmpeg files used. have you ever customized that?

[mueslo 16]

Oh yeah, I had, to be able to manually run an 11.1-compatible version. I completely forgot about that.

 

Commented out

emby_server_ffmpeg="/usr/local/bin/ffmpeg"
emby_server_ffprobe="/usr/local/bin/ffprobe"

in my /etc/rc.conf, now everything works again.

 

Sorry for the trouble and thanks a lot for the help!

Edited July 3, 2019 by mueslo

[Luke 37008]

Thanks for the feedback.