Error when transcoding movie - Conversion Failed

[ctaranto 16]

When media is transcoded off the emby server LAN (for example, on cellular or my sons house), after a minute or two of playing, there is an error in the transcoding logs:

16:35:05.313 Too many packets buffered for output stream 1:1.
16:35:05.352 Conversion failed!

This seems like the same issue reported here: https://emby.media/community/index.php?/topic/59419-error-when-playing-movie/

My internet is VZ FiOS 500/500 and my son has VZ FiOS 300/300.  About 20 miles total distance from each location.

This happens with both Hardware and Software transcoding.  Using Hardware, it happens around 2 minutes into playing something.  With Software, around 5 minutes.

Has there been any progress with resolving this in ffmpeg or allowing for overrides as suggested in the thread above?

Attached transcode and server logs.

Thanks...

 

embyserver (2).txt ffmpeg-transcode-40d9e791-813c-40e0-9235-3042b40b8afa_1.txt

Edited May 10, 2024 by ctaranto

[ctaranto 16]

After testing more, every media stream that needs to transcodes (playing media outside of the LAN) hits this ffmpeg issue that is fixed by bumping -max_muxing_queue_size to a higher value.

Unfortunately, I don't see how this can be changed in Emby Server.

Any other workarounds?  Any beta builds of emby-ffmpeg where this option is bumped up to something higher than default?

 

Edited May 14, 2024 by ctaranto

[Luke 42077]

9 hours ago, ctaranto said:

After testing more, every media stream that needs to transcodes (playing media outside of the LAN) hits this ffmpeg issue that is fixed by bumping -max_muxing_queue_size to a higher value.

Unfortunately, I don't see how this can be changed in Emby Server.

Any other workarounds?  Any beta builds of emby-ffmpeg where this option is bumped up to something higher than default?

 

Hi, how did you determine this?

[ctaranto 16]

3 hours ago, Luke said:

Hi, how did you determine this?

Hi Luke,

Thanks for replying.  I've come to this likely conclusion based on various links (including in this forum) that points to this particular setting.  While waiting, I've tried to do a lot of research to fix this on my own, but no luck.

• https://stackoverflow.com/questions/49686244/ffmpeg-too-many-packets-buffered-for-output-stream-01
• https://stackoverflow.com/questions/66880338/ffmpeg-error-too-many-packets-buffered-for-output-stream
• https://superuser.com/questions/1811544/too-many-packets-buffered-for-output-stream-01-audio-stream
• https://github.com/jellyfin/jellyfin/issues/3960
• https://trac.ffmpeg.org/ticket/6375

Any many other links.

 

 

Edited May 14, 2024 by ctaranto

[Luke 42077]

You could test that theory by installing the diagnostic plugin. It has a search and replace feature to edit the ffmpeg command line that gets used.

[ctaranto 16]

7 minutes ago, Luke said:

You could test that theory by installing the diagnostic plugin. It has a search and replace feature to edit the ffmpeg command line that gets used.

I had that plugin installed but never went into it to see what could be done.  Thanks for that tip!

Looking at the log, it appears that "-max_muxing_queue_size" isn't sent to ffmpeg by Emby Server.  Do you suggest I search an existing parameter that's always constant (such as -write_header_trailer 0) and replace it with itself plus the -max_muxing_queue_size parameter?

Edited May 15, 2024 by ctaranto

[ctaranto 16]

I was able to inject the -max_muxing_queue_size parameter into the ffmpeg command.  Thanks again, Luke, for that tip!

Though it didn't solve the issue, I am seeing a pattern.

When setting to 2048, the conversion fails on Segment 24

When setting to 20000, the conversion fails on Segment 276

When setting to 40000, the conversion fails on Segment 554

I ran that last one twice (moving the -max_muxing_queue_size from the middle of the parameter list to the end as recommended elsewhere).  Both failed on the same segment.

Attached is a Full Trace of the transcode (reduced back to 2048 for size reasons) if this helps with any debugging.

ffmpeg-transcode-9c318bac-e263-479e-9b78-bfe64017c63b_1.zip

Edited May 15, 2024 by ctaranto

[ctaranto 16]

For the heck of it, I set the size to 100000.  This appears to mitigate the "Too many packets buffered for output stream" error.  I still have the test movie playing.  12 minutes in, with transcoding humming along at 270fps, and Segment at 2850.  

Is this a viable solution?

I'd like to understand more about the -max_muxing_queue_size ffmpeg parameter in the context of how it affects Emby transcodes.

 

For those who are having the same issue, here's what I did to fix this in my environment:

1. Install the Diagnostic Options plug-in
2. Scroll down to the Parameter Adjustment section
3. In Text to Replace:  -segment_write_temp 1
4. In Replacement Text: -segment_write_temp 1 -max_muxing_queue_size 100000
5. Save

I do not know the impact to an Emby Server (memory, disk, concurrent streams, etc).  But for one stream that requires transcoding which used to always fail with "Too many packets buffered for output stream", this fixed it for me.  I only tested with one movie and will continue testing.

 

Edited May 15, 2024 by ctaranto

[ctaranto 16]

Following up with more testing. The movies tested were 4k high bit rate; Emby player was configured with a max bit rate setting of 10, 20, and 40 in various tests to force transcoding.  Tests included Emby running on Android TV (nVidia Shield), Android (on a Samsung Galaxy S24 Ultra), and Browser.

Before the change: 5 out of 5 movies tested failed with "too many packets..." 

After the change: Those 5 movies no longer fail

Interestingly, my son (remotely) texted me last night that the movie he was playing froze.  I had restarted the Emby Server yesterday after noon to update a plugin, and apparently restarting the server clears the overrides in the Diagnostic Plugin.  Yea, I didn't read that part that clearly says that at the top.  I re-applied the override, and it then played the entire way through.

I would like to find out a few things:

1. What can be done on a more permanent basis?
2. What effect does setting -max_muxing_queue_size 100000 have on the server?
3. Am I the only customer experiencing this?

Thanks...

[ctaranto 16]

While waiting for answers, I've continued to attempt workarounds for this.

I created a shell script ("ffmpeg-shell") that reads in the ffmpeg parameters, injects the max_muxing parameter, and calls the real emby-server ffmpeg with the new parameter list.  I changed the emby setting so it calls the ffmpeg shell script instead of ffmpeg itself).  Unfortunately, that did not work.  In the emby log, I see a call to /opt/emby-server/bin/ffrmpeg-shell and all the parameters, but the shell script appears to never run.  I went so far as to only have one line in the shell script:  "echo "foo" > /tmp/foo.txt".  The file never appeared in /tmp.

Looking forward to hearing from @Lukeor @softworkzabout the ffmpeg issue I'm seeing and any more permanent fixes.

 

[Luke 42077]

On 5/18/2024 at 8:02 AM, ctaranto said:

Following up with more testing. The movies tested were 4k high bit rate; Emby player was configured with a max bit rate setting of 10, 20, and 40 in various tests to force transcoding.  Tests included Emby running on Android TV (nVidia Shield), Android (on a Samsung Galaxy S24 Ultra), and Browser.

Before the change: 5 out of 5 movies tested failed with "too many packets..." 

After the change: Those 5 movies no longer fail

Interestingly, my son (remotely) texted me last night that the movie he was playing froze.  I had restarted the Emby Server yesterday after noon to update a plugin, and apparently restarting the server clears the overrides in the Diagnostic Plugin.  Yea, I didn't read that part that clearly says that at the top.  I re-applied the override, and it then played the entire way through.

I would like to find out a few things:

1. What can be done on a more permanent basis?
2. What effect does setting -max_muxing_queue_size 100000 have on the server?
3. Am I the only customer experiencing this?

Thanks...

Hi @ctarantois this the lowest value that resolves the issue for you?

[ctaranto 16]

Hi Luke,

I tested at 60,000 and it worked.  Then tested at 50,000 and it also worked.  Then re-tested at 40,000 and it failed.

For reference, the server it's running on has 32GB memory and an Intel Core i7-4790K - oldie but goodie.  Transcoding is done on an nVidia 1660 Super using NVENC/NVDEC.

 

Edited May 25, 2024 by ctaranto

[Luke 42077]

Did you see high memory usage with it?

[ctaranto 16]

14 minutes ago, Luke said:

Did you see high memory usage with it?

Depends on the definition of high.  Before playing a movie, the server was sitting at 3.7GB memory used.  During playing, it went up to 11.7GB and settled there.  Stopping the movie has the memory use immediately jump back down to 3.7GB.  I'm using htop to monitor.

Interesting observations:

• It takes about 2:40 of playing the movie for the "red bar" to show up in the Emby Dashboard
• It's at this time (when the red bar first appears) when memory use hits the 11.7GB mark.  The memory use stops growing and holds there for the duration of the movie
• The green hardware acceleration symbols next the 4K HEVC and Trandcode (in the Video section of the Dashboard) also show up at this same time (red bar showing up at 2:40 into the movie)

Edited May 25, 2024 by ctaranto

[Luke 42077]

OK thanks for the info, we’ll take a look at it.

[ctaranto 16]

Thanks, Luke.  Let me know if you need any other info or logs.

I'm only looking for a more sticky setting that doesn't reset on a reboot.  Anything that can be provided is appreciated

[ctaranto 16]

Checking in to see if anything is in the works for this.  Thanks.

[Luke 42077]

It's a tricky one because while it helps with this one video, increasing max_muxing_queue_size is not necessarily something that should be done all of the time. So we're still looking into it.

It's also possible that a future update to ffmpeg may resolve it without even having to do that.

Thanks.

[ctaranto 16]

Thanks for getting back to me, Luke.  To be clear, this happens with every video that needs transcoding.  I tested 7 videos, and all 7 failed when I forced transcoding (by setting a lower quality in the player).

I do hope a future ffmpeg resolves this - that would be the best solution!

 

[BuzStringer 33]

On 5/15/2024 at 8:04 PM, ctaranto said:

For the heck of it, I set the size to 100000.  This appears to mitigate the "Too many packets buffered for output stream" error.  I still have the test movie playing.  12 minutes in, with transcoding humming along at 270fps, and Segment at 2850.  

Is this a viable solution?

I'd like to understand more about the -max_muxing_queue_size ffmpeg parameter in the context of how it affects Emby transcodes.

 

For those who are having the same issue, here's what I did to fix this in my environment:

1. Install the Diagnostic Options plug-in
2. Scroll down to the Parameter Adjustment section
3. In Text to Replace:  -segment_write_temp 1
4. In Replacement Text: -segment_write_temp 1 -max_muxing_queue_size 100000
5. Save

I do not know the impact to an Emby Server (memory, disk, concurrent streams, etc).  But for one stream that requires transcoding which used to always fail with "Too many packets buffered for output stream", this fixed it for me.  I only tested with one movie and will continue testing.

 

 

Hi i am also getting this now on a few video file, is this still the workaround?  or is there a Permanent fix now? thanks

[ctaranto 16]

I'll let Luke chime in for a definitive answer, but I still use the setting to avoid problems when transcoding content.  It has to be applied every time I restart (or update) the Emby server.  It's an inconvenience for sure.

This is what I settled on, trying to minimize the memory impact yet still be able to transcode all media:

Text to replace:
-segment_write_temp 1

Replacement text:
-segment_write_temp 1 -max_muxing_queue_size 50000

 

[BuzStringer 33]

On 5/6/2024 at 8:09 AM, Leinad86 said:

@BlackDub

Try activating these two options. You need the Diagnostic plugin for this, you can activate it there. After that everything was working again for me. You have to set these options again after every restart. - Hope it helps you

 

thank you, but I've actually been using this workaround, still need to be re-applied each reboot, but ticking boxes is easier, i have tested a bunch of files that where having issues and it's stable so far.

 

Edited December 1, 2024 by BuzStringer

[ctaranto 16]

Interesting.  I'm not familiar with the impact of those settings - does checking those boxes prevent subtitles from working?

[BuzStringer 33]

that is good question, i haven't tried. i will do some testing later

[Luke 42077]

3 hours ago, ctaranto said:

Interesting.  I'm not familiar with the impact of those settings - does checking those boxes prevent subtitles from working?

Yes they can cause that.