Bug Report: VAAPI Transcoding gives Shredded Video on Roku on AMD Phoenix3

[mchahn 22]

My new ryzen server can't play some files that played fine on old intel server. I spent hours with chatgpt chasing this problem.  Then I asked chatgpt to produce this bug report.  I reviewed this and it matches what I remember happening.  If you don't want extreme chatgpt reporting let me know.  I can ask it to make the report more succinct or not use it at all.  I thought the more the better for now.  A log is included.

Summary
Hardware-accelerated transcoding (VAAPI) with Emby produces “shredded” or corrupted video for Roku clients when streaming AV1 content (transcoded to H.264), but only when using Emby’s internal pipeline.  All manual ffmpeg tests output clean files that play in VLC. The issue occurs only with hardware transcoding.

Emby Server:   emby-server-deb_4.9.1.14_amd64.deb
OS:   Ubuntu 24.04.2 LTS
GPU:   AMD Phoenix3 (RDNA3 iGPU)
Mesa drivers:   25.0.7
System ffmpeg:   6.1.1
Roku model:   Roku Express 4K+

What Was Tested ...

Works (Local Playback):
Manual VAAPI transcode to mp4: Clean output in VLC; not tested on Roku directly.
Manual VAAPI HLS (m3u8/ts segments): Clean output in VLC; not tested on Roku directly.
Manual test using Emby’s filter: Also clean output in VLC; not tested on Roku directly.

Fails (Via Emby to Roku):
Transcoding through Emby to Roku (VAAPI enabled): Video is “shredded” (severe corruption/artifacts) when streamed to Roku via Emby.  Disabling hardware transcoding (using libx264/software): Video streams perfectly to Roku—no corruption.

Things Checked
VAAPI working: vainfo as user and as emby user—hardware support and permissions verified.
FFmpeg versions: System ffmpeg is 6.1.1, Mesa is latest for Ubuntu 24.04.
Permissions: All test files and directories are readable by Emby and test users.
Manual HLS/TS testing: All output files play fine in VLC. 

Suspected Cause
Emby’s ffmpeg environment, runtime options, or output muxing for HLS/TS segments is causing the issue, but only with hardware VAAPI transcoding on this hardware/software combo. The issue cannot be reproduced outside Emby, even using the same filter chain and segmenting.

Possibly due to:
Old or incompatible bundled ffmpeg.
Internal pipeline differences when running live streaming vs single file.
How Emby hands off .ts segments or buffers for Roku, especially when using VAAPI on AMD.

Request
Please advise if there is a way to:
Set Emby’s ffmpeg path from the UI or config file (to use system ffmpeg).
Tweak Emby’s HLS or segment output options.
Patch or override the VAAPI filter/scale handling.

LOG Enclosed
Ran The.Big.Bang.Theory.S08E08 using hardware transcoding shortly after rotating this log.  The video was shredded.

embyserver (1).txt

[mchahn 22]

What are the chances of this being dealt with in the near future?  I can return this pc and replace it with intel but I'd have to do it soon.

[Neminem 1518]

Transcoding logs ?

Does you AMD GPU support AV1 ?

On top of that the audio is OPUS 5.1.

AMD transcoding just sucks use a iGPU, Its still king of transcoding.

Edited August 6, 2025 by Neminem
Adding info

[mchahn 22]

Are Transcoding logs different from the normal log I included above?

Yes, the AMD GPU supports AV1.

> AMD transcoding just sucks

Would the intel  ffmpeg cmd chain be different and have a better chance of working?  I'd hate to go to the trouble of swapping the pc to  intel and find out transcoding still doesn't work.

[Neminem 1518]

The Transcoding logs should be in the logs folder, if Emby did transcode.

Are there any there ?

If it did not transcode then its a client issue.

[mchahn 22]

> If it did not transcode then its a client issue.

It does transcode but the video is shredded.  See my OP.  I would be surprised if the modern Roku was at fault.  Does anyone else use the AMD Phoenix3 GPU?

Log is included.  This was captured while shredded video was running.

BTW, I'm getting a lot of spinning circles and startup for a video takes up to 15 secs.  I assume the circles mean it is buffering up the video before starting. Will GPU transcoding fix this?  My old server was 4X slower and didn't even have a GPU and the video started much faster, This kind of problem is why I got the new server.  My wifi is very fast and probably not the problem.

 

ffmpeg-transcode-b56ede66-a84f-4d52-a417-7c2899806c62_1.txt

[mchahn 22]

Every show I've tried works with chromecast because they all use direct play.  So apparently chomecast supports more formats than Roku. This solves my immediate problem. I can't test transcoding though.  I might get a file with higher resolution to test transcoding on chromecast. I know ffmpeg transcoding works outside of emby so I guess there's no need to swap out the server. 

Thanks for taking the trouble to help me.

[Luke 42077]

HI, we'll have a new ffmpeg build on the server beta channel in the near future that should help with this. so stay tuned for that. Thanks.