HW acceleration (decode + encode) with RK3568?

[danergo 37]

Hi!

I'm curious (will test a bit later) if Emby can utilize RK3568's HW capabilities for HW decode/encode of H264 and HEVC files.

Will post here my updates and findings, in case anyone have more information as of now, please share it, I'd appreciate it.

 

Thank you very much

Edited September 10, 2024 by softworkz

[Luke 42077]

Hi, what were the results?

[danergo 37]

Hi, I'm still building the system, and didn't yet installed Emby, but will do it probably next week and for sure I'll post here all relevant logs and information.

[danergo 37]

Hi, @Luke:

 

I installed Emby onto this board, and copied a few footage from my action camera (HEVC, ~50Mbps).

In my local network, I can directplay these, but when I want to limit the bitrate (to 4Mbps), Emby tries to transcode, but eventually fails.

I'm attaching my logs, please review.

If you have any idea what would worth checking, let me know.

 

I wanted to run Emby's ffmpeg manually to print the builtin available decoders and encoders, but I couldn't:

/opt/emby-server/bin/ffmpeg: error while loading shared libraries: libdrm.so.2: cannot open shared object file: No such file or directory

 

Either HEVC decoder and/or H264 encoder would be awesome, and this processor is well capable of doing both of these:

 

Here is my media info for quick look:

Thank you!

ffmpeg-transcode-2ecb39e1-7f8d-4b59-ae7f-7a961a7ff01d_1.txt embyserver (3).txt

Edited April 15, 2023 by danergo

[danergo 37]

Here are some references for these CPUs:

https://github.com/blakeblackshear/frigate/pull/1814#issuecomment-939294837

https://github.com/rockchip-linux/kernel/issues/224

We might have to rebuild the ffmpeg Emby uses to support HW accel on these architectures, what do you think?

[danergo 37]

I got another intention:

https://github.com/rockchip-linux/mpp

And I also found an ffmpeg based on this:

https://github.com/jjm2473/ffmpeg-rk

 

This one seems very promising, it would be awesome if you could also check this.

Edited April 16, 2023 by danergo

[danergo 37]

It's strange that ffmpeg is not supporting this architecture out-of-the-box.

Also, very strange Emby is not aware of these HW accelerations, as they marketed Emby on their website (https://www.friendlyelec.com/index.php?route=product/product&product_id=290):

 

Just for curiosity: although Emby can be perfectly usable without HW acceleration, what's the point to market it on a strong architecture as Rockchip which supports 4K HEVC  acceleration by hardware without actually supporting HW acceleration?

Anyway I'll try to solve this, any help would be highly appreciated.

[danergo 37]

And one more reasonable question: if I manage to build an ffmpeg which can utilize this CPU's hw capabilities, (how) can I ask Emby to use my version of ffmpeg?

[danergo 37]

Can Emby use gstreamer instead of ffmpeg for transcoding?

[Luke 42077]

4 hours ago, danergo said:

Can Emby use gstreamer instead of ffmpeg for transcoding?

No, but supporting this hardware is something we can look into. Thanks.

[danergo 37]

Thanks Luke, that would be awesome! I'll keep posting here any new infos I'll find.

[danergo 37]

There are two more ffmpeg repos worth looking into:

https://github.com/JeffyCN/FFmpeg

https://github.com/hbiyik/FFmpeg

[danergo 37]

Hi, dears,

Do we have any news?

If I manage to build an ffmpeg which does support this architecture, how can I use it with Emby?

[Luke 42077]

Hi, I don’t have any news yet, sorry. There’s no Official support for running custom ffmpeg builds in the server.

[danergo 37]

Thx, Luke.

So in theory, if I can compile a version of ffmpeg which can transcode with this shitty architecture, what would be the path to make it available for emby?

[Luke 42077]

1 hour ago, danergo said:

Thx, Luke.

So in theory, if I can compile a version of ffmpeg which can transcode with this shitty architecture, what would be the path to make it available for emby?

I'm afraid I don't know. It's been a while since we've tested that. On windows you can just drop in and replace, but on other platforms it's not so easy.

[them8os235 47]

I have an Radxa Rock 5B [RK3588 SBC based] looking for a purpose. Serving as  my most power efficient Emby Server is the best use for it that I've come across. 

In the same manner that Jellyfin has added hardware decoding & reencoding support for the RK3588 I am requesting Emby support it as well.  -Thnx

https://www.cnx-software.com/2024/02/01/jellyfin-rockchip-rk3588-mpp-hardware-acceleration/?amp=1

 

[Luke 42077]

Hi, yes it's certainly possible for future updates. Thanks.

[kbx81 1]

Second this! 

[chj915 10]

Just to give a push on this topic:

The Jellyfin 10.9 has included a HWA support for Rockchip devices. I have verified the setup on my Khadas Edge 2 device, it can perform hardware transcoding for HEVC (H.265) videos (CPU usage is low ).

I wonder when Emby can have the HWA support included in the offical build. After all, there are lots of Pi-like SBCs on the market right now that are equipped with Rockchip (ARM). It would be great to host Emby Server on those devices (Rockchip runs on low power yet could be powerful).

Here is reference page from Jellyfin: https://jellyfin.org/docs/general/administration/hardware-acceleration/rockchip/

[janmemby 7]

as well here, thx for looking into this emby team

[danergo 37]

Hi @Luke:

I'm getting back to this topic. Do we have any news regarding this arch? Maybe @softworkz?

I can provide more logs, or system info if necessary. 

thx!

[softworkz 5066]

You found some repos which pretend to provide ffmpeg with HWA support for this platform, right?

Have you managed to get any of those versions to run on your device with HWA (ffmpeg via command line)?

If no, then you got an answer already.
If yes, then you should already know why it's pointless (even though kind of working).

The point is: those devices are technically not really suited for hardware accelerated transcoding.
The video acceleration capabilities that they are advertising cannot be compared to what Intel or Nvidia GPUs can do. The actual meaning of those advertised capabilities must be understood like:

• For video decoding: This is solely about video playback, that means they have a hardware decoding unit which decodes directly to the video output for display - that's all! 
  Even when there are ways to read out the output from video memory, the device would never be able to copy and process the amount of data that a 4k video has.
  Also, those figures are not realisitc. No way that this SOC can decode and output 4kP60 in any realistic scenario.
• Video encoding: This is intended for video conferencing (without battery drain) but nothing more

HWA Transcoding requires more than just a decoder and an encoder. It also requires video processing that is done in hardware and in a way that the data is kept in video memory during all those steps. 
The best option for a low-cost Emby Server with HWA transcoding is still Android. Buy a used Samsung or Google phone and you get something that works fairly well (still not comparable to a PC). But those SOCs are a dead end, both from hardware side as well as from software support (reg. hwa).

[softworkz 5066]

Another problem with ffmpeg and those devices is that the ffmpeg logic doesn't fit well to those hardwares. Their design is more oriented on mobile devices architectures.

One thing you could check is whether there's an Android image for your device which has video drivers with OMX/MediaCodec support for hwa. For earlier RockChip SOCs with MALI GPUs there weren't any available, but maybe they have now..

[danergo 37]

I have found some ffmpeg clones which seems to be able to do pretty good transcoding from hevc 4K to h264 FHD.

Although I haven't compiled it for myself yet, because as Luke said its somewhat hard to replace ffmpeg on Linux (Debian) instances and since my original question dates back to last year I first wanted to know whether has been any progress on this architecture yet.

So now, I can do the build on this architecture and see the results myself. But in case this can really do the proper transcoding (and rescaling), can you @softworkzplease describe what is exactly needed on Debian to replace stock ffmpeg with my own version?

Thank you!