Jump to content

How to speed up Emby on Linux


Dan64

Recommended Posts

Assuming that there is memory to spare, placing Emby's data and/or cache to RAM offers significant advantages. Even though opting for the partial route is an improvement by itself, the latter can make Emby even more responsive compared to its stock configuration. Benefits include, among others:

- reduced drive read/writes;
- heightened responsive feel;
- many operations within Emby, such as quick search are nearly instantaneous.

 

To do so we can make use of a tmpfs.

 

Because data placed therein cannot survive a shutdown, a script responsible for syncing back to drive prior to system shutdown is necessary if persistence is desired. On the other hand, only relocating the cache is a quick, less inclusive solution that will slightly speed up user experience with Emby.

 

In this post is provided an example of script to move and sync the "data" folder (where is stored the emby db) to RAM.  Be sure that rsync is installed and save the "executable" script to /opt/emby-server/bin/emby-data_sync.sh, for example:

#!/bin/sh
# The script will first move emby's data to a new static location,
# make a sub-directory in /dev/shm, softlink to it and later populate it with the contents of data.
# The second time the script runs, it will then preserve the RAM profile by copying (sync) it back to disk.

PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

static=data-static
link=data
volatile=/dev/shm/emby-data

IFS=
set -efu

cd /var/lib/emby

if [ ! -r $volatile ]; then
        mkdir -m0700 $volatile
fi

if [ "$(readlink $link)" != "$volatile" ]; then
        echo "--> Emby: Init RAM Disk"
        mv $link $static
        ln -s $volatile $link
fi

if [ -e $link/.unpacked ]; then
        echo "--> Emby: Sync Disk"
        rsync -av --delete --exclude .unpacked ./$link/ ./$static/
else
        echo "--> Emby: Sync RAM"
        rsync -av ./$static/ ./$link/
        touch $link/.unpacked
fi

It is best to add a cron job to guarantee the sync between RAM and disk, for example the following entry will run the sync process every 30min.

*/30 * * * * /opt/emby-server/bin/emby-data_sync.sh

Then it is necessary to change the service: /lib/systemd/system/emby-server.service to guarantee the initial and final sync, as follow:

[Unit]
Description=Emby Server is a personal media server with apps on just about every device.
After=network.target

[Service]
TimeoutSec=5min
EnvironmentFile=/etc/emby-server.conf
WorkingDirectory=/opt/emby-server
ExecStartPre=/opt/emby-server/bin/emby-data_sync.sh
ExecStart=/opt/emby-server/bin/emby-server
ExecStopPost=/opt/emby-server/bin/emby-data_sync.sh
RestartForceExitStatus=3
User=emby

[Install]
WantedBy=multi-user.target

After this change it is necessary reload the service with the command:

systemctl daemon-reload

Finally it is possible to activate the shared-memory for Emby with the command

systemctl restart emby-server

It is possible to verify the execution of the script with the command:

systemctl status emby-server

● emby-server.service - Emby Server is a personal media server with apps on just about every device.
   Loaded: loaded (/lib/systemd/system/emby-server.service; enabled; vendor preset: enabled)
   Active: active (running) since Tue 2019-01-01 09:46:44 CET; 7min ago
  Process: 8310 ExecStopPost=/opt/emby-server/bin/emby-data_sync.sh (code=exited, status=0/SUCCESS)
  Process: 8315 ExecStartPre=/opt/emby-server/bin/emby-data_sync.sh (code=exited, status=0/SUCCESS)
 Main PID: 8320 (EmbyServer)
    Tasks: 23 (limit: 4915)
   Memory: 676.5M
   CGroup: /system.slice/emby-server.service
           └─8320 /opt/emby-server/system/EmbyServer -programdata /var/lib/emby -ffmpeg /opt/emby-server/bin/ffmpeg

More details can be found with the command:

journalctl | grep emby-data_sync.sh

emby-data_sync.sh[23844]: sending incremental file list
emby-data_sync.sh[23844]: activitylog.db-wal
emby-data_sync.sh[23844]: authentication.db-wal
emby-data_sync.sh[23844]: library.db
emby-data_sync.sh[23844]: library.db-wal
emby-data_sync.sh[23844]: users.db-wal
emby-data_sync.sh[23844]: ScheduledTasks/6330ee8f-b4a9-57f3-3981-f89aa78b030f.js
emby-data_sync.sh[23844]: ScheduledTasks/81267bb8-bb49-51de-4b1b-446262cc7365.js
emby-data_sync.sh[23844]: ScheduledTasks/d459f39d-7abb-81f5-fcce-03048a7ecf4d.js
emby-data_sync.sh[23844]: SyncData/Emby.Kodi.SyncQueue.I.1.40.json
emby-data_sync.sh[23844]: sent 255,467,606 bytes  received 227 bytes  30,055,039.18 bytes/sec
emby-data_sync.sh[23844]: total size is 257,320,410  speedup is 1.01
emby-data_sync.sh[23852]: sending incremental file list
emby-data_sync.sh[23852]: sent 4,561 bytes  received 44 bytes  9,210.00 bytes/sec
emby-data_sync.sh[23852]: total size is 257,320,410  speedup is 55,878.48

The paths used in this post are the default ones for a Debian installation, in the case the distribution used is different could be necessary to change some path. But the main concept remain the same. On my installation I was able to speed up the "quick searches" by 100 times. This improvement is mainly due to the increase of speed obtained by moving the SQLite databases to RAM. In this article: https://www.directionsmag.com/article/3794   is reported an interesting comparison on memory access vs disk access speed and is stated that: "If you have a problem with slow database  access, placing the entire database in RAM can instantly provide a factor-of-100-thousand increase in access speed with no other programming or data structure changes required".

Edited by Dan64
  • Like 2
Link to post
Share on other sites

The problem is that the access to sqlite database is generally slow. Unless you decided to implement an in memory cache solution for access the database, moving it to RAM can speed up the the "searches" by more than 100 times. In any case I'm waiting for the new version, as the proposed solution is not able to speed up all the Emby's features.

Link to post
Share on other sites
frabe8378910

Assuming that there is memory to spare, placing Emby's data and/or cache to RAM offers significant advantages. Even though opting for the partial route is an improvement by itself, the latter can make Emby even more responsive compared to its stock configuration. Benefits include, among others:

- reduced drive read/writes;

- heightened responsive feel;

- many operations within Emby, such as quick search are nearly instantaneous.

 

 

 

Thanks for your valueable idea. I'll always appreciate easy to moderate suggestions which could lead to (maybe significant) improvement.

My 8 years old 7x24 debian server running the latest emby Beta (with two SAT cards) has only 4GB RAM - so not to much to spend for caching. But taking your idea a little bit different, I might test this concenct with just adding one fast SSD card and throw cache and DB on that. Limitation for my server could be the speed of the older SATA interface and no free PCIE slot. Do someone already have experience with performance improvements with such an setup?

Thanks in advance.    

Edited by frabe8378910
Link to post
Share on other sites

If your library is quite big, I can guarantee that the improvement is significant. Previously I had to wait between 20 to 200 sec to get an answer to my searches, now the answers are nearly instantaneous. Recently I decided to upgrade the hardware of my 7x24 debian server and since I was tired about the slow behavior of Emby with my big library; I decided to use, as server disk for OS and programs, a SSD. I was very impressed how much improved Emby (actually all the programs running on my server improved in speed, especially KDE). Unfortunately since all my 4 SATA ports were full, I had to use a USB to SATA bridge to add an internal SATA port using the onboard USB 3 adapter (my only PCIe port was already employed by a TV/turner). Doing so I discovered that the SSD disks when are connected to USB to SATA bridge in Linux are unreliable (It seems a combination of  bridge-firmware/OS issues not yet solved). So I had to remove the SSD and install the OS on a standard 2.5" 5400 rpm disk. As expected Emby and KDE started to run slow using the hard disk setup. I tried to figure out a way to have an SSD like speed using a standard hard disk, and I noted that the amount of space of folder "/var/lib/emby/data" was only 250 mb, small enough to stay in memory without problems and so I tested the approach described in this post, and I can guaranteed that the speed improvement is quite the same of the one previously observed using the SSD (with the only exception of images that, since are not in RAM, aren't loaded instantaneously). If you have low memory I suggest to use "zram" or "zswap" (better in my opinion) to improve the memory usage (data in RAM are compressed).  

Edited by Dan64
  • Like 1
Link to post
Share on other sites
  • 1 month later...
  • 1 year later...

I can't get this to work. I have 8 gigs of RAM is that not enough? When I do "systemctl restart emby-server" I get this error

"Job for emby-server.service failed because the control process exited with error code.
See "systemctl status emby-server.service" and "journalctl -xe" for details." I have no idea how to fix it

 

Link to post
Share on other sites
Luke
7 hours ago, Mykids said:

I can't get this to work. I have 8 gigs of RAM is that not enough? When I do "systemctl restart emby-server" I get this error

"Job for emby-server.service failed because the control process exited with error code.
See "systemctl status emby-server.service" and "journalctl -xe" for details." I have no idea how to fix it

 

Hi, what exactly are you trying to do?

  • Like 1
Link to post
Share on other sites
10 hours ago, Luke said:

Hi, what exactly are you trying to do?

It's okay. I rebooted my PC and it worked, Thanks! I was trying to follow this guide to speed up Emby. I have a large library, emby was taking 10 minutes to load up, search function didn't work, overall everything was slow. This guide fixed everything, Emby loads up instantly now, search has instant results and everything is perfect. Thanks

Link to post
Share on other sites
Luke
1 hour ago, Mykids said:

It's okay. I rebooted my PC and it worked, Thanks! I was trying to follow this guide to speed up Emby. I have a large library, emby was taking 10 minutes to load up, search function didn't work, overall everything was slow. This guide fixed everything, Emby loads up instantly now, search has instant results and everything is perfect. Thanks

Thanks for the feedback.

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...