Emby killing my server

[aenima99x 8]

I've recently migrated from a MariaDB/MySQL server backend to Emby and am running into an issue. I absolutely love Emby and the integration with all of my Kodi setups and would like to keep this setup instead of going back to a MySQL setup. Any suggestions would be appreciated. 

 

 

Server/NAS Info

Intel Core i7-3770 3.4GHz

32GB RAM

Ubuntu 16.10 installed on mSSD

(8) HDD setup in MDADM RAID for storage (20TB)

Emby server version 3.0.8500

No Emby plugins installed besides Kodi Sync Queue

 

 

Client Info

Various Chromeboxes, laptops running LibreElec/Kodi 16.1 with Emby for Kodi addon version 2.3.6

 

 

Issue

Primary issue is extremely high cpu usage during library scan task, refresh people task and transcoding. 

The high cpu usage is actually causing the cpu core temps to spike all the way to 105C and then the CPU is throttling down.

 

 

Troubleshooting steps taken that haven't resolved issue

I've uninstalled/reinstalled emby-server/mono/ffmpeg - no change. 

Disabled the Kodi plugin - no change

Turned off the real-time monitoring for all libraries - no change

DLNA is disabled

Auto-Organize is disabled

 

 

Current scenario test

Emby is idle with no tasks running and no clients connected.

CPU hovering around 165% usage, RAM is at 8G used by emby and all 4 cores of CPU over 100C temp

 

Stop emby server ( systemctl stop emby-server.service) -

CPU usage drops to 0%, RAM usage drops to 500MB, and all 4 cores of CPU are now at 50C temp

 

Start emby server (systemctl start emby-server.service) -

CPU usage at 4%, RAM usage at 800MB and all 4 cores of CPU at 50C temp

 

Start the Refresh people task - 

within 1 minute the CPU usage has jumped to 140%, RAM usage now up to 1.2GB, all 4 cores of CPU now at 70C temp

task at 20% complete - CPU usage now 280%, RAM at 1.6GB, all 4 cores of CPU at 80C temp

task at 75% complete - CPU usage now 330%, RAM at 1.8GB, all 4 cores of CPU over 100C temp 

task at 99% complete - CPU usage now 510%, RAM at 1.9GB, all 4 cores of CPU over 103C temp (screenshot below)

task finished after 22min & 48 seconds. CPU usage still hovering around 250%, RAM at 1.7GB, all 4 cores of CPU at 100C temp. AT this point, server is idle with no tasks running.

 

 

 

 

 

 

[Luke 42077]

Thanks for the report. With the next release of Emby Server the refresh people task will be considerably faster and lighter so these numbers will no longer be applicable.

[aenima99x 8]

Thanks Luke, but the problem exists beyond the refresh people task. The library scan and transcoding also show the same cpu usage.

[Guest plexman]

I know it's complicated but if I were you, I would try ubuntu 16.04 to see. Just as a test, maybe is something related to the OS.

[aenima99x 8]

I know it's complicated but if I were you, I would try ubuntu 16.04 to see. Just as a test, maybe is something related to the OS.

I had been running 16.04.1 with these issues and just upgraded to 16.10 to see if it would help. 

[Luke 42077]

Transcoding will have cpu impact but the are several areas of the server that will be lighter for the next release, including people refresh.

[aenima99x 8]

Yes understood about the transcoding, I'm not so worried about that. I seem to have gotten it under control for the time being by a combination of things including - disbaling real-time monitoring, disabling MusicBrainz lookups for Music and setting all the clients to use the "Enable database locked fix". 

Is there an ETA for the next release? 

[Luke 42077]

What is this?

Enable database locked fix

[aenima99x 8]

It's a setting in the Emby for Kodi plugin that is on the client machines

 

[Luke 42077]

Ok, I was just curious. I don't have an ETA on the next release, maybe next weekend. There's just a lot of internal changes that need more testing.

[jon@spoonamore.net 2]

Just wanted to give my 2-cents on this discussion.

 

I've been a Linux Mint person a long time.  Before that, Slackware.  Until Ubuntu 16.x came along, I was happy with Mint up to 17.x (Mint 18 is based on Ubuntu 16.x) .  Ubuntu 16.x is having a lot issues across the board from stability to optimizations.

 

I switched to the Arch based distro Manjaro two months ago.  I can tell you I have had better results with Emby on Arch over Ubuntu.  I am not going back to a Ubuntu based distro any time soon.

 

I would also highly recommend that you transcode all of your video-media to "x264/mp3 .mp4".  By doing this, you will almost eliminate on-the-fly transcoding.  Stay away from x265!!!!  Even on my CPU, it still eats up a lot of horsepower during playback.  x265 does save drive space, but it's not worth the CPU load!  I've had up to 6 people access my Emby Server at the same time with no performance issues.  I could do more, but I am limited to my out-going bandwidth.

 

My desktop/server specs out as:

 

E5-2658 v3 - 24-Threads @ 2.6Ghz  ($160 on eBay!!!)

32GB of DDR4

Manjaro Mate 16.10 Fringilla

4 2TB HDs

1 4TB HD

[SkyBehind 23]

I have pretty much the same setup, 16.04 though, only 16GB RAM, and no RAID.  Stream all the same types of things and have some with Kodi (Emby for Kodi).  No issues.  I'm running Mono Stable 4.6.1.5.  What are you running?  I seem to remember that mine did that before, with a different version of Emby, but it's since been fixed.  I'm also running Emby 3.0.8500.0.

[Angelblue05 4132]

Yes understood about the transcoding, I'm not so worried about that. I seem to have gotten it under control for the time being by a combination of things including - disbaling real-time monitoring, disabling MusicBrainz lookups for Music and setting all the clients to use the "Enable database locked fix". 

Is there an ETA for the next release? 

Whats the issue you are having in Kodi? This option will drastically reduce the performance of the add-on. This is usually only needed if you have other addons or kodi content which causes conflict with Emby for Kodi adding things to the Kodi database at the same time. Let me know

[aenima99x 8]

Whats the issue you are having in Kodi? This option will drastically reduce the performance of the add-on. This is usually only needed if you have other addons or kodi content which causes conflict with Emby for Kodi adding things to the Kodi database at the same time. Let me know

I was running into an issue where when the sync was happening on the clients, if I tried to go into a video node (Movies or TV), it would completely lock up the UI and the only way to fix it was by ssh'ing into it and rebooting. 

[Angelblue05 4132]

How many items were syncing, a huge amount?

[aenima99x 8]

It happened several times, but I think it was a large amount each time. When i get home I can turn off the option and try again.

[Angelblue05 4132]

Your server is generating a huge amount of updates, then. You should fix the issue at the source, and it will lighten the amount of updates sent to emby for kodi.

 

Things you can try:

 

In the emby dashboard > metadata > nfo settings > uncheck save image paths to nfos.

Do you have third party software that changes the last modified dates of your nfos?

 

When nfos are modified, it forces the server to refresh the entry of that content. If all nfos are modified, this generates a pile of update for content, even though, nothing actually changed. In turn, the add-on is forced to update all those entries, putting a lot of strain on your clients.

 

In reality, updates should only be for content you've watched, content you've added or actually updated (artwork, metadata), and content you've removed. Anything else that doesn't fit in these categories is superfluous and should be looked into to see why the server is updating those entries.

Edited November 22, 2016 by Angelblue05

[aenima99x 8]

That would explain it then....I had a cron job running a bash script hourly against my media folders to fix the permissions. I've just disabled this in the last day after I finally set the emby user/group to the same account I'm using for other things on the server. I think I just hit several unrelated hiccups during my migration from MariaDB to Emby as my backend for all the Kodi clients. 

[Luke 42077]

The server will be a lot lighter on its' feet with the next release. I think you will see better results then.