Huge database performance issues kodi

[quickmic 1703]

Hello

 

I've some performance issues. Some are general, some are (I guess) specific to Kodi plugin.

 

First of all my setup.

I run Emby by a LXC container via Proxmox on Debian 8.

Resources are 6 core Xeon nativ and 16GB of ram assigned to the container. So it's imho not a hardware problem, but I could assign even more.

HDD is Raid 5 -> performance around 400MB/Sec r/w

 

My database includes about:

3500 movies

10000 series

25000 musicvideos

300000 music

10000 not specified

...so let's say around 400.000 items

The whole emby-data folder (not the media folder) includes: "798307 items, totalling 21.9 GB"

 

After several days, finally the DB was initialized by Emby.

btw, I came from Subsonic which uses MySQL and is much faster in my special case. I checked, and Emby uses sqlite and sticks to that. No change possible using mysql according to this thread -> https://emby.media/community/index.php?/topic/31103-mysql-support/

Anyway, I read something about 2GB DB limitations of sqlite. Well, I check my DB and it's exactly 2GB at the moment. Could this be a coincidence, or is sqlite already messing up?

Going on...

I installed Kodi plugin and starts sync.

It took again several days and never finished the music shares. I also guess based on size. The other parts were synced.

I tried the sync... I don't know... 10 times, 20 times but no success. Even tried different configs. Music only, movies and tv-shows only...

One behavior I recognized is Emby became slower and slower without restart.

e.g. When I start sync by Kodi, it begins quite fast. Then I restart Kodi (cause Music sync never ends after hours/days) and start the sync again it's damn slow. Even by syncing tv-series, movies etc. which before works fine.

This is reproducible. If I restart Emby, it starts fast again and became slower and slower... btw, can I turn off the logs completely? In other circumstances (other tools I use), this was an issue, cause the logs became also quite big.

 

I think this is a general issue of Emby. Imho there is something messing with the recourses.

btw, I played a bit with the GC settings of mono. Beside I hate these GC crap in general (imho a good coded program takes care about resources by itself and don't needs help from automated functions), I never figured out how to force Emby taking all of my assigned Ram.

... No offense to the coders, just a general rant about GC. I think it's even not possible turning off GC completely for mono/Net framework?

Is there a "performance" option available for the mono-environment which boosts caching?

Are all parts compiled to amd64 or are there limitations cause of i386 code?

 

Back to Kodi...

As mentioned before sqlite could have issues with large databases and Kodi uses them too.

I checked the Kodi-local DB and it's not nearly that big as Emby's DB. Anyway, maybe still an issue here.

Usually I use MySql from Kodi site, but I read something that's not possible with Emby. Is this correct? If I give it a shot, it should cover the transencoding paths.

I cannot use it if only absolute pathes (NFS/SMB/FTP whatsoever) are stored by Kodi->mysql. Final configuration should run also via Internet.

 

 

Finally, thx to all developers for their afford. Great job so far.

Edited March 3, 2017 by quickmic