mgworek 121 Posted April 17, 2019 Share Posted April 17, 2019 (edited) never mind. I realized I wasn't running the nightly build with the new wizard. Everything good now. Thanks log20190417.txt Edited April 17, 2019 by mgworek Link to comment Share on other sites More sharing options...
mgworek 121 Posted April 17, 2019 Share Posted April 17, 2019 (edited) Is anyone having a problem getting it to open in Safari? It just sits on the splash screen cycling through the logo. Runs fine with Chrome and Firefox. I used to use it with safari. Update: Works fine in Safari Tech Preview. I will investigate more. It's either the current safari version or something on my end. Thanks. Edited April 17, 2019 by mgworek Link to comment Share on other sites More sharing options...
Pog22 50 Posted April 17, 2019 Share Posted April 17, 2019 Right I got this running behind Nginx reverse proxy and all scanned in which took a while (~100TB). It's running in docker so I've linked the Emby server by hostname and local port to bypass Nginx. One thing, could you possibly add in the option to use the Emby wan address in the library links Link to comment Share on other sites More sharing options...
mgworek 121 Posted April 18, 2019 Share Posted April 18, 2019 (edited) My first scan scan crashed yesterday but 2nd try everything was good. This morning I noticed my docker was crashed. I started again and had no movies anymore. Ran the scan again and it crashes each time. I get about 5 of these in the log and then the docker crashes again. "Microsoft.Data.Sqlite.SqliteException (0x80004005): SQLite Error 5: 'database is locked'. obviously my db got locked, any idea how or how to unlock it? Thanks. 2019-04-18.log Edited April 18, 2019 by mgworek Link to comment Share on other sites More sharing options...
jaycedk 402 Posted April 18, 2019 Share Posted April 18, 2019 Check out the git https://github.com/mregni/EmbyStat/issues/645 Same issue but on windows. 1 Link to comment Share on other sites More sharing options...
mgworek 121 Posted April 18, 2019 Share Posted April 18, 2019 Check out the git https://github.com/mregni/EmbyStat/issues/645 Same issue but on windows. Sorry about that, I did a quick search in this topic for "locked" but had no results. Link to comment Share on other sites More sharing options...
Spaceboy 2507 Posted April 19, 2019 Share Posted April 19, 2019 hi reggi, i'm still having difficulty getting embystat to complete the initial sync with emby. sometimes it seems like it has completed but then it wipes everything and starts again. i had one similar error in emby to that which you pointed out on hitman 47 but i can't see any other things to look at and it seems to stop at different places. the longest it has ran for so far is 37 hours. i do have a fair size library but i imagine other people have much more. i wondered if you had thought about breaking the initial sync down in some way? either by batch or perhaps media type? last 2 days logs attached. 2019-04-18.log2019-04-19 (1).log cheers Link to comment Share on other sites More sharing options...
Pog22 50 Posted April 19, 2019 Share Posted April 19, 2019 I have 100TB, a full sync is taking 6.5 hours Link to comment Share on other sites More sharing options...
reggi 409 Posted April 19, 2019 Author Share Posted April 19, 2019 Hey guys, sorry for not responding sooner. I have some busy days now with my work and will have time on sunday to check the thread again. As for @@Spaceboy, said to hear it's not running! I do have to see how I can refactor the initial sync into smaller pieces and make it incremental as well so EmbyStat doesn't DDOS your Emby every night. 1 Link to comment Share on other sites More sharing options...
Spaceboy 2507 Posted April 19, 2019 Share Posted April 19, 2019 I have 100TB, a full sync is taking 6.5 hourscool, I have more Link to comment Share on other sites More sharing options...
adrianwi 238 Posted April 19, 2019 Share Posted April 19, 2019 Is anyone noticing an alarming increase in the HDD space used by the Docker host when running this? I seem to have something consuming HDD at an alarming rate, and whilst it's easy enough to increase the space, I don't think anything I'm dockerising requires that much space. It might not be related to this, but seems to have started around the same time I installed the latest version to test. Link to comment Share on other sites More sharing options...
Pog22 50 Posted April 19, 2019 Share Posted April 19, 2019 I like this, but I've had to give up on it as my server is a remote server behind an Nginx reverse proxy and I'm also using Cloudflare DNS. To get images and links working I would need to use the Emby Wan URL which with a 100TB library creates a huge amount of traffic out through Cloudflare and back when Embystat runs in docker on the same host. Feature request, It would be great if I could use the local port and hostname and have the option to amend the public domain URL to the image and library links, Ombi has this Externally Facing Hostname setting Link to comment Share on other sites More sharing options...
reggi 409 Posted April 20, 2019 Author Share Posted April 20, 2019 @adrianwi Strange that the HDD space is increasing fast in your docker, this shouldn't happen because the only thing that is written to DB is the database file. But it looks like everyone is having major performance issues with the current version, so best thing is to restructure the media sync job and see what can be done. Currently I'm looking to switch SQLite with LiteDB because it seems more thread/process safe. And I have to create some paging in the http calls to Emby so the memory doesn't blow up every time you do a sync. Or you guys just have way to much media, hehehe Sorry for having this kind of major issues still, I hope we will get to a better version soon. Or you guys just have way to much media, hehehe 2 Link to comment Share on other sites More sharing options...
reggi 409 Posted April 21, 2019 Author Share Posted April 21, 2019 (edited) So, yesterday I was working on a new solution for the performance and memory issues with EmbyStat and SQLite and I came u with LiteDB, it's a file database but document based and it looks very promising. I did some tests with my collections (1061 movies and 195 shows) and it takes me now just over 9 minutes instead of > 30 minutes. So that looks very good. Maybe I can create a standalone app for this console application so people with huge collections can test the performance as well before I implement this in EmbyStat. I'll create a package later today. EDIT: forgot to add test result screenshot Edited April 21, 2019 by reggi 3 Link to comment Share on other sites More sharing options...
jaycedk 402 Posted April 21, 2019 Share Posted April 21, 2019 Nice Looking forward to testing that Link to comment Share on other sites More sharing options...
reggi 409 Posted April 22, 2019 Author Share Posted April 22, 2019 So, I have 2 version of the testing tool packages. The packages are self contained so you don't need to install the dotnet core framework yourself. Usage of the test app is fairly easy tester --username <Emby username> --url <url>:<port> --pass <Emby user password> Let it run for a while, at the end it will show some sync timings. The app will generate a data.db file, you can always use this viewer to check the data => https://github.com/falahati/LiteDBViewer It's possible you get a 404 when checking for missing episodes, that's probably because there is a unknown thetvdb id in the Emby database. It would be create if some of you could run this app and share the results. That way I get a better idea if it is a good idea to switch to LiteDB. Tester win-x64.zip Tester linux-x64.zip 1 Link to comment Share on other sites More sharing options...
kesm 25 Posted April 22, 2019 Share Posted April 22, 2019 (edited) I had to launch this command before the test, also your command should be Tester and not tester : chmod +x Tester but here is the result : Collecting run statistics --------------------------------------------------- RESULTS Count Time Collection 6 278 ms Movies 996 6587 ms Shows 156 172411 ms Episodes 12561 157093 ms (count => total missing episodes, time => time to check for missing episodes) People 30813 (no time, people are synced while syncing media Total 336369 ms --------------------------------------------------- All done! with some 404 before Edited April 22, 2019 by kesm 1 Link to comment Share on other sites More sharing options...
reggi 409 Posted April 22, 2019 Author Share Posted April 22, 2019 Thx for the output and extra instructions! Your sync time looks very good as well. Link to comment Share on other sites More sharing options...
jaycedk 402 Posted April 22, 2019 Share Posted April 22, 2019 Hi there Just finished with your test script of liteDB. Collecting run statistics---------------------------------------------------RESULTS Count TimeCollection 11 330 msMovies 4086 10050 msShows 381 11067205 msEpisodes 27 308001 ms (count => total missing episodes, time => time to check for missing episodes)People 86626 (no time, people are synced while syncing mediaTotal 11385586 ms---------------------------------------------------All done! 1 Link to comment Share on other sites More sharing options...
Happy2Play 8361 Posted April 22, 2019 Share Posted April 22, 2019 With 404 error Collecting run statistics --------------------------------------------------- RESULTS Count Time Collection 8 119 ms Movies 3976 13742 ms Shows 360 2172045 ms Episodes 3278 879808 ms (count => total missing episodes, time => time to check for missing episodes) People 129017 (no time, people are synced while syncing media Total 3065714 ms --------------------------------------------------- All done! 1 Link to comment Share on other sites More sharing options...
neik 838 Posted April 22, 2019 Share Posted April 22, 2019 With one 404 error: Collecting run statistics --------------------------------------------------- RESULTS Count Time Collection 9 577 ms Movies 625 11805 ms Shows 121 743127 ms Episodes 1494 139092 ms (count => total missing episodes, time => time to check for missing episodes) People 30475 (no time, people are synced while syncing media Total 894601 ms --------------------------------------------------- All done! 1 Link to comment Share on other sites More sharing options...
hshah 19 Posted April 22, 2019 Share Posted April 22, 2019 I run EmbyStat in Docker on my Synology NAS, but the below test was done in Windows where both my Plex and Emby servers run: Start syncing collections Start syncing movies Start syncing shows Start syncing missing episodes Collecting run statistics --------------------------------------------------- RESULTS Count Time Collection 5 210 ms Movies 1395 4506 ms Shows 229 1253548 ms Episodes 7516 209482 ms (count => total missing episodes, time => time to check for missing episodes) People 56640 (no time, people are synced while syncing media Total 1467746 ms --------------------------------------------------- All done! 1 Link to comment Share on other sites More sharing options...
reggi 409 Posted April 24, 2019 Author Share Posted April 24, 2019 Thanks guys for the responses. I'll have a look today but seems like a good solution. Shows are still some kind of a bottleneck. @@jaycedk, I see the shows sync itself is taking just over 3 hours on your machine. Can you tell me how long it took with the current EmbyStat implementation? Link to comment Share on other sites More sharing options...
jaycedk 402 Posted April 24, 2019 Share Posted April 24, 2019 Im current at work, but will have a look in about 9 hours Link to comment Share on other sites More sharing options...
jaycedk 402 Posted April 24, 2019 Share Posted April 24, 2019 Hmm well the total sync takes 2h50m with the old sql lite. Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now