Jump to content

Developing a standalone embystat server


reggi

Recommended Posts

Junglejim
On 17/05/2022 at 00:52, reggi said:

Hi everyone, after a long time I created another beta version. I'm getting the project up to speed again. I have refactored a lot of code (backend and frontend) and the sync time should be a lot lower. Also, very basic user statistics (table) is added to this release.

You can find the release here: Release v0.2.0-beta.30 · mregni/EmbyStat (github.com)

Changes:

  • Refactored the whole job code
  • Incremental jobs supported for Movies and Shows
  • Moved back to Sqlite as database
  • Updated Nuget packages
  • Added very basic user page
  • Moved to different chart library
  • UI loading improved
  • Translation base file cleanup
  • Updated to NET6.0
  • Missing episode list added (+export to CSV)
  • Movie and Show table with filters

Let me know if you like it or find any bugs.

Wow cheers mate! 👍 Got the .32 beta up and running in a docker on my NAS (OMV).

Tried the update container path but that didn't work so I purged the container from the file system and ran the compose stack from scratch. Works and looks great! :)🍻

Only had one error and that was the logs tab threw a "EXCEPTIONS.UNHANDLED".

Thanks again for your work on this! Much appreciated. 👍

Edit: What plans do you have for the dashboard tab, a mix of TV/Movie info?

Also I just noticed what I think is a bug with your jobs "Last run about" times, I can run a job then move to another tab.. when I come back (maybe a couple of minutes later) it says "Last run about 10 hours ago". :)

Edited by Junglejim
Link to comment
Share on other sites

reggi

 @Painkiller8818 can you PM me your log file, so I can see what is going wrong with your Movie sync?

And thanks for letting me know about the 403 GitHub issue. I'll log a bug for that on GitHub to track.

 

@Junglejim thanks for testing the application. Update is still not really safe, so recreating is a better option. Can you go into your log file manually and check for exceptions?

In the end I would like to have some custom dashboarding, so you can drag and drop graphs and stats into your dashboard and create your own layout. But that is still a long way to go. In the past, I made the mistake of implementing new stuff when the old stuff was not really 100% working. This created a lot of bugs everywhere for everyone. So I'm taking it step by step from now on.  

Link to comment
Share on other sites

neik

Just tested the latest beta but whenever I try to open the logs tab I get an "Exceptions unhandled" in the bottom right corner.

Also, during syncing shows I get this:

Quote

9:38 PM 05/24/2022SHOW-SYNCJob failed, check logs for more info.

9:38 PM 05/24/2022SHOW-SYNCAn error occured deserializing the response.

Edit: Just clicking through the available tabs and I have to say this is getting quite nice looking.
Looking forward to further development you have planned for this @reggi (e.g. current activity would be nice).

Edited by neik
  • Agree 2
Link to comment
Share on other sites

Painkiller8818

@reggi Do you mean the embyserver log? I don't have logs from the embystat as i can't access it because of the error. so i don't know where to take the logs from.

  • Agree 1
Link to comment
Share on other sites

Junglejim

Hi @reggi Thanks for the clarification on the dashboard.

As @neik stated I have also run into this error while running a show sync..

13 hours ago, neik said:

9:38 PM 05/24/2022SHOW-SYNCJob failed, check logs for more info.

9:38 PM 05/24/2022SHOW-SYNCAn error occured deserializing the response.

I initially thought it was a problem with some of my files but ruled that out.

I'm also not sure which logs your after as @Painkiller8818 mentioned, embystat logs tab throws a "EXCEPTIONS.UNHANDLED" error.

If you want the embystat logs I'll have to ssh into OMV and navigate to the docker config folder I guess, copy and post here?

Man I'm dumb, I should be able to copy the logs from portainer.. :)

stat_logs.txt

In this log I tried the logs tab then a show scan that threw the above error. If you need more let me know. 👍

Edited by Junglejim
  • Thanks 1
Link to comment
Share on other sites

reggi

Thanks for the logs and feedback! It's always handy to get the logs one way or the other. 

I'm working on the log page fix and the sync error. Seems like I missed a stream type somewhere. 

Link to comment
Share on other sites

Junglejim

Yea no probs @reggi, glad to help! Got a new bug I think, maybe related...

I just added some new episodes to "Ghost in the Shell SAC" season 4.

The episode missing count was fine before I added these. Now it's a bit funky!

This is my "Ghost in the Shell SAC" in Emby, as you can see 4 seasons with 76 episodes.Screenshot_2022-05-25_18-43-38.thumb.png.38dda4c72b5c4e84eb28e86bdb9933c4.png

Embystat shows just 12/64 episodes and only 3 seasons, maybe due to there dodgy API.. :)

Screenshot_2022-05-25_18-49-34.thumb.png.aaef8c3e850a3ba9da7dd334b72ef895.png

 

All good anyway! I know this is a work in progress and it will take time to sort out some of the bugs. Great to see that your working on this again. 👍😀

 

Edited by Junglejim
Link to comment
Share on other sites

reggi

@Junglejimthanks for the screens. Did you try and run the show sync? Normally Emby should flag your show as changed so the increment should pic up the new episodes. If not, my incremental sync is broken already 😛 

Link to comment
Share on other sites

adrianwi

It's been a while since I tested this, but it does seem to have made some good steps forward.  Some thoughts from me:

  • I struggle to read the white font on the movie backdrops - they look nice and I like that they change, but would it be better with more transparency or a different font?  My 50 year old eyes need glasses now, so maybe its just me!
  • Not sure where the most watched stats come from, but I find them hard to believe
  • User stats a good start, but this to be is where the focus should be
  • I'm getting a EXCEPTIONS.UNHANDLED error on the Log page

It's still no plexpy or tautulli or whatever it's called now, but it's looking more like it every time I check, which is a good thing!

Good work

 

 

  • Like 1
Link to comment
Share on other sites

Junglejim
20 hours ago, reggi said:

@Junglejimthanks for the screens. Did you try and run the show sync? Normally Emby should flag your show as changed so the increment should pic up the new episodes. If not, my incremental sync is broken already 😛 

Hi @reggi, yep yesterday I tried a show sync with the result above for "Ghost in the Shell SAC".

Today I got home from work and it's sorted.. It fixed its self with no intervention on my part :)

They are new episodes so maybe the info from the API was lagging behind some.

Link to comment
Share on other sites

Ametz

The new version (0.2.0-beta.32) can only sync 16 tv-shows then it errors out.

6:18 PM 05/27/2022 SHOW-SYNC Job failed, check logs for more info.
6:18 PM 05/27/2022 SHOW-SYNC SQLite Error 19: 'FOREIGN KEY constraint failed'.

 

I tried to add an API to themoviedb, by the way there is 2 different API keys from there:
v3 auth
and
v4 auth
I used v3

And also every time i add it, it vanishing, so i don't see it the next time i look at settings.

 

And when syncing movies i get this (it can't sync all movies):
 

7:54 PM 05/27/2022 MOVIE-SYNC Job failed, check logs for more info.
7:54 PM 05/27/2022 MOVIE-SYNC An error occured deserializing the response.
7:54 PM 05/27/2022 MOVIE-SYNC Processed 650 / 1231 movies
Edited by Ametz
Link to comment
Share on other sites

Junglejim

Can't get this container to deploy now.. :(

Not sure why, hopefully it gets sorted out. Trying to deploy "image: lscr.io/linuxserver/embystat:latest".

 
Link to comment
Share on other sites

reggi

Contacting the guys from linuxserver to update their image. I had to change the no-update parameter.

  • Thanks 1
Link to comment
Share on other sites

Junglejim
13 minutes ago, reggi said:

Contacting the guys from linuxserver to update their image. I had to change the no-update parameter.

Thanks for the quick reply. I thought something was wrong, usually I can deploy this fairly quickly but it's just not happening. :)

I tried to update the container and that didn't work so I nuked the hole thing from the file system and tried from scratch, that also just sits there and does nothing.

Hopefully the updated image will get me going again with the new beta. ;)

Cheers. 🍻

  • Like 1
Link to comment
Share on other sites

Ametz

After i updated i can't even log in anymore (don't exist). I get lots of errors in CMD, one is: SQLite Error 10: 'disk I/O error
As i understand it Embystat dosen't have access to write on the disk now.

Also first time i run it it went to the wizard and then it reloaded that page over and over and over, then i close and opened it again, then no user and this I/O error message.

 

Update:
After running it as administrator i get no I/O messages (lots or errors tho) and i run the wizard again, no problem.
But it still only sync 16 tv-shows. Log sent in message.

This time all moves was indexed. nice

Edited by Ametz
Link to comment
Share on other sites

reggi

@Ametz, make sure to always install a new version of the tool instead of updating because I'm still moving a lot of settings and database tables around and I can't write migrations scripts for everything. I hope that will minimize the errors you are having.

  • Like 2
Link to comment
Share on other sites

t123thomas

Apology if this question is silly, my emby is on widows 10, I have downloaded the stat Windows zip  from Github, how do I intsall,

any suggestions

Thanks

 

Link to comment
Share on other sites

reggi

I just uploaded beta.36 with some path fixes for everyone. So now normally the log file page should work again correctly.

 

@t123thomas, thanks for testing the application first of all! If you downloaded the windows zip file you have to open a command window in the folder and run EmbyStat.exe. If you want to pass in some startup flags you can run EmbyStat.exe --help for more information about the flags. 

After that the application will start running and you can access is on http://localhost:6555. You can also download the installer https://github.com/mregni/EmbyStat/releases/download/0.2.0-beta.36/EmbyStat_0.2.0.36_windows-x64.exe and install embystat as a service in the install wizard. 

If you have any problems or you are finding bugs feel free to message here or send me a PM.

  • Like 1
Link to comment
Share on other sites

Junglejim
12 hours ago, reggi said:

I just uploaded beta.36 with some path fixes for everyone. So now normally the log file page should work again correctly.

 

@t123thomas, thanks for testing the application first of all! If you downloaded the windows zip file you have to open a command window in the folder and run EmbyStat.exe. If you want to pass in some startup flags you can run EmbyStat.exe --help for more information about the flags. 

After that the application will start running and you can access is on http://localhost:6555. You can also download the installer https://github.com/mregni/EmbyStat/releases/download/0.2.0-beta.36/EmbyStat_0.2.0.36_windows-x64.exe and install embystat as a service in the install wizard. 

If you have any problems or you are finding bugs feel free to message here or send me a PM.

Still cant deploy this in a docker container.. :(

Getting: Option 'no-updates' is unknown. Then the server shutdown loop.

Link to comment
Share on other sites

Painkiller8818

Hi @reggi and thanks for the Update.

I am running Embystat on a Synology NAS in a Docker container.

Version beta .36

I have a Problem with one Show, but the logs does not really give me more infos why:

 

[2022-06-11 06:13:18.017] []  [INF] --------------------------------------------------------------------
[2022-06-11 06:13:18.196] []  [INF] System info:
[2022-06-11 06:13:18.197] []  [INF] 	Environment	Production
[2022-06-11 06:13:18.197] []  [INF] 	Debugger	False
[2022-06-11 06:13:18.205] []  [INF] 	Process Name	dotnet
[2022-06-11 06:13:18.206] []  [INF] 	Version		0.2.0.0
[2022-06-11 06:13:18.206] []  [INF] 	Log level:	Information
[2022-06-11 06:13:18.207] []  [INF] 	Port:		6555
[2022-06-11 06:13:18.207] []  [INF] 	SSL Port:	6556
[2022-06-11 06:13:18.207] []  [INF] 	SSL Enabled:	False
[2022-06-11 06:13:18.207] []  [INF] 	URLs:		*
[2022-06-11 06:13:18.207] []  [INF] 	Config dir:	/app/config
[2022-06-11 06:13:18.207] []  [INF] 	Log dir:	/app/logs
[2022-06-11 06:13:18.208] []  [INF] 	Data dir:	/app/data
[2022-06-11 06:13:18.208] []  [INF] 	Can update:	True
[2022-06-11 06:13:18.208] []  [INF] 	As service:	False
[2022-06-11 06:13:18.208] []  [INF] --------------------------------------------------------------------
[2022-06-11 06:13:24.152] [Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository]  [WRN] Storing keys in a directory '/root/.aspnet/DataProtection-Keys' that may not be persisted outside of the container. Protected data will be unavailable when container is destroyed.
[2022-06-11 06:13:24.246] [Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager]  [WRN] No XML encryptor configured. Key {c8586688-3cda-42e8-bdbc-8802c056fdc6} may be persisted to storage in unencrypted form.
[2022-06-11 06:13:24.693] [Microsoft.Hosting.Lifetime]  [INF] Now listening on: http://[::]:6555
[2022-06-11 06:13:31.771] [Microsoft.Hosting.Lifetime]  [INF] Application started. Press Ctrl+C to shut down.
[2022-06-11 06:13:31.772] [Microsoft.Hosting.Lifetime]  [INF] Hosting environment: Production
[2022-06-11 06:13:31.773] [Microsoft.Hosting.Lifetime]  [INF] Content root path: /app
[2022-06-11 06:15:09.531] [EmbyStat.Jobs.Jobs.Maintenance.PingEmbyJob]  [WRN] Job not running because wizard is not finished
[2022-06-11 06:20:09.874] [EmbyStat.Jobs.Jobs.Maintenance.PingEmbyJob]  [WRN] Job not running because wizard is not finished
[2022-06-11 06:25:10.241] [EmbyStat.Jobs.Jobs.Maintenance.PingEmbyJob]  [WRN] Job not running because wizard is not finished
[2022-06-11 06:29:10.454] [EmbyStat.Jobs.Jobs.Sync.MovieSyncJob]  [WRN] Job not running because wizard is not finished
[2022-06-11 06:30:10.623] [Microsoft.EntityFrameworkCore.Query]  [WRN] The query uses the 'First'/'FirstOrDefault' operator without 'OrderBy' and filter operators. This may lead to unpredictable results.
[2022-06-11 06:30:10.656] [Microsoft.EntityFrameworkCore.Query]  [WRN] The query uses the 'First'/'FirstOrDefault' operator without 'OrderBy' and filter operators. This may lead to unpredictable results.
[2022-06-11 06:37:21.582] [EmbyStat.Jobs.Jobs.Sync.ShowSyncJob]  [INF] Starting job
[2022-06-11 06:37:21.693] [EmbyStat.Jobs.Jobs.Sync.ShowSyncJob]  [INF] Processing information from 87180 people
[2022-06-11 06:37:28.224] [EmbyStat.Jobs.Jobs.Sync.ShowSyncJob]  [INF] Processing genres
[2022-06-11 06:37:28.248] [EmbyStat.Jobs.Jobs.Sync.ShowSyncJob]  [INF] Processing shows
[2022-06-11 06:37:28.249] [EmbyStat.Jobs.Jobs.Sync.ShowSyncJob]  [INF] 1 libraries are selected, getting ready for processing
[2022-06-11 06:37:28.264] [EmbyStat.Jobs.Jobs.Sync.ShowSyncJob]  [INF] Found 158 changed shows since last sync in Serien
[2022-06-11 06:37:29.844] [EmbyStat.Jobs.Jobs.Sync.ShowSyncJob]  [INF] Found 0 missing episodes for show 2 Broke Girls
[2022-06-11 06:37:30.251] [EmbyStat.Jobs.Jobs.Sync.ShowSyncJob]  [INF] Found 0 missing episodes for show Absentia
[2022-06-11 06:37:31.022] [EmbyStat.Jobs.Jobs.Sync.ShowSyncJob]  [INF] Found 0 missing episodes for show Akte X - Die unheimlichen Fälle des FBI
[2022-06-11 06:37:36.407] [EmbyStat.Jobs.Jobs.Sync.ShowSyncJob]  [INF] Found 0 missing episodes for show Alias - Die Agentin
[2022-06-11 06:38:03.764] [EmbyStat.Jobs.Jobs.Sync.ShowSyncJob]  [ERR] Can't seem to process show Dragon Ball Z, check the logs for more details!
[2022-06-11 06:38:03.773] [EmbyStat.Jobs.Jobs.Sync.ShowSyncJob]  [ERR] Exception details
EmbyStat.Common.Exceptions.NotFoundException: Could not find show Dragon Ball Z with id 81746
   at EmbyStat.Jobs.Jobs.Sync.ShowSyncJob.ProcessMissingEpisodesAsync(Show show)
   at EmbyStat.Jobs.Jobs.Sync.ShowSyncJob.GetMissingEpisodesFromProviderAsync(Show show)
[2022-06-11 06:38:04.109] [EmbyStat.Jobs.Jobs.Sync.ShowSyncJob]  [INF] Found 0 missing episodes for show Dunkle Leidenschaft
[2022-06-11 06:38:04.346] [EmbyStat.Jobs.Jobs.Sync.ShowSyncJob]  [INF] Found 0 missing episodes for show Elfen (2021)
[2022-06-11 06:38:11.026] [EmbyStat.Jobs.Jobs.Sync.ShowSyncJob]  [INF] Found 60 missing episodes for show Family Guy
....

Maybe you can see whats wrong, as the show is working and recorgnized by emby.

thanks

Link to comment
Share on other sites

Painkiller8818

Another thing i already asked a few pages ago: from where exactly are you taking the shows season/episode infos?

They aren't correct in many cases here:

e.g pretty little liars it shows me i have 1 missing episode on season 5 (episode 26) but there are only 25 episodes for season 5, i checked this on IMBD and TVDB so i don't know where it takes the infos.

image.thumb.png.32ba72eaea2717c131ca20f3b99a68e2.png

 

https://www.imdb.com/title/tt1578873/episodes?season=5
https://thetvdb.com/series/pretty-little-liars/seasons/official/5

Same is for shows like dragonball it tells me i have missing shows but i have the exact same sorting and season numbers like thetvdb does

 

 

Link to comment
Share on other sites

Junglejim

@Painkiller8818 Nice to see you've got it running in a docker..

I just can't get it to spin up..🙃 I'm in OMV which is Debian based.

Edited by Junglejim
Link to comment
Share on other sites

Painkiller8818
1 hour ago, Junglejim said:

@Painkiller8818 Nice to see you've got it running in a docker..

I just can't get it to spin up..🙃 I'm in OMV which is Debian based.

Hi, i don't update all my docker containers on my own, i have Watchtower doing this for me so i never have to worry about new versions, they get pulled and installed when they are available:

https://github.com/containrrr/watchtower

Link to comment
Share on other sites

reggi

Don't try to update EmbyStat at the moment, always perform a clean install. The update process is not really stable because I'm still changing a lot of config structure.

@Junglejim try with the '--update-enable false' flag when starting your container. That should work normally. I changed the naming. More info can be found with the --help flag.

@Painkiller8818 I'm taking my info from tmdb.com since thetvdb changed their API policies. Here you can see that season 5 has 26 episodes: Pretty Little Liars: Season 5 (2014) — The Movie Database (TMDB) (themoviedb.org). Looks like S05E13 (We Love You to DeAth) does not exist on thetvdb. Not sure which of the two is wrong here. That's why it is stating you have a missing episode. 

 

Also, someone asked before how I get the "watched count" stats. Emby keeps track of how many times a certain user pushed the play button on a media item. Problem here is that when a user paused a movie and start playing it again, it will record it as playCount = 2. So I can't be sure how many times the user really watched the full length of the movie. What I do at the moment is just count a certain playCount as 1. Meaning that if a user watched a movie 3 times full or watched the movie once but paused the movie 3 times, it will always count as 1. If it seems better to count that playCount as 3 then I can change that easily. Not sure what the best option is here. 

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...