Jump to content

Developing a standalone embystat server


Recommended Posts

@candyman 

 

I do prefer a github issue. But if you don't have a github account you can post them here yes.

 

@@reggi no problem will get it reported on github

  • Like 1
Link to post
Share on other sites

Hey guys,

 

Just release a new beta version (still no update mechanism, so update manually), version 0.1.0.22-beta with some bug fixes.

Feel free to give your opinion on the tool!

 

Next step is to sync the live user data with EmbyStat and fix some graph issues for both shows and movie statistics.

Edited by reggi
Link to post
Share on other sites
adrianwi

I've got this running in Docker using the beta-linux image and it starts up ok and appear to connect to my emby server, but when I try and start a sync task it errors.  The log shows this:

2018-07-16 18:24:32 [INF] Booting up server
2018-07-16 18:24:33 [INF] Migrating database started
2018-07-16 18:24:34 [INF] Migrating database ended
2018-07-16 18:24:35 [INF] Daily trigger for TASKS.SMALLEMBYSYNCTITLE set to fire at 07/17/2018 00:30:00, which is 365.41545797 minutes from now.
2018-07-16 18:24:35 [INF] Daily trigger for TASKS.MEDIASYNCTITLE set to fire at 07/17/2018 00:00:00, which is 335.414757933333 minutes from now.
2018-07-16 18:24:43 [INF] Getting server configuration from database.
2018-07-16 18:24:46 [INF] Getting server configuration from database.
2018-07-16 18:24:51 [INF] Getting all background tasks.
2018-07-16 18:24:57 [INF] Daily trigger for TASKS.MEDIASYNCTITLE set to fire at 07/17/2018 00:00:00, which is 335.041048185 minutes from now.
2018-07-16 18:24:59 [INF] Executing TASKS.PINGEMBYSERVERTITLE
2018-07-16 18:24:59 [INF] TASKS.PINGEMBYSERVERTITLE "Completed" after 0 minute(s) and 0 seconds
2018-07-16 18:24:59 [INF] ExecuteQueuedTasks
2018-07-16 18:25:03 [INF] Executing TASKS.MEDIASYNCTITLE
2018-07-16 18:25:03 [INF] First delete all existing media and root media collections from database so we have a clean start.
2018-07-16 18:25:03 [INF] Lets start processing movies
2018-07-16 18:25:03 [INF] Asking for all root views for admin user with id 79ad4cab3c4845869e1c5652acb34476
2018-07-16 18:25:03 [INF] Found 1 movie root items, getting ready for processing
2018-07-16 18:25:03 [INF] Asking Emby all movies for parent (f137a2dd21bbc1b99aa5c0f6bf02a805) Movies
2018-07-16 18:25:09 [INF] Ready to add movies to database. We found 631 movies
2018-07-16 18:25:09 [ERR] Error
System.ArgumentNullException: Value cannot be null.
Parameter name: source
   at System.Linq.Enumerable.Select[TSource,TResult](IEnumerable`1 source, Func`2 selector)
   at EmbyStat.Common.Converters.MovieHelper.ConvertToMovie(BaseItemDto x) in /opt/teamcity/work/aa794b64fcf59423/EmbyStat.Common/Converters/MovieHelper.cs:line 15
   at System.Linq.Enumerable.SelectArrayIterator`2.ToList()
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at EmbyStat.Tasks.Tasks.MediaSyncTask.<ProcessMovies>d__21.MoveNext() in /opt/teamcity/work/aa794b64fcf59423/EmbyStat.Tasks/Tasks/MediaSyncTask.cs:line 105
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at EmbyStat.Tasks.Tasks.MediaSyncTask.<Execute>d__19.MoveNext() in /opt/teamcity/work/aa794b64fcf59423/EmbyStat.Tasks/Tasks/MediaSyncTask.cs:line 75
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at EmbyStat.Tasks.ScheduledTaskWorker.<ExecuteInternal>d__54.MoveNext() in /opt/teamcity/work/aa794b64fcf59423/EmbyStat.Tasks/ScheduledTaskWorker.cs:line 231
2018-07-16 18:25:09 [INF] TASKS.MEDIASYNCTITLE "Failed" after 0 minute(s) and 5 seconds
2018-07-16 18:25:09 [INF] ExecuteQueuedTasks

Link to post
Share on other sites
jaspalgill

I had a similar issue on the previous beta version but on Windows 64-bit, where it would connect to the server but never start the sync.

 

However, I've now gone to the latest docker image using Ubuntu and I'm now getting further. I have a couple of media libraries for Documentaries, Comedy, Fitness and Movies.

It's a large library but not massive compared to what I've seen on the forums, how long are people seeing their initial sync process take to complete?

 

I assume the below log is saying it took 6 minutes to add the 30,735 people, is this the process where the local database is being created/populated? It actually took another 5 minutes before it started to import the movies.

My other libraries such as Documentaries, Comedy and Fitness took only seconds, however there are only at most 200 items there.

 

23:27:50 - Ready to add movies to database. We found 1586 movies
23:27:51 - Need to add 10 genres first (Fantasy, Horror, Musical, Mystery, News, Romance, Science Fiction, Sci-Fi, War, Western)
23:33:49 - Need to add 30735 people first
...import
23:52:03 - Media syncronisation is finished.
Edited by jaspalgill
Link to post
Share on other sites
Happy2Play

I believe EmbyStats currently only looks at content type Movies and TV, as my Unset/Mixed Content library is not detected.

Link to post
Share on other sites
pir8radio

This topic shows promise.   What are your end goals for stats?   User, and server stats too? Maybe a list of stats you plan to supply?  

  • Like 1
Link to post
Share on other sites

@@adrianwi

Did you update your image to the latest version yesterday? If you did, I will create an issue for this on Github.

 

@@jaspalgill

Importing the people from Emby to EmbyStat is always the longest process. But only at the first sync, next time it will find only import people that are not yet in your EmbyStat db.

As for the timings, syncing 873 movies and 157 shows took me 11 minutes and 58 seconds (with an I7-7700 at 2,8Ghz). This will decrease more when this bug is fixed. (at the moment all shows are always synced with theTVDB and not just updated)

 

@@pir8radio

Thanks for your interest in the project! Here is a small list of stats I still want to include

- Movies boxset count

- File size ( + graph)

- Views/users

- View/time of day

- Views/day of week

...

 

There is still a lot to be done before I can go to a stable release. (update service, notifications, system info, ...)

 

But if you have an idea about something that is missing, feel free to create a new GitHub issue or post it on the forum here.

Edited by reggi
  • Like 1
Link to post
Share on other sites

Yes indeed, I will remove the latest from docker since that's only used when I build a master release. 

 

Problem you are having has to do with converting the Emby movie objects to EmbyStat movie objects. 

I'll create a new docker image with some more logging enabled (haven't had the time to implement it properly).  

Link to post
Share on other sites
pir8radio

@@adrianwi

Did you update your image to the latest version yesterday? If you did, I will create an issue for this on Github.

 

@@jaspalgill

Importing the people from Emby to EmbyStat is always the longest process. But only at the first sync, next time it will find only import people that are not yet in your EmbyStat db.

As for the timings, syncing 873 movies and 157 shows took me 11 minutes and 58 seconds (with an I7-7700 at 2,8Ghz). This will decrease more when this bug is fixed. (at the moment all shows are always synced with theTVDB and not just updated)

 

@@pir8radio

Thanks for your interest in the project! Here is a small list of stats I still want to include

- Movies boxset count

- File size ( + graph)

- Views/users

- View/time of day

- Views/day of week

...

 

There is still a lot to be done before I can go to a stable release. (update service, notifications, system info, ...)

 

But if you have an idea about something that is missing, feel free to create a new GitHub issue or post it on the forum here.

 

I guess my question is are you going to focus just on media?    Or will you show things like library size, qty and data?   Data transferred?  (not sure you can get that info from the api)   Users login history/stats, devices associated with that user, things like that?   The media info sounds like a great start though.  I would love to get down to one stats source.  I can give examples if that data is even available to you..       

 

on a side note, i've always wanted a "forum signature" that had real time stats in it! lol    You know maybe an html link to an image that is dynamically updated to show our server stats to use in this forum?   X movies, X music, X music videos, X users per day or something cool.     :D

Edited by pir8radio
  • Like 2
Link to post
Share on other sites

Well at first my focus will be on the media, but at the end, I would love to have more info from the server itself like you mentioned already (size, data transfer, uptime, hardware...)

But I have to start somewhere :) 

 

I will add all year ideas to my notepad list, but if you have a GitHub account feel free to create some feature request here: https://github.com/mregni/EmbyStat/issues/new?template=Feature_request.md

You don't have to add a request for every stat! But it's always handy to have a place to discuss all stats in one place. 

 

I do like your idea for the forum widget, building such an API is not that hard but that would mean that you have to make EmbyStat public accessible. Maybe in the future, I come up with another solution for that. I'll keep it in mind!

Link to post
Share on other sites
pir8radio

Well at first my focus will be on the media, but at the end, I would love to have more info from the server itself like you mentioned already (size, data transfer, uptime, hardware...)

But I have to start somewhere :)

 

I will add all year ideas to my notepad list, but if you have a GitHub account feel free to create some feature request here: https://github.com/mregni/EmbyStat/issues/new?template=Feature_request.md

You don't have to add a request for every stat! But it's always handy to have a place to discuss all stats in one place. 

 

I do like your idea for the forum widget, building such an API is not that hard but that would mean that you have to make EmbyStat public accessible. Maybe in the future, I come up with another solution for that. I'll keep it in mind!

 

 

Cool, I look forward to your progress...    I was just shooting cool ideas out there...   I'm sure you know, or will soon find out, you can't make everyone happy!    :)

Link to post
Share on other sites

Hehehe, true that :D

 

But it's always cool to get new ideas from different people so thanks for that!

Link to post
Share on other sites

Not sure what's going on.  I got the latest docker running. I point to my server, it tells me that

 

Emby server is running admin is an Emby administrator

 

Getting Emby token failed. Please check your credentials and address

 

 

It won't go on, the stable did this too.  What am I missing?  Thanks.

Link to post
Share on other sites

Did you type in the server address correctly ?

you need to start with http or https :// your ip : port

Edited by jaycedk
  • Like 2
Link to post
Share on other sites

@@meddaugh

Looks indeed like EmbyStat can't find your Emby server (or wrong credentials). 

 

Like @@jaycedk suggested, add http:// or https:// in front of the url/ip:port. If it stil fails you can check your Emby log files, probably you will find a "wrong authentication error" in there.

  • Like 1
Link to post
Share on other sites

@@meddaugh

Looks indeed like EmbyStat can't find your Emby server (or wrong credentials). 

 

Like @@jaycedk suggested, add http:// or https:// in front of the url/ip:port. If it stil fails you can check your Emby log files, probably you will find a "wrong authentication error" in there.

 

Hm.. Yea, I added http:// and the port at the end, it was able to login.  You might want to change the wording, I just use what it asked for, the "ip address".  Not the full url.  Thanks, I'll be testing it out more.

  • Like 1
Link to post
Share on other sites
adrianwi

Not sure if you'd made some changes, but I tried this again today using the nightly-linux image and it connected to my emby server and scanned in around 600 movies :D

 

A couple of questions:

 

  • It looks like it's missed anything in a Collection - do you plan to include these?
  • Is there a way of storing the database outside the Docker container (in its own volume) to persist the data when rebuilding?
  • Will you be looking to add user stats?  Knowing more about what media I have is useful, but knowing how my server is being used is much more useful

Looking good though, so keep up the great work!

Link to post
Share on other sites

Thanks @@adrianwi, a lot needs to be done before I can release the first version of EmbyStat. 

  • Normally it searches recursively in collections as well. But some bugs have been reported about this indeed. So I added it to my backlog to give it another check and see what is going wrong here.
  • At this moment there is no way to store the database outside of the container. I also should not advise doing so because I'm still setting up the full database structure. And I found out that too complicated database migration scripts fail very fast. But when I have a more stable database structure I will add a database export function somewhere. Until I have a decent update mechanism of course. Sorry for the inconvenience.
  • I will add user stats later on so you can see who watched what and also see when are the busier and calm hours of the Emby server. Also hard disk stats and run time hours are usefull maybe. If you have more ideas you may always post them or create feature request on my github page. The more the better :D

 

At the same time I'm also started building a smaller website to promote EmbyStat. This will of course come online when the product is more stable.

  • Like 1
Link to post
Share on other sites

Hey guys,

 

I published a new beta release on Github and docker. Feel free to test it an supply me with feedback.

Github release can be found here: https://github.com/mregni/EmbyStat/releases/tag/0.1.0.24-beta

Docker tags are still uping/embystat:beta-linux and uping/embystat:beta-win

 

I'm still working on getting EmbyStat working as stable as possible before moving on to new stats (user, server, events, ...)

 

edit: release is done.

 

So I was preparing to release a new beta version but it's still way to buggy so it will take some more days before a new update is released. Sorry about that!

I added support for mixed libraries as well which made it a bit more complex very fast. But I'm almost there!

 

I'm also including some small UI changes and better overall feeling. Hang on just a couple more days (2 max) and I will upload a new beta release.

 

Thanks for the testing and feedback so far!

Edited by reggi
  • Like 2
Link to post
Share on other sites
jaspalgill

Hi reggi

 

Just wanted to say I just updated my linux docker and upgraded to the latest beta. All worked fine and the initial wizard worked perfectly. Navigating the graphs and statistics all feels a lot smoother and I've not had any errors pop up so far :) 

  • Like 1
Link to post
Share on other sites

Hey Reggi, 

 

Having a few issues.

 

1. If i access the emby stat server using a domain name then the images (posters etc) dont load. just blank. everything else works, graphs etc.

 

2. In the settings, if i put my emby server as a URL using my domain name it cant connect to it. It works fine when i use the local LAN IP:port. 

 

Any ideas?

 

Also any plans to have a authentication setting, to log into the stats server?

 

Pete

  • Like 1
Link to post
Share on other sites

@jaspalgill 

Thanks for testing EmbyStat and your positive feedback!

 

@@Swynol

I created a new GitHub issue for your problem so I won't lose track of it. When I get home I will test connecting with domain myself and see what error I get.

If you have a GitHub account, feel free to add some screenshots of your console output here: https://github.com/mregni/EmbyStat/issues/218

 

As for the authentication, since I don't support SSL connections yet using Bearer tokens as Authentication seems a bit dangerous. But I do have an issue to investigate the local authentication.

 

Thanks for testing and your feedback as well! 

  • Like 1
Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...