Developing a standalone embystat server

[gridrunner 1]

Hi @reggi, thanks for your time in making this. 

When i was your reading your first post I was thinking about a name. I know its a few months on now

 

but what about  Em-Boss

[ddurdle 75]

Nice work

[reggi 415]

Hey everyone,

 

@@gridrunner Good idea! I'll take that into consideration. 

 

To give you all some more information about the server. The next update is a bit delayed because I've updated to .NET CORE 2.1 and I have to rewrite my docker files from scratch.

Als I'm updating the following at the same time

- Better database migrations (Fluent Migrations)

- WebSocket connection to Emby. This will allow me to receive life use data.

- Update system. So you guy's don't have to install a new server every time there is an update.

- Some smaller UI issues are also going to be fixed.

 

I'll let you all know when the update is ready and pushed to Github and docker.

 

Thanks for the patience and sorry for the delay! 

[mgworek 124]

Hey, just wanted to say awesome job on this. I started playing around with it last week.

 

Sorry if this has been asked before but are you planning on changing the Suspicious Movies logic? 

 

#1 it says I have 2032 duplicate movies, which makes no sense.

#2 the ones it shows as duplicates, the names are the same but the years are different. Example. Casino Royale. One was from 1967 and one is from 2006. Another are christmas movies, tv christmas movies often use the same name a few years apart.

 

Maybe if the name is the same, check date and if they are different, don't report them?

 

Thanks!

[reggi 415]

@@mgworek

Thanks for the input. Strange it's telling that you have 2032 duplicate movies indeed. 

 

As for #2, a good idea indeed. Didn't think about that before. I created an issue for it and it will be fixed in the next beta release. Maybe I can check on name and premiere date. But I have to reevaluate the logic because maybe looking at the IMDB id is enough. 

[mgworek 124]

Awesome. Thanks.

 

Ignore #1. I just discovered its a problem in safari and I thought it was telling me I had that many Duplicate Movies. The Blue Boxes are staying on the screen for some reason. In Chrome, it works fine, they go away.

 

Screen shot added so it makes sense.

[reggi 415]

Thanks, that's some very strange behavior. Do you have the same problem when navigating to the People or Graphs tab on that page?

[ddurdle 75]

Can you use this to monitor multiple emby servers?

[reggi 415]

Sorry for the silence, but I didn't had the time to work on it anymore. The update mechanism is almost done but needs some more tweaking before I can create a new release.

I'll post an update when that is ready

 

@@ddurdle

Thanks the idea yes, but not in the first stable release. 

 

I do keep in mind when I program that multiple servers should be supported, but when I implement it for real there will be a lot of changes needed anyway.

So for now it's just one Emby server.

 

What you can do is setup multiple docker containers in parallel.

[adrianwi 250]

It's been completely broken for me since switching to the 3.6.x.x betas, although I think along with the .db changes there are some changes to the APIs too?

 

I'd just like to say again that I think what you're doing is great, and having this standalone makes sense to me coming from Plex and PlexPy (now Tautulli).

 

That said, I think as the development currently stands you are missing a trick.  It looks to be replicating the Reporting plugin, which provides lots of numbers about the media library (e.g. how many of this and that) but not much about how the media library is being used, which for me at least is much more valuable.  Knowing I have 8 Marvel movies (which I probably know without a reporting app) is of very little value, but knowing Iron Man has been watched by all my users and Spiderman hasn't been watched by anyone is far more useful.  

 

Anyway, keep up the great work.  Having something that comes close to what PlexPy did, but for emby would be fantastic.

Edited September 23, 2018 by adrianwi

[reggi 415]

@@adrianwi

 

Thanks for your feedback and you are a 100% correct! At the moment the beta version only holds some static data about the media. I choose to implement the static data first because of the simplicity. At the moment I'm working on the update system. When that is finished I will start working on the live user data "who watched what and when data".

 

I already have websocket connection implemented to Emby, so getting the correct data is just a matter of mapping events into my db and calculating stats from it.

 

I do have a lot of work to do so my time working on EmbyStat is limited, but I hope in a month I will be able to move forward again a lot. Sorry for the delay!

[gleep52 16]

Is there an install guide for this?  

[reggi 415]

@@gleep52

 

You can find a small install guide here: https://github.com/mregni/EmbyStat/wiki/docker 

Easiest for now is to host EmbyStat in a Docker container, just run it with this command "docker run -p <port>:80 uping/embystat:<tag>"

 

If you still have problems with booting up the server let me know!

[gleep52 16]

Do I run this on the same server that Emby is running on?  What if port 80 is already taken by another service?

 

Edit:  How do we install docker?  I don't see any information about that on the github page...

Edited September 28, 2018 by gleep52

[jurmb84 281]

I have a fairly large movie database with about 8100 movies.... It seems to roll through the collections ok but when it asks for the movies it dies about 30 seconds in with the following error:

 

I have tried to run this 5 or 6 times and it always dies at this point...

2018-10-06 21:03:05 [INF] EMBY-CLIENT	Sending GET: http://{emby-server}/emby/Users/33e98a6f677b49f5b5582bcd034f71ef/Items?sortOrder=Ascending&SeriesStatuses=&fields=Genres,DateCreated,MediaSources,HomePageUrl,OriginalTitle,Studios,MediaStreams,Path,Overview,ProviderIds,SortName,ParentId,People&Filters=&ImageTypes=&VideoTypes=&AirDays=&EnableImageTypes=Banner,Primary,Thumb,Logo&ParentId=f137a2dd21bbc1b99aa5c0f6bf02a805&SortBy=sortname&recursive=True&Genres=&StudioIds=&IncludeItemTypes=Movie&Years=&format=json
2018-10-06 21:03:35 [ERR] TASK-WORKER	Error
MediaBrowser.Model.Net.HttpException: One or more errors occurred. (The request was aborted: The request was canceled.) ---> System.AggregateException: One or more errors occurred. (The request was aborted: The request was canceled.) ---> System.Net.WebException: The request was aborted: The request was canceled.
   at System.Net.HttpWebRequest.CheckAbort()
   at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
   --- End of inner exception stack trace ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at EmbyStat.Api.EmbyClient.Net.HttpWebRequestClient.<GetResponse>d__2.MoveNext() in C:\BuildAgent-DarthVader\work\aa794b64fcf59423\EmbyStat.Api\EmbyClient\Net\HttpWebRequestClient.cs:line 54
   --- End of inner exception stack trace ---
   at EmbyStat.Api.EmbyClient.Net.HttpWebRequestClient.<GetResponse>d__2.MoveNext() in C:\BuildAgent-DarthVader\work\aa794b64fcf59423\EmbyStat.Api\EmbyClient\Net\HttpWebRequestClient.cs:line 80
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at EmbyStat.Api.EmbyClient.Net.HttpWebRequestClient.<SendAsync>d__5.MoveNext() in C:\BuildAgent-DarthVader\work\aa794b64fcf59423\EmbyStat.Api\EmbyClient\Net\HttpWebRequestClient.cs:line 116
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at EmbyStat.Api.EmbyClient.BaseClient`1.<SendAsync>d__57.MoveNext() in C:\BuildAgent-DarthVader\work\aa794b64fcf59423\EmbyStat.Api\EmbyClient\BaseClient.cs:line 305
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at EmbyStat.Api.EmbyClient.EmbyClient.<GetItemsAsync>d__6.MoveNext() in C:\BuildAgent-DarthVader\work\aa794b64fcf59423\EmbyStat.Api\EmbyClient\EmbyClient.cs:line 122
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at EmbyStat.Tasks.Tasks.MediaSyncTask.<GetMoviesFromEmby>d__22.MoveNext() in C:\BuildAgent-DarthVader\work\aa794b64fcf59423\EmbyStat.Tasks\Tasks\MediaSyncTask.cs:line 158
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at EmbyStat.Tasks.Tasks.MediaSyncTask.<ProcessMovies>d__21.MoveNext() in C:\BuildAgent-DarthVader\work\aa794b64fcf59423\EmbyStat.Tasks\Tasks\MediaSyncTask.cs:line 112
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at EmbyStat.Tasks.Tasks.MediaSyncTask.<Execute>d__19.MoveNext() in C:\BuildAgent-DarthVader\work\aa794b64fcf59423\EmbyStat.Tasks\Tasks\MediaSyncTask.cs:line 85
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at EmbyStat.Tasks.ScheduledTaskWorker.<ExecuteInternal>d__54.MoveNext() in C:\BuildAgent-DarthVader\work\aa794b64fcf59423\EmbyStat.Tasks\ScheduledTaskWorker.cs:line 232
2018-10-06 21:03:35 [INF] TASK-WORKER	TASKS.MEDIASYNCTITLE Failed after 7 minute(s) and 27 seconds
2018-10-06 21:03:35 [INF] TASK-MANAGER	ExecuteQueuedTasks

Any ideas?

 

Thanx in advance...

[CobraTG2000 0]

When trying to setup the stats server i continually receive the error "EmbyStats is an Emby administrator" - I have gone in and verified its not an admin, also tried with an admin account. No matter what account i use i always get this message. Any ideas?

 

So it seems to be something with permissions. Moved it from my external drive to my desktop and all appears to work correctly. Still cant determine were the issues was - still need help if possible as id like to keep it off the C drive. 

 

Also it seems that after getting it connected, when doing the 1st initial sync, it times out or errors out after about 45 seconds or so while doing the media sync.

Edited October 16, 2018 by CobraTG2000

[reggi 415]

Hey @@CobraTG2000

 

Sorry to hear you got some problems with the server.

 

So you are running the server and Emby on the same machine? And I presume you are running EmbyStat without docker?

I'll test it when I get later this week. At the moment I don't have a lot of time to work on the project (still working on the promised updating system first)

[CobraTG2000 0]

Hey @@CobraTG2000

 

Sorry to hear you got some problems with the server.

 

So you are running the server and Emby on the same machine? And I presume you are running EmbyStat without docker?

I'll test it when I get later this week. At the moment I don't have a lot of time to work on the project (still working on the promised updating system first)

 

I am using the Docker version. Didn't see any other option besides Linux. 

Emby Server - 2012r2 (HTTPS)

Emby Stats - I have tried it on the same server as the Emby and also on my Windows 10 64 machine. Both with same results.

Edited October 17, 2018 by CobraTG2000

[CobraTG2000 0]

I have a fairly large movie database with about 8100 movies.... It seems to roll through the collections ok but when it asks for the movies it dies about 30 seconds in with the following error:

 

I have tried to run this 5 or 6 times and it always dies at this point...

2018-10-06 21:03:05 [INF] EMBY-CLIENT	Sending GET: http://{emby-server}/emby/Users/33e98a6f677b49f5b5582bcd034f71ef/Items?sortOrder=Ascending&SeriesStatuses=&fields=Genres,DateCreated,MediaSources,HomePageUrl,OriginalTitle,Studios,MediaStreams,Path,Overview,ProviderIds,SortName,ParentId,People&Filters=&ImageTypes=&VideoTypes=&AirDays=&EnableImageTypes=Banner,Primary,Thumb,Logo&ParentId=f137a2dd21bbc1b99aa5c0f6bf02a805&SortBy=sortname&recursive=True&Genres=&StudioIds=&IncludeItemTypes=Movie&Years=&format=json
2018-10-06 21:03:35 [ERR] TASK-WORKER	Error
MediaBrowser.Model.Net.HttpException: One or more errors occurred. (The request was aborted: The request was canceled.) ---> System.AggregateException: One or more errors occurred. (The request was aborted: The request was canceled.) ---> System.Net.WebException: The request was aborted: The request was canceled.
   at System.Net.HttpWebRequest.CheckAbort()
   at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
   --- End of inner exception stack trace ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at EmbyStat.Api.EmbyClient.Net.HttpWebRequestClient.<GetResponse>d__2.MoveNext() in C:\BuildAgent-DarthVader\work\aa794b64fcf59423\EmbyStat.Api\EmbyClient\Net\HttpWebRequestClient.cs:line 54
   --- End of inner exception stack trace ---
   at EmbyStat.Api.EmbyClient.Net.HttpWebRequestClient.<GetResponse>d__2.MoveNext() in C:\BuildAgent-DarthVader\work\aa794b64fcf59423\EmbyStat.Api\EmbyClient\Net\HttpWebRequestClient.cs:line 80
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at EmbyStat.Api.EmbyClient.Net.HttpWebRequestClient.<SendAsync>d__5.MoveNext() in C:\BuildAgent-DarthVader\work\aa794b64fcf59423\EmbyStat.Api\EmbyClient\Net\HttpWebRequestClient.cs:line 116
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at EmbyStat.Api.EmbyClient.BaseClient`1.<SendAsync>d__57.MoveNext() in C:\BuildAgent-DarthVader\work\aa794b64fcf59423\EmbyStat.Api\EmbyClient\BaseClient.cs:line 305
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at EmbyStat.Api.EmbyClient.EmbyClient.<GetItemsAsync>d__6.MoveNext() in C:\BuildAgent-DarthVader\work\aa794b64fcf59423\EmbyStat.Api\EmbyClient\EmbyClient.cs:line 122
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at EmbyStat.Tasks.Tasks.MediaSyncTask.<GetMoviesFromEmby>d__22.MoveNext() in C:\BuildAgent-DarthVader\work\aa794b64fcf59423\EmbyStat.Tasks\Tasks\MediaSyncTask.cs:line 158
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at EmbyStat.Tasks.Tasks.MediaSyncTask.<ProcessMovies>d__21.MoveNext() in C:\BuildAgent-DarthVader\work\aa794b64fcf59423\EmbyStat.Tasks\Tasks\MediaSyncTask.cs:line 112
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at EmbyStat.Tasks.Tasks.MediaSyncTask.<Execute>d__19.MoveNext() in C:\BuildAgent-DarthVader\work\aa794b64fcf59423\EmbyStat.Tasks\Tasks\MediaSyncTask.cs:line 85
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at EmbyStat.Tasks.ScheduledTaskWorker.<ExecuteInternal>d__54.MoveNext() in C:\BuildAgent-DarthVader\work\aa794b64fcf59423\EmbyStat.Tasks\ScheduledTaskWorker.cs:line 232
2018-10-06 21:03:35 [INF] TASK-WORKER	TASKS.MEDIASYNCTITLE Failed after 7 minute(s) and 27 seconds
2018-10-06 21:03:35 [INF] TASK-MANAGER	ExecuteQueuedTasks

Any ideas?

 

Thanx in advance...

 

After i looked back though the posts, i realized, im having the exact same issues as you. I too have a large library - 12k items or so. Dies 30-45 secs into scan.

[gleep52 16]

After i looked back though the posts, i realized, im having the exact same issues as you. I too have a large library - 12k items or so. Dies 30-45 secs into scan.

 movies?  12K?  where do you store all of these??

[CobraTG2000 0]

 movies?  12K?  where do you store all of these??

 

Well, i have 4 servers that are on full time. 2 Poweredge 2900's 10x4TB drives in each (Raid-5 for now). 2 PowerEdge 2950, with 6x4TB drives (raid-5 as well - 1 is dedicated to emby and 1 to plex + storage. Those drives will soon be upgraded to 12 or 14TB drives, not sure which yet. 

 

But yes, 12k movies, around 25k tv episodes, and around 60k songs. 

[jurmb84 281]

I have one server that reads them all with a 144TB NAS... (obviously used for other things than just Emby)... However, I was able to make it work by downloading the source code changing all timeouts to 120sec (120000 ms) in the following files:

 EmbyStat\EmbyStat.Web\ClientApp\protractor.conf.js(7):  allScriptsTimeout: 120000,
 EmbyStat\EmbyStat.Web\ClientApp\protractor.conf.js(19):    defaultTimeoutInterval: 120000,
 EmbyStat\EmbyStat.Api\EmbyClient\Net\HttpRequest.cs(22):		    Timeout = 120000;

and adding the following line to all '.csproj' files inside of the <PropertyGroup> tags:

<CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies>

open vs studio 2017 and open the EmbyStat solution... right click EmbyStat.Web and click Set as Startup Project... then right click the solution at the top and select "Build All"...

 

grab all the files from the output bin folder (except ClientApp) after compilation.... (EmbyStat\EmbyStat.Web\bin\Debug\netcoreapp2.1) and put them in your production EmbyStat folder overwriting the originals...

 

then, make sure you on have the latest LTS version of node installed (8.12.0) ... open a cmd window and change directory to the EmbyStat project folder and drill down to EmbyStat.Web -> ClientApp...

 

type (without the quotes) "npm install" and press enter...

 

type (without the quotes) "npm run build" and press enter...

 

this will create a dist folder in the EmbyStat\EmbyStat.Web\ClientApp folder...

 

rename the dist folder in your production EmbyStat\ClientApp folder to dist.old and copy the dist folder from EmbyStat\EmbyStat.Web\ClientApp to production EmbyStat\ClientApp

 

i use the dotnet method of running EmbyStat so I then go to my production EmbyStat and use the command "dotnet EmbyStat.Web.dll" and connect via "localhost:5000"

 

Using this method I was able to do not just movies but also all TV shows... Make sure you have a TVDBapi code to enter into the setup as well so you can find missing episodes when you sync... the sync took almost 4 hours for me but was a HUGE help in finding duplicate imdb numbers due to having the same movie in different folders... 

 

Hope this helps...

 

Cheers 

Edited October 18, 2018 by jurmb84

[reggi 415]

You guys have some very nice servers there

 

I was thinking about the timeout as well, thanks @@jurmb84 for taking the time to rebuild the code and test it!

I will increase the http timeouts in the next release so huge collections will not crash the sync.

 

Nice to hear the server helped you out to find duplicates in your collection!

[nobody01 87]

When will there be another linux docker version available? Can we get a nightly for testing?

[reggi 415]

@@jaketame

 

At the moment there are no nightly build to test really since I haven't got the time anymore to work on it.

I'll post an update here when I'll be able to work on it again. I think I will be able to start working on it at the beginning of December.