New Statistics plugin

[DarWun 266]

@hackthis02In one of the recent updates, a "Most Watched Shows" category was added under "Show Statistics". For me, it seems to be showing the first three series listed in my library alphabetically. They are definitely not the most watched series. One of them only has a single season with 24 episodes. There are other series in my library with over a hundred episodes watched within the last six months that I would have expected to show up instead of it.

[hackthis02 93]

1 hour ago, DarWun said:

@hackthis02In one of the recent updates, a "Most Watched Shows" category was added under "Show Statistics". For me, it seems to be showing the first three series listed in my library alphabetically. They are definitely not the most watched series. One of them only has a single season with 24 episodes. There are other series in my library with over a hundred episodes watched within the last six months that I would have expected to show up instead of it.

It's based off number of episodes finished in your library. So, I'm betting if you only have 24 episodes you might have a lot more people having watched all 24 compared to the series with 100s of episodes.

You can check my logic. The algorithm I use is
I get the number of episodes a user has watched per show.
Then I divide that by the number of total collected episodes (episodes you have in your library, not total aired).
I added that number to the percentage of watched episodes per show of all the other users.
Then I divide that by the number of users on the server.
 

foreach (var user in users)
{
  foreach (var show in showList)
  {
    SetUser(user);
    var totalEpisodes = tvdbData.IdList.FirstOrDefault(x => x.ShowId == show.GetProviderId(MetadataProviders.Tvdb))?.Count ?? 0;
    var collectedEpisodes = GetOwnedEpisodesCount(show);
    var seenEpisodes = GetPlayedEpisodeCount(show);
    var totalSpecials = GetOwnedSpecials(show);
    var seenSpecials = GetPlayedSpecials(show);

    if (collectedEpisodes > totalEpisodes)
      totalEpisodes = collectedEpisodes;

    decimal watched = 0;
    decimal collected = 0;
    if (totalEpisodes > 0)
    {
      collected = collectedEpisodes / (decimal)totalEpisodes * 100;
    }

    if (collectedEpisodes > 0)
    {
      watched = seenEpisodes / (decimal)collectedEpisodes * 100;
    }

    var index = showProgress.FindIndex(x => x.Name == show.Name);

    if (index != -1)
    {
      showProgress[index].Watched += Math.Round(watched, 1);
    }
    else
    {
      showProgress.Add(new ShowProgress
	  {
 		Name = show.Name,
 		SortName = show.SortName,
 		Score = show.CommunityRating,
 		Status = show.Status,
 		StartYear = show.PremiereDate?.ToString("yyyy"),
 		Watched = Math.Round(watched, 1),
 		Episodes = collectedEpisodes,
 		SeenEpisodes = seenEpisodes,
 		Specials = totalSpecials,
 		SeenSpecials = seenSpecials,
 		Collected = Math.Round(collected, 1),
 		Total = totalEpisodes
 	   });
    }
  }
}


foreach (var show in showProgress)
{
  show.Watched = Math.Round(show.Watched / users.Count(), 1);
}
List<ShowProgress> SortedList = showProgress.OrderByDescending(o => o.Watched).ToList();

You can also turn on debug logging and look for "CalculateMostWatchedShows" and it will show you the shows and what percentage of your users have watched it.

Edited November 8, 2022 by hackthis02

[DarWun 266]

34 minutes ago, hackthis02 said:

It's based off number of episodes finished in your library. So, I'm betting if you only have 24 episodes you might have a lot more people having watched all 24 compared to the series with 100s of episodes.

You can check my logic. The algorithm I use is
I get the number of episodes a user has watched per show.
Then I divide that by the number of total collected episodes (episodes you have in your library, not total aired).
I added that number to the percentage of watched episodes per show of all the other users.
Then I divide that by the number of users on the server.
 

foreach (var user in users)
{
  foreach (var show in showList)
  {
    SetUser(user);
    var totalEpisodes = tvdbData.IdList.FirstOrDefault(x => x.ShowId == show.GetProviderId(MetadataProviders.Tvdb))?.Count ?? 0;
    var collectedEpisodes = GetOwnedEpisodesCount(show);
    var seenEpisodes = GetPlayedEpisodeCount(show);
    var totalSpecials = GetOwnedSpecials(show);
    var seenSpecials = GetPlayedSpecials(show);

    if (collectedEpisodes > totalEpisodes)
      totalEpisodes = collectedEpisodes;

    decimal watched = 0;
    decimal collected = 0;
    if (totalEpisodes > 0)
    {
      collected = collectedEpisodes / (decimal)totalEpisodes * 100;
    }

    if (collectedEpisodes > 0)
    {
      watched = seenEpisodes / (decimal)collectedEpisodes * 100;
    }

    var index = showProgress.FindIndex(x => x.Name == show.Name);

    if (index != -1)
    {
      showProgress[index].Watched += Math.Round(watched, 1);
    }
    else
    {
      showProgress.Add(new ShowProgress
	  {
 		Name = show.Name,
 		SortName = show.SortName,
 		Score = show.CommunityRating,
 		Status = show.Status,
 		StartYear = show.PremiereDate?.ToString("yyyy"),
 		Watched = Math.Round(watched, 1),
 		Episodes = collectedEpisodes,
 		SeenEpisodes = seenEpisodes,
 		Specials = totalSpecials,
 		SeenSpecials = seenSpecials,
 		Collected = Math.Round(collected, 1),
 		Total = totalEpisodes
 	   });
    }
  }
}


foreach (var show in showProgress)
{
  show.Watched = Math.Round(show.Watched / users.Count(), 1);
}
List<ShowProgress> SortedList = showProgress.OrderByDescending(o => o.Watched).ToList();

You can also turn on debug logging and look for "CalculateMostWatchedShows" and it will show you the shows and what percentage of your users have watched it.

Thanks for the explanation. I'm going to take some time to digest this. I am the only user accessing media on my server. So for the series in question, I only ever watched 24 episodes. There are no other users. With restarts after partial rewatches, I may have played episodes from the series 50 times over the watchthough several years ago. But I just rewatched all three season of Twin Peaks over the last couple months, and the number of playbacks from that watchthrough were way above that.

I'll post back after some more troubleshooting.

[hackthis02 93]

3 minutes ago, DarWun said:

Thanks for the explanation. I'm going to take some time to digest this. I am the only user accessing media on my server. So for the series in question, I only ever watched 24 episodes. There are no other users. With restarts after partial rewatches, I may have played episodes from the series 50 times over the watchthough several years ago. But I just rewatched all three season of Twin Peaks over the last couple months, and the number of playbacks from that watchthrough were way above that.

I'll post back after some more troubleshooting.

My understanding of emby's watched function is it's either watched or unwatched. So an episode you've watched 100 and one you've only watched once, come back the same.

[Happy2Play 8282]

14 minutes ago, hackthis02 said:

My understanding of emby's watched function is it's either watched or unwatched. So an episode you've watched 100 and one you've only watched once, come back the same.

Correct here is 4 episodes of S.T. Picard as it is listed first on my test machine.  But will assume playcount is a factor.

 

 

[hackthis02 93]

2 minutes ago, Happy2Play said:

Correct here is 4 episodes of S.T. Picard as it is listed first on my test machine.  But will assume playcount is a factor.

 

 

I'll double check but I believe I'm using the played variable.

[jordy 284]

Hi,

 

For a while now I have been getting this error msg:

Object reference not set to an instance of an object.
at Statistics.Helpers.Calculator.CalculateMovieQualities()
at Statistics.ScheduledTasks.CalculateStatsTask.MediaBrowser.Model.Tasks.IScheduledTask.Execute(CancellationToken cancellationToken, IProgress`1 progress)
at Emby.Server.Implementations.ScheduledTasks.ScheduledTaskWorker.ExecuteInternal(TaskOptions options)

Any idea what is wrong?

 

Thanks jordy

[hackthis02 93]

9 minutes ago, jordy said:

Hi,

 

For a while now I have been getting this error msg:

Object reference not set to an instance of an object.
at Statistics.Helpers.Calculator.CalculateMovieQualities()
at Statistics.ScheduledTasks.CalculateStatsTask.MediaBrowser.Model.Tasks.IScheduledTask.Execute(CancellationToken cancellationToken, IProgress`1 progress)
at Emby.Server.Implementations.ScheduledTasks.ScheduledTaskWorker.ExecuteInternal(TaskOptions options)

Any idea what is wrong?

 

Thanks jordy

You need to find the unidentified items in your library.

https://emby.media/community/index.php?/topic/65998-new-statistics-plugin/&do=findComment&comment=1180098

[DarWun 266]

On 11/7/2022 at 7:39 PM, hackthis02 said:

It's based off number of episodes finished in your library. So, I'm betting if you only have 24 episodes you might have a lot more people having watched all 24 compared to the series with 100s of episodes.

You can check my logic. The algorithm I use is
I get the number of episodes a user has watched per show.
Then I divide that by the number of total collected episodes (episodes you have in your library, not total aired).
I added that number to the percentage of watched episodes per show of all the other users.
Then I divide that by the number of users on the server.
 

You can also turn on debug logging and look for "CalculateMostWatchedShows" and it will show you the shows and what percentage of your users have watched it.

@hackthis02Your algorithm seems fine. But the results aren't what I would expect based on it. First off, I am the only user on my server. So that removes the user part of the algorithm. One thing I'm not clear on in your algorithm is the total collected episodes. Is that the total collected episodes on the server, or the total collected episodes for the Series? If it is the latter, then the estimated value for all series is always going to be 100 after all episodes in a series have been watched (i.e. if you have a series with 24 episodes in the library and you watch all 24 of them then (24/24)*100=100). And from what I'm seeing after enabling debug mode, that is exactly what is being reported on lines in the log for "CalculateMostWatchedShows".

Shouldn't the plugin be calculating the percentage based on the total number of episodes on the server? 

Edited November 10, 2022 by DarWun

[hackthis02 93]

2 hours ago, DarWun said:

@hackthis02Your algorithm seems fine. But the results aren't what I would expect based on it. First off, I am the only user on my server. So that removes the user part of the algorithm. One thing I'm not clear on in your algorithm is the total collected episodes. Is that the total collected episodes on the server, or the total collected episodes for the Series? If it is the latter, then the estimated value for all series is always going to be 100 after all episodes in a series have been watched (i.e. if you have a series with 24 episodes in the library and you watch all 24 of them then (24/24)*100=100). And from what I'm seeing after enabling debug mode, that is exactly what is being reported on lines in the log for "CalculateMostWatchedShows".

Shouldn't the plugin be calculating the percentage based on the total number of episodes on the server? 

It is calculating based on the total number of episodes on the server.

watched = seenEpisodes / (decimal)collectedEpisodes * 100;

I think you might be a fringe case. Since you're the only user and you seem to watch all the shows on your server to completion. In that case play count would need to be taken into account. But if I did that, on servers where there was more than one user, a single user could "put their thumb on the scale" by watching and episode/season/show over and over again. It wouldn't be the most watched show across the whole server, it would just be the show(s) that one user likes. I may add most watched shows in the User-Based statistics page in the future, which would act more akin to how you're expecting it should.

[DarWun 266]

9 minutes ago, hackthis02 said:

It is calculating based on the total number of episodes on the server.

watched = seenEpisodes / (decimal)collectedEpisodes * 100;

I think you might be a fringe case. Since you're the only user and you seem to watch all the shows on your server to completion. In that case play count would need to be taken into account. But if I did that, on servers where there was more than one user, a single user could "put their thumb on the scale" by watching and episode/season/show over and over again. It wouldn't be the most watched show across the whole server, it would just be the show(s) that one user likes. I may add most watched shows in the User-Based statistics page in the future, which would act more akin to how you're expecting it should.

Not the first time I've been referred to as a "fringe case" ;-). Thanks for explaining how this feature is intended to work.

[jordy 284]

On 11/9/2022 at 5:47 PM, hackthis02 said:

You need to find the unidentified items in your library.

https://emby.media/community/index.php?/topic/65998-new-statistics-plugin/&do=findComment&comment=1180098

@hackthis02, Finally got around to attending to this. It turned out to be a movie, correctly named but in a TV Show folder.

 

Thanks for your help and thanks to @Happy2Play for his code.

 

cheers Jordy

[Happy2Play 8282]

Is there a issue with codecs video appears ok but audio is way off?

[hackthis02 93]

32 minutes ago, Happy2Play said:

Is there a issue with codecs video appears ok but audio is way off?

In that area it should only be displaying the video codecs. I just noticed in mine that there is one ac3 being reported. So, I'll look into what's causing this.

[hackthis02 93]

@Happy2PlayI have an update that should fix this issue. But I'm unable to upload it. When I go to Emby's Admin site and try to upload the new file, I get an "Unauthorized" error? Any idea why this is happening? Or know who I bug about it? (If it's ebr, I've already hit him up and am awaiting a reply.)

[Happy2Play 8282]

2 hours ago, hackthis02 said:

@Happy2PlayI have an update that should fix this issue. But I'm unable to upload it. When I go to Emby's Admin site and try to upload the new file, I get an "Unauthorized" error? Any idea why this is happening? Or know who I bug about it? (If it's ebr, I've already hit him up and am awaiting a reply.)

Sorry only the admins can help with this, but believe another plugin dev had a similar issue recently.

[Luke 37066]

3 hours ago, hackthis02 said:

@Happy2PlayI have an update that should fix this issue. But I'm unable to upload it. When I go to Emby's Admin site and try to upload the new file, I get an "Unauthorized" error? Any idea why this is happening? Or know who I bug about it? (If it's ebr, I've already hit him up and am awaiting a reply.)

One thing you could try while waiting for him is signing out and back in again.

[hackthis02 93]

2 hours ago, Luke said:

One thing you could try while waiting for him is signing out and back in again.

I already tried that, tried Edge & Chrome (regular and incognito). Also Tried moving the file to my phone and uploading from a different network.

[Luke 37066]

51 minutes ago, hackthis02 said:

I already tried that, tried Edge & Chrome (regular and incognito). Also Tried moving the file to my phone and uploading from a different network.

OK I'm sure he'll get you squared away tomorrow. Thanks.

[hackthis02 93]

@Happy2Play ebr got me sorted and you should be able to get the fix now.

[DarWun 266]

@hackthis02The Statistic plug-in updated to version 3.0.6.0 earlier today on Emby Beta 4.8.0.21. Since the update, the calculate statistics task fails repeatedly.

I've attached a log that captures a failed scan. For testing purposes, I rolled back to version 3.0.5.2. The scan completed fine. Updated to 3.0.6.0 again and the scan failed.

embyserver.txt

[Happy2Play 8282]

I can say it is working on my primary beta server but see it fails on another test server and my stable server.

 

2023-02-14 14:50:07.218 Error TaskManager: Error
	*** Error Report ***
	Version: 4.7.11.0
	Command line: A:\Emby-Server\system\EmbyServer.dll -noautorunwebapp
	Operating system: Microsoft Windows 10.0.19045
	Framework: .NET 6.0.10
	OS/Process: x64/x64
	Runtime: A:/Emby-Server/system/System.Private.CoreLib.dll
	Processor count: 16
	Data path: A:\Emby-Server
	Application path: A:\Emby-Server\system
	System.NullReferenceException: System.NullReferenceException: Object reference not set to an instance of an object.
	   at Statistics.Helpers.Calculator.GetMediaResalution(MediaStream typeInfo)
	   at Statistics.Helpers.Calculator.CalculateMovieQualities()
	   at Statistics.ScheduledTasks.CalculateStatsTask.MediaBrowser.Model.Tasks.IScheduledTask.Execute(CancellationToken cancellationToken, IProgress`1 progress)
	   at Emby.Server.Implementations.ScheduledTasks.ScheduledTaskWorker.ExecuteInternal(TaskOptions options)
	Source: statistics
	TargetSite: System.String GetMediaResalution(MediaBrowser.Model.Entities.MediaStream)

 

[hackthis02 93]

I just pushed an update. Tell me if that fixes the issue.

[Happy2Play 8282]

7 minutes ago, hackthis02 said:

I just pushed an update. Tell me if that fixes the issue.

Appears to work for me on all my systems now.

No sure about the unknown or 4K though.  Looks like I have some searching to do.

[hackthis02 93]

2 minutes ago, Happy2Play said:

Appears to work for me on all my systems now.

No sure about the unknown or 4K though.  Looks like I have some searching to do.

If you check your logs, it should till you which medias are throwing those.