New Statistics plugin

[hackthis02 121]

I'll ask  @DarWunand @crossfire13148since they seem to have the most interest in it. How should I display this info?

A few facts first. 

Here is the code if you're interested.

foreach (var show in showList)
{
    var totalEpisodes = tvdbData.IdList.FirstOrDefault(x => x.ShowId == show.GetProviderId(MetadataProviders.Tvdb))?.Count ?? 0;
    var collectedEpisodes = GetOwnedEpisodesCount(show);
    var seenEpisodes = GetPlayedEpisodeCount(show);
    var totalSpecials = GetOwnedSpecials(show);
    var seenSpecials = GetPlayedSpecials(show);

    if (collectedEpisodes > totalEpisodes)
        totalEpisodes = collectedEpisodes;

    decimal watched = 0;
    decimal collected = 0;
    if (totalEpisodes > 0)
    {
        collected = collectedEpisodes / (decimal)totalEpisodes * 100;
    }

    if (collectedEpisodes > 0)
    {
        watched = seenEpisodes / (decimal)collectedEpisodes * 100;
    }

    showProgress.Add(new ShowProgress
    {
        Name = show.Name,
        SortName = show.SortName,
        Score = show.CommunityRating,
        Status = show.Status,
        StartYear = show.PremiereDate?.ToString("yyyy"),
        Watched = Math.Round(watched, 1),
        Episodes = collectedEpisodes - totalSpecials,
        SeenEpisodes = seenEpisodes,
        Specials = totalSpecials,
        SeenSpecials = seenSpecials,
        Collected = Math.Round(collected, 1),
        Total = totalEpisodes - totalSpecials,
        Id = show.Id.ToString()
    });
}

Specials are included when getting the total number of episodes and collected for a series.
If you don't have specials it doesn't count negatively to your total number. Also specials are removed from the count after the percentage is calculated. 

So I have it currently set if your collected number of episodes is larger then the total number of episodes, the collected is now the new number.
ie.
 if (collectedEpisodes > totalEpisodes)
     totalEpisodes = collectedEpisodes;

This is in the event you have episodes that aren't included in the TVDB count.

When I display the collected column it's collected episodes -specials / total episodes(if greater then collected) -specials (percentage of collected) +number of specials
example:
total: 215
collected: 215
specials: 1
214 / 214 (100%) +1 sp

Watched is a little different. it's watched(plus specials) / collected episodes(if greater then collected) -specials (percentage of watched) +number of specials

Example 1:
total: 69
collected: 61
specials: 8
watched: 54 + 1sp
55 / 61 (79.7%) +8sp

Example:
total: 69
collected: 58
specials: 8
watched: 58 +8sp
66 / 58 (100%) +8sp

Edited May 7, 2024 by hackthis02

[DarWun 310]

@hackthis02Thanks for asking about my thoughts on this! I'm "away-from-keyboard" for the next week or so on vacation. So I haven't reviewed your response in detail yet. But I will do so when I get back in front of my keyboard. I'll provide my thoughts then.

Edited May 7, 2024 by DarWun

[crossfire13148 29]

23 hours ago, hackthis02 said:

When I display the collected column it's collected episodes -specials / total episodes(if greater then collected) -specials (percentage of collected) +number of specials
example:
total: 215
collected: 215
specials: 1
214 / 214 (100%) +1 sp

 

so this is what i am thinking:

using this example that seems perfect.
where the prob seems to be, from my end, and correct me if i am wrong:

***EDIT***
rereading code....so this section was changed,ill get back to you here!
***EDIT***

as long as Episodes is the collected count and Collected is the percentage
 

but what i believe it used to do, was more "correct" in NOT counting specials in collected: as to me "collected" would only consist of the ACTUAL show collected.

in the past it was the specials that kept breaking the "complete" count! 

so basically i am thinking, and hope others chime in
"Collected" = actual episodes collected
"specials" = specials
"total" = Collected +specials
collected display = Collected then the # +SP

example:

total: 21
collected: 20
specials: 1
20 / 20 (100%) +1 sp

You have a total of 21 which is 20 episodes and 1 special

which worked GREAT when you had lets say 2 folders, 1 with 10 in then and 1 with 10, and 1 folder with specials. Verifying you had all the episodes and specials very easily!
S0 = 1
S1 = 10
S2 = 10

but that was also broken by emby, as it shows specials IN the folders, so now you would see the green circles
S0 = 1
S1 = 11
S2 = 10

(counting the green circles on the folders now give you 22!) when you only have 21 (21 episodes and 1 special)

right now running a scan, then will test out some things. may take a day or 2

i hope this makes sense, as my brain is on 40 things right now as i am working and typing!

Edited May 8, 2024 by crossfire13148
read something wrong, looking more in depth

[crossfire13148 29]

ok so from what i said before, (i missed reading a variable so i removed that part!)

i feel "collectedEpisodes" should not include specials, as they are "special"!

so this:
if (totalEpisodes > 0) { collected = collectedEpisodes / (decimal)totalEpisodes * 100; }

should be this:
if (totalEpisodes > 0) { collected = (collectedEpisodes - totalSpecials) / (decimal)totalEpisodes * 100; }

then it would show this:

total: 21
collected: 20
specials: 1
20 / 20 (100%) +1 sp


now as far as watched, i dont use that part of the tables, so i do not have an opinion on if specials SHOULD be considered as watched episodes?
although you could prob do the same here with episodes/specials?

XX number / YY number (%%) + ZZ sp

as you already have the var for them

var seenEpisodes = GetPlayedEpisodeCount(show);
var seenSpecials = GetPlayedSpecials(show);

using like above

if (collectedEpisodes > 0) { watched = (seenEpisodes-seenSpecials) / (decimal)collectedEpisodes * 100; }

[DarWun 310]

Back in front of the computer...sorry for the late response. @hackthis02Everything you (and @crossfire13148) have said pertaining to how collected episodes are displayed makes sense.

How watched episodes are displayed is weird to me though. I have to look at the code and the math a bit more. But let me offer some thoughts I have based on what I'm seeing in the plugin prior to me digging in to the math:

For the series Dollhouse, I have collected all 26 episodes of the series, plus one special (the unaired pilot). After watching all 26 episodes and the pilot, what I expected to see in the watched column was "26 /26 (100%) +1 sp". But what is displayed seems to double count the special that was watched (i.e. 27 total episodes watched plus the watch of one special). Plus "27 /26 (100%)" makes no sense as "27 / 26" is 104%.

The play results for Doctor Who (2005) are even stranger. I have collected four specials, and two episodes out of the 151 total episodes in the series. I watched all four specials. But I have not watched the most recent aired episodes. What would make sense to me to see in the watched column would be "0 / 2 (0%) + 4 sp". "4 /2" being equal to 67% makes no sense. Once I've watched the two episodes, what I would expect to see in the watched column is "2 /2 (100%) +4 sp".

I'll post back with some additional thoughts once I've dug in to the math from your posts.

Edited May 12, 2024 by DarWun

[DarWun 310]

19 hours ago, DarWun said:

Back in front of the computer...sorry for the late response. @hackthis02Everything you (and @crossfire13148) have said pertaining to how collected episodes are displayed makes sense.

.

I'll post back with some additional thoughts once I've dug in to the math from your posts.

Okay. I've reviewed the math behind the watched statistics, and it makes sense. I still find it a bit confusing as to how the statistics are displayed in the plugin though. Would it make more sense to just show the number of items played out of the total items collected (including specials), and then not show the "+x sp"? I know from the collected items column how many specials have been collected.

For example, with Dollhouse in my previous post just display "27 / 27 (100%)" in the watched column. I know based on the what is presented in the collected column that that includes the 26 main episodes plus one special. For Doctor Who in my previous post, the watched column would show "4 / 6 (66.7%)". That may not be ideal, as the user doesn't know what mix of main and special episodes have been watched.

Anyway, I'm good if you leave things as they are now that I understand the math behind it. I just found it weird seeing "27 / 26 (100%). I read that as 27 / 26 = 100%, and the engineer in me said "no it does not" .

 

Edited May 13, 2024 by DarWun

[crossfire13148 29]

so here is an example i found...

new episode, which would be the 250th episode....and i have 249+1 special BUT

so somewhere it is correctly showing i have 249 episodes, but is not seeing that there is 250, OR it is counting my 249+1sp as 250 for the math

this SHOULD be saying

249/250 (99.6%) +1 sp

stats just ran 3 hours ago (after midnight), but i am running it again to see if the results are the same ill update in around 2 hours after scans are done

i may just strip the stats plugin out, just to have the "collected" in series as the only thing that is processed!

also, is there a chance to add "collections" on movies as a stat too?

ie, scans collections to see that i am missing a movie from a collection?

just a thought!


 

[crossfire13148 29]

2 hours ago, crossfire13148 said:

stats just ran 3 hours ago (after midnight), but i am running it again to see if the results are the same ill update in around 2 hours after scans are done

stats finished running, and no change

[crossfire13148 29]

and house of the dragon is doing this also. have 12 episodes and 1 special.

showing 12/12 100 %, but i am missing episode 13...
 

seems the math is using specials as episodes, but displaying correctly 12

Edited June 30, 2024 by crossfire13148

[Happy2Play 9441]

1 hour ago, crossfire13148 said:

and house of the dragon is doing this also. have 12 episodes and 1 special.

showing 12/12 100 %, but i am missing episode 13...
 

seems the math is using specials as episodes, but displaying correctly 12

Specials can't be missing as they are not really counted that way, just if you have them.

10 episodes from season 1 and 2 episodes from season 2 so it looks correct to me.

Unless you are trying to count TODAY's episode that it should not do in my opinion.

Edited June 30, 2024 by Happy2Play

[crossfire13148 29]

On 6/30/2024 at 4:23 PM, Happy2Play said:

Specials can't be missing as they are not really counted that way, just if you have them.

Never said specials were missing, said it was counting the specials as episodes, and misrepresenting a series as "100%" when it is not

Quote

Unless you are trying to count TODAY's episode that it should not do in my opinion.

that is EXACTLY what it does, and that is one of the bonuses i use.

Here is upcoming (today):

and here is what stats show (after it scans 1215am on sunday):

whats IS missing right now, IS house of the dragon, because:

S02E04 is the 14th episode, YET stats is showing this as being complete at 13 of 13

Spinning up a test server to confirm (as main server takes 2 hours to run stats)

[Happy2Play 9441]

10 minutes ago, crossfire13148 said:

Never said specials were missing, said it was counting the specials as episodes, and misrepresenting a series as "100%" when it is not

that is EXACTLY what it does, and that is one of the bonuses i use.

Here is upcoming (today):

and here is what stats show (after it scans 1215am on sunday):

whats IS missing right now, IS house of the dragon, because:

S02E04 is the 14th episode, YET stats is showing this as being complete at 13 of 13

Spinning up a test server to confirm (as main server takes 2 hours to run stats)

According to Emby Today's episode can not be missing as it is still Future/Upcoming.  So I do not believe this plugin should call it missing either.

To me this is totally correct for this example as it can not be missing until tomorrow 7/8.

 

Edited July 7, 2024 by Happy2Play

[crossfire13148 29]

Test Server: House of the Dragon:

 

WITHOUT specials:

 

With 1 Special added:



The ONLY difference is 1 Special was added

a) episodes airing on the day of the scan IS counted as missing
b) Specials ARE being counted as episodes in the count for "complete"
and looks like
c) that "episodes - specials" are being done AFTER the math to bring the display of episodes count back to 13

any questions/doubts you can run this is a test server like i did

 

[crossfire13148 29]

Quote

According to Emby Today's episode can not be missing as it is still Future/Upcoming.  So I do not believe this plugin should call it missing either.

To me this is totally correct for this example as it can not be missing until tomorrow 7/8.

A.) this isnt EMBY, this is the STATS plugin... (and i like this feature to know the day of, what i am needing to get on the server)
B.) you are arguing semantics

it doesnt matter if this is done today or a week from now, it is STILL going to read this the same way, as it has done this twice now, including in my previous examples:
1) ancient aliens was DAYS after, where emby also had it as missing, and not in upcoming
2) house of the dragon, where again, emby also had it as missing, and not in upcoming

to prove this, i can use a series that is ended to prove the point but i think the issue is obvious without more examples

[crossfire13148 29]

And here it is, in EMBY as missing:

And still reporting the same:

 

which completely negates:

Quote

According to Emby Today's episode can not be missing as it is still Future/Upcoming.  So I do not believe this plugin should call it missing either.

To me this is totally correct for this example as it can not be missing until tomorrow 7/8.

and shows Specials are being processed wrong in the episode count

 

[Happy2Play 9441]

Sorry did not check as mine is now 14 of 14 which is completely correct.

Edited July 8, 2024 by Happy2Play

[crossfire13148 29]

4 hours ago, Happy2Play said:

Sorry did not check as mine is now 14 of 14 which is completely correct.

so then take out the last episode, then run the stats, then see what it says

just like i did in the test server

[crossfire13148 29]

so is this plug in dead? or is no one looking at the issue?

[DarkStar1977 93]

2 hours ago, crossfire13148 said:

so is this plug in dead? or is no one looking at the issue?

Since the missing episodes are not anymore in the database of Emby the plugin is providing wrong random data, and it's because the information is not anymore in the database, one example below, but I've identified at least 40 shows with the same problem, 4 that shows are listed in the plugin with 0/0 (0%):

Example:
Plugin Info:

Library:

Missing Episodes:

Since the missing episodes have been removed from the database, the plugin is not working properly.

 

[hackthis02 121]

6 hours ago, crossfire13148 said:

so is this plug in dead? or is no one looking at the issue?

No, the plug-in is not dead. I've just been busy with life/work and haven't had a chance to sit down and debug what's happening.

Reading your posts, there are a lot of moving factors. The plug-in doesn't do any data collection, only aggregation, from Emby. Emby in turn pulls some info from the local DB and other data from the TVBD plugin. So in some examples I could see where the local DB has an episode that hasn't been updated via TVDB. I also need to review how I'm handling specials. Since in some places specials are included with episodes and others it's not.

[noybman 9]

Not sure if I should ask this here, or in a new thread, please advise (I'll happily delete and recreate if thats best).

I'm running v 3.3.1.1 of the plugin. this appears to be the latest.

I'm wondering if there is a place I can get some "advanced" documentation on its configuration, perhaps there's something I can do in the scheduled tasks area? What I'm specifically looking for is a way to FORCE the plugin to look at OLD logs or database files to pull stats from it?

I had some issues a few months ago and basically had to start the database over, but I have backups upon backups as well as logs upon logs. I would ****REALLLLLLLLY**** like to rebuild this data and get all the stats that used to be there back.

All the users are the same so I'm really hoping there is something I can do, even if it requires some manual work, or building entries for sql..... Thanks in advance!

[Happy2Play 9441]

11 minutes ago, noybman said:

Not sure if I should ask this here, or in a new thread, please advise (I'll happily delete and recreate if thats best).

I'm running v 3.3.1.1 of the plugin. this appears to be the latest.

I'm wondering if there is a place I can get some "advanced" documentation on its configuration, perhaps there's something I can do in the scheduled tasks area? What I'm specifically looking for is a way to FORCE the plugin to look at OLD logs or database files to pull stats from it?

I had some issues a few months ago and basically had to start the database over, but I have backups upon backups as well as logs upon logs. I would ****REALLLLLLLLY**** like to rebuild this data and get all the stats that used to be there back.

All the users are the same so I'm really hoping there is something I can do, even if it requires some manual work, or building entries for sql..... Thanks in advance!

Sounds like you are talking about userdata correct?

Might be able to get the backup and restore plugin to extract the data from backup db with a little trickery placing the back db in a full backup.

Otherwise you can look at exporting the userdatas table from backup database and importing it into the new database.

As this plugin is only reading Emby database.

[noybman 9]

9 hours ago, Happy2Play said:

Sounds like you are talking about userdata correct?

Might be able to get the backup and restore plugin to extract the data from backup db with a little trickery placing the back db in a full backup.

Otherwise you can look at exporting the userdatas table from backup database and importing it into the new database.

As this plugin is only reading Emby database.

But, if Im not mistaken, doesnt the statistics plugin only go back 90 days at most when it reads the DB?

[Happy2Play 9441]

2 minutes ago, noybman said:

But, if Im not mistaken, doesnt the statistics plugin only go back 90 days at most when it reads the DB?

Not that I am aware of.

 

But 90 days would never work looking a watching shows for the first time.

[ChaoOos 2]

On 7/29/2024 at 4:53 PM, hackthis02 said:

No, the plug-in is not dead. I've just been busy with life/work and haven't had a chance to sit down and debug what's happening.

Reading your posts, there are a lot of moving factors. The plug-in doesn't do any data collection, only aggregation, from Emby. Emby in turn pulls some info from the local DB and other data from the TVBD plugin. So in some examples I could see where the local DB has an episode that hasn't been updated via TVDB. I also need to review how I'm handling specials. Since in some places specials are included with episodes and others it's not.

Hi, you had time to look into this "bug"?

I checked it myself and found a strange behavier.

1. Checking a series in the statistics plugin

2. Look in missing episodes

3. new scan for statistics plugin

4. Checking a series in the statistics plugin again

 

I think each series need a new trigger.

Maybe it can help you to fix it