New Statistics plugin

[DarkStar1977 119]

@hackthis02Last version completes scanning but maintain previous issues:

- Shows presented with no episodes:

- Shows with wrong episode and missing counting:

[hackthis02 129]

3 hours ago, DarkStar1977 said:

@hackthis02Last version completes scanning but maintain previous issues:

- Shows presented with no episodes:

- Shows with wrong episode and missing counting:

Enable debug logging and send me the log.

[DarkStar1977 119]

2 minutes ago, hackthis02 said:

Enable debug logging and send me the log.

attached

embyserver (1).txt

[hackthis02 129]

5 minutes ago, DarkStar1977 said:

attached

embyserver (1).txt 2.69 MB · 0 downloads

So the first thing I see is you're missing the TVDB files. Confirm you have the TVDB plugin and then 'Refresh Metadata' on your TV library.
Without the TVDB files, the plugin has no idea how many episodes there are.
 

2025-02-12 03:15:06.458 Error Statistics: Could not find a part of the path '/mnt/HD/HD_a2/emby/cache/tvdb/426080/episodes-official.json'.
2025-02-12 03:15:06.459 Error Statistics: Could not find a part of the path '/mnt/HD/HD_a2/emby/cache/tvdb/412826/episodes-official.json'.
2025-02-12 03:15:06.459 Error Statistics: Could not find a part of the path '/mnt/HD/HD_a2/emby/cache/tvdb/438130/episodes-official.json'.
2025-02-12 03:15:06.484 Error Statistics: Could not find a part of the path '/mnt/HD/HD_a2/emby/cache/tvdb/406592/episodes-official.json'.
2025-02-12 03:15:06.484 Error Statistics: Could not find a part of the path '/mnt/HD/HD_a2/emby/cache/tvdb/276890/episodes-official.json'.
2025-02-12 03:15:06.506 Error Statistics: Could not find a part of the path '/mnt/HD/HD_a2/emby/cache/tvdb/405718/episodes-official.json'.
2025-02-12 03:15:06.533 Error Statistics: Could not find a part of the path '/mnt/HD/HD_a2/emby/cache/tvdb/401920/episodes-official.json'.
2025-02-12 03:15:06.534 Error Statistics: Could not find a part of the path '/mnt/HD/HD_a2/emby/cache/tvdb/410466/episodes-official.json'.
2025-02-12 03:15:06.534 Error Statistics: Could not find a part of the path '/mnt/HD/HD_a2/emby/cache/tvdb/388445/episodes-official.json'.

 

Edited February 12, 2025 by hackthis02

[DarkStar1977 119]

50 minutes ago, hackthis02 said:

So the first thing I see is you're missing the TVDB files. Confirm you have the TVDB plugin and then 'Refresh Metadata' on your TV library.
Without the TVDB files, the plugin has no idea how many episodes there are.
 

2025-02-12 03:15:06.458 Error Statistics: Could not find a part of the path '/mnt/HD/HD_a2/emby/cache/tvdb/426080/episodes-official.json'.
2025-02-12 03:15:06.459 Error Statistics: Could not find a part of the path '/mnt/HD/HD_a2/emby/cache/tvdb/412826/episodes-official.json'.
2025-02-12 03:15:06.459 Error Statistics: Could not find a part of the path '/mnt/HD/HD_a2/emby/cache/tvdb/438130/episodes-official.json'.
2025-02-12 03:15:06.484 Error Statistics: Could not find a part of the path '/mnt/HD/HD_a2/emby/cache/tvdb/406592/episodes-official.json'.
2025-02-12 03:15:06.484 Error Statistics: Could not find a part of the path '/mnt/HD/HD_a2/emby/cache/tvdb/276890/episodes-official.json'.
2025-02-12 03:15:06.506 Error Statistics: Could not find a part of the path '/mnt/HD/HD_a2/emby/cache/tvdb/405718/episodes-official.json'.
2025-02-12 03:15:06.533 Error Statistics: Could not find a part of the path '/mnt/HD/HD_a2/emby/cache/tvdb/401920/episodes-official.json'.
2025-02-12 03:15:06.534 Error Statistics: Could not find a part of the path '/mnt/HD/HD_a2/emby/cache/tvdb/410466/episodes-official.json'.
2025-02-12 03:15:06.534 Error Statistics: Could not find a part of the path '/mnt/HD/HD_a2/emby/cache/tvdb/388445/episodes-official.json'.

 

The TVDB Plugin is installed, it's the first source of metadata in all libraries and the metadata is present in those shows ... so I don't know why this specific json files appears as missing.

In fact I've deleted the metadata of this shows several times and forced to be re-downloaded from TVdb and the plugin stills presenting the issue.

BTW, those issues start to happen when emby was upgraded to 4.8 version removing missing episodes from the db.

Edited February 12, 2025 by DarkStar1977

[hackthis02 129]

5 minutes ago, DarkStar1977 said:

The TVDB Plugin is installed, it's the first source of metadata in all libraries and the metadata is present in those shows ... so I don't know why this specific json files appears as missing.

In fact I've deleted the metadata of this shows several times and forced to be re-downloaded from TVdb and the plugin stills presenting the issue.

BTW, those issues start to happen when emby was upgraded to 4.8 version removing missing episodes from the db.

Based on your screenshots it looks like your primary language is Spanish. Since mine is English, My first thought would be that might be the issue.
I don't know how your server is setup but if you could go into any of the subfolders in `/mnt/HD/HD_a2/emby/cache/tvdb` and give me the names of the json files in there, I can try and figure out what's going on.

[DarkStar1977 119]

2 minutes ago, hackthis02 said:

Based on your screenshots it looks like your primary language is Spanish. Since mine is English, My first thought would be that might be the issue.
I don't know how your server is setup but if you could go into any of the subfolders in `/mnt/HD/HD_a2/emby/cache/tvdb` and give me the names of the json files in there, I can try and figure out what's going on.

There are no files, are folders, and inside folders files:

Will try to check if I can do it with SSH, as this installation is on a WD NAS so I have no access to file system if it's not from FTP client or SSH Terminal connection.

BTW, there are several tvdb folders:

which ones you need ?

[DarkStar1977 119]

@hackthis02those are all folders in tvdb

[hackthis02 129]

3 minutes ago, DarkStar1977 said:

which ones you need ?

it looks inside the 'tvdb` folder. You could try deleting that folder then refreshing metadata.
 

 

2 minutes ago, DarkStar1977 said:

@hackthis02those are all folders in tvdb

Look inside one of those numbered folders and there should be one or more .json files.

[Happy2Play 9780]

Yes by default the server deletes these cache files every 30 days so there will be times when they don't exist, and this plugin will throw the errors shown.  So old shows that do not get updates will eventually have issues unless you manually touch or refresh metadata.

[DarkStar1977 119]

14 minutes ago, Happy2Play said:

Yes by default the server deletes these cache files every 30 days so there will be times when they don't exist, and this plugin will throw the errors shown.  So old shows that do not get updates will eventually have issues unless you manually touch or refresh metadata.

I have more than 1.9K shows, if this is true, then most of them that have not been updated in years will not appear on the plugin, and the plugin sees those shows:

[hackthis02 129]

The plugin aggregates data from different parts of the server. Some parts come from Emby while other parts come from TVDB.
I don't know where which part comes from off the top of my head. But I do know without the TVDB json files the User-based Show Progress page doesn't display correctly.

[Happy2Play 9780]

6 minutes ago, DarkStar1977 said:

I have more than 1.9K shows, if this is true, then most of them that have not been updated in years will not appear on the plugin, and the plugin sees those shows:

Plugin will see all shows/episode but if Emby has deleted the cached metadata there is no way for it to do calculations.  So you may have to touch your Shows for the server to get metadata cache again.

This plugin changed a long time ago to use Emby data instead up Emby and this plugin hitting TVDB for the exact same information.  Only draw back is Emby cache clean up of everything over 30 days olds as it does eventually get it again when needed by the server.

[DarWun 349]

5 hours ago, Happy2Play said:

Plugin will see all shows/episode but if Emby has deleted the cached metadata there is no way for it to do calculations.  So you may have to touch your Shows for the server to get metadata cache again.

This plugin changed a long time ago to use Emby data instead up Emby and this plugin hitting TVDB for the exact same information.  Only draw back is Emby cache clean up of everything over 30 days olds as it does eventually get it again when needed by the server.

That doesn't seem to be an issue for me though. I've had errors about missing parts in my log ever since the statistics plugin was changed to aggregate data from the cached metadata.

The above is a snapshot of the log with a statistics plugin scan I did a few minutes ago. The first error in the list is for The Adventures of Briscoe County, Jr. (TVDB ID72364). I have not touched that series in at least a couple of years. I just checked and there is most definitely not a file for the series in the /var/cache/tvdb folder.

But the data is still being populated correctly by the Statistics plugin.

I always assumed there was a failsafe where the plugin would access the metadata on TVDB if the cached files were missing. So I didn't report it.

 

Edited February 12, 2025 by DarWun

[hackthis02 129]

The way the code works, if you have more episodes collected then TVDB is reporting then it assumes the collected number is correct. In shows that have ended, this won't be an issue, since you would have the maximum number of episodes either way. 

example:
collection = 18
TVDB = 20
shows 18/20 (90%)

collection = 18
TVDB = 0 (no json file)
shows 18/18 (100%)

With that said, I'm still not sure how @DarkStar1977is getting 0/0 (0%) listings, unless he has created empty series folders that Emby has picked up.

[crossfire13148 29]

Created a new emby test server
Installed the "test" version of this plug in. (3.3.2.3)
installed 1 library with 1 show in it (Lib 1 containing House of the Dragon (2022)
in said library, i have 1 special, season 1 has 10 episodes, season 2 has 4 episodes. i removed S02E5-E8 as seen in the following:

Then ran the scan:
The results seem to be the same as before any updates as seen in the following:

According to this, i have 15 of the 18 and 1 special - meaning i am only missing 3 instead of the 4 episodes.

Running this again without the specials yields:

Showing that it is NOT subtracting the specials from the "total episode listing" before doing calculations to report "collected"(ie missing) episodes.

Now i will add another library and run scans to see how it is reacting to multiple libraries

[crossfire13148 29]

So now i have the following:
Lib 1 containing House of the Dragon (2022) 1 special, season 1 has 10 episodes, season 2 has 4 episodes. (S02E05-E08 Missing)
Lib 2 containing Angel Beats! (2010) containing Season 1 with 13 episodes (Complete)

Which does in fact look like it is reading multiple libraries on my test server

something the present one installed on my main server does not seem to do as i have a tv series folder and a tv series incomplete folder
 

Edited February 13, 2025 by crossfire13148
typo on (S02E05-E08 Missing) mistyped S01

[crossfire13148 29]

Third test:
So now i have the following:
Lib 1 containing House of the Dragon (2022) 1 special, season 1 has 10 episodes, season 2 has 4 episodes. (S01E05-E08 Missing)
Lib 2 containing Angel Beats! (2010) containing Season 1 with 13 episodes (Complete)
Lib 3 containing House of the Dragon (2022) Season 2 containing the missing S02E05-E08
First as all 3 libraries as stand alone:

Results as expected like previous

now making emby library 1 use both paths (lib 1 and lib 3):

WITHOUT the default option checked:

same result as above

now WITH the default option checked:

This did in fact Scan and keep Libraries separate when they were supposed to
and DID put them together when they were supposed to:
The ONLY issue i am still seeing on these tests were the Specials being counted as normal episodes for the Collected Display
 

 

 

[crossfire13148 29]

Test Four:
i deleted the "angle beats" video files from the folder in lib 2, leaving Lib 2 and the folder.
and did recreate the 0/0


 

[crossfire13148 29]

i do not know if this is affected by left over data in emby`s metadata or cache folder
but i know it has no bearing on NFO files being in the series folder or not there. Just having the folder there was enough for it to show 0/0 in my testing

[Junglejim 385]

17 hours ago, Happy2Play said:

Plugin will see all shows/episode but if Emby has deleted the cached metadata there is no way for it to do calculations.  So you may have to touch your Shows for the server to get metadata cache again.

This plugin changed a long time ago to use Emby data instead up Emby and this plugin hitting TVDB for the exact same information.  Only draw back is Emby cache clean up of everything over 30 days olds as it does eventually get it again when needed by the server.

As @Happy2Playhas mentioned you will need manually refresh (Metadata Manager/Views/Missing Episodes) to keep this plugin running smoothly.

That's if your like me and haven't used it in over 2 mounths..  

[DarkStar1977 119]

45 minutes ago, Junglejim said:

As @Happy2Playhas mentioned you will need manually refresh (Metadata Manager/Views/Missing Episodes) to keep this plugin running smoothly.

That's if your like me and haven't used it in over 2 mounths..  

And that's why removing missing episodes from the DB is making this plugin unusable ...

[DarkStar1977 119]

An btw, @Happy2Playrefreshing library metadata just for this is insane, I've only refreshed 1 of my TVshows libraries (I have 7), and took 3 hours and as result of this, the server is since 10 am this morning "Detecting intros" again because of this.

When the emby team will asume that the best way to handle episodes is to have the ones you have, the ones that are missing and the upcoming in the Emby database as it was on version 4.7 ?

This plugin worked like a charm in that version .....

Edited February 13, 2025 by DarkStar1977

[crossfire13148 29]

1 hour ago, DarkStar1977 said:

When the emby team will asume that the best way to handle episodes is to have the ones you have, the ones that are missing and the upcoming in the Emby database as it was on version 4.7 ?

same reason they have removed many of the functional things we used. such as removing the choice of "showing specials IN seasons"
 

i paid for lifetime premium cause i was happy with how emby worked, and its features. just to have those features removed.

and it takes me almost 5 hours to scan for missing episodes because it scans every movie, user, etc with no choices on what to scan and what not to.

and the question of "showing the list of what media there is" ie codec etc... the response of it would take too long to process...
i find that funny, cause the "reports" plug in does that perfect. just wish it did episodes separate.

as far as the cache issue. i have not noticed anything with that. does that mean shows that have "fallen off" of the cache, wont show me missing episodes anymore?
if that is the case, then this plug in is 100 percent useless to me now.
which is also misleading, as i thought by everything i have seen, that it is using theTVDB,

what is the point of having the TVDB plugin, and this plug in having code to use TVDB, if it doesnt use it?
 

[hackthis02 129]

@crossfire13148 Thank you for doing the extensive testing on the test version of the plugin. It seems I've fixed the issue of libraries across multiple folders issue. 

12 hours ago, crossfire13148 said:

Test Four:
i deleted the "angle beats" video files from the folder in lib 2, leaving Lib 2 and the folder.
and did recreate the 0/0


 

So I think the 0/0, which I don't see in your screenshot, would be the culmination of a empty folder and lack of a TVDB json file.

As for the specials getting added to the regular episode count. That is a tricky thing, as I have yet to see a variable that identifies an episode as a special.
The closest I've found is if season number is zero. But I'm still testing to see if that is a reliable indicator.

As for the json files management. How often they clear them out or update them is on on the Emby team and outside the scope of this plug-in.
I will tell you the reason we switched to using Emby's TVDB plugin over and internal one is because TVDB changed there API structure and started requiring people to pay per instance.
And if I recall it was a lot of money to ask each user to pay just for this single plug-in.

If you would like them to update how they handle it, you should reach out to @Lukeor @ebr. I don't know if he manages it, but I'm sure he could point you in the right direction.

As for now, I'm going to spend this weekend looking for a better way to handle specials. I may reach out to some of you to test dev builds.

Edited February 13, 2025 by hackthis02