metadata-collection not working as expected on separated folders

[RokeWayne 1]

Hi,

following situation:
I'm trying to organize my "big anime about a rubber pirate" tv show/anime files. Since that tv show / anime is such big pile of episodes, I tried to organize my files in different folders separated by a finite episode numbers. I started with around 100 files per folder. i constructed this folder structure:

[upper directories] \tv show name (year){tvdb=XXXXX}
[upper directories] \tv show name (year){tvdb=XXXXX}\tv show name (year)- 001 - 100\[100 files]
[upper directories] \tv show name (year){tvdb=XXXXX}\tv show name (year)- 101 - 200\[100 files]
[...]
[upper directories] \tv show name (year){tvdb=XXXXX}\tv show name (year)- 1001 - 1100\[100 files]
[upper directories] \tv show name (year){tvdb=XXXXX}\tv show name (year)- specials\specials\[special episode or movie files]

shown here:

This file organization is correctly recognized by emby by sorting all files correctly into the tv show. Since I use the "absolute" numbering scheme, the files are usually numbered  0001 - 1100 and I chose the following name scheme:

tv show name (year) eXXXX_[episode name]- [resolution][group].[file-extension]

All files are correctly put into season 1 (because of absolute numbering) and specials which includes OVAs, Specials and Movies. Perfectly done until here.
 

But now comes what i found strange:

When i tried to add metadata to the tv show, only the first folder that contained the first 100 episodes was correctly enriched by metadata. Every other folder (except for the specials material) was ignored by the metadata collection and now only appears by file name.

What I did to help analyzing / reproduce:

Since I wanted to reproduce the problem, I moved some episodes from the second folder (episodes 101-105 in "101-200"-folder) into the first folder ("001-100"). After retriggering the meta data collection now episodes 101 to 105 are also enriched by the correct metadata, which means, that the folder separation works fine with recognition of the tv show relation, but not for meta data collection.

result:

Since the tv show relation clearly is found across the separate folders, I (maybe wrongly) expected the meta data collection to work just as fine.

Am I doing something wrong or did I find a special behaviour of the meta data collection algorithm?
 

Kind regards

Roke

[Abobader 3243]

Hello RokeWayne,

** This is an auto reply **

Please wait for someone from staff support or our members to reply to you.

It's recommended to provide more info, as it explain in this thread:

Thank you.

Emby Team

[RokeWayne 1]

Since I forgot to mention.

Emby runs on a Synology NAS in the most recent version 4.8.7.0.

[Luke 38803]

Hi, did you configure series in emby to use absolute file order?

[RokeWayne 1]

Hi, Luke.

Yeah, I did.
It was too much hassle in this special case to use airdate or dvd order.

Thanks for your help.

[Luke 38803]

Did that help resolve your issue? Or had you already done that?

[RokeWayne 1]

Hi, Luke.

I did that from the beginning, so it didn't help/ is not part of any solution/ expected behaviour.

My presumption still is, that sepatate folders are - as long as they have the correct identifier- recognized for a tv show, but the meta data algorithm is not able to continue, when a jump into a next folder would be necessary.

Kind regards

Roke

 

[Happy2Play 9140]

Will have to do some testing but will guess these folder organization will be an issue as absolute would be one folder with all episodes.

test files 001-200 + 807 in single folder.

[RokeWayne 1]

HI, together.

Don't get me wrong - in every other case, i would put all files into one folder and it would not cause any problem. But with this special tv show / anime, where the file count exceeeds 1100 files

[Happy2Play 9140]

But retesting this with structrure appears to work fine also.

Will test adding episodes 1000-1100.

[RokeWayne 1]

Edit from my previous post:

HI, together.

Don't get me wrong - in every other case, i would put all files into one folder and it would not cause any problem. But with this special tv show / anime, where the file count exceeeds 1100 files, its a bit difficult to navigate in a folder with >1100 files, so I chose this structure.

I think that my problem would still persist, even if i were to create just 2 folder with 600+ episodes each... I think, that the combination "absolute numbering" and "separate folders" is just not working how it could, since the tv show recognition already works good but in my case the metadata algorithm is not able to jump into the next folder.

But:
@Happy2Playi recognize already, that you're getting a different result than me. I just don't see why.... On which layer did you trigger the metadata collection (or maybe file scan)?

[Happy2Play 9140]

Adding 1000-1100 added without issue also.

 

So my next quest would be what is your metadata provider order?

 

2 minutes ago, RokeWayne said:

i recognize already, that you're getting a different result than me. I just don't see why.... On which layer did you trigger the metadata collection (or maybe file scan)?

I added the show folder Emby recognized Aired order, I changed to Absolute and Refreshed w/replace images to correct the issue with Aired and everything feel into place as I this test was all db only data.

[RokeWayne 1]

Hi, pls dont be mad at me. On my side it's 3.30 a.m.:

Thats my configuration for the collection:

 

@Happy2PlayI noticed, that I have one folder layer more than you, since above the separate folders with 100 files each resides the one folders that was meant to build the opening bracket:

[upper directories] \tv show name (year){tvdb=XXXXX}\tv show name (year)- 001 - 100\[100 files]

I will test later, if it works if I remove this folder from the equation...

Regardless I want to say thank you very much by supporting me in such an edge case!

But now I will go rest.

I'll update you later with any result I'll stumble upon!

Thx and kind regards!

Roke

 

 

 

Edited May 26, 2024 by RokeWayne

[Happy2Play 9140]

1 minute ago, RokeWayne said:

@Happy2PlayI noticed, that I have one folder layer more than you, since above the separate folders with 100 files each resides the one folders that was meant to build the opening bracket:

[upper directories] \tv show name (year){tvdb=XXXXX}\tv show name (year)- 001 - 100\[100 files]

No I just did not add provider id to folder naming.  So we have same structure with slightly different naming on Show folder

E:\Media\TV Shows - DB\One Piece (1999)\One Piece (1999)-001 - 100\[100 files] 

Everything looks correct per library settings so I am not entirely sure unless there is somet

[Happy2Play 9140]

Sorry I will have to retest as I just noticed your naming scheme is slightly different tv show name (year) eXXXX and mine is only three digit minus the e.

So it may be in the 4 digit code for the lower numbers.

[RokeWayne 1]

HI, @Happy2Play

you are right about having as many folder layers like me, sorry, i was not attentive enough.

But yeah. My episode naming sheme is being 4 digit wide because of this huge anime. It runs for more than 20 years already xD So it's a rather unique case. There aren't any other tv show with so many episodes!

Kind regards

Roke

[Happy2Play 9140]

30 minutes ago, RokeWayne said:

I think that my problem would still persist, even if i were to create just 2 folder with 600+ episodes each... I think, that the combination "absolute numbering" and "separate folders" is just not working how it could, since the tv show recognition already works good but in my case the metadata algorithm is not able to jump into the next folder.

From a view standpoint correct as they will be virtually grouped into one folder.  These will not be broken into multiple subfolders like organization as Absolutle is one folder/season showing all just like you see it on the metadata provider sites.  So yes this is awkward scrolling 1100+ episodes.

 

Also did not notices any difference changing to exxxx this naming.

Edited May 26, 2024 by Happy2Play
added test

[RokeWayne 1]

Short update, while I try to continously experiment with the folder structure and naming:
My theory with 600 files per folder was just validated as it is as fruitless as my previous setups. So with my actual configuration it's really the jump into the second folder that causes the metadata additon to halt for whichever reason. It's plain reproducable within my setup.

I'll try to experiment a bit more, until I can deduct the relevant configuration or mechanism.

 

[RokeWayne 1]

I'll edit this post with my experiments.

 

Experiment 1:
I decided to remove the additional upper level of folder, so now I have two folders.

The paths are now :

[Upper directories] \ Animes \ One Piece (1999)- First Part \ Some files
[Upper directories] \ Animes \ One Piece (1999)- Second Part \ Some files

instead of

[Upper directories] \ Animes \ One Piece (1999) \ One Piece (1999)- First Part \ Some files
[Upper directories] \ Animes \ One Piece (1999) \ One Piece (1999)- Second Part \ Some files

Note for consideration:

I also removed the folder structure with the specials, so in the next step I'll add that to gradually regrow to my old structure to find out when it breaks apart.

Result 1:

Metadata is added correctly over all folders.

 

Experiment 2:
I decided to remove the additional upper level of folder, so now I have two folders.

The paths are now :

[Upper directories] \ Animes \ One Piece (1999)- First Part \ Some files
[Upper directories] \ Animes \ One Piece (1999)- Second Part \ Some files
[Upper directories] \ Animes \ One Piece (1999)- specials \ Some files

instead of

[Upper directories] \ Animes \ One Piece (1999) \ One Piece (1999)- First Part \ Some files
[Upper directories] \ Animes \ One Piece (1999) \ One Piece (1999)- Second Part \ Some files
[Upper directories] \ Animes \ One Piece (1999) \ One Piece (1999)- specials \ specials \ Some files

Note for consideration:

In my old structure there was a folder layer more for specials (the distinct "specials" folder below the tv show folder, cause its usually needed on other tv shows and I thought it would be needed here , too. This assumption seems to be wrong, since in this experiment the specials-files were correctly recognized even when virtually on the same layer as the normal episodes.

Result 2:

Metadata is added correctly over all folders.

Edited May 26, 2024 by RokeWayne
Addition for more folder structures

[RokeWayne 1]

Experiment 3:
I added the additional below level of specials folder that i had before.

The paths are now :

[Upper directories] \ Animes \ One Piece (1999)- First Part \ Some files
[Upper directories] \ Animes \ One Piece (1999)- Second Part \ Some files
[Upper directories] \ Animes \ One Piece (1999)- specials \ specials \ Some files

instead of

[Upper directories] \ Animes \ One Piece (1999) \ One Piece (1999)- First Part \ Some files
[Upper directories] \ Animes \ One Piece (1999) \ One Piece (1999)- Second Part \ Some files
[Upper directories] \ Animes \ One Piece (1999) \ One Piece (1999)- specials \ specials \ Some files

Result 3:

 

Metadata is added correctly over all folders

 

Experiment 4:
So the last thing would be to readd the top level folder I had that contained my (now three) subfolders.

The paths are now :

[Upper directories] \ Animes \ One Piece (1999) \ One Piece (1999)- First Part \ Some files
[Upper directories] \ Animes \ One Piece (1999) \ One Piece (1999)- Second Part \ Some files
[Upper directories] \ Animes \ One Piece (1999) \ One Piece (1999)- specials \ specials \ Some files

Everything looks now as before, when the metadata was not added:

[Upper directories] \ Animes \ One Piece (1999) \ One Piece (1999)- First Part \ Some files
[Upper directories] \ Animes \ One Piece (1999) \ One Piece (1999)- Second Part \ Some files
[Upper directories] \ Animes \ One Piece (1999) \ One Piece (1999)- specials \ specials \ Some files

Note of considereation:
Since I was suspicious of the folder structures until now, i reduced the file count for the experiments. I'm really curious, what this test will show.

Result 4:

With reduced files the chosen folder structure works just fine. → Could it be the immense file count?

[RokeWayne 1]

Well, I think the file count could be the cuplrit ar at least involved in my problem:

With now restoring the old structure and putting almost 1150 files in it, the metadata collection broke already after episode 61...

 

[Happy2Play 9140]

Unless a timing issue as it does take a while to update 1100+ files I have yet to reproduce an issue here.  But have not tested most of the experiments above as they are incorrect as Emby must have a parent folder for every show/series.

So any of these should provide broken results as this is technically three separate shows/series.  Unless the parser by chance eliminates all trailing info and only uses One Piece (1999) but that is/should be hit and miss.  But will require the library merge folders option.

[Upper directories] \ Animes \ One Piece (1999)- First Part \ Some files
[Upper directories] \ Animes \ One Piece (1999)- Second Part \ Some files
[Upper directories] \ Animes \ One Piece (1999)- specials \ Some files

Edited May 26, 2024 by Happy2Play

[RokeWayne 1]

Well. That merge folders option you named is surely activated, and in my eyes it looked as if emby does merge the folders even when I remove the top folder. But maybe it does it "wrong"?

I could show you how it was displayed in the "directory" view to me before I made the recent changes. Do you want me to?

[Luke 38803]

19 minutes ago, RokeWayne said:

Well. That merge folders option you named is surely activated, and in my eyes it looked as if emby does merge the folders even when I remove the top folder. But maybe it does it "wrong"?

I could show you how it was displayed in the "directory" view to me before I made the recent changes. Do you want me to?

Hi, yes, more information would be great. Thanks !

[RokeWayne 1]

I'll revert my changes and try to let the metadata collection work over night. Tomorrow I'll post pictures and whatever else you wish on how it looks in my emby gui.

Thanks for your help!