metadata cache purge feature / force update metadata

[Riezel 15]

i've been got some issue atm (recently, i think after upgrade to 4.8.1.0, previously is fine in 4.7 i think, kinda forget)

like this for example

this happens with auto organize grabbed the wrong ones
for example the episode 6 was detected as multiple files, EP6 - EP8, hence it got ,TBA,TBA
episode 7 was way worse, it got ~ E179 or something (filename from filebot before imported to emby is something like "xxxxx 07 [179XXX].mkv") 

now, i've tried refresh meta data (already renamed the files to correct ones and delete the old nfo), but the nfo generated is still same, while check in tvdb online it was normal without issue
this happens in lot of series i have, sure you can do manage each one by one case, but if this happens to hundred or thousand files, it's become a hassle

i've tried server restart, check internal emby var folder (even tried to rename metadata folder, to attempt purge them) but still not working, hence my conclusion must be, this is from the database level, and its somehow got "cached", is it possible to fetch them online or a way to purge the metadata cached in database or even add feature like "force refresh metadata" ?

emby version 4.8.1.0

 

[Luke 37068]

Hi, what metadata fetchers are enabled on the library, and in what order?

[Riezel 15]

5 minutes ago, Luke said:

Hi, what metadata fetchers are enabled on the library, and in what order?

 

Library is in mixed content

Edited February 20 by Riezel

[Luke 37068]

do the episodes have a tvdb id in the metadata editor? If so, are they correct?

[Riezel 15]

2 minutes ago, Luke said:

do the episodes have a tvdb id in the metadata editor? If so, are they correct?

yep, all good

https://thetvdb.com/series/tsukimichi-moonlit-fantasy/episodes/10180829
^ when pressing that button

metadata in tvdb is fine, but on my emby it's kinda cached no matter what i do
i also attached the nfo generated by emby (i didnt edit this file directly, nor i did lock the metadata)

* also correction, it was until E3897 (due the hash code similar to emby file naming)

S02E07 - [SubsPlease] Tsuki ga Michibiku Isekai Douchuu S2 - 07 (1080p) [E03897F1].nfo

[Happy2Play 8282]

@Lukelooks like multi-episode but without end episode per other topics.

Have not tested but will guess the naming scheme. S02 - 07 being seen as Episodes 2 - 7?

[Luke 37068]

3 hours ago, Happy2Play said:

@Lukelooks like multi-episode but without end episode per other topics.

Have not tested but will guess the naming scheme. S02 - 07 being seen as Episodes 2 - 7?

I don't think so, no. The fact that the overview is blank tells me that maybe it's not getting any tvdb metadata at all.

@Riezelas a test, if you remove the anidb plugin, then restart the server and refresh metadata on the episode, does that resolve it?

[Riezel 15]

4 hours ago, Happy2Play said:

@Lukelooks like multi-episode but without end episode per other topics.

Have not tested but will guess the naming scheme. S02 - 07 being seen as Episodes 2 - 7?

no, i've been configuring it that way so it wont missmatch like that, also i've been using that kind of naming type since 2 years ago

 

14 minutes ago, Luke said:

I don't think so, no. The fact that the overview is blank tells me that maybe it's not getting any tvdb metadata at all.

@Riezelas a test, if you remove the anidb plugin, then restart the server and refresh metadata on the episode, does that resolve it?

i just woke up, sorry
and it seems the overview is filled now probably due some other plugins (although the title still has TBA things)

i also tested disabled anidb and try to refresh meta data + replace all including image, seems the title as well didnt changed
i attached the log as well, seems there is no hit to tvdb except the cover only

also, i just tried auto organize (manual input season and episode via auto organize), seems it has another TBA things in my new added files (you can ignore overview, since there is no overview atm even in the tvdb online, but has lot of TBA there)

 

embyserver (2).txt

[Riezel 15]

since my speculation is about "caching"
i turned on this in the library in case it's help (although it is from my understanding only cache the cover / picture, not the metadata)


also the affected TBA on title currently on my emby

[Happy2Play 8282]

But naming interpretation has changed and brackets can potentially be irrelevant.

But just guessing as "S02E07 - [SubsPlease] Tsuki ga Michibiku Isekai Douchuu S2 - 07 (1080p) [E03897F1]" is this episode 7 thru 3897?

But @Lukewould have to potentially run tests against it.

But as I suspected is happening multi-episode.

S02E06 - [SubsPlease] Tsuki ga Michibiku Isekai Douchuu S2 - 06 (1080p) [605EA3E8]

Edited February 21 by Happy2Play

[Riezel 15]

1 minute ago, Happy2Play said:

But naming interpretation has changed and brackets can potentially be irrelevant.

But just guessing as "S02E07 - [SubsPlease] Tsuki ga Michibiku Isekai Douchuu S2 - 07 (1080p) [E03897F1]" is this episode 7 thru 3897?

But @Lukewould have to potentially run tests against it.

But as I suspected is happening.

yeah this is what happened in the first place (first try), hence the lot of TBA, but i already tried renaming it manually
AH i got it, just realized now

@Lukeit seems the meta data still read the file naming instead of the manual set from auto organize? (especially title)
for example, episode 6

S02E06 - xxxxxxxxxx S2 - 06 xxxx [605EA3E8]
hence 6 - 8. Title episode 6 metadata, title episode 7 metadata, title episode 8 metadata

i think this only happens recently tho, the old way seems fine, CMIIW 

[Luke 37068]

8 minutes ago, Happy2Play said:

But naming interpretation has changed and brackets can potentially be irrelevant.

But just guessing as "S02E07 - [SubsPlease] Tsuki ga Michibiku Isekai Douchuu S2 - 07 (1080p) [E03897F1]" is this episode 7 thru 3897?

But @Lukewould have to potentially run tests against it.

But as I suspected is happening multi-episode.

S02E06 - [SubsPlease] Tsuki ga Michibiku Isekai Douchuu S2 - 06 (1080p) [605EA3E8]

Ah yes you are correct. 3897 becomes the ending episode number.

[Riezel 15]

7 minutes ago, Luke said:

Ah yes you are correct. 3897 becomes the ending episode number.

can we limit the naming detection (as option maybe) ?
for example, it only limit to the first 2 or 3 words it's found (from left to right) instead the whole name (the red area only, for example) to avoid this things happens
S02E07 - [SubsPlease] Tsuki ga Michibiku Isekai Douchuu S2 - 07 (1080p) [E03897F1]

or just back to old naming check (as option, so everyone still can use the new one)

Edited February 21 by Riezel

[Revenent 9]

Just chiming in, having the same issue.  And to add on, it also causes library refreshes and scans to grind to a halt, as it tries to grab metadata for all the episodes, regardless of whether they exist or not.

In my case, it was trying to find metadata for 25,000+ episodes, basically stalling the scan for hours.

[Riezel 15]

8 minutes ago, Revenent said:

Just chiming in, having the same issue.  And to add on, it also causes library refreshes and scans to grind to a halt, as it tries to grab metadata for all the episodes, regardless of whether they exist or not.

In my case, it was trying to find metadata for 25,000+ episodes, basically stalling the scan for hours.

as for temporary solutions (still waiting for proper fix), i made a small script to rename all my files (well i only do the 2024 and ahead for now)
if your in linux OS, here is the script

for file in *; do [[ -f $file ]] || continue; delim=$(echo $file | grep -o "\[" | wc -l); if [[ $((delim)) < 3 ]]; then tcount=2; else tcount=2; fi; ext=$(echo $file | awk -F . '{print $NF}'); base=$(echo ${file%.*}); new_name=$(cut -d '[' -f 1-$tcount <<< "$base" | sed 's/[[:blank:]]*$//'); mv "$file" "$new_name"."$ext"; done

what it did, is removing the red section, so it only keep the blue only (asuming the problem is  inside " [ ] " (mostly checksum hash)

[Fanrelease] Title Show - 01 (720p) [AB2990CC][X265][BD][Multi-Sub].mkv 

use the script in root of your files folder (make sure to test it first before execute) (it also rename all .nfo and thumbnail files), just refresh the series afterward (scan and refresh metadata)
my output later will be like this, so no misleading metadata due some of checksum hash have "E0xxx" 

 

[Luke 37068]

hi, we are looking into this. Thanks guys.

[Luke 37068]

OK we have a fix and it will be on the beta channel this week. Thanks guys.

[Revenent 9]

7 hours ago, Luke said:

OK we have a fix and it will be on the beta channel this week. Thanks guys.

Thanks!  Will this be backported to a 4.8 release or will it only be available in the 4.9 release?

[Luke 37068]

1 minute ago, Revenent said:

Thanks!  Will this be backported to a 4.8 release or will it only be available in the 4.9 release?

Yes, but it's unlikely to make 4.8.2. I think it needs to sit in the beta channel for a bit to see if it produces any negative side effects.  So maybe some update after that.

[adminExitium 173]

Same issue as reported here? 

It will definitely help to go through the list in the shared link once and see if any current regexes used for matching need to be expanded upon or improved because Sonarr, being meant for media management, has gone through a lot of revisions and testing for such stuff.

[Revenent 9]

On 2/24/2024 at 10:56 PM, Riezel said:

as for temporary solutions (still waiting for proper fix), i made a small script to rename all my files (well i only do the 2024 and ahead for now)
if your in linux OS, here is the script

for file in *; do [[ -f $file ]] || continue; delim=$(echo $file | grep -o "\[" | wc -l); if [[ $((delim)) < 3 ]]; then tcount=2; else tcount=2; fi; ext=$(echo $file | awk -F . '{print $NF}'); base=$(echo ${file%.*}); new_name=$(cut -d '[' -f 1-$tcount <<< "$base" | sed 's/[[:blank:]]*$//'); mv "$file" "$new_name"."$ext"; done

what it did, is removing the red section, so it only keep the blue only (asuming the problem is  inside " [ ] " (mostly checksum hash)

[Fanrelease] Title Show - 01 (720p) [AB2990CC][X265][BD][Multi-Sub].mkv 

use the script in root of your files folder (make sure to test it first before execute) (it also rename all .nfo and thumbnail files), just refresh the series afterward (scan and refresh metadata)
my output later will be like this, so no misleading metadata due some of checksum hash have "E0xxx" 

 

Thanks!  I wrote something similar in PowerShell (on a Windows box) that replaces the E in the CRC with a Greek Epsilon.  It looks the same visually, but it doesn't trigger the multi-episode logic.  And then I'll just write a script that finds all occurrences of the Epsilon and rename it back to an E when the fix goes in.

param (
    [Parameter(Mandatory = $true)]
    [string] $RootPath,

    [Parameter(Mandatory = $false)]
    [switch] $WhatIf
)

$files = (Get-ChildItem -Path $RootPath -Recurse) | Where-Object { $_.Name -match ".*(?<crc>[A-Fa-f0-9]{8}).*" -and $Matches.crc -match ".*[Ee].*" }

foreach ($file in $files)
{
    $file.FullName -match ".*(?<crc>[A-Fa-f0-9]{8}).*" | Out-Null
    $crcSection = $Matches.crc
    $newCrcSection = $crcSection -ireplace "E", "Ε"
    $newName = $file.Name -replace $crcSection, $newCrcSection

    if ($WhatIf)
    {
        Write-Host $("[WHATIF] Renaming " + $file.FullName + " to " + $newName)
    }
    else
    {
        Write-Host $("Renaming " + $file.Name + " to " + $newName)
        Rename-Item -Path $file.FullName -NewName $newName
    }
}

 

[Happy2Play 8282]

Looks to be resolved in 4.9.0.8 performing my same test as above.

[Revenent 9]

Not fixed yet in the 4.8.3.0.  Guess we'll have to wait a bit longer.

[Riezel 15]

3 minutes ago, Revenent said:

Not fixed yet in the 4.8.3.0.  Guess we'll have to wait a bit longer.

it probably will be in 4.9

4.8.x until 4.9.0 probably scheduled for other bug / feature i think
so yeah, it will be a bit longer, probably month or more, so just use your temp measure for now

[Happy2Play 8282]

Doesn't appear so but was this supposed to be in 4.8.3.0?  @Luke