Jump to content

Audio Metadata - Database Duplications & Uncorrectable Errors


Heckler

Recommended Posts

Heckler

Noticed an issue with either the metadata for FLAC files or the way emby is displaying.

 

 

Went to play something this afternoon whilst painting and noticed that the track listings were a little off... Tried to refresh the data with no luck, still displayed them in a weird manner... Then went through all of the albums for that artist which are in FLAC and all are displayed oddly as you can see in the images. Either additional numbering, or track title/number listed twice.

 

So I went back and checked a few albums that were MP3 and all of those are displayed correctly.

 

It only appears to affect FLAC files.

 

audioflaclisting01.JPG

audioflaclisting02.JPG

Link to comment
Share on other sites

How does the metada shows in the metadata editor of emby? also try to check the metadata with another software like mp3tag. Chances are that your metadata is really missing something like adding track number info in metadata and the title of the song. A good part of mi music collection is in FLAC and is showing correctly.

Edited by Mibok
Link to comment
Share on other sites

Happy2Play

Sorry all my flac files appear the same as my mp3 files.  As mentioned above I would verify embedded metadata.

Link to comment
Share on other sites

Heckler

I already use mp3tag for all audio, and can be a little OCD about ensuring that tags are correct... I rechecked and none of the tags show the data doubled. as in pic 2... But some do show the song number in the title... So one down... the other still unexplained.

Link to comment
Share on other sites

Happy2Play

Well that is where you will have to break it down.  What does it look like in metadata manager?  What happens if you refresh that album?  What server version is this?

Link to comment
Share on other sites

Heckler

Started noticing other odd things in audio metadata after digging a little deeper.

 

Bands like The Wonder Stuff are listed as that, and also the Wonder Stuff despite the fact that ALL metadata is identical and correct, and where the track titles are duplicated in the pic above... on these it's the band name that is duplicated... yet not a single track/album has them labelled as 'the Wonder Stuff'

 

I even went through after double checking mp3tag and deleted all the artist nfo and album nfo files, did a full rescan and then a manual refresh, after I had make certain every single track was labelled correctly before doing a full refresh on the artist... only to have the exact same data show up... either it never erased the old info, or it's repeating the same issue.

 

So I delved further and found dozens of artists so far with the same issue... mp3tag is showing the artist & album artists as correctly listed/spelled and emby is adding the artist twice with a slightly different spelling (one of which doesn't exist anywhere in the artists name, either folder, album title or any track)

 

A few more pics to highlight what I mean.

audiolisting01.JPG

audiolisting02.JPG

audiolisting03.JPG

audiolisting04.JPG

 

Given the 20,000+ music files I have or the 800+ artists.... It's a task that I am not looking forward to resolving... and there's no way in hell I'm doing another clear out and full refresh of all data for the 3rd time in 6 months due to issues created by 'updates'

Edited by Heckler
Link to comment
Share on other sites

PenkethBoy

There a few threads on the dup bands - i had a few - i did two things

 

i edited the band name in the library view - did a library refresh and the double did not come back and has not done so in several rebuilds - library deletes - because

 

the second thing i did was use Muzic Brainz Picard app to compare the metadata i had (i.e. Mp3Tag) and found that some of the data i had was fine but on MBz the odd track would has Ace Of Base and not Ace of Base

 

This combination has solved it for me - it may explain why your data looks fine but the duplicates appear.

 

Took me an afternoon to sort and have been error free since - you have double the album artists and 15 times the songs i have so it will take longer but only a one time job :)

Link to comment
Share on other sites

Heckler

There a few threads on the dup bands - i had a few - i did two things

 

i edited the band name in the library view - did a library refresh and the double did not come back and has not done so in several rebuilds - library deletes - because

 

the second thing i did was use Muzic Brainz Picard app to compare the metadata i had (i.e. Mp3Tag) and found that some of the data i had was fine but on MBz the odd track would has Ace Of Base and not Ace of Base

 

This combination has solved it for me - it may explain why your data looks fine but the duplicates appear.

 

Took me an afternoon to sort and have been error free since - you have double the album artists and 15 times the songs i have so it will take longer but only a one time job :)

 

I'll look into the musicbrainz app when I've got some free time next week... It's not every artist that is duplicated.. but on a quick visual scan through my album artists list in emby browser UI, I spotted a couple of dozen... many of which are duplicated because some data lists tracks/albums with 'The' at the beginning and some don't.

 

If the data that's being grabbed is at fault, then obviously emby isn't to blame... so thanks for the insight

 

 

 

However.... some data CANNOT be changed... even after correcting the metadata for the Band 'The Cult'  where the mp3tag details are correctly, but emby shows every single track as 'The Cult;the Cult' for both Artist & Album Artist... When I manually correct each track and save... when I go back to it, the metadata has been reverted to it's incorrect state once more.

 

 

This is the INDENTICAL bug that I've reported regarding a TV show Red Dwarf, where it's not grabbing the correct data, and is missing actors and so forth from the metadata regadless of what you do... ID's for the data are correct, and manually adjusting the metadata info to correct it... simply isn't being recorded in the .nfo file properly.  You can adjust it, and immediately check the nfo and it shows the change... but even though the nfo file isn't locked/read only or restricted in any way... the next time it is accessed by emby it seems to overwrite it it to the old one.

 

Unless you visually inspect EVERY SINGLE piece of data, there is simply no way to tell how widespread this issue is.

Link to comment
Share on other sites

PenkethBoy

interesting i found last night after that AC/DC for all tracks was listed as ac/dc;I - checked the tags etc no odd extra data - its in the nfo - deleted it and emby notices the change - refresh and its back - so went to check MBz/audiodb - nothing obvious from their pages - i am going to do some more checking - but i think possibly the metadata downloaded has a problem

  • Like 1
Link to comment
Share on other sites

Heckler

The album artist and artist tags are only listed once in the album nfo file.... and if you look at the actual nfo in notepad it clearly shows that it's listed correctly as in the pic below (5th &6th lines from the bottom)

 

So where it's getting this extra data from and where it's storing it... is a mystery because it's NOT in the nfo, and the nfo files aren't read/write protected, so there's no reason for the correct data not to be displayed in emby unless there's a bug somewhere.

audiolisting05.JPG

audiolisting06.JPG

Link to comment
Share on other sites

Heckler

So further investigating has turned up the following bands that are throwing up the same problems... nfo files don't show anything wrong, nor do the mp3tags... yet they are duplicated in the same manner as before... album artist/artist fields populated twice with a variation of the same name/spelling

 

I'm listing them as emby lists them, only one of the names is correct (the first one in a line) and mp3tags and nfo files show only one band name... yet emby shows multiples that cannot be removed by manually editing the metadata as it just magically reappears from out of thin air (it's being stored somewhere else and overwriting any changes is my guess)

 

AC/DC;ac/dc

Blink-182;Blink 182;blink-182;blink 182

Bowling For Soup;Bowling for Soup

The Cult;the Cult

The Cure;Cure

Chvrches;CHVRCHES

The Human League;Human League

The Jesus & Mary Chain;Jesus & Mary Chain

The La's;La's

Men at Work;Men At Work

Nine Inch Nails;NINE INCH NAILS;NIN

Noah & the Whale;Noah & The Whale

Puddle of Mudd;Puddle Of Mudd

R.E.M.;R.E.M

The Smashing Pumpkins;Smashing Pumpkins

Tears for Fears;Tears For Fears

Voice of the Beehive;Voice Of The Beehive

The Wonder Stuff;the Wonder Stuff

 

 

 

That's just a quick glance over album artists to see which ones are duplicated... Not delved any further than checking mp3tags and nfo files for any mismatch... and none are found that could explain the naming errors that cannot be removed.

Link to comment
Share on other sites

Hi,

 

You may have already done this but .......Another piece of the puzzle to double check is also the naming of the artist on the file system. I have found that generaly if all of the tags are correct you can get wierd dups if the file naming ( in this case the artist directory) is not a exact match.

 

580b95caba272_songs.png

 

 

580b95dd9fb26_direct.png

 

 

-vicpa

  • Like 1
Link to comment
Share on other sites

PenkethBoy

The "extra" info is in the database

 

For me and the AC/DC;I problem 

 

post-134563-0-60280600-1477156084_thumb.png

 

Column Names

ItemId (binary Blob), Type, Value, Clean Value

 

This is from the ItemValues Table in the library DB 

 

looking at the typedbaseitems table i can see that the "data" column has the error 

 

post-134563-0-31561300-1477163481_thumb.png

 

and finally linking with the providerids table

 

we have

post-134563-0-05620100-1477163651_thumb.png

 

either emby is importing it wrong or its coming wrong from MBz as the data field appears to store the orginal data - or its been processed???

 

if you want to look at your data - i used DB Browser for SQL Lite (very basic and buggy) but for a quick hack it ok - close emby server before you try and open the db - if you not sure use a copy of the db

 

Have fun

 

however having checked the artists that i have from your listi see no issues like yours sorry

 

This might be due to when it was imported vs the error

 

  • Like 1
Link to comment
Share on other sites

Happy2Play

When was the last time you Refreshed your music library?  When ever errors are fixed like this a used usually has to manually refresh the library to correct any database issues.  It is resolve for all new installations.

  • Like 1
Link to comment
Share on other sites

Heckler

I'm not gonna go digging in the database, chances are I'd mess something up as it's not something I'm familiar with.

 

@Happy2Play  when you say a refresh on the music.. do you mean a manual refresh of the individual artist, or a full media scan?  Because I've done both and for the Alanis Morrissette issue... that seems to have been resolved and it looks like the issue is nothing to do with Flac files at all... But rather data stored within the emby database that's incorrect and seemingly unchangeable.

 

With anything by The Cult... no matter how many times you adjust the metadata to the correct name and save... it always reverts to it's previous state regardless of a manual refresh or a media rescan... The same issue I suspect that is happening with the TV show Red Dwarf.

 

I'm going to try and manually adjust a single track and then save it... then immediately do a manual refresh on it... and see what happens.

 

There needs to be a way to wipe this data from within the database to allow a full and clean refresh of all data.... because with just the artists listed in my previous posts... that could amount to over a thousand tracks.

 

It is entirely possible that I have added music to my mediaserver, and it's been initially added via a scheduled scan, BEFORE I've been able to get in and adjust the mp3tags to their correct settings... and that's where some incorrect data has come from... But the fact you cannot seemingly undo it seems very counter productive when you have a problem.

Link to comment
Share on other sites

Happy2Play

There needs to be a way to wipe this data from within the database to allow a full and clean refresh of all data

Well removing the media from the library performing a library scan, then re-adding the media and performing a library scan should do that.  You will probably want to remove any metadata in the folder before reading.

Edited by Happy2Play
  • Like 1
Link to comment
Share on other sites

Heckler

Well removing the media from the library performing a library scan, then re-adding the media and performing a library scan should do that.  You will probably want to remove any metadata in the folder before reading.

 

I'll give it a shot

Link to comment
Share on other sites

Heckler

Hi,

 

You may have already done this but .......Another piece of the puzzle to double check is also the naming of the artist on the file system. I have found that generaly if all of the tags are correct you can get wierd dups if the file naming ( in this case the artist directory) is not a exact match.

 

580b95caba272_songs.png

 

 

580b95dd9fb26_direct.png

 

 

-vicpa

 

I can understand this, as it's something that's caused issues with movies in the past... It does however flag up a problem that cannot be resolved if it's the cause... Artists that have non standard characters in their names that windows refuses to allow in folder/file names.

 

An example would be the group B*Witched.... I have to have the folder titled B Witched because it won't allow * in the folder name.

 

Another would be R.E.M.  because windows won't allow that final  '.' in the folder name... but is fine with the others.

Edited by Heckler
Link to comment
Share on other sites

Heckler

@Happy2Play  could you change the thread subject to something like   Audio Metadata - Database Duplications & Uncorrectable Errors

 

 

Now that I have more info about the issue... the current subject is misleading.

Link to comment
Share on other sites

PenkethBoy

I can understand this, as it's something that's caused issues with movies in the past... It does however flag up a problem that cannot be resolved if it's the cause... Artists that have non standard characters in their names that windows refuses to allow in folder/file names.

 

An example would be the group B*Witched.... I have to have the folder titled B Witched because it won't allow * in the folder name.

 

Another would be R.E.M.  because windows won't allow that final  '.' in the folder name... but is fine with the others.

yes remeber something recently on the a : in a title name - IIRC it was to do with a movie

Link to comment
Share on other sites

Heckler

Well, I tried removing the affected files from the music directory and stored them elsewhere... removed all traces of metadata files, rescanned, cleaned the database... restarted the server, cleaned the database again, rescanned again.... Checked all artists were no longer present.

 

Added one artist back, a single album... Allowed it to rescan and make completely new metadata files.

 

Did exactly the same thing again... Duplicated the artist and album artist name exactly as it did before... and once again it cannot be altered.

 

 

The only thing I can think of to do next is delete the entire DB library... and allow it to rebuild... but as it's close to 1GB in size... I imagine that's gonna take the better part of a day to repopulate.

Link to comment
Share on other sites

Happy2Play

Just added some AC/DC and I see the ",|" being added.  I can't get the duplicate Artist issue.  @@Luke is the "/" being seen as a multi-artist separator in this situation?
 
  
580bf69795553_artist.jpg

 

@@Heckler most of the issues with duplicate Artist issues, have been resolved by deleting and allowing the library.db to rebuild have resolved the issues, but every ones issue can be different.

  • Like 1
Link to comment
Share on other sites

Heckler

Just added some AC/DC and I see the ",|" being added.  I can't get the duplicate Artist issue.  @@Luke is the "/" being seen as a multi-artist separator in this situation?

 

  

580bf69795553_artist.jpg

 

@@Heckler most of the issues with duplicate Artist issues, have been resolved by deleting and allowing the library.db to rebuild have resolved the issues, but every ones issue can be different.

 

Just about to delete the library.db and allow it to rebuild overnight... I've had enough of it today and I'm going to bed.   :)

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...