Emby still going rogue on Music Library imports

[wordlover 103]

I am so confused. If you can confirm the steps above are correct, I will do them again in the morning and send you new log.

[Luke 37095]

I actually think the best thing to do is test quickly on a portable install and see what the results are. 

[wordlover 103]

Isn't it going.to cause problems because it uses the same localhost port?

Edited February 6 by wordlover

[Luke 37095]

Well this should only take a few minutes right? so just shutdown the current server, run the portable one, and then you can delete it afterwards.

[wordlover 103]

@Luke I ran portable install and it correctly imports these files. Screenshots below. For AlbumArtist screen it has retrieved a MusicBrainz id, correctly shows name with diacritical, and has set its own E without diacritical mark as Sort title (I'm OK with that, if that's what Emby needs).

For individual Albums, it correctly shows name with diacritical, has also removed diacritical from first letter for Sort title, and correctly not retrieved any external IDs. So what does this tell us about how Emby misbehaves when importing new files that were part of the database in the past (or have file/folder creation dates before a previous import, or some other factor internal to Emby)? 

 

[wordlover 103]

@LukeNeed me to do any other debugging? Is this related to new "Library... Advanced..." setting that allows users to choose "Date added behavior for new content"?

Or is it related to where/how metadata is saved, i.e. could removal of these artist/album directories maybe have NOT removed all AlbumArtist traces in other metadata save locations?

Weirdly, in Emby's Metadata Manager, AlbumArtist is CORRECTLY listed as Éthiopiques (with diacritical) but not in standard album or artist view screens. Is any of this helpful for isolating the problem?

[Luke 37095]

Quote

So what does this tell us about how Emby misbehaves when importing new files that were part of the database in the past

I gave instructions on that earlier, right?

[wordlover 103]

I don't understand. I did what you asked:

On 2/5/2024 at 11:02 PM, Luke said:

I actually think the best thing to do is test quickly on a portable install and see what the results are. 

Now I'm asking what that indicates and if that helps you solve this problem. What more would you like me to do to help debug this?

[Luke 37095]

I would suggest removing all music and audiobook libraries, then running a scan of all libraries to flush out the database. Then try adding it back.

[wordlover 103]

10 minutes ago, Luke said:

I would suggest removing all music and audiobook libraries, then running a scan of all libraries to flush out the database. Then try adding it back.

I did that! As noted above, the steps followed:

1. Rotate Emby log
2. Full full scan (from Library page in Settings) - all files/albums are in Emby, ALBUMARTIST is displayed inconsistently with what we see in FFProbe and MP3Tag and other metatag apps.
3. Remove entire Éthiopiques directory and all subdirectories from Music Library directory
4. Full full scan (from Library page in Settings)
5. Confirm NONE of those files are in Emby - a search shows nothing that even begins with "ethio"
6. In MP3Tag, change ALBUM names to add diacritical to first character (e.g. "Éthiopiques 01: Golden Years of Modern Ethiopian Music 1969-1975")
7. Move full directory back into Music Library directory
8. Full full scan (from Library page in Settings)
9. Now all album names HAVE been changed! (screenshot below)
10. BUT... ALBUMARTIST still hasn't been changes

You are saying for some reason Emby still isn't flushing the database. Why is that? Why isn't Emby operating properly to do the same full, clean replacement of changed ALBUMARTIST that it is doing for changed ALBUMTITLE, when a user does the Library dance? What more can you do to fix this problem? Again, what seems like a major clue:

Weirdly, in Emby's Metadata Manager, AlbumArtist is CORRECTLY listed as Éthiopiques (with diacritical) but not in standard album or artist view screens. Is any of this helpful for isolating the problem?

Edited February 7 by wordlover

[wordlover 103]

I have now done the full Library Dance process again. 

Here's a sample .nfo - every album in the directory has the same correct <albumartist> (Éthiopiques) and correct <dateadded>.

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<album>
  <review />
  <outline />
  <lockdata>false</lockdata>
  <dateadded>2024-02-07 10:59:05</dateadded>
  <title>Éthiopiques 01: Golden Years of Modern Ethiopian Music 1969-1975</title>
  <year>1998</year>
  <sorttitle>Ethiopiques 01: Golden Years of Modern Ethiopian Music 1969-1975</sorttitle>
  <runtime>70</runtime>
  <genre>World</genre>
  <genre>African</genre>
  <artist>Mulugèn Mèllèssè</artist>
  <artist>Mahmoud Ahmed</artist>
  <artist>Mulatu Astatqé</artist>
  <artist>Sèyfu Yohannès</artist>
  <artist>Tèshomé Meteku</artist>
  <artist>Tèsfa-Maryam Kidané</artist>
  <artist>Gétatchèw Kassa</artist>
  <albumartist>Éthiopiques</albumartist>
</album>

After the full Library dance and rescans, you can see everything is imported:

And at least all albums are properly sorted as most recent Library additions, based on correct Date Added.

But Album Artist name is still incorrect, DESPITE being correct in the .nfo file. This is what's so mystifying. Emby is displaying data that conflicts with its own .nfo files.

 And I have attached latest log. What do you find?

embyserver.txt

 

 

[Happy2Play 8296]

It is still specific to your database containing this artist wrong spelling or media somewhere with the wrong spelling.  If it does not purge when removing all data by this artist then unfortunately you will have to manually remove if form that database.

Does the wrong spelling the artist appear in your /metadata/musicartist folder?

Edited February 7 by Happy2Play

[wordlover 103]

It is correct in the metadata folder and in all the .nfo files. That's what's so mystifying. Emby doesn't seem to be using the data it knows it has.

 

Edited February 7 by wordlover

[Happy2Play 8296]

It does but something happened to your database when stuff was imported and will most likely require your manual intervention.

Does changing the spelling in the UI correct your issue?

[wordlover 103]

Two things:

1. Emby devs should figure out and fix this problem - full, clean removal and re-add of everything by an artist (as confirmed in logs and screenshots above) shouldn't break Emby's database - there should be better internal trash collection.

2. Could you be more precise about what you mean "change spelling in the UI"? Of course I can go into metadata screen for albumartist and manually change it. But I don't want to do that until @Lukesays everything else has been investigated, so as not to muddy the debugging waters.

[Happy2Play 8296]

1 minute ago, wordlover said:

Two things:

1. Emby devs should figure out and fix this problem - full, clean removal and re-add of everything by an artist (as confirmed in logs and screenshots above) shouldn't break Emby's database - there should be better internal trash collection.

2. Could you be more precise about what you mean "change spelling in the UI"? Of course I can go into metadata screen for albumartist and manually change it. But I don't want to do that until @Lukesays everything else has been investigated, so as not to muddy the debugging waters.

I don't know the issue but appears isolated to your system and it is impossible to correct a defect on everyone's system when it is impossible to say when it happens but one could pm the library.db and have it looked at.

But to me a simple edit in UI should solve the issue.  As the would be the same as editing the DB.

Since Luke does not have the issue with the files you provided him it has to be in one of your existing files.  But only you can test. 

[wordlover 103]

Agreed - as I reported above, even *I* didn't have this issue when I spun up a portable server and imported these and a bunch of other Library directories. So clearly: in this situation where even the full Library Dance was done (which shouldn't even be necessary when changing metadata, but that's a separate issue), Emby apparently is importing and seeing the correct Album Artist info (as reflected in all the .nfo files and in the metadata directory) but for some reason isn't using/displaying it. That's a bug.

[Happy2Play 8296]

I don't know as I have corrected a few artists today by removing the content, doing a full scan not an individual library scan as is does not clear the database, and readded media and it corrected all issues previously had with artist duplication as the name is special characters "()" that Emby use to create 4 artists for.

[wordlover 103]

Dear friend, you're killing me. That's what I did. I appreciate you chiming in, but I have reported in excruciating detail above that this didn't work for me in this instance. Let's wait for Luke to take a look at the logs and the code to see if he can figure out the problem. 

[Luke 37095]

Can you tell us exactly what you did?

[Happy2Play 8296]

But if it were in the code then it would have been reproducible and you me and Luke have already shown it isn't on three different setups it works correctly.  So what is broke on the system it is wrong on?

[wordlover 103]

Luke, are you referring to me? I spelled it out in incredible detail above, and posted the log. What isn't clear?

[wordlover 103]

1 hour ago, wordlover said:

I have now done the full Library Dance process again. 

Here's a sample .nfo - every album in the directory has the same correct <albumartist> (Éthiopiques) and correct <dateadded>.

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<album>
  <review />
  <outline />
  <lockdata>false</lockdata>
  <dateadded>2024-02-07 10:59:05</dateadded>
  <title>Éthiopiques 01: Golden Years of Modern Ethiopian Music 1969-1975</title>
  <year>1998</year>
  <sorttitle>Ethiopiques 01: Golden Years of Modern Ethiopian Music 1969-1975</sorttitle>
  <runtime>70</runtime>
  <genre>World</genre>
  <genre>African</genre>
  <artist>Mulugèn Mèllèssè</artist>
  <artist>Mahmoud Ahmed</artist>
  <artist>Mulatu Astatqé</artist>
  <artist>Sèyfu Yohannès</artist>
  <artist>Tèshomé Meteku</artist>
  <artist>Tèsfa-Maryam Kidané</artist>
  <artist>Gétatchèw Kassa</artist>
  <albumartist>Éthiopiques</albumartist>
</album>

After the full Library dance and rescans, you can see everything is imported:

And at least all albums are properly sorted as most recent Library additions, based on correct Date Added.

But Album Artist name is still incorrect, DESPITE being correct in the .nfo file. This is what's so mystifying. Emby is displaying data that conflicts with its own .nfo files.

 And I have attached latest log. What do you find?

embyserver.txt 366.64 kB · 0 downloads

 

 

[wordlover 103]

Here's the post again:

https://emby.media/community/index.php?/topic/125670-emby-still-going-rogue-on-music-library-imports/&do=findComment&comment=1324050

[Luke 37095]

Can you describe each of the steps?