Group Versions Not Showing All Versions

[dwyatt 2]

This is a very peculiar issue I have been unable to solve. I don't have many movies with multiple versions, but one that I do is the original Star Wars trilogy. See photo of the naming below. Files are named identical to the folder they are in plus extra at the end in [  ] to indicate the version. Emby picks up all 3 of these versions just fine as shown in the second photo. Going into each of these shows that they are in fact the 3 different versions. BUT! If I select all 3 of them and use the Group Versions option, I get a single item that replaces the 3 but within that item there are only 2 of the versions available. No matter what I do I can't get the third to show up. I can group any 2 versions together just fine (leaving a second entry for the third version) but as soon as I try to group 3 of them I only get 2. 

I've tried using the more official ' - ' naming scheme to indicate version information at the end but that doesn't make a difference. I have scoured the logs but can't see anything of interest in there. 

Emby Server 4.8.1.0 running in docker on Linux. Server log attached - in this log I did the following:

1. restarted server to get a clean log
2. navigated to my Star Wars collection
3. selected the 3 versions and used option to group them
4. went into that new grouped item and saw there are only 2 versions available
5. used the option to split versions apart
6. went back to Star Wars collection and we're back where we started (3 versions showing when not grouped together)

 

embyserver.txt

[jaycedk 382]

I think the dash in No-DNR is messing it up

Try this, remove the dash - and see what happens.

[dwyatt 2]

15 hours ago, jaycedk said:

I think the dash in No-DNR is messing it up

Try this, remove the dash - and see what happens.

Unfortunately that doesn't help (I did try that before but it didn't make a difference). With or without the dash it will let me group it with another movie just fine. It's only when I try to group all 3 that the one with dash (or without if I remove it) always disappears. The exact same thing happens to episode VI where I also have 3 versions. I don't think have 3 versions of any other movies in my library so I haven't noticed this before. 

[tedfroop21 38]

Stupid question but is there a limit for the length of the name and you are exceeding it?  That one name is 4 or 5 characters longer than the others....

Windows name spaces are huge but Emby may not be coded using Windows name spaces.....

Edited February 17 by tedfroop21

[dwyatt 2]

1 hour ago, tedfroop21 said:

Stupid question but is there a limit for the length of the name and you are exceeding it?  That one name is 4 or 5 characters longer than the others....

Windows name spaces are huge but Emby may not be coded using Windows name spaces.....

These files originally didn't have the 1080p in the name and had the same problem. And besides, it does show up and work in Emby just fine, and groups with a single other movie. It's just when I group 3 of them it disappears.

[Happy2Play 8290]

Not sure as testing this in 4.8 and 4.9 it groups correctly with OP naming scheme.

[Happy2Play 8290]

On 2/15/2024 at 7:49 PM, dwyatt said:

Emby Server 4.8.1.0 running in docker on Linux. Server log attached - in this log I did the following:

1. restarted server to get a clean log
2. navigated to my Star Wars collection
3. selected the 3 versions and used option to group them
4. went into that new grouped item and saw there are only 2 versions available
5. used the option to split versions apart
6. went back to Star Wars collection and we're back where we started (3 versions showing when not grouped together)

Only guess would be something broke in your database if they are not grouping automatically and manually grouping them  is not working also.

Have you removed the media from the library, did a full server scan, and readded them media?

Have you tested without the AutoCollections plugin?

 

[dwyatt 2]

On 2/16/2024 at 9:06 PM, Happy2Play said:

Only guess would be something broke in your database if they are not grouping automatically and manually grouping them  is not working also.

Have you removed the media from the library, did a full server scan, and readded them media?

Have you tested without the AutoCollections plugin?

 

I've done some more testing. This doesn't just affect Star Wars. ANY movie I try to group more than 2 items it doesn't show the 3rd. 

I had AutoCollections installed before, I've since removed it and tested again and it makes no difference. Oddly things are still being grouped together automatically though. Even after a server restart, and there's no mention of AutoCollections in the log anymore, but still things get grouped. 

[pwhodges 1532]

Do you use .nfo files?

Paul

[Luke 37085]

@dwyattplease start by removing the auto collections plugin and then restarting the server. As we don't know what that's doing, it's going to be difficult to troubleshoot as long as it is involved. 

Then please remove your A New Hope folder altogether and run a scan of all libraries to clean out the database. Then add it back.

At that point, how do things compare?

[dwyatt 2]

Sorry for the long delay responding - got sidetracked to more important things and forgot to keep working on this.

On 2/23/2024 at 12:41 AM, pwhodges said:

Do you use .nfo files?

Paul

Yes I do. nfo and all images are stored with the media files. 

On 2/23/2024 at 12:53 AM, Luke said:

@dwyattplease start by removing the auto collections plugin and then restarting the server. As we don't know what that's doing, it's going to be difficult to troubleshoot as long as it is involved. 

Then please remove your A New Hope folder altogether and run a scan of all libraries to clean out the database. Then add it back.

At that point, how do things compare?

I guess the first thing to solve is why movies are still auto-grouping after I've removed the AutoCollections plugin. I've removed the plugin entirely and Emby Plugins page doesn't show it (nor does the log show it loading) but my movies are still being grouped automatically. See below for sequence of events that correspond with the attached log file (I have debug turned on). 12 Angry Men is a movie I have never had in my collection so there shouldn't be any old information in the database to mess anything up. 

1. Restarted server
2. Created 12 Angry Men movie folder with single version inside and rescanned library -> this resulted in the movie correctly showing up in Emby. No issues yet.
3. Added second version of movie to folder and rescanned library -> this results in second version being found and automatically grouped with the first. 
4. Added third version to movie to folder and rescanned library -> this results in third version being found and automatically grouped with the first. The versions dropdown now shows [v1] and [v3]. [v2] is hidden. 
5. Click the Split Versions Apart button -> now I have all 3 movies visible as their own movies. 
6. Select the 3 movies and group them -> now we're back to a single entry showing only [v1] and [v3]. [v2] is missing. 

 

 

embyserver.txt

[Happy2Play 8290]

Unless dev sees something in the log I still cannot reproduce per the steps above unless this is a platform prevalent issue.

Edited March 8 by Happy2Play

[dwyatt 2]

11 minutes ago, Happy2Play said:

Unless dev sees something in the log I still cannot reproduce per the steps above unless this is a platform prevalent issue.

Fascinating that it works for you. For context, I'm running Emby in a docker container on an Ubuntu host. I've been doing it that way for a few years, prior to that it was in docker on a Synology NAS, and prior to that it was just running on a Windows server. This database and config is over 10 years old at this point. I really don't want to have to set it up again from scratch if I can avoid it. I suppose maybe I should spin up a second instance for testing to see if it does the same thing ... 

Does yours auto-group as you add each version? or you have to do that manually?

Edited March 8 by dwyatt

[crusher11 851]

If they're all in the same folder and are named appropriately, Emby will autogroup them without the plugin.

[Happy2Play 8290]

3 minutes ago, dwyatt said:

Does yours auto-group as you add each version? or you have to do that manually?

Since this is a Emby Core structure/naming scheme the server automatically groups them.

 

Only test I can think of would a New Library with a New path to see if it an existing database issue with current library set.

[dwyatt 2]

2 hours ago, Happy2Play said:

Since this is a Emby Core structure/naming scheme the server automatically groups them.

 

Only test I can think of would a New Library with a New path to see if it an existing database issue with current library set.

This does seem to be something about my particular Emby database/config. I just created a new root folder for a new library with just the one movie folder in it (containing the 3 versions from before). Creating a new library in Emby and pointing to that folder results in same thing: only 2 versions shown. BUT! I also created a new Emby instance, pointed it to that exact same folder and it shows me all 3 versions! 

Library settings between the new instance and the new library on existing instance are identical. So it must be something outside of Library settings. I'm off to compare all the other settings and see if I can find something.

 

[Happy2Play 8290]

Might have to look a the libraries for the api as not all settings are in the UI.  But really can't think of anything that would cause this.  As from a versioning standpoint I believe Emby is hardcoded to a limit of 8 versions.

[dwyatt 2]

15 minutes ago, Happy2Play said:

Might have to look a the libraries for the api as not all settings are in the UI.  But really can't think of anything that would cause this.  As from a versioning standpoint I believe Emby is hardcoded to a limit of 8 versions.

I can't see any obvious differences between the Library config on the new instance vs the original one. Not just the Library config either, the entire server config is basically identical. I don't know where else to look for differences - is the library config itself stored in the database? Is there a way for me to inspect that manually to compare old and new? Just wondering if my config is so old it has some legacy setting that hasn't been updated to whatever the new default is. 

Alternatively, is there an easy way to migrate users (playstate and login credentials) to a new server instance? I can recreate the libraries easily enough since my images/metadata are all stored with files. But I need to retain the user data. 

[Happy2Play 8290]

26 minutes ago, dwyatt said:

Alternatively, is there an easy way to migrate users (playstate and login credentials) to a new server instance? I can recreate the libraries easily enough since my images/metadata are all stored with files. But I need to retain the user data.

Since this is a potential db issue it is a little difficult but yes.  Make a full backup on old system via the plugin.

You need to copy the users.db to new system and point the backup and restore to the old backup and should see an option to extract userdata.  Do not restore from backup.

[dwyatt 2]

7 minutes ago, Happy2Play said:

Since this is a potential db issue it is a little difficult but yes.  Make a full backup on old system via the plugin.

You need to copy the users.db to new system and point the backup and restore to the old backup and should see an option to extract userdata.  Do not restore from backup.

Do you know where the version grouping is stored in the database? I've copied my library.db databases (original plus the new instance one) and am inspecting them with DB Browser. They again appear identical for the media items in question (old database still shows all 3 items after they've been grouped but UI only shows the 2). What table is the grouping linking stored in? 

[Happy2Play 8290]

1 hour ago, dwyatt said:

Do you know where the version grouping is stored in the database? I've copied my library.db databases (original plus the new instance one) and am inspecting them with DB Browser. They again appear identical for the media items in question (old database still shows all 3 items after they've been grouped but UI only shows the 2). What table is the grouping linking stored in? 

Not positive but in mediaitems table look at the PresentationUniqueKey or UserDataKeyId column.

Edited March 8 by Happy2Play

[dwyatt 2]

1 hour ago, Happy2Play said:

Not positive but in mediaitems table look at the UserDataKeyId column.

In both old and new databases this column is populated with matching values for the 3 versions (271455 on old database, 26 on new one). If I then look in the UserDataKeys2 table and find that ID then in both old and new databases it gives me 389 as the corresponding UserDataKey. After a bit of investigation this appears to be the tmdbid for this movie, hence it being the same in both databases. I can then look this 389 up in the UserDatas table and see that this is where it tracks if users have watched it or not, etc. But clearly the fact that both old and new databases correctly seem to be linking all 3 together seems like it should be working, but clearly it's not. 

As an aside (or maybe related?), this led to me discovering that my UserDatas table is a bit different in old and new databases. See below for their definitions. Note the new one has some FOREIGN KEY and PRIMARY KEY statements at the end that are missing from old one and things are ordered a bit differently. No idea if this is relevant ... just providing all the information I discover!

Old: 

CREATE TABLE UserDatas (
    userId INT not null, 
    rating float null, 
    played bit not null, 
    playCount int not null, 
    isFavorite bit not null, 
    playbackPositionTicks bigint not null, 
    lastPlayedDate datetime null, 
    AudioStreamIndex INT, 
    SubtitleStreamIndex INT, 
    LastPlayedDateInt INT NULL, 
    UserDataKeyId INT NULL, 
    RatingLastModified INT NULL, 
    PlaystateLastModified INT NULL, 
    HideFromResume bit NULL)

New:

CREATE TABLE UserDatas (
    UserDataKeyId INT not null, 
    userId INT not null, 
    rating float null, 
    played bit not null, 
    playCount int not null, 
    isFavorite bit not null, 
    HideFromResume bit not null, 
    playbackPositionTicks bigint not null, 
    LastPlayedDateInt INT null, 
    AudioStreamIndex INT, 
    SubtitleStreamIndex INT,
    RatingLastModified INT not null, 
    PlaystateLastModified INT not null, 
    FOREIGN KEY (UserDataKeyId) REFERENCES UserDataKeys2(Id), 
    PRIMARY KEY (UserDataKeyId,userId)
    ) WITHOUT ROWID

 

EDIT: I think UserDatas table seems to be purely to track watch state/etc for users so I don't think this has anything to do with the grouping of the items itself. The items having the same UserDataKey in UserDataKeys2 table is what should group them together? Although interestingly if I split the versions then they still stay with same UserDataKey as when grouped so maybe this doesn't actually control if they are grouped or not. 

Edited March 9 by dwyatt

[dwyatt 2]

@LukeAny chance you can stop by this thread and provide any insight or ideas? I'm pretty stuck at this point and short of migrating to a new fresh install (which I'd prefer not to do) I'm not sure what to do. 

[Luke 37085]

Can you try splitting them all apart and then joining them manually?

[dwyatt 2]

2 hours ago, Luke said:

Can you try splitting them all apart and then joining them manually?

I can do that all day with no results. See my post below from earlier in this thread which includes a logfile if that helps.