Automated Tagging

[roaku 795]

10 minutes ago, bakes82 said:

If you tag the base item with 4k, and it has 1080p option also, the movie wont appear because its been tagged 4k also and the 4k tag is blocked, thus the whole base item movie is blocked even if it has sources of 720 or 1080.    Tags are being applied at a base item not a media source.   This prevents having duplicate IDs in db with the exact same info.  Its a 1 to many relationship, but the 1 has the tag holder.

I understand that.

The key though, is that Tags and other metada can easily be applied to each BaseItem regardless of versioning. The only reason it looks like they can't is because the MediaManager  (edit: I meant MetadataManager) doesn't currently support it (support is on the way, though).

So, there is no problem modifying the tags for each BaseItem programmatically (you can even do it by modifying the particular file's nfo file and rescanning your library).

Emby will behave correctly and honor the tags on each BaseItem once that metadata is in place on the item.

 

Edited January 22, 2021 by roaku

[bakes82 90]

No idea, with the SDK the mediasource doesnt have a baseitem, but baseitems have mediasources. otherwise youre in a multi to multi relationship

[roaku 795]

2 minutes ago, bakes82 said:

They shouldn't be.  Once you merge them the other ref should be deleted, and its now just pointing to 1 base item.  This is what I would be expecting in a 1 to many DB relation ship.

That's not how it actually is, though.

I went down this rabbit hole previously. Basically, all versions are fully fledged entities behind the scenes. If you modify their metadata, it will get honored throughout the Emby system.

If you don't get each version's metadata modified, you'll see problems like this:

 

[bakes82 90]

Well if the API/SDK doesnt expose it what you want me to do?

[roaku 795]

Just now, bakes82 said:

Well if the API/SDK doesnt expose it what you want me to do?

It does expose it if you use InternalItemsQuery() without a User argument at the start or over straight rest, dropping the User and their id from the path.

From then everything else is the same, you're just now able to act on each movie version as its own BaseItem instead of as multi-version group.

Emby's backend doesn't care about the multi-versioning when you're updating metadata. It will update the metadata of the BaseItem you tell it to.

[roaku 795]

Alright, I've verified that my Reports customization still works the way I described.

In this example, 'Airplane!' has two different versions grouped with multi-versioning on the frontend:

But my customization to Reports means each version is presented separately, with no confusion over which MediaStreams belong to which BaseItem:

 

I'll put together the code in a minute, but it's just doing what I've been describing in this thread.

[bakes82 90]

will be interesting to see whats diff

[roaku 795]

16 minutes ago, bakes82 said:

will be interesting to see whats diff

Here's a full on diff file.

It also includes a change to make the loop through all the Audio streams instead of just grabbing the first one, but the change to InternalItemQuery() is enough to get all the versions of an item as their own BaseItem, ready to have their tags manipulated.

 

Just remove 'User' from this line here:

https://github.com/MediaBrowser/EmbyReports/blob/master/EmbyReports/Api/ReportsService.cs#L182

And your versions come back independent of however they might be grouped on the front end.

Just tweak the way it looks at, say, the Audio MediaStream, and you can check all of them instead of stopping at the first one (again, this is change isn't directly related to what we're talking about, it can be ignored), but it does demonstrate GetMediaStreams() and how it only shows the MediaStreams for the current, ungrouped BaseItem:
https://github.com/MediaBrowser/EmbyReports/blob/master/EmbyReports/Api/Common/ReportBuilderBase.cs#L72

            if (detailed)
              //the custom way
            {
                string audioText = "";
                item.GetMediaStreams().ForEach(stream => {
                    if (stream.Type == MediaStreamType.Audio)
                    {
                        if (stream.Title != null)
                        {
                            audioText += stream.Title + " - ";
                        }
                        audioText += stream.Codec.ToUpper() == "DCA" ? stream.Profile : stream.Codec.
                        ToUpper() +" | ";
                    }
                });
                return audioText;
            }
            else {
              //the normal way
                var stream = item.GetMediaStreams().FirstOrDefault(i => i.Type == MediaStreamType.Audio);
                if (stream != null)
                {
                    if (stream.Title != null)
                    {
                        return stream.Codec.ToUpper() == "DCA" ? stream.Profile : stream.Codec.
                        ToUpper() + " " + stream.Title;
                    }
                    else
                    {
                        return stream.Codec.ToUpper() == "DCA" ? stream.Profile : stream.Codec.
                        ToUpper();
                    }
                }
            }

 

reports-changes.txt

Edited January 22, 2021 by roaku

[roaku 795]

And yes, I could have kept the streaming lambda filtery stuff but I was just using Reports as a reference for something else I was working on, so no code judgment please.

[bakes82 90]

Looking at this code whats the root folder?

 

This is what the code would need to be based off what you are saying.  But IDK what "RootFolder" really maps to, but thats how the code is calling that private method.

 

var items = LibraryManager.RootFolder.GetItems(new InternalItemsQuery()
                {
                    Recursive        = true,
                    IncludeItemTypes = new[] { rule.Profile.Type },
                    AudioCodecs = new []{ rule.Profile.AudioCodec ?? "" },
                    Containers = new []{ rule.Profile.Container ?? "" },
                    OfficialRatings = new []{ rule.Profile.Rating ?? "" }
                });

                var item = items.Items;

 

[roaku 795]

And just to reemphasize, managing the metadata, including tags for each version is supported now,  and the Metadata Manager will eventually be updated to handle it.

The tag based whitelist/blacklist already honors the Item's tags regardless of grouping.

 

[roaku 795]

3 minutes ago, bakes82 said:

Looking at this code whats the root folder?

 

This is what the code would need to be based off what you are saying.  But IDK what "RootFolder" really maps to, but thats how the code is calling that private method.

 

var items = LibraryManager.RootFolder.GetItems(new InternalItemsQuery()
                {
                    Recursive        = true,
                    IncludeItemTypes = new[] { rule.Profile.Type },
                    AudioCodecs = new []{ rule.Profile.AudioCodec ?? "" },
                    Containers = new []{ rule.Profile.Container ?? "" },
                    OfficialRatings = new []{ rule.Profile.Rating ?? "" }
                });

                var item = items.Items;

 

InternalItemsQuery() is public and in Media.Controller.Browser.Entities

 

[bakes82 90]

This might do the same thing no idea, would need to log the counts:

 

var itemQuery = LibraryManager.QueryItems(new InternalItemsQuery()
                {
                    Recursive        = true,
                    IncludeItemTypes = new[] { rule.Profile.Type },
                    AudioCodecs = new []{ rule.Profile.AudioCodec ?? "" },
                    Containers = new []{ rule.Profile.Container ?? "" },
                    OfficialRatings = new []{ rule.Profile.Rating ?? "" }
                });

 

[roaku 795]

Ya, my expectation is that anything that accepts the InternalItemsQuery class as its argument should behave the same way.

And if not, Reports is just another plugin, so we should be able to use RootFolder or whatever else we need to.

Edited January 22, 2021 by roaku

[bakes82 90]

He was already calling it with internal, just passing an admin user.

[bakes82 90]

Ill let him log the numbers, watch them be diff

[roaku 795]

Ya, the distinction is 'sort of intentional' to quote Luke. I asked about it when I came across the differences between the two endpoints:
 

 

[bakes82 90]

No I mean the rootfolder items vs the normal getitems method.  Same internal query, I wonder if they would return two diff counts, and how avg joe dev would know to drill to the rootfolder to get items when the manager has its own.  This is what Im saying these things get wonky and dont always align.

[roaku 795]

5 minutes ago, bakes82 said:

No I mean the rootfolder items vs the normal getitems method.  Same internal query, I wonder if they would return two diff counts, and how avg joe dev would know to drill to the rootfolder to get items when the manager has its own.  This is what Im saying these things get wonky and dont always align.

They definitely do return two different counts. That can be seen in the customized Reports plugin results and through the rest API.

For example, I have 601 movies in my Movie collection if I ask with a User or just look through a client app. I have 832 if I omit the User.

Edited January 22, 2021 by roaku

[bakes82 90]

1 minute ago, roaku said:

They definitely do return two different counts. That can be seen in the customized Reports plugin results and through the rest API.

For example, I have 601 movies in my Movie collection if I ask with a User. I have 832 if I omit the User.

I didnt say user dork.  Watch them return diff counts.

I said this:

var items = LibraryManager.RootFolder.GetItems(new InternalItemsQuery()
                {
                    Recursive        = true,
                    IncludeItemTypes = new[] { rule.Profile.Type },
                    AudioCodecs = new []{ rule.Profile.AudioCodec ?? "" },
                    Containers = new []{ rule.Profile.Container ?? "" },
                    OfficialRatings = new []{ rule.Profile.Rating ?? "" }
                });

vs

var itemQuery = LibraryManager.QueryItems(new InternalItemsQuery()
                {
                    Recursive        = true,
                    IncludeItemTypes = new[] { rule.Profile.Type },
                    AudioCodecs = new []{ rule.Profile.AudioCodec ?? "" },
                    Containers = new []{ rule.Profile.Container ?? "" },
                    OfficialRatings = new []{ rule.Profile.Rating ?? "" }
                });

[roaku 795]

Dorky enough to show you how to get ungrouped BaseItems so the plugin will work I guess.

[ebr 14921]

I apologize that I did not read all the updates here but, in case you haven't figured it out yet, the logical layout is:

BaseItem -> can contain one or more MediaSources -> which can then contain multiple MediaStreams.

And the tag is a property of the BaseItem as a whole.

So, to satisfy conditions, you would need to query all the media streams on each of the media sources for any particular base item and then add the tag to the base item if the condition was met by the underlying streams.

[bakes82 90]

7 minutes ago, ebr said:

I apologize that I did not read all the updates here but, in case you haven't figured it out yet, the logical layout is:

BaseItem -> can contain one or more MediaSources -> which can then contain multiple MediaStreams.

And the tag is a property of the BaseItem as a whole.

So, to satisfy conditions, you would need to query all the media streams on each of the media sources for any particular base item and then add the tag to the base item if the condition was met by the underlying streams.

Do all Sources/Streams only point to 1 base item?  Thus if you add a 4k tag will it block the entire item from being shown or just the 1 stream?

[roaku 795]

24 minutes ago, ebr said:

I apologize that I did not read all the updates here but, in case you haven't figured it out yet, the logical layout is:

BaseItem -> can contain one or more MediaSources -> which can then contain multiple MediaStreams.

And the tag is a property of the BaseItem as a whole.

So, to satisfy conditions, you would need to query all the media streams on each of the media sources for any particular base item and then add the tag to the base item if the condition was met by the underlying streams.

So, the problem with this approach is that each movie version will potentially match different rules and therefore need different tags. If this isn't accommodated, Users who shouldn't have access to MovieVersion B might be able to see it because Movie Version C matched a rule and so the plugin added the tag to the 'primary' Item, which is Movie Version A. Or vice versa. That problem is documented here.

The alternative approach we're trying now is to query for the items *without a User*. This query returns all the Items ungrouped by their version as full fledged BaseItems.

This way, we can trust BaseItem.GetMediaStreams() to have all the MediaStreams we care about *for that version*. We don't need to use GetMediaSources() this way (although, the plugin might need to look at GetMediaSources() for other reasons, and its data is equally trustworthy with this approach).

We also now have direct access to the BaseItem we need to update the tags for. And Emby tag white listing will already behave properly if each version's tags are different.

Edited January 22, 2021 by roaku

[chef 3746]

4 hours ago, roaku said:

The value we need is exposed as 'Id' through the rest api:

/emby/Items?api_key={apikey}&Recursive=true&IncludeItemTypes=Movie&Fields=Path,TagLines,Overview,PremiereDate,Genres,MediaStreams&SortBy=Name&SortOrder=Ascending

[
  {
    "Name": "Airplane!",
    "ServerId": "84be18a2e4db4328ab7b6fe164491562",
    "Id": "1402",
    "Container": "mkv",
    "PremiereDate": "1980-07-01T06:00:00.0000000+00:00",
    ...
  },
  {
    "Name": "Airplane!",
    "ServerId": "84be18a2e4db4328ab7b6fe164491562",
    "Id": "1403",
    "Container": "mkv",
    "PremiereDate": "1980-07-02T05:00:00.0000000+00:00",
    ...
  }
]

 

So both those baseItems represent two different resolutions?