Jump to content

Sort By Bitrate/HDR


storelogan

Recommended Posts

awdspyder

I think you may be misinterpreting what I'm saying here.  I definitely do not want Emby to conflate functional with display-related info.  I want Emby to continue to make the best decisions regarding playback based on what ffprobe sees - and I fully understand why this is necessary.  

In fact, I don't even care what Emby displays in the audio pick list - clearly that's related to the functional aspect of ffmpeg, as it should be.  What I'd like to see is for Emby to simply parse the file header as you mentioned, pull out the relevant video range info, dump that info into fields in the database and allow the user to sort, filter, etc. based on this info. And clearly a single file could have multiple values - i.e. HDR10 & DV or HDR10+ & DV, etc.

Maybe it would even be nice if Emby displayed some watermark type logos of the audio/video/aspect ratio info on the title details page as XBMC/Kodi used to (maybe still does, haven't used it in years).  This would literally be for informational purposes only.  I don't see the functional info that ffprobe gathers and displaying informational data from the header as being mutually exclusive.

Re: Dolby Vision vs. HDR10+.  We can discuss the merits of each, but it appears that the slight technical advantage goes to DV, at least on paper (with mastering displays and consumer hardware being what they are).  DV, however, has a clear edge in both studio and hardware adoption at this point.  Throw in the fact that the current leader in picture quality,  LG's OLEDs, still do not support HDR10+ and it further muddies the water.  Ultimately, I hope that HDR10+ wins the dynamic metadata war, as it appears to have most of the technical advantages of DV without the draconian licensing model (and I don't think consumer displays will hit north of 4000 nits any time soon).  However, it seems that HDR10+'s slow adoption, lack of name differentiation, and the fact that's playing significant catch-up to DV will unfortunately make it an uphill battle.

 

Edited by awdspyder
Link to comment
Share on other sites

rbjtech
On 30/10/2021 at 18:27, awdspyder said:

I think you may be misinterpreting what I'm saying here.  I definitely do not want Emby to conflate functional with display-related info.  I want Emby to continue to make the best decisions regarding playback based on what ffprobe sees - and I fully understand why this is necessary.  

In fact, I don't even care what Emby displays in the audio pick list - clearly that's related to the functional aspect of ffmpeg, as it should be.  What I'd like to see is for Emby to simply parse the file header as you mentioned, pull out the relevant video range info, dump that info into fields in the database and allow the user to sort, filter, etc. based on this info. And clearly a single file could have multiple values - i.e. HDR10 & DV or HDR10+ & DV, etc.

Maybe it would even be nice if Emby displayed some watermark type logos of the audio/video/aspect ratio info on the title details page as XBMC/Kodi used to (maybe still does, haven't used it in years).  This would literally be for informational purposes only.  I don't see the functional info that ffprobe gathers and displaying informational data from the header as being mutually exclusive.

Re: Dolby Vision vs. HDR10+.  We can discuss the merits of each, but it appears that the slight technical advantage goes to DV, at least on paper (with mastering displays and consumer hardware being what they are).  DV, however, has a clear edge in both studio and hardware adoption at this point.  Throw in the fact that the current leader in picture quality,  LG's OLEDs, still do not support HDR10+ and it further muddies the water.  Ultimately, I hope that HDR10+ wins the dynamic metadata war, as it appears to have most of the technical advantages of DV without the draconian licensing model (and I don't think consumer displays will hit north of 4000 nits any time soon).  However, it seems that HDR10+'s slow adoption, lack of name differentiation, and the fact that's playing significant catch-up to DV will unfortunately make it an uphill battle.

 

Hi - This is what everyone is saying - the current ffmpeg stack does not report (as codec's) the details you desire - thus emby cannot read them.

Once it does, then they will add - but until then, you have several options. 1) Use the embedded media file stream 'Titles'  or 2) Use Emby Tag's - Neither are particularly good options and are high maintenance unless you automate the process to ensure they are consistent. 

Link to comment
Share on other sites

awdspyder
1 hour ago, rbjtech said:

Hi - This is what everyone is saying - the current ffmpeg stack does not report (as codec's) the details you desire - thus emby cannot read them.

I completely understand that point - what I was attempting to convey (rather poorly it seems) is that I, and perhaps others, would love to see this implemented via another mechanism if necessary.  It's a shame it can't all be done by a single toolset, but it would seem ffprobe is lagging in this regard. Clearly it's programmatically doable - MediaInfo does it, MakeMKV does it (at least to some degree), pretty sure DVDFab can do it, and I'm sure there are others.

Now maybe there's a technical limitation that prevents using other methods to do this in Emby.  Perhaps there are licensing or other constraints.  Perhaps not enough people care about it like I do or it would require too large a time investment for the relatively small payout of simply displaying some extra metadata.  All of those are valid reasons for not implementing a software feature, but there seems to be a sense of, "ffmpeg/ffprobe can't do it, therefore it cannot be done in Emby." 

Perhaps that wasn't the intended message; perhaps the intended message was, "ffmpeg can't do it, and ffmpeg is our tool of choice, therefore we're not implementing a new mechanism just to satisfy your OCD. You'll just have to wait to see if/when they implement this." Of course that's disappointing, but totally understandable - Emby is a business and business decisions must be made.

1 hour ago, rbjtech said:

Use Emby Tag's - Neither are particularly good options and are high maintenance unless you automate the process to ensure they are consistent. 

Agreed - I have enough of that in my life.  Keeping up with an 80TB DIY fiber channel SAN, backing a 2-node Hyper-V cluster with two dozen VMs, keeps me doing more than enough mundane maintenance.

Edited by awdspyder
  • Agree 1
Link to comment
Share on other sites

rbjtech
8 minutes ago, awdspyder said:

I completely understand that point - what I was attempting to convey (rather poorly it seems) is that I, and perhaps others, would love to see this implemented via another mechanism if necessary.  It's a shame it can't all be done by a single toolset, but it would seem ffprobe is lagging in this regard. Clearly it's programmatically doable - MediaInfo does it, MakeMKV does it (at least to some degree), pretty sure DVDFab can do it, and I'm sure there are others.

Now maybe there's a technical limitation that prevents using other methods to do this in Emby.  Perhaps there are licensing or other constraints.  Perhaps not enough people care about it like I do or it would require too large a time investment for the relatively small payout of simply displaying some extra metadata.  All of those are valid reasons for not implementing a software feature, but there seems to be a sense of, "ffmpeg/ffprobe can't do it, therefore it cannot be done in Emby." 

Perhaps that wasn't the intended message; perhaps the intended message was, "ffmpeg can't do it, and ffmpeg is our tool of choice, therefore we're not implementing a new mechanism just to satisfy your OCD. You'll just have to wait to see if/when they implement this." Of course that's disappointing, but totally understandable - Emby is a business and business decisions must be made.

I'm in agreement with you ;) 

I just use mediainfocli with a custom codec template to extract the codec data in a simple script - that's the easy part - I have a CSV export of all my media listing all the codec's ready to go.    The more challenging part is to match the media name and then write that data into the emby system so it can be useful.     I think the only reliable way of doing that, is to embed mediainfocli in an emby plugin - writing the tag's to each emby itemId via the emby API.

It just needs somebody with the time and inclination to do this - tbh I think it would be a pretty popular plugin ... ( @chef ... 🤪)

Link to comment
Share on other sites

awdspyder
19 minutes ago, rbjtech said:

I just use mediainfocli with a custom codec template to extract the codec data in a simple script

LOL, a man after my own heart.  I was doing the same and then dumping the data into a CSV then importing into a MySQL database and serving it up via a LAMP stack.  It was working, albeit very time consuming.  Especially because then I was like, "wait, I need to make the CSS look pretty," and, "oh, I really need search and filtering functionality," and "ooh, what if I query IMDB and pull in metadata and coverart."

And pretty soon I realized, not only am I not a developer, but my wife is going to leave me if don't at least emerge from my office on weekends.  Plus, Emby already does most of this for me. 

Ah well, here's to hoping ffmpeg comes around!

  • Haha 1
Link to comment
Share on other sites

1 hour ago, awdspyder said:

Now maybe there's a technical limitation that prevents using other methods to do this in Emby

Hi.  The limitation is the fact that we use this information to make playback decisions so it must be accurate.  Therefore, we can't make it editable nor rely on simple header information that can also be edited/inaccurate.

Now, you'll then likely say - then just give us some display values - but then the issue becomes what happens when the display values conflict with the ones we derive from the actual media?  Now we've got not only duplicates but also possible conflicts and we spend more time troubleshooting and trying to help people understand the differences and/or why they aren't seeing what they expect.

  • Like 2
Link to comment
Share on other sites

rbjtech
1 hour ago, awdspyder said:

LOL, a man after my own heart.  I was doing the same and then dumping the data into a CSV then importing into a MySQL database and serving it up via a LAMP stack.  It was working, albeit very time consuming.  Especially because then I was like, "wait, I need to make the CSS look pretty," and, "oh, I really need search and filtering functionality," and "ooh, what if I query IMDB and pull in metadata and coverart."

And pretty soon I realized, not only am I not a developer, but my wife is going to leave me if don't at least emerge from my office on weekends.  Plus, Emby already does most of this for me. 

Ah well, here's to hoping ffmpeg comes around!

I actually have just had a VERY quick play and there is actually a Tag API function (I knew it could be done, but didn't realise there is a dedicated function to do it :) ) - so you just pass in the Tag info and it populates the entry with a tag or multiple tags - easy peasy.  

So the next question is how to match the mediainfocli output to the correct emby ItemId - the only common value we have is the filepath, which I just happened to write in the CSV .. 

Again this is easy enough in the API using a Recursive Path search - I can get an exact match to the Emby ItemId. 👍

So I just need to match the two and bingo, a fully automated way to write the output file from mediainfocli into emby tags .. 😎

I'll have a look later as this should be reasonably easy to do via curl ...

Link to comment
Share on other sites

awdspyder
1 hour ago, ebr said:

Now, you'll then likely say - then just give us some display values - but then the issue becomes what happens when the display values conflict with the ones we derive from the actual media?  Now we've got not only duplicates but also possible conflicts and we spend more time troubleshooting and trying to help people understand the differences and/or why they aren't seeing what they expect.

Right, this is what I was driving at - just having display values.  I've been the IT field for over 20 years, so I fully understand regarding not opening up the user confusion can of worms.  As rbjtech mentioned, perhaps a plugin is the better workaround.  Then OCD nuts like me can willingly deal with the duplication/discrepancies and it's not forced on the masses.

Thanks for your time!

Edited by awdspyder
  • Like 2
Link to comment
Share on other sites

rbjtech
34 minutes ago, GrimReaper said:

@rbjtech @awdspyder

Would using a 3-rd party tool for batch-tagging those do? 

.. such as this one ;)

https://emby.media/community/index.php?/topic/99809-a-tool-for-tagging-emby-content/

Yea I thought about this too haha - take the CSV as the input file and run it through Vic's HTML tagging tool - it may be an easy solution .. :)

Link to comment
Share on other sites

GrimReaper
6 minutes ago, rbjtech said:

.. such as this one ;)

https://emby.media/community/index.php?/topic/99809-a-tool-for-tagging-emby-content/

Yea I thought about this too haha - take the CSV as the input file and run it through Vic's HTML tagging tool - it may be an easy solution .. :)

Nope, said "3-rd party", not "community-made". 😉

Knowing me, you know it'll be TMM (Filter there is insanely powerful, it's like an app in itself, you'd be hard-pressed to think of something you can't filter on), as you can Filter by any 'overlying' codec: Dolby Vision, HDR10+, Dolby Atmos, DTS:X, TrueHD, whatever - and bulk-edit, tagging all filtered items in one swoop. It should take you roughly 15-20 seconds per term.

Though idea with utilizing Vic's tool is a nice one, as well. 

Edited by GrimReaper
Link to comment
Share on other sites

rbjtech

.. you and your TMM.. haha.

Getting the HDR (and HD Audio in my example) codec data out your collection, really is not an issue - it's a 1 line 'script' using mediainfocli ..

FOR /F "delims=" %%x in ('dir /a-s /b /s "M:\Films 4k\*.m??"') DO (mediainfocli --Output=file://hdr.txt "%%x" >> output.txt )

hdr.txt is just a template file .. you can grab whatever fields you like from MediaInfo ..

General;"""%CompleteName%"""
Video;,V#%StreamKindID%,%HDR_Format%,%HDR_Format_Profile%,%HDR_Format_Level%,%HDR_Format_Settings%
Audio;,A#%StreamKindID%,%Format/String%,%Format_Commercial_IfAny%,%Channel(s)%,%Language%
\r\n

The 'hard part' is getting the results into emby tag's and matching the ItemId - although the 'Path' option in the API makes that a lot easier than I first thought as the Path should be unique.

This would probably take the likes of @chef or @Cheesegeezer seconds to do, but as I have no experience of writing Emby Plugin's ..  I had a quick look at Scripter-X, but that seemed to be more 'Operational' type scripts rather than changing Metadata.  @VicMoore HTML is Javascript - so I got lost pretty quickly lol.

If I can get some time - I'll maybe take a look at fudging something together - having emby tags of all the specialist MediaInfo info does sound enticing .. :)

btw All the fields you can get out of MediaInfo are listed in the attached text file .. 

parameters.txt

Edited by rbjtech
  • Haha 1
  • Thanks 1
Link to comment
Share on other sites

Cheesegeezer
13 minutes ago, rbjtech said:

.. you and your TMM.. haha.

Getting the HDR (and HD Audio in my example) codec data out your collection, really is not an issue - it's a 1 line 'script' using mediainfocli ..

FOR /F "delims=" %%x in ('dir /a-s /b /s "M:\Films 4k\*.m??"') DO (mediainfocli --Output=file://hdr.txt "%%x" >> output.txt )

hdr.txt is just a template file .. you can grab whatever fields you like from MediaInfo ..

General;"""%CompleteName%"""
Video;,V#%StreamKindID%,%HDR_Format%,%HDR_Format_Profile%,%HDR_Format_Level%,%HDR_Format_Settings%
Audio;,A#%StreamKindID%,%Format/String%,%Format_Commercial_IfAny%,%Channel(s)%,%Language%
\r\n

The 'hard part' is getting the results into emby tag's and matching the ItemId - although the 'Path' option in the API makes that a lot easier than I first thought as the Path should be unique.

This would probably take the likes of @chef or @Cheesegeezer seconds to do, but as I have no experience of writing Emby Plugin's ..  I had a quick look at Scripter-X, but that seemed to be more 'Operational' type scripts rather than changing Metadata.  @VicMoore HTML is Javascript - so I got lost pretty quickly lol.

If I can get some time - I'll maybe take a look at fudging something together - having emby tags of all the specialist MediaInfo info does sound enticing .. :)

btw All the fields you can get out of MediaInfo are listed in the attached text file .. 

parameters.txt 113.68 kB · 1 download

I’m game for doing this, @chef what you think?  
cheesyChefProductions strike again?? 
 

Ok so can someone list out the following 

  1. Primary function - output goal
  2. Where/how this info is preferred to be used
  3. Benefits for users
  4. server/client/user? integration - which one(s) & why (use cases)

cheers

Link to comment
Share on other sites

rbjtech
15 minutes ago, Cheesegeezer said:

I’m game for doing this, @chef what you think?  
cheesyChefProductions strike again?? 
 

Ok so can someone list out the following 

  1. Primary function - output goal
  2. Where/how this info is preferred to be used
  3. Benefits for users
  4. server/client/user? integration - which one(s) & why (use cases)

cheers

Are you trying to push @chef over the edge ?! 🤣

  1. Primary function - output goal

Emby does not currently list any of the more specialist Codec details for the more discerning emby user - ie Dolby Vision, HDR10+, Dolby Atmos, DTS X etc etc.  The limitation is what ffmpeg/probe can use.   The information is readily available with open source utilities - such as mediainfocli - it's just a matter of getting it into emby.

2. Where/how this info is preferred to be used

The only current option to use this info is in Tag's - as we cannot change the core emby codec information for the reasons Eric have given above.

3. Benefits for users

Tag's can be filtered/sorted and searched - so while it's not perfect and ideally should be displayed alonside the main codecs, it's significantly better than not having the info at all.

If I wanted all my 'Dolby Vision' films with 'DTS:X' as an example - then I could easily filter by those two items and get a filtered list.

4. server/client/user? integration - which one(s) & why (use cases)

I believe all the clients can already utilise tag's now - so it could be used without any further Client development.

Edited by rbjtech
  • Like 1
Link to comment
Share on other sites

Cheesegeezer
12 minutes ago, rbjtech said:

Are you trying to push @chef over the edge ?! 

Of course... i can get this started, it’s just a metadata adder. Any cool plugin names spring to mind? Like Codec:X or DeepMeta or....

cmonFFmpeg lol 😝 

12 minutes ago, rbjtech said:
  1. Primary function - output goal

Emby does not currently list any of the more specialist Codec details for the more discerning emby user - ie Dolby Vision, HDR10+, Dolby Atmos, DTS X etc etc.  The limitation is what ffmpeg/probe can use.   The information is readily available with open source utilities - such as mediainfocli - it's just a matter of getting it into emby.

2. Where/how this info is preferred to be used

The only current option to use this info is in Tag's - as we cannot change the core emby codec information for the reasons Eric have given above.

3. Benefits for users

Tag's can be filtered/sorted and searched - so while it's not perfect and ideally should be displayed alonside the main codecs, it's significantly better than not having the info at all.

If I wanted all my 'Dolby Vision' films with 'DTS:X' as an example - then I could easily filter by those two items and get a filtered list.

4. server/client/user? integration - which one(s) & why (use cases)

I believe all the clients can already utilise tag's now - so it could be used without any further Client development.

Glad to have you on the team again to keep us on the windy blurry road lol

Link to comment
Share on other sites

GrimReaper
30 minutes ago, rbjtech said:

If I wanted all my 'Dolby Vision' films with 'DTS:X' as an example - then I could easily filter by those two items and get a filtered list.

Slight correction there: no, you couldn't, seeing both of those woud be Tags and values within same property get ORed, whereas narrowing it down as per example would require them to be ANDed. 

That ties back to a request of mine from some time ago to at least have an AND/OR switch on Genre and Tag property, those being the most prominent ones for such use:

 

Edited by GrimReaper
Typo
Link to comment
Share on other sites

Cheesegeezer
14 minutes ago, GrimReaper said:

Slight correction there: no, you couldn't, seeing both of those woud be Tags and values within same property get ORed, whereas narrowing it down as per example would require them to be ANDed. 

That ties back to a request of mine from some time ago to at least have an AND/OR switch on Genre and Tag property, those being the most prominent ones for such use:

 

So how would this be solved if there is no core functionality for and  && or ||.

plugins are there to solve something the core doesn’t offer. Would we actually produce a viable and user solution without core integration 

Link to comment
Share on other sites

GrimReaper
Just now, Cheesegeezer said:

So how would this be solved if there is no core functionality for and  && or ||.

Well, I've been trying to get core support there... 

Just now, Cheesegeezer said:

Would we actually produce a viable and user solution without core integration 

Depends on the usage case, it would still be viable to search/filter for a single (or multiple) term there, with all items with either being listed - unfortunately any combo ain't feasible atm. 

  • Agree 1
Link to comment
Share on other sites

rbjtech

Yep - Grim is right - we can't AND Tag's - but we can OR them.

tbh - I was trying to keep this as simple as possible and just automate adding tags from a CSV file and/or output from mediainfocli.

If the core functionality for tag filters being AND'd is not there yet, then I don't think there is anything we can do without using 'Combo' tag's to do the logic for us - and I'm not sure I wanna go there to cover up the shortfalls of emby.

 

Logic for this is simply scan new files as they are added and tag any of the 'codecs' listed (selectable I guess in the Plugin config) -

for Video there will only be ~3 - HLG, HDR10+, Dolby Vision 

for Audio there will be ~3 - DD+ with Dolby Atmos, DTS:X HD Master Audio, TrueHD with Dolby Atmos 

There may be some more, but I don't think we are talking a huge number of combo's here.

 

Link to comment
Share on other sites

7 hours ago, awdspyder said:

I completely understand that point - what I was attempting to convey (rather poorly it seems) 

I had gotten your point actually. My previous reply was targeted rather at the whole subject (covered in the referenced topics) than being a direct response to your message. Sorry for the confusion.

7 hours ago, awdspyder said:

I, and perhaps others, would love to see this implemented via another mechanism if necessary. 

That's not as easy as it sounds. Whatever "mechanism" we would pull in - it would need to work on all platforms we deliver for, and we would take care of compiling those binaries for all those platforms and test it properly. 

EDIT: I forgot to mention the processing time this would add for scanning large libraries.

Then there's all the things that @ebr and I already mentioned before and finally the question for what information?
Just for recognizing Dolby Vision profiles and a few more audio codec details?

OK, it's unfortunate that we can't provide that information right now, but given the fact that ffmpeg will have those things most likely within 6-12 months, why should we spend time on this? Eventually we'd have to work twice on it then: 1. for employing another "mechanism" 2. for removing it again and switch to the ffmpeg provided output (which will surely differ in some ways, creating additional mismatch).

In software development, you can't make substantial progress and innovation when you work in a way that you do a little here and a little there all the time.
We are much smaller than others but still technically leading in a number of areas. This is only possible by proper planning, like bundling certain related development tasks into "epics" and choosing the right moment to get into it. But when getting into something, get into it deeply and try to get the most benefit out of it - ideally doing it better than the others.

I can also tell what the next "epic" is planned to be in this area: Just a while ago, we have delivered tone mapping capabilities with hw and software implementations. All those are based on static light level data (HDR10). The next step will be to be able to use dynamic lighting information as provided by HDR10+ and certain(!!) Dolby Vision profiles (those which are backward compatible with HDR10). Then there are other DV profiles that use custom color transfer functions (e.g. the DV "Containers Demo"), which not even VLC can display properly yet (it's being worked on, though). Also, there's an open issue with Quicksync, which doesn't even provide the basic HDR10 lighting information yet, and some other things that need to be done at the side of ffmpeg.
When those prerequisites are fulfilled, we'll be ready to go for this.

Proper display of HDR metadata will just be a minor side effect of this 😉 


Disclaimer: Plans are always subject to change!

Edited by softworkz
Link to comment
Share on other sites

GrimReaper
15 minutes ago, rbjtech said:

then I don't think there is anything we can do without using 'Combo' tag's to do the logic for us - and I'm not sure I wanna go there to cover up the shortfalls of emby.

Lol believe me, you don't, here was my comment upon engaging in such endeavour 🤣:

On 9/20/2021 at 9:01 AM, GrimReaper said:

I have thousands upon thousands of tags, still not all combinations covered. Likewise for Genres, I have basic combos tagged, and STILL wife comes up with the combo that no sane person would ever think of. 

 

 

  • Haha 1
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...