Jump to content


Photo

Improve reporting features

errors filename title tool reports year path misidentified identification

  • Please log in to reply
25 replies to this topic

#1 topbanana OFFLINE  

topbanana

    Advanced Member

  • Members
  • 150 posts
  • Local time: 09:57 PM

Posted 05 October 2018 - 08:38 AM

Your libraries of movies, TV, etc. almost certainly have lots and lots and lots of misidentified items, errors...  5%, perhaps more is possible...
And 5% of a large collection, is a lot of items.  Wrongly identified.
 

Emby relies on the various databases on the internet, and between them, they inevitably make the odd mistake...   That's fair enough.  Many factors are involved, bad filenames, rare titles, foreign character, etc, etc... Some are just stupidly wrong. They're going to happen...  

BUT.

We still don't have any tools to find them.
You have no way of knowing how many errors are in your libraries.
We have to rely on stumbling upon the error, or if you have an intimate knowledge of what titles you do have, you'll spot a few more of the errors perhaps.

As attempted a couple of times, some of us have tried combining the grossly, needlessly inadequate 'Reports' feature (now a plugin), with data pulled from the database files with external tools.  This has worked OK, but is a royal PITA, and not slick, not quick to do.   But it does show you how many errors emby holds.  Which might horrify/surprise you.

If you want some fun, reinstall emby from scratch, rebuilding your libraries... You'll very quickly see some errors, with many more being there without you knowing.

ALL WE WANT is the ability to, at least, list the path, filename, against the title and year from metadata downloaded...   
This simple feature, ability, would make finding these numerous errors possible, if a little bit slow to process (takes a lot of concentration to check 5,000 filenames, titles, years).

Later, yeah, it would be sweet to have it highlight non-exact matches, that would speed up the process immensely. But we need the minimum now.

 

These tools need to be in the core of emby...  It's not good enough that this media database application doesn't have the most basic data analysis tool!


Path, filename, title, year.

Please.


Edited by topbanana, 07 October 2018 - 02:18 PM.

  • jlr19 likes this

#2 PenkethBoy OFFLINE  

PenkethBoy

    Advanced Member

  • Members
  • 4312 posts
  • Local time: 02:57 PM
  • LocationWarrington,UK

Posted 05 October 2018 - 09:14 AM

LOL - bit Ranty!

 

You can do this easy enough with Db Browser for Sqlite (free) - yes it would be nice if this were built in but its not so....

 

Shut down you Emby Server and open the library.db file

 

Then use the SQL tab

 

e.g

5bb7628d8b48c_Capture2.jpg

 

You can then export this to a csv and do your analysis (hint look for the icon in the bottom right of the screen)

 

This is for the beta server only and i think i got all the relevant types but easy enough for you to add them

 

Only took 5 mins to write and test


  • CBers and Happy2Play like this

#3 TylerV76 OFFLINE  

TylerV76

    Advanced Member

  • Members
  • 485 posts
  • Local time: 09:57 AM

Posted 05 October 2018 - 09:48 AM

I don't have any errors or misidentified items in my library, zero. When I add new content, I see it in the "new" section and if it doesn't match I correct the metadata. This is EXTREMELY rare and typically only on an obscure file.

 

As long as the file is properly tagged you shouldn't be having any issues.  



#4 topbanana OFFLINE  

topbanana

    Advanced Member

  • Members
  • 150 posts
  • Local time: 09:57 PM

Posted 06 October 2018 - 01:56 AM

LOL - bit Ranty!

It's been going on for years...  We've asked for the most basic of tools (just adding the path, filename columns to the reports, absolute basic db task). But still we have nothing.
 

 

You can do this easy enough with Db Browser for Sqlite (free) - yes it would be nice if this were built in but its not so....

Yes, we've detailed this method in another thread.
It works a treat, i found many errors this way...  but is very time consuming and not the kind of thing every, or many emby users wants to do.

It would be much better to get the software to do this for us...  That's the point of software like this, wrangling huge amounts of data and automating tasks... Emby is excellent as a media server, but it doesn't yet have any feature to help us find the errors that unavoidably, inevitably happen.

All we want to do is, say, go to the report, compare the path and filename with the title and year that emby has gotten from the internet databases.

 

 

Only took 5 mins to write and test

So much of this software can be done externally in only 5 minutes...
But, we're better off paying a programmer to spend that 5 minutes writing the code and adding it to the software itself...  That way all the users can make use of the feature over and over and over.  Adding value to the software.

 

 

 

You can then export this to a csv and do your analysis (hint look for the icon in the bottom right of the screen)

Interested: Did you look through your library this way?
How many errors did you spot?

 

 

 

I don't have any errors or misidentified items in my library, zero. When I add new content, I see it in the "new" section and if it doesn't match I correct the metadata. This is EXTREMELY rare and typically only on an obscure file.

You seem very confident.
And this is entirely possible, sure. But it looks like you've invested a lot of effort up front to ensure this.  Great work.

But.  You have no quick or easy way of finding out if you are correct.
 

If you were to reinstall emby and rebuild your libraries from scratch, how many errors might be made?

If you were given 100+ movie files, with no nfos, just the correctly named files, how many errors would emby make iding them all?
 

 

As long as the file is properly tagged you shouldn't be having any issues.  

I don't use nfo files.  As don't many others.
Instead, i just have 'perfectly' named files.  I mostly just get filebot to name them 'title (year) (720p).mkv'...  And i correct the mistakes made manually, pretty much the same way you do.    All are correct, as far as i know (i'm realistically only 99% sure though), compared with tmdb, omdb, imbd, etc...  
But even with 'perfectly' named files, emby makes mistakes.  Some are slightly wrong, some are totally wrong.  Many have been linked to the infamous premiere date bug that's been going on for years too... 

A stupid example.  Given these two movies:
Live Free or Die (2006).avi
Live Free or Die Hard (2007).avi
Similar names, different years, both not obscure...  emby identifies them BOTH as 'Live Free or Die Hard (2007)'...  So an 'easy' error to spot if you happen to browse past the movies in emby, as the posters are obviously identical... and either you have two files for the same film, or emby has made a mistake.

This it just one example.
Emby mostly makes mistakes such that a completely different title is identified, so you have no way of knowing the error exists unless you browse past the movie poster and recognise it as not a film you thought you had...  But that takes intimate knowledge of your libraries.

If i'd kept a log of all the mistakes i'd stumbled across, or found using the Db Browser for Sqlite tool, you'd find it horrifying or hilarious.

You, I, we just need a feature in emby to easily find these errors.



#5 Happy2Play OFFLINE  

Happy2Play

    Trial and Error

  • Moderators
  • 19200 posts
  • Local time: 06:57 AM
  • LocationWashington State

Posted 06 October 2018 - 02:21 AM

Or have a look at EmbyStats Server, it is a work in progress as links need updated with url "/index.html#!/" change.

 

5bb8539c1e02f_suspicious.jpg



#6 topbanana OFFLINE  

topbanana

    Advanced Member

  • Members
  • 150 posts
  • Local time: 09:57 PM

Posted 06 October 2018 - 03:09 AM

Or have a look at EmbyStats Server, it is a work in progress as links need updated with url "/index.html#!/" change.

Again, it's an external tool.
Does it find the blatantly wrong mistakes, that are fairly common?  i.e. wrong movie title and year altogether.
One that i couldn't get to run, and i don't have docker, but anyway...

Emby makes mistakes, will always make mistakes.  They're unavoidable.
So emby should include at least the minimum feature/tool to help us find them.
All emby users should have this at hand.

Remember, there's probably a very high percentage of us pendantic, ocd spectrum emby users out there!   We love to sort and maintain our collections! And we can't live knowing theirs blatant misidentified movies in there!  ;-)

Adding a database column or two to a report is the absolute basic of basic tasks for this software.
 

 

Path, filename, title, year.

Pretty Please



#7 topbanana OFFLINE  

topbanana

    Advanced Member

  • Members
  • 150 posts
  • Local time: 09:57 PM

Posted 06 October 2018 - 03:29 AM

Stumbled across another of the easy to spot errors.

Emby was given these two movies:

New York (2009) (720p).mkv

Synecdoche, New York (2008) (720p).mkv

And it identified them both as 'Synecdoche, New York (2008)'.  Saw the two identical movie posters next to each other.

Again, fairly easy to spot when passing them, by chance.  But i had no way of knowing or finding the error otherwise.



#8 Happy2Play OFFLINE  

Happy2Play

    Trial and Error

  • Moderators
  • 19200 posts
  • Local time: 06:57 AM
  • LocationWashington State

Posted 06 October 2018 - 04:02 AM

Stumbled across another of the easy to spot errors.

Emby was given these two movies:

New York (2009) (720p).mkv

Synecdoche, New York (2008) (720p).mkv

And it identified them both as 'Synecdoche, New York (2008)'.  Saw the two identical movie posters next to each other.

Again, fairly easy to spot when passing them, by chance.  But i had no way of knowing or finding the error otherwise.

 

Good example of providers returning a wrong result.

 

 

.
Does it find the blatantly wrong mistakes, that are fairly common?  i.e. wrong movie title and year altogether.
One that i couldn't get to run, and i don't have docker, but anyway...

 

From a single movie aspect, I would say no as the only way to compare is title to path assuming path is named correctly.

 

As for your example above they would be linked to the same metadata id so yes the stat server will catch it as shown above.  I don't use docker either, I got the windows zip, but that won't work either if your not on windows.

 

Things like this will always require additional tools.



#9 topbanana OFFLINE  

topbanana

    Advanced Member

  • Members
  • 150 posts
  • Local time: 09:57 PM

Posted 06 October 2018 - 04:34 AM

Using DB Browser for SQLite for a very, very quick n dirty scan (just looking for first letters that don't follow the alphabet listing):
 

Miles (2017) (720p).mkv   ...   Emby IDed as '600 Miles (2016)'
Game of Death (死亡遊戲) (1978) (720p).mkv   ...   Emby IDed as 'The True Game of Death (1978)'
Princess (2014).mkv   ...   Emby IDed as 'Barbie in Princess Power (2015)'

The Experiment (Das Experiment) (2001) (720p).mkv   ...   Emby IDed as 'Acceptable Risk (2001)'
The Son (Le fils) (2002).mkv   ...   Emby IDed as 'Son of the Bride (2001)'
 

So all these were blatantly wrongly identified.

But i had to go to a website, download a software tool, copy some code from a forum post i could track down only because i participated in it, copy the database file out of my user folder, open it up in the software, run the copied code in the tool... Only then to get the 3 columns that allowed me to carefully scan down through them, to see if anything didn't match.  Only then did i find these errors with a few minutes of scanning.



#10 topbanana OFFLINE  

topbanana

    Advanced Member

  • Members
  • 150 posts
  • Local time: 09:57 PM

Posted 06 October 2018 - 04:51 AM

Good example of providers returning a wrong result.

Absolutely.
Emby is only as good as the providers it's using to id the movies.  So we can't blame emby, but the error exist within emby's libraries.  So we should have the ability to find the mistakes made by whoever.
 

 

From a single movie aspect, I would say no as the only way to compare is title to path assuming path is named correctly.

Yes.  It's up to us to present the movie to emby with the correct path/filenames.
But emby (it's providers) make mistakes even when the path/filename is correct.  So we should have the ability to find the mistakes made.

 

 

Things like this will always require additional tools.

The request, which goes back years, has been to, as a minimum, add the path/filename columns to the reports feature.
Adding a column to the reports is dead easy.  The most basic of requests for a piece of software built on a database.
The Reports feature is now a plugin, ok.  But this additional feature should be included.  And i think it should be in the core of emby.  Finding the errors that unavoidably happen should be a task that emby deals with.

We want emby to be awesome.  It presents lots of beautiful posters and info about our movie collections, allows us to browse them, and to play them on all our devices.  But we also need it to help us find the errors that always happen.



#11 Happy2Play OFFLINE  

Happy2Play

    Trial and Error

  • Moderators
  • 19200 posts
  • Local time: 06:57 AM
  • LocationWashington State

Posted 06 October 2018 - 06:05 AM

Not that I really have any idea how I did it but are you wanting this in the Reports plugin, paths for Movies and Episodes.  I will have to document all the changes I did to the download.

 

5bb886f14fc3d_reports.jpg


  • topbanana and jlr19 like this

#12 topbanana OFFLINE  

topbanana

    Advanced Member

  • Members
  • 150 posts
  • Local time: 09:57 PM

Posted 06 October 2018 - 06:24 AM

Not that I really have any idea how I did it but are you wanting this in the Reports plugin, paths for Movies and Episodes.

Awesome job!
Yep, that's pretty much it!

For the task in hand, it's best if the title and year are together, and the path on one side or the other, so that you can compare the two. Reports doesn't allow column reordering i think.
I've been using DB Browser with the path to the left, and the name, year to the right. With the columns as such, you could probably get by without having to export anything to excel, etc, you could compare them right there in emby. Which is desirable.

Thanks Happy2Play!

Edited by topbanana, 06 October 2018 - 06:25 AM.

  • jlr19 likes this

#13 Happy2Play OFFLINE  

Happy2Play

    Trial and Error

  • Moderators
  • 19200 posts
  • Local time: 06:57 AM
  • LocationWashington State

Posted 06 October 2018 - 08:21 AM

So like this.

 

5bb89f6c22a1f_reports.jpg

 

Submitted pull request on GitHub.

 

@Luke is there a way to make Columns options persistant, to maintain column selection after browser refresh?


  • topbanana likes this

#14 topbanana OFFLINE  

topbanana

    Advanced Member

  • Members
  • 150 posts
  • Local time: 09:57 PM

Posted 06 October 2018 - 09:28 AM

So like this.

...

 

Submitted pull request on GitHub.

Yep!
Awesome.



#15 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 157196 posts
  • Local time: 09:57 AM

Posted 06 October 2018 - 10:28 AM

So like this.

 

5bb89f6c22a1f_reports.jpg

 

Submitted pull request on GitHub.

 

@Luke is there a way to make Columns options persistant, to maintain column selection after browser refresh?

 

A new contributor could help add this to the reports plugin, yes.



#16 Happy2Play OFFLINE  

Happy2Play

    Trial and Error

  • Moderators
  • 19200 posts
  • Local time: 06:57 AM
  • LocationWashington State

Posted 06 October 2018 - 10:30 AM

A new contributor could help add this to the reports plugin, yes.

 

I submitted a pull request, is there something else I need to do?


  • topbanana likes this

#17 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 157196 posts
  • Local time: 09:57 AM

Posted 06 October 2018 - 10:39 AM

No, that's good, thanks.


  • topbanana likes this

#18 TylerV76 OFFLINE  

TylerV76

    Advanced Member

  • Members
  • 485 posts
  • Local time: 09:57 AM

Posted 06 October 2018 - 08:53 PM

Edit: NM 


Edited by TylerV76, 06 October 2018 - 08:57 PM.


#19 jlr19 OFFLINE  

jlr19

    Sr Noob

  • Members
  • 1079 posts
  • Local time: 08:57 AM
  • LocationLower Canada

Posted 07 October 2018 - 07:35 AM

Not that I really have any idea how I did it but are you wanting this in the Reports plugin, paths for Movies and Episodes.  I will have to document all the changes I did to the download.

 

5bb886f14fc3d_reports.jpg

I too have been crying out for error reporting forever, I had all but resigned due to no support or even mild interest from other users or so it seemed...

I am really interested by this added feature, how will I get it eventually?

 

And I salute @topbanana's persistence and well presented case and arguments.


  • topbanana likes this

#20 jlr19 OFFLINE  

jlr19

    Sr Noob

  • Members
  • 1079 posts
  • Local time: 08:57 AM
  • LocationLower Canada

Posted 07 October 2018 - 07:52 AM

I don't have any errors or misidentified items in my library, zero. When I add new content, I see it in the "new" section and if it doesn't match I correct the metadata. This is EXTREMELY rare and typically only on an obscure file.

 

As long as the file is properly tagged you shouldn't be having any issues.  

I am glad you have no errors/misidentified, and yes, sorting your library by descending date added is useful to catch recent stuff, but not so much on larger collections.

As far as your statement file (and path) are properly tagged warrants no issues, I disagree. I am constantly searching movies on themoviedb and tv show on tvdb.com and I am always astonished by the numbers of search results I am getting (some so far off it's almost funny) from an otherwise PERFECTLY named search. What I really don't get is sometime you find a perfect match, but only scrolling down to the 5th results for instance. How can anyone expect Emby to always grab the right data when the website it depends upon is so "loose".


Edited by jlr19, 07 October 2018 - 07:58 AM.






Also tagged with one or more of these keywords: errors, filename, title, tool, reports, year, path, misidentified, identification

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users