Cast & Crew Images Don't Display As They Should

[freddyfinn 9]

On 10/12/2020 at 12:51 PM, cayars said:

My problem was the length of time it took to run but would never finish without an error.  I just bit the bullet and reloaded since that worked on my test system.

Yeah the plugin is a dud. 

I fixed the issue directly by editing the database manually.

It was straight forward and didn't take much time to do. It would be great if emby had a built in function to do the same. 

 

 

[freddyfinn 9]

 

I take no responsibility if this fails so make a backup!

Superfast way to fix this issue. 

Shutdown Emby Server

Download SQLite .  I used the portable version.

Go into C:\Users\(YourAccountName)\AppData\Roaming\Emby-Server\programdata\data

And make backup of library.db

Open SQLite Select 'Open Database' and navigate to C:\Users\(YourAccountName)\AppData\Roaming\Emby-Server\programdata\data

and open 'library.db'.

Download the attached 'Emby.sql'

In SQLite go to the 'Execute SQL' tab and load Emby.sql

Press the play button to Execute SQL

Close the database and SQLite.

Start Emby Server go into server settings and open 'Library'

Do a Metadata refresh by clicking on the three buttons on each folder.

Select 'Search For Missing Metadata' and you can leave 'Replace Existing Images unchecked' 

Done!

 

 

Emby.sql

[Carlo 4330]

Why do you want to update every row in the table with null values?

[freddyfinn 9]

28 minutes ago, cayars said:

Why do you want to update every row in the table with null values?

Nope not every row and it works perfectly. 

Carried out on five databases so far and it only takes 10 minutes to setup and run from start to finish plus the folder refresh on top of that. 

Once the rows are set to null they update correctly. 

Edited October 24, 2020 by freddyfinn
Edit.

[Carlo 4330]

Yes every row.

UPDATE MediaItems
SET ProviderIds = NULL,
    PresentationUniqueKey = NULL,
    Images = NULL,
    UserDataKeyID = NULL,
    UserDataKey = NULL

There is no where clause so it's updating every row/record.

[freddyfinn 9]

Just now, cayars said:

Yes every row.

UPDATE MediaItems
SET ProviderIds = NULL,
    PresentationUniqueKey = NULL,
    Images = NULL,
    UserDataKeyID = NULL,
    UserDataKey = NULL

There is no where clause so it's updating every row/record.

Its nullifying column records not row. 

[Carlo 4330]

I didn't mention columns.  I said rows.

It's updating or nulling specific columns for every record and I'm asking why?

[freddyfinn 9]

 

25 minutes ago, cayars said:

I didn't mention columns.  I said rows.

It's updating or nulling specific columns for every record and I'm asking why?

To ensure all potentially corrupt data is removed. The database structure remains intact and all pointers and links are removed. 

In other words the rows stay intact less the column values. Fastest and most efficient way to ensure the db is cleared. 

I didn't see anyone else even giving this approach a modicum of thought. Feel free to edit the sql however it works fine in its current format. This is the most basic SQL given there is no need to write anything overly complex and much like the plugin that takes hours to run and does not work.  

Edited October 24, 2020 by freddyfinn

[Carlo 4330]

I understand what it does.

But this forces a full reload vs just the images that won't load and doesn't "fix" the problem because the images aren't reloaded.  Now you won't get an image until the system needs it.

You can end up right back in the same spot again later if more URL change (how this happened).

See what I'm getting at?

[freddyfinn 9]

Once you do the metadata refresh everything is back where it should be hence the metadata refresh function. The names are retained in the db and the correct values are rewritten during the refresh including the people links. If they aren't use the refresh people task. 

 

[Carlo 4330]

You're not understanding what I'm saying.  You're doing a "forced reset" and I get that but you aren't pulling in the posters or artwork but just the info to pull them.

So what I'm saying is no graphics are loaded until you visit a page where they're needed.  Now what happens if the meta-provider has reordered things again so the URL isn't valid at that point in time?

You've got bad data again.  This is what the plugin was trying to fix and handle differently.

Edited October 24, 2020 by cayars

[freddyfinn 9]

41 minutes ago, cayars said:

You can end up right back in the same spot again later if more URL change (how this happened)  

So what your saying is this could happen again. Well yes it could and it could happen to anyone period. Even you after your library rebuild. 

It can happen as it has happened and because emby has the inability to detect bad links then yes but I will be able to fix it yet again quite swiftly. 

And if it doesn't guess what my people images are working properly and as they should instead of blank boxes and corrupt data which in the interim nobody has introduced a method or straight forward fix for the current issue. 

And yes emby is working the way its claimed to work by grabbing the links and displaying the links. Nothing stops the third party data changing in the future but this is the first time I've had issues since the mediabrowser days. 

If it does happen again a lot of people will be in the same boat again. 

At least with this method everything works as it should and its quick and relatively easy to employ. Could be employed im emby itself. 

Why pissfart around for hours with the plugin that doesn't work and errors out. This could be employed as one button. Clear db and refresh meta data. Not let's wait for months for a fix that might not come. 

When I posed the original question none of you had the answer and you still don't nor is there a fix on the horizon. 

And as a reminder you yourself in your own words "I bit the bullet and rebuilt my library" says it all. 

Emby in it's current state nobody is immune even if you took the long way around rebuilt your library you are not immune. 

 

Edited October 24, 2020 by freddyfinn

[Carlo 4330]

OK so since I've been down this road myself and been there, done that, let me ask.
If I have 80% of my pics showing up. Delete 100% and reload and get 75% back because URLS and messed up did that help or hurt me?

The changes to the DB could hurt you right now as much as help you. You can end up worse off since you wipe existing things that actually have images on the system.

A far better solution is to only touch rows in the DB that aren't correct, but that's a far more complex process.

This type of thing may work ok on a system with minimal data but when you have 250K+ rows of data from movies to episodes it won't be pretty.

So my basic question would be this.

Why should I remove 80% of data that is working correctly to try to gain from that without a guarantee it will improve things?

 I jus t don't sew how this  helps.

 

[PenkethBoy 2063]

I dont have this issue but....

i modified the sql a bit so it's more specific to the problem rather than nuking all images from all items!

UPDATE MediaItems
SET Images = NULL
WHERE type=23

type 23 are people

so this removes the references for all image for just people

running the scheduled task - refresh people - makes emby look again and pulls the images from the cache i believe

Now just doing that may not be enough to fix the problem - but you can add in the other Set X - Null if they are really needed

note if you need to do other media types - say Artists then they are type 13

So..

UPDATE MediaItems
SET Images = NULL
WHERE type=23 or type=13

A library scan brings the Artist Images back

Edited October 24, 2020 by PenkethBoy
typo

[freddyfinn 9]

6 minutes ago, cayars said:

OK so since I've been down this road myself and been there, done that, let me ask.
If I have 80% of my pics showing up. Delete 100% and reload and get 75% back because URLS and messed up did that help or hurt me?

As i have said previously all pictures and library's were returned as they were. No missing images once the full scan was complete.

Gain all people restored. No blank images all covers backdrops tv series as they were including recorded TV. Fail to see the loss. 

You basically did nothing different by reloading everything from scratch except having to go through all the hassle of recreating libraries and rescanning everything. 

15 minutes ago, PenkethBoy said:

I dont have this issue but....

i modified the sql a bit so it's more specific to the problem rather than nuking all images from all items!

UPDATE MediaItems
SET Images = NULL
WHERE type=23

type 23 are people

so this removes the references for all image for just people

running the scheduled task - refresh people - makes emby look again and pulls the images from the cache i believe

Now just doing that may not be enough to fix the problem - but you can add in the other Set X - Null if they are really needed

note if you need to do other media types - say Artists then they are type 13

So..

UPDATE MediaItems
SET Images = NULL
WHERE type=23 or type=13

A library scan brings the Artist Images back

Now that's that's useful.

I was not sure of the image types as it looks like I restored an old db and there seemed to be left overs from previous emby versions. I've done five dbs using my script and everything has restored to where it was and should be and all the new people links have downloaded. 

When I started I manually filtered people images and removed the links however for some reason I still had blanks so I went nuclear and it worked. 

Next db I will use your script. Should work no problem. It shows quickly if it doesn't work.  Blanks will still be there and some old place holders that will only update using the search function manually. 

[Carlo 4330]

This is what I was getting at.  You were updating all rows without a where clause.

But again this doesn't fix the problem only your DB at this point in time. But it can and will have issues again in the future if one meta provider keep changing URLs for it's objects.

[freddyfinn 9]

7 hours ago, cayars said:

This is what I was getting at.  You were updating all rows without a where clause.

But again this doesn't fix the problem only your DB at this point in time. But it can and will have issues again in the future if one meta provider keep changing URLs for it's objects.

Again I know this and while it isn't a permanent solution it is one that can be deployed quickly and restore functionality. Better than reloading everything. And yes fixing the db is the only quick viable option at the moment to get things working again. 

Easily restoring functionality for the time being is better than saying we don't know or try a plugin that runs for days and doesn't work or we are working on a solution that may not get here for months. Customers don't want to hear that. The people I have setup with Emby only want their images displaying as they should.

The where clause is not there because as I said when I did a manual filter and only deleted the people links it did not work. I still had blank on blank images in the search area that didn't update. I therefore didn't pursue using a where clause or looking at the IDs.

Remote desktop, transfer SQLite portable with SQL, run nuclear option Clean db, quick rescan. Emby off-line 2 minutes. 

Happy friends, happy family. 

Done. 

 

 

 

[Carlo 4330]

I guess you still don't see the drastic difference between your sql and what Happy2Play showed you.

If effects far less rows in the db that didn't have a problem and doesn't null out other fields that didn't need changing.

[Happy2Play 8281]

If I were going to null that many items I would just start a new database, as nulling all ProviderIDs basically nukes everything identified.  Then restore all my userdata.

[freddyfinn 9]

You can do what you like. You were as useful at the start of this thread as tits on a man. 

The method worked. Didnt involve reloading anthing including libraries and within two hours everything was restored.

If this is what you call help @Lukeneeds to look at who is actually helping. The end user is me your customer and the fact people sit and wait getting answers like this with no solution is extremely disappointing. You all have the ability to criticise but you never said here run this sql on your database and it will all be good. 

Your thing to do is bitch like old moles about something you could have come up with instead of saying run a plugin thats useless or just delete everything. 

Stellar help when customers fix their own issues. 

 

[Carlo 4330]

Because we don't want other people doing the same thing thinking it's a fix for people/cast the way you were doing it. You' shouldn't NULL those ID but only the image field for people only.  You are orphaning records by nulling the IDs and you shouldn't do that!

We CAN'T fix the issue because it's not OUR ISSUE. The problem is a meta-data provider changing things and make pre-existing URLs invalid. All we can do it try to work around the issue.

Luke has made a couple of changes to help with this but until the meta-data provider resolves/finish these changes it's still going to be a problem.

I don't have an issue with the way you went about this with SQL and editing the database.  My issue was the extra field's your nulling and the fact you nulled every record in the table but only needed to null one type with a where clause and ONLY the image column.

Switch to the SQL Happy2Play gave you and you're good.

Edited October 25, 2020 by cayars

[freddyfinn 9]

10 minutes ago, cayars said:

Because we don't want other people doing the same thing thinking it's a fix for people/cast the way you were doing it. You' shouldn't NULL those ID but only the image field for people only.

We CAN'T fix the issue because it's not OUR ISSUE. The problem is a meta-data provider changing things and make pre-existing URLs invalid.

Luke has made a couple of changes to help with this but until the meta-data provider resolves/finish these changes it's still going to be a problem.

I don't have an issue with the way you went about this with SQL and editing the database.  My issue was the extra field's your nulling and the fact you nulled ever record in the table but only needed to null one type with a where clause.

Switch the SQL Happy2Play gave you and you're good.

You can't fix the issue and it is your issue. Providers change things from time to time and thats a given you have no control over that however emby should be able to flush its links and rebuild its images. The problem with emby is once a link is added it never gets updated during a rescan if data is present in the db. Here's an idea refresh people refresh links. 

I will reiterate this again for the umpteenth time. Removing the people  links only doesn't fix the problem. It still leaves blanks here and there until you physically go into search delete the blank and correct image shows. 

There was another issue this fixed on my fathers machine. He has a custom image for live tv that would just disappear or you would load it and it would be corrupt. You would have to load it over and over until it displayed properly then eventually it would go blank. This is a local image. On top of that one series he had would only show corrupt thumbs no matter how many times they were deleted updated etc. That also now works. 

At the moment fields nulled makes what difference when they are rewritten and work?

Hmm none I have seen.  

Fix let's look at that word. Did something go from not working to working yes. Permenent fix no. Will people like the end user care probably not. They get what they want now and not in six months. Can they rerun it if it happens again yes and If they backup their db they can always restore it. 

 

Only other options use a dud plugin or rebuild everything over and over until a permanent fix becomes available.   I won't hold my breath. Still waiting for a ff/rew fix for Android TV app.

If you think you can just delete the people links go for it. Don't let me stop you. I guarantee there will still be blanks. 

[Carlo 4330]

I'm sorry but you just don't understand what's taken place or you wouldn't be saying what your saying as it's not true. You think you found "gold" because a reset of some fields fixed a current issue for you but you don't understand what that code would do on other systems with much larger data sets. I do which is why I questioned you on updating all rows and orphaning the records.

Imagine for a moment if I'm a wholesaler and give you access to my backend.  You download the master list of product info and the links to pictures for your use.  You go off and build your website incorporating all of this.  While you're doing this we reorganize our database and change IDs of products and do no referral URL mapping on our website. You now have dead links or pictures on your site because of changes I made.  Is this your fault or mine (besides choosing me as vendor)?

Would you accept that, it's your fault? Of course not and you would know the wholesaler messed up and made changes without thinking of how it's customers use the data.  No different here.  We play by the rules they give us and load the graphics on use and if they change from when we grabbed them to when you need them...

This hasn't affected only Emby but all tools using their data! They could/should have left a redirect to the new location of the data but didn't do this.

So I repeat it's NOT an Emby code issue. Emby is trying to handle this, but isn't going to keep wiping things out on your system while the meta data provider is still making all these changes. We'll address it a step at a time as it makes sense. We have to take into consideration the different sizes of system and how long a "wipe" would take on some systems to refresh.  I've got a pretty large system personally and if I ran your SQL on my system I'd be nearly half a week recovering from it.  The size of your library comes into play.  You can't just wipe info on everything when someone has a quarter million videos on their system like I do. But your code didn't wipe just videos but albums and songs as well, actually all graphics.  Add 2 or 3 million rows on top of that as well for music artists.  My personal system would limp along for a good solid week trying to recover from that SQL.

On a smallish system it can rebuilt in a hour or two but on a system with a decent amount of videos/tracks you have a massive problem.

Can Emby handle this?  Surely we can and Luke has already put some code in place to help with this. We refresh people on a 30 day basis so it will improve over time but we can't bombard the provider all at once.

A "fix" or solution just isn't as easy as you think it is especially while the underlying problem is still taking place and likely will till around EOY when they (meta data providers) expect to be done remapping things.

You threw in other issues which aren't related and I don't disagree with a few of them, but they are different issues and there are threads on those.

I'll say it again, if you want temporary relief use your process as a whole but use Happy2Play's SQL. This will not cause DB problems with nulled IDs and will reset only people.

 

 

Edited October 25, 2020 by cayars

[PenkethBoy 2063]

4 hours ago, cayars said:

Switch to the SQL Happy2Play gave you and you're good.

Cough cough

[PenkethBoy 2063]

Wow this escalated quickly

There are two things going on here that need to be separated

1. getting rid of incorrect image references from the db

2. Missing images for people

They are not the same issue - although to an end user they appear the same

for 1 the nulling of the image links with the sql i uploaded with the where clause fixes those images as it makes emby go an look in its cache and reload them.

for 2 the nulling by just image type is not enough as emby never had the images in its cache so they come back blank for those people - i suspect ,as have not tested this, but one of the other null'd fields kicks emby to go and have another look for the images with the refresh people task or a metadata refresh of the movie/tv show.

this is very likely PresentationUniqueKey = NULL or less likely ProviderIds = NULL

Somebody would have to test systematically to confirm my assumptions.

 

However, none of these work arounds are for the average user - thats on @Luke to add extra functionality to address both issues which have been outstanding for sometime. But if people are careful and to some extent adventurous then an enhanced version of the where sql will work - as i would not advise the whole sale nuking of 5 columns of data - as we dont know in detail, as not tested or supported what other implications it would have. If there are other issues that crop up then you are faced with using a backup or starting from scratch each has its obvious downsides.