Question about replace existing images

[Deathsquirrel 741]

The issue it the re-indexing specific content types and users that don't store with media.  All the items get new itemids so all the existing media in the previous itemid folder is invalidated and has to be re-acquired just like chapter images as they existed in the folder that now gets deleted do to it no longer being a valid itemid folder. 

 

Every time there is a major change in the database or you have a database corruption and you do not save with media you will have this issue.

 

So all users that do not save metadata and images with media need to ensure they backup there entire Emby-Server folder to prevent issue like this as it will happen again in the future.

 

Example 50012 of why you should REALLY store your metadata with your movies.

[Luke 37116]

Does it say delete or Replace? so if we are to understand it correctly.... replace is 1:1 delete isnt. For a person that obviously doesnt speak english as a first language you really should consider his point of view. but that all aside, the real problem here was the issue with 4.0.0.2 to begin with. the delete and replace is all syntax and really just added frustration to the problem caused by the rushed update.

 

I'll update the help text.

[tillithz 1]

7 days? Thats it? You weren't ever a Plex user were you? Those guys wait at least a year for a bug fix. 7 days is pretty darn good. Its not like it wasn't beta tested. There were something like 86 versions of the beta over the past 5 or so months. 

 

Edit: The replace function DID NOT CHANGE with 4.0. Its been that way for a long time. Stop blaming an update for a user error. He would have done the exact same thing on 3.5.3.

yes 7 days! Being a Plex user has nothing to do with any of this...well except if coming to Emby was to improve..... Being a Plex user is no excuse for a user to make setting changes on things they dont understand nor a developer releasing such a flawed version. Considering this wasnt a simple ooops, but a huge blunder that should of been easily identified in the past 6 months. But to dismiss the headache that its caused when people have thousands and thousands of movies/media is short sighted.

 

I will point out that the op wouldnt of done the same thing, in fact he DIDNT do the same thing, he was already on 3.5. IT WAS SOLELY BECAUSE OF v4. that he went and poked around and clicked things that werent clear to him. May be clear to us, to you, to many people but many of us dont want to have to stay up on this stuff. If Emby is designed for the developer or beta tester or even the hobbyist that needs to devote daily time to keeping up with updates and configuration then that should be stated. But then, I imagine many users wouldnt support the developers or even use Emby. 

 

a side note: I do think the appalled comment earlier was a bit far, I was upset as well and its irritating to see thousands of movies be "messed up" and when you post only to get a single sentence response. Then, think about how this issue could of..... SHOULD of .... been easily avoided, it only makes the frustration worse. But, in the end the developers/Luke stepped up and helped with what appears to be the best they could. So thank you for that. It cant be easy to keep composure when so many are frustrated that their media appears to be messed up. maybe thats why the short responses.... easier to apologize for not saying something then saying the wrong thing (or saying too much like this post lol) anyways, I'm done it can be discussed without me. theres life to live. 

[Carlo 4330]

7 days is extremely fast to get updates.  You're not going to change people's mind around here about that, period.  We've dealt with the competition (multiple vendors) and just know better.  Plex is still dealing with it's last client (not even a server update) 60 days after it's release and it's "being looked into" and the "devs are talking about ways to fix/change it".  Not even working on the coding solution yet.  So 7 days out the door with fixes is awesome!!!  Besides Plex, no other vendor really has anything close to Emby or it's functionality.  This really is a 2 vendor market (at this level).

 

But as has been pointed out the basic functionality you call a blunder is the proper functionality and it does what it says it's going to do. 

 

Moving forward I'd suggest:

 

First just as good practice if you had your environment (programs and data) backed up then even the worst of updates could easily be countered by restoring the folders that got changed putting the system back the way it was.  It's too easy these days to back things up that there is no excuse to not do it.

 

Take some time and read the WIKI start to finish (it's not that big or that many pages) to learn the different features and functions of the server so you can make informed decisions without trying to guess what a function does.  It's the old saying I'll just abbreviate as "RTFM". Anything you don't understand, come to the forums and ask.  You WILL get quick responses to these types of questions and if the information is not clear the wiki will get updated to make it more clear.

 

Most importantly the 4.0 upgrade is working great for 99%+ of users.  Overall there are very few users in the scheme of things with problem that haven't been addressed.  Most of the initial issues reported were from people jumping the gun and not letting the processes of the upgrade to take place which can take time. They thought things were broken but they just weren't finished yet.

[hermann_x5 0]

I’ll try to explain myself a last time – only partially because I need to win an internet argument J, but mostly because maybe this can help make emby even better and to avoid surprises on some user’s parts.

 

 

So, first of all:

 

 

1.) Why did I even chose “replace all images” ?

 

I did it because after the update all my cover pictures were gone in emby (without me doing anything, just updating) and replaced with either a thumb or fanart. Do not know which, but clearly all posters and covers were gone. I thought this function would repair my emby pictures.

 

2.) What did I think would happen given the settings that I had chosen (which were: 1. Not store artwork with media and 2. Store the metadata in a non-default directory)

 

• Emby would either
  • parse my artwork (because I have the feeling that this is what emby does normally - which is great since I carefully chose my artwork and also carefully fill my nfos with the tool MediaElch and so I was always happy that emby just takes the info from my nfo and also uses my artwork. Which btw. Is the much, much better that what plex does )
  • or scrap artwork from the internet
  • and replace it’s own artwork (which was missing because of the most likely faulty update and which I thought would be stored in the non-default folder I had chosen)

 

Clearly, all of this did not happen in the way I imagined it would.

 

So,

1. yes there was a problem with the update because part of my metadata was missing from emby
2. I then did something “wrong” which in my opinion should be handled in a different way by the program or explained more clearly in the UI (not only in a Wicki)
3. This led to my own artwork being deleted without being replaced (there was no artwork in emby or on my disk anymore !)
4. This led to me panicking and asking for help in this forum and not getting too much of it except explanations, what I did wrong. Thanks, for that J

In the end it’s (mostly) ok:

1. As I explained: I had everything backed up, so except for an initial panic and some work, all was fine.
2. I now know how to handle this and
3. I think that the fixes that were rolling out quickly (thanks for that) helped
4. The problem that I still have that was introduced also by the update is that emby always wants to install VC++ 2015 but cannot because 2017 is already installed.

[Statick 9]

bumping this as I've had the same problem. The terminology is confusing and there is no warning or indication that data will be permanently deleted.

 

I replaced some folder.jpg files and wanted emby to refresh the thumbnails. I wanted to avoid a full library scan as this takes a very long time and I only needed to refresh like 5 thumbnails. There doesn't seem to be a way to refresh individual thumbnails, so I refreshed metadata on the library in question and told it to "replace all images" because I wanted existing images to be replaced with new ones.

 

To the end user it seems as though image files are scanned and imported into a database or cache of some kind - so you think "replace all images" would do another scan and replace the images in the database or cache, with the new images found on disk.

 

This is clearly not what happens, and it's only because of this thread that I now understand what happens!

 

I have just lost a lot of data with no warning or explanation. It's all replaceable but will take a long time.

 

These options should be better described - choosing "replace all images" is potentially destructive and so it should pop up a warning "This process will replace all image files at '//server/share/library/' with newly generated images - proceed? yes/no". It should never be possible for a user to lose data simply because they do not understand what a scan is about to do.

 

Also a way to explicitly refresh thumbnails on their own would be really helpful - other media servers have this feature.

[Luke 37116]

We can look at improving it in future updates. thanks for the feedback.

[webjunky 1]

I'm just sick right now. I thought it would replace my posters with updated ones but it deleted all of my fanart that I painstaking downloaded over months and months of work is gone.  It didn't even put the deleted items in the trashcan where I could "undelete" them. They are just totally gone.   Had the toggle been "Delete all existing poster, background, fan art, etc. and hopefully download a few new ones" then I would have never touched it!

Bottom line is REPLACE means just that--REPLACE. Your app didn't do that. You app DELETED all artwork from my media folders and then attempted to download new content. It should only replace, not delete. 

In tears here and really wondering if I can go through the effort with Emby to go to fan sites and download my posters and artwork again. 

This problem was brought to your attention over 2 years ago and you still didn't change the terminology or a warning that all existing content will be permanently deleted and unrecoverable. That had to be a fairly minor coding thing to accomplish and you chose to let more new users commit the same mistake. Very disappointed. 

My reasoning is the same has HermanX5 above, by the way.  I built a new computer, installed Windows 11, moved over my raid control and drives, installed a fresh install of emby and loaded my library.  After 2 days none of my movie's posters where showing in the library, or any of the fanart---but they were saved in the media folders---they were right there.  So I asked Emby to replace the image (replace the images in the cache, which were none, with the images on my hard drive.)  I'm really shocked the app deleted media without warning.  Cache's can be rebuild, but delete a file from the drive... without warning... not cool. 

Edited February 7, 2022 by webjunky

[Luke 37116]

On 2/7/2022 at 3:54 PM, webjunky said:

I'm just sick right now. I thought it would replace my posters with updated ones but it deleted all of my fanart that I painstaking downloaded over months and months of work is gone.  It didn't even put the deleted items in the trashcan where I could "undelete" them. They are just totally gone.   Had the toggle been "Delete all existing poster, background, fan art, etc. and hopefully download a few new ones" then I would have never touched it!

Bottom line is REPLACE means just that--REPLACE. Your app didn't do that. You app DELETED all artwork from my media folders and then attempted to download new content. It should only replace, not delete. 

In tears here and really wondering if I can go through the effort with Emby to go to fan sites and download my posters and artwork again. 

This problem was brought to your attention over 2 years ago and you still didn't change the terminology or a warning that all existing content will be permanently deleted and unrecoverable. That had to be a fairly minor coding thing to accomplish and you chose to let more new users commit the same mistake. Very disappointed. 

My reasoning is the same has HermanX5 above, by the way.  I built a new computer, installed Windows 11, moved over my raid control and drives, installed a fresh install of emby and loaded my library.  After 2 days none of my movie's posters where showing in the library, or any of the fanart---but they were saved in the media folders---they were right there.  So I asked Emby to replace the image (replace the images in the cache, which were none, with the images on my hard drive.)  I'm really shocked the app deleted media without warning.  Cache's can be rebuild, but delete a file from the drive... without warning... not cool. 

Hi there, apologies for the disruption. We'll improve the help text underneath the option. Thanks for the feedback.

[wkl_lau 0]

@LukeI actually have the same problem recently. I have a library using the type "Home video and photo" (because I have mixed content), in which there is a folder structure like this:

Main folder -> [Video title] -> ep1, ep2, ep3, etc

Each ep folder has the video (named ep1.wmv) and a picture (ep1.jpg)

I DID run the refresh metadata and checked the box "Replace existing metadata"

After that SOME of the pictures are gone.

I think the problem is the "Video image fetcher" (I don't see any metadata downloader for "Home video and photo"). It SEEMS that it can't tell the ep1.jpg is actually user data rather than metadata (since it has the same name as the video itself).

I have to bulk rename the picture to something else to avoid being overwritten. I am still testing it. Luke if you can give us some detail of the logic of how the metadata is generated and if we should avoid having the picture using the same name as the video itself (or any other sensitive word like "cover.jpg", it will be much helpful.

I don't think Emby will somehow delete user data, just the user need to have the proper expectation of how it works.

 

 

[Luke 37116]

So you didn't want it to replace the image even though you checked the box?

[wkl_lau 0]

7 hours ago, Luke said:

So you didn't want it to replace the image even though you checked the box?

I didn't want to replace the image because I consider those as user data (I generate those picture, not by Emby). But when I check the box and refresh, it remove my picture without generate anything new.

 

So I am suspecting there are 2 problems:

1. Emby thought the picture is metadata since it has the same name as the video

2. Emby first remove the picture without generate any new pictures. (I tried it with the box "safe pictures in local folders" checked and also tried it without. Both give same result)

[Luke 37116]

Right currently there is no distinction. That function will replace all.

[wkl_lau 0]

@LukeIs there any specify keyword that make Emby identify the image as metadata instead of user data? Like it has the same name as the video?

Or it just delete all the image it can find in the library since it assume they are all metadata (I highly doubt that since I have a folder with image and it was not touched)

[Luke 37116]

6 hours ago, wkl_lau said:

@LukeIs there any specify keyword that make Emby identify the image as metadata instead of user data? Like it has the same name as the video?

Or it just delete all the image it can find in the library since it assume they are all metadata (I highly doubt that since I have a folder with image and it was not touched)

When images are imported we don't keep a record of where they come from, so no, not currently. Maybe we could have a lock for images like we do with other fields.

[wkl_lau 0]

Ok @Luke this is what I have observed so far:

When using "Home videos and photos" If the folder that contains the video also has an image, then screengraber will not try to grab a screen from the video to use it as primary image of the folder. Instead the following will happen:

1. If the image has the same name as the video (video.wmv/video.jpg) it will be used as primary image of the folder, but if you trigger a refresh metadata + replace existing image check box, then it will be deleted regardless it is user data or not

2. If the image has different name as the video (video.wmv/front.jpg) then it will not be used as primary image, and screengraber will not attempt to grab an image from the video either, so your folder will have the plain look.

I am using Emby 4.7.10.0 on TrueNAS-13.0-U1.1, Emby User (UID 989) has full permission to the library, but files are owned by root.

[Luke 37116]

Quote

 

1. If the image has the same name as the video (video.wmv/video.jpg) it will be used as primary image of the folder, but if you trigger a refresh metadata + replace existing image check box, then it will be deleted regardless it is user data or not

You are correct.

[Luke 37116]

Quote

2. If the image has different name as the video (video.wmv/front.jpg) then it will not be used as primary image, and screengraber will not attempt to grab an image from the video either, so your folder will have the plain look.

Yes it will, but it depends on the name of the image and whether or not the folder contains any other videos. If the video is the only file in the folder, then the image can be named poster, cover, default, folder, or movie.

[wkl_lau 0]

18 hours ago, Luke said:

Yes it will, but it depends on the name of the image and whether or not the folder contains any other videos. If the video is the only file in the folder, then the image can be named poster, cover, default, folder, or movie.

@Luke But now the issue is screengrabber does not take any screenshot for the folder icon. The only way to do this (for me) is to remove all image from the folder...
I am seeing this in the folder view. All folder is plain unless I trigger a metadata refresh with the replace option checked...

This is reproducible, you can try it on your side

[Luke 37116]

3 hours ago, wkl_lau said:

@Luke But now the issue is screengrabber does not take any screenshot for the folder icon. The only way to do this (for me) is to remove all image from the folder...
I am seeing this in the folder view. All folder is plain unless I trigger a metadata refresh with the replace option checked...

This is reproducible, you can try it on your side

Hi there, can you please provide a specific example? thanks.

[wkl_lau 0]

48 minutes ago, Luke said:

Hi there, can you please provide a specific example? thanks.

OK, first create a TrueNAS (13.0-U1.1) VM, install Emby 4.7.10.0, mount the filesystem so Emby can access the media files directory and full control permission for user 989

Then in the media files directory, create 3 directories, namely movie_1, movie_2, and movie_3

In movie_1, put a video file (ex: video_1.wmv), a picture (1.jpg) and another picture (front.jpg)

In movie_2, put a video file (ex: video_2.wmv), a picture (video_2.jpg - notice the same file name as the video)

In movie_3, put a video file (ex: video_3.wmv) and nothing else

Then create a new library using the "Home videos and photos" type, add the media directory to it.

After the initial scan, go to the Folder view of your library, you should notice the following:

Movie_1: Folder is plain, no primary image

Movie_2: video2_jpg is used as folder's primary image since it has the same name as the video, but if I trigger a metadata refresh with replace option checked, the picture will be deleted (even if that picture belongs to me)

Movie_3: screengrabber took a screenshot and use it as folder primary image

So my issue is:

1. In movie_1, the presences of pictures with different filename block screengrabber to add folder image

2. In movie_2, picture with same file name is used as primary image of folder, but prone to be delete during metadata refresh

3. The only successful case of screengrabber, when the directory has only the video file

 

 

 

[Luke 37116]

Quote

2. In movie_2, picture with same file name is used as primary image of folder, but prone to be delete during metadata refresh

Hi, that's what replace existing images does, and then it generates a new one. if it did not, then let's look at an example.

How to Report a Problem

Thanks.

[Happy2Play 8296]

On 12/6/2022 at 10:42 AM, wkl_lau said:

2. In movie_2, picture with same file name is used as primary image of folder, but prone to be delete during metadata refresh

Would think that is expected behavior as properly name primary image would be deleted with replace images option.  

But yes would like to see real examples as not all of that really makes senses.

Now if I am understanding correctly in "Home Videos"

1 becomes a folder do to multiple images and videos (first image becomes folder image)

2 is a item that uses proper named item image

3 is a item the creates screen grab.

Limited test

 

[wkl_lau 0]

@Happy2Play are you checking this under folder view?

Also, according to your signature, you are using windows. Are you testing this with TrueNAS?

Thx

[wkl_lau 0]

12 hours ago, Luke said:

Hi, that's what replace existing images does, and then it generates a new one. if it did not, then let's look at an example.

How to Report a Problem

Thanks.

Hi @Lukethank, depending on the filename of the image, if it is identical to the video, it will be deleted (as expected) but nothing is generated.

The only case where a folder primary image is generated is when there is no image in the folder, or when the image has the same filename as the video

Are you observing the same thing? (Make sure you are in folder view and testing is done on TrueNAS)

Thx