Additional Options for metadata folder and storing images etc. next to media

[Clackdor 109]

Basically the idea is to expand on the current options to either use the metadata folder only (default behavior), store images, BIF files, NFO files, etc. next to media and optionally whether or not to copy that back to the metadata folder (essentially using the metadata folder as a read cache). The current options/behaviors could be put into a drop down menu along with the aditional options.

The first additional option would be an inverse of the current option to make a copy of images to the metadata folder. Basically the metadata folder will be the primary location to write and read images etc. from, but any changes will still be copied over to the media folders. This makes the metadata folder the "source of truth" and prevents any unwanted alterations/deletions/corruption of images occuring externally as emby could just replace them from the metadata folder. On the initial library creation or new library addition do a one-time copy of images/etc. from the media folder to the metadata folder. Afterwards any further changes will then go from the metadata folder to the library folder and any external alterations will be overwritten by emby unless the item is deleted/renamed, etc. 

The second option I propose would basically use the metadata folder as a write cache, but still use the media folders as the primary location to read from. Like the first option, the metadata folder becomes the "source of truth"  for any changes, but reads should still happen from the library folders. I'll admit this one may be a bit more niche than the first option, but I still see some utility to this assuming it wouldn't cause too much additional complexity or conflicts. 

These options would expand on the existing ones to allow better fine tuning of performance, data flow, and consistency for different storage setups whether it be separate NAS & Emby server, or a single machine with a mix  storage that is pooled, tiered, or split into separate arrays, volumes, etc. I personally use Stablebit Drivepool on my server and have a mix of HDD's and SSD's and NVME drives in various pools and subpools.

I think they would also be useful for others like myself that are running multiple Emby instances pointed to the same library folders for separation, testing, or a variety of other reasons.

[Luke 42077]

Quote

The first additional option would be an inverse of the current option to make a copy of images to the metadata folder.

HI, we have this already with the caching option on each library.

[Clackdor 109]

24 minutes ago, Luke said:

HI, we have this already with the caching option on each library.

 

39 minutes ago, Clackdor said:

The first additional option would be an inverse of the current option to make a copy of images to the metadata folder

The current behavior writes to the media folder first, then caches a copy to the metadata folder, basically just using the metadata folder as a read cache. As an example, with the existing option if I make change to a poster next to the media the version in the metadata folder gets updated with that image. 

The first option I suggested reads/writes from/to the metadata folder as the primary location, but also writes a copy to the media folder. In this situation the metadata folder would be acting as a write cache as well as read cache.  As an example, if the poster stored next to the media gets changed, that change is not seen by emby and will later be overwritten by a libary scan/scheduled task, etc with the poster from the metadata folder. This prevents any unwanted changes being permanently made to images stored next to media by another emby instance, other software monitoring media folders, accidental deletion, etc. 

 

[Luke 42077]

Quote

The first option I suggested reads/writes from/to the metadata folder as the primary location, but also writes a copy to the media folder. In this situation the metadata folder would be acting as a write cache as well as read cache.

I don't think there's really a difference though.

Quote

As an example, if the poster stored next to the media gets changed, that change is not seen by emby

Yes it will because the server captures the date modified timestamp of the file.

[mrmixed 77]

Continuing this thread, I'd like to see an option to allow NFO files (as well as BIF and images, etc.) to be saved in a location other than the media folders.  For my own compulsive reasons (mostly to promote ease of finding files using alternative media players), I want the media folders to stay clean without a lot of superfluous files.  So I prefer to only store video files within the media folders, and am willing to concede subtitle files as well, since they have utility as most other media players can pick them up. 

I really want to save NFO files because I've had very bad experiences where metadata refreshes within Emby have undone hours of configuration (I discovered too late that it's not enough to simply identify a title that Emby misidentified; if you don't lock in the metadata after manual identification, a refresh will destroy it).  But I'd like to be able to allow the NFO files to live in a parallel location managed by Emby, and not in my media folders.  It makes sense to do the same thing for images as well (this would also help prevent similar disasters when my custom image selections got blown out upon refresh when Emby again re-misidentified a title that wasn't locked).  And doing the same for BIF files just seems like a natural progression to keep the original media folders clean.

Ultimately, I have learned to not to ever refresh metadata, but my prior experiences, along with several comments relating to NFO files in this forum, really reinforce the idea that not having an NFO backup is a recipe for eventual doom.  I'd just like to be able to do it neatly, without dumping the NFO files into the media folders, and to place them instead in a parallel structure, whether it's metadata, cache, or something new.