MOVE LIBRARY while retaining PLAYED COUNTS and ADDED DATES and UPDATE NFO without Re-Download also CHECKSUM

[HTCM842091 0]

Please LIKE THIS POST to UP-VOTE its implementation!!!  Thank You.

Libraries sometimes need to be moved due to unforseen circumstances or optimizations or system migrations, doing so manually often loses the link between the database and the media items, such as play-count loss (all the items are now Unwatched), added-date loss (all items are now considered "latest"), etc.  Suggestions in this post may guide a motivated developer to successfully create and implement a method of LIBRARY MIGRATION.

 

EDIT (this was going to be near the bottom of this post but it should be led with instead, you may figure out CAPITALIZED words contextually by reading below the --- line):  CHECKSUMS would be easiest and BEST way to VERIFY the MEDIA ITEMS missing from an OLD library snapshot are the SAME as found in a NEW library snapshot, and should be used as the COMPARISON METHOD.  This could be expanded to include METADATA IMAGES, TRAILERS, and additional folder contents (assuming items in each folder are classified as relative to that folder's TITLE).  Media items assumed the same would have their paths changed within database, and their METADATA would be verified for checksum match to switch THEIR paths in the NFO files additionally.  Option should be added to "Avoid over-writing NFO files which EMBY did NOT write" to avoid complications.  LOGS should ALWAYS be created and ACTIONS should be made REVERSIBLE.

 

(The remainder of this post may seem confusing or irrelavent, as it was the building-of-thoughts which resulted in what is above)

-----------------------------------------------------------------------------------------------------------------------------------------

CURRENTLY, if moving library contents, the MEDIA FOLDER containing MEDIA ITEM and NFO gets moved, and the paths therein for the metadata files and images becomes UNLINKED (problem 01), when "Update metadata" is selected then the images are re-downloaded or the movie info is re-downloaded and could possibly be erroneous new data if no tt ID exists (or other provider).  The solution outlined below resolves this issue by taking two SNAPSHOTS of the library (OLD and NEW; Before and After, respectively, of a MOVE operation).  If MOVE is simply a directory alteration for each item within a library, then NFO contents should stay the same, so should media item FOLDER contents, and anything relative to the BASE folder of the LIBRARY.  HOWEVER, within each NFO file, contains PATHS which have now CHANGED from OLD to NEW, so THESE are the items which need to be edited while avoiding a re-download of [possibly erroneous] metadata or images. 

Editing a database file manually proved initially to look successful but quickly became apparent that EMBY could no longer update the database with new additions (database error or permissions error created by saving file by a program running under admin maybe?)  I used a database editor from PortableApps, cannot recall which one but atm)

 

User would PAUSE ALL AUTOMATIC LIBRARY UPDATES.

A simple button to backup the CURRENT library (libraries), to compare them with a FUTURE library once MOVE operations have been done.

USER will move the CONTENTS of a library (WITHOUT RENAMING THE PATHS OF THE INDIVIDUAL MEDIA ITEMS).

USER will EDIT the LIBRARY to ADD a NEW FOLDER, optionally removing the OLD FOLDER.

A simple button to COMPARE the two once MOVE operation has completed (from hereon, CURRENT becomes OLD, and FUTURE becomes NEW)

COMPARE will identify what items are MISSING from the OLD library, and what items are FOUND in the NEW library, and show a RESULTS page (for easy approval).

RESULTS page will show ANOMALIES FIRST (Items which do not seem to be in either [lost], or in both [copied not moved], or in NEW only [renamed or new]), followed by NEEDS APROVAL (Items which are SIMILAR from OLD to NEW but do not quite line up) (a mass-approve button would be well-suited here, or checkboxes with a "check all/none/invert" and TASKS to perform on checked items).  Lastly the RESULTS page will show items which MATCHED PERFECTLY (just so users can see, with option to UNCHECK item or whatever).  LOGS of whatever TASKS should be made, in a non-overwriting fashion, path of logs should be provided so USER can manually delete them SERVER SIDE if they feel so inclined. 

TASKS would include an "Export Problems to CSV or other" so someone can manually correct things or run it through an automated script or whatever.

APPROVED items will have DIRECTORIES/PATHS re-mapped within the database, (for each ROW in database, find PATH and UPDATE it in a loop from OLD snapshot to NEW snapshot, this will retain PLAYED boolean, IMPORT DATE, Etc, even across multiple users).  NFO of each APPROVED item will then have its contents reviewed, and ONLY PATHS will be updated if necessary between OLD and NEW snapshots.

USER WOULD THEN RESUME ANY PAUSED AUTO-WATCHED FOLDER SCANNING.

 

THIS METHOD OF COMPARE WILL HAVE LIMITATIONS, being that RENAMING of MEDIA ITEMS is NOT supported.  Additional methods would need to be added for ALTERNATIVE DIFFERENCES to seek out and use to re-map NEW items to OLD items. 

 

PERHAPS (Additional Improvement Thoughts):

An option to run a CHECKSUM task could be added to the base functionality of EMBY, such that a USER may CHECKSUM the MEDIA ITEMS of [FOLDER/PATH/LIBRARY/CHECKBOXED-ITEMS/SELECTION (Basically add a SCOPE-OF-OPERATION limitation so as not to overwhelm a user if time is limited for transfer) with option to "INCLUDE RELATED FILES WITHIN EACH MEDIA ITEM FOLDER" to also checksum things like SRT/JPG/PNG/SAMPLE/TRAILER/TXT/NFO-ORIG/XML/etc for MORE precision when comparing], or could be added to the basic functions of each Library in ADVANCED SETTINGS to "Run a checksum on NEW media items" or a "Run a checksum on ALL media items within X".  CHECKSUM for each MEDIA ITEM would then be added to the DATABASE (and could possibly even be used as Unique Keys though it would eliminate the ability to run a DEDUPLICATION task thereupon),  and DEDUPLICATION could be performed (or at least the duplicate items LIST could be exported and a USER could manually do some things in their filesystem and then run a library update).  If the OLD snapshot is created WITH the CHECKSUMS, then the NEW library FOLDER could be SCANNED and CHECKSUMMED, the CHECKSUMS could then be run through the OLD snapshot to IDENTIFY MATCHES between the OLD snapshot and NEW snapshot, and UPDATE the PATH(s) within the DATABASE creating a correct pointer to the NFO file which will also be updated with correct pointers to the NEW paths of the METADATA ITEMS.

 

ADDITIONAL THOUGHTS:

NFO files would not need path updates if RELATIVE paths were used within them, RELATIVE meaning in relation to the NFO actual PATH (or PRECEIVED PATH for Network shares and SymLinks or Libraries), all METADATA files would then be RELATIVE to whatever PATH the USER is viewing the MEDIA ITEM FOLDER from.

if COPYING (rather than MOVING) from one FOLDER to another FOLDER within the SAME library, then updated, the LIBRARY should see two VERSIONS of the same item, then the ORIGINAL can be removed or deleted, and the library updated again, then the library would only see ONE version and likely retain PLAY-COUNT and ADDED-DATE but may still try to place the metadata for the NEW path BACK into the OLD path (Images showing up in a directory which contains no media items), MIGHT lose play-count and possibly added-date, and play-count-per-user info.  COPYING may not be available in all situations, especially if a Library is simply being changed from "X:\LONG\PATH\TO\LIBRARY\BASE\FOLDER\MOVIES\" to "X:\MOVIES\".

 

 

Apologies if this has been suggested already, I tried a forum search but after three pages of irrelevant posts I gave up looking through its results.

Edited June 11, 2021 by HTCM842091