Changing Path of Library to new one without Rescanning

[roshanconnor 0]

Hello there

Like My Title suggest I would love to Change the Folder path of My library without rescanning
The issue is that I have around 4500 TV shows in my Library and 15000 Movies
So its really a pain in the ass to change the folder path and let it scan for days

Last time it took around 5 days for me to scan the whole thing

Is there anything I could try by tinkling with Database and if so which File should I edit?
Thank you

PS : My Old Movies Path -

/mnt/media/Movies

     My New Movies Path -

/mnt/media/Library/Movies

        My Old TV-Show Path -

/mnt/media/TV-Shows

       My New TV-Show Path -

/mnt/media/Library/TV

 

      

Edited November 4, 2020 by roshanconnor

[Luke 37054]

HI, you might be able to edit the database to accomplish this, but we don't currently have a documented way of doing this.

[roshanconnor 0]

1 hour ago, Luke said:

HI, you might be able to edit the database to accomplish this, but we don't currently have a documented way of doing this.

Thanks for responding
I have seen some posts mentioning the same with Plex and how its done
Could u suggest which File I should be editing

Thanks

[Happy2Play 8279]

1 minute ago, roshanconnor said:

Thanks for responding
I have seen some posts mentioning the same with Plex and how its done
Could u suggest which File I should be editing

Thanks

Updating the library.db and the paths in your libraries options.xml (Emby-Server\programdata\root\default) the path will be slightly different on each platform.

Ideally maintain a backup of your entire programdata folder incase something goes wrong.

[roshanconnor 0]

2 minutes ago, Happy2Play said:

Updating the library.db and the paths in your libraries options.xml (Emby-Server\programdata\root\default) the path will be slightly different on each platform.

Ideally maintain a backup of your entire programdata folder incase something goes wrong.

Thank you

Will try this and update here

 

[Guest topbanana]

I have this exact same issue.
I have libraries that have changed HDDs due to running out of space...  The items, dates and data structure remains exactly the same, just the disk letter changed....

I'll play with 'DB Browser for SQLite' to achieve this i guess....  tips???

Is the path stored in full for each and every library item?  Full path, not a relative path?
Could/should this instead have the root path(s) to the library set as a Variable(s), and each item having relative paths to that variable?
Then a location change could just mean the variable being updated, and at most a restart of emby server perhaps.

I can imagine this is a fairly common occurrence and one that will never stop... until the infinite capacity HDD comes along!

[Luke 37054]

10 hours ago, topbanana said:

I have this exact same issue.
I have libraries that have changed HDDs due to running out of space...  The items, dates and data structure remains exactly the same, just the disk letter changed....

I'll play with 'DB Browser for SQLite' to achieve this i guess....  tips???

Is the path stored in full for each and every library item?  Full path, not a relative path?
Could/should this instead have the root path(s) to the library set as a Variable(s), and each item having relative paths to that variable?
Then a location change could just mean the variable being updated, and at most a restart of emby server perhaps.

I can imagine this is a fairly common occurrence and one that will never stop... until the infinite capacity HDD comes along!

The library paths for media items are stored as full paths.

[Guest topbanana]

On 14/01/2021 at 02:07, Luke said:

The library paths for media items are stored as full paths.

To enable changing paths to libraries, relative paths will be needed???

Perhaps something to follow in a future DB upgrade?

[Luke 37054]

Yes, something to consider for the future. thanks.

[Guest topbanana]

Having just outgrown another HDD, I've had to add another HDD and split my movie library across them.
Needless to say, emby had to rescan and redownload all the metadata and images for those several thousand items.

Could it be possible to achieve the OPs valid request by adding a '2nd' location to the Library, and tell emby to see if the items that were originally in the 1st location, have moved to the 2nd, and avoid the current, removing and scanning in new items fresh?

If when adding the 2nd location, there could be a button to tell emby that some/all of the items have migrated to that 2nd location.  Emby would then scan the whole library, checking the presence of the items and rewriting the file location as needed.

This part/full migration job might take emby a few minutes, but hey, Migration jobs are often a headache in the IT world...  At least we could have it where it simply takes a little time, but no other effort.

This could be done today?  With emby's db's absolute paths... So without needing the change of the db to relative paths, that might come in the future.

Edited August 21, 2021 by topbanana

[XcOM9876 9]

Not sure for today other than either modifying the dB manually to replace the paths or adding symbolic links between the two paths, but in future you could create a mount location for all of your media then in future when you replace a hdd as long as you mount it in the same location all will be well, I've replaced a few drives now and migrated from local storage to network storage and all has been well using this method

[Guest topbanana]

2 minutes ago, XcOM9876 said:

Not sure for today other than either modifying the dB manually to replace the paths or adding symbolic links between the two paths, but in future you could create a mount location for all of your media then in future when you replace a hdd as long as you mount it in the same location all will be well, I've replaced a few drives now and migrated from local storage to network storage and all has been well using this method

Yeah, I've bought bigger HDDs in the past, and just swapped everything over and given the new HDD the same drive letter...  But alas, they don't make single 2.5" external HDD bigger than 5TB currently...  So for me, i have to split the movie library i have.
There are workarounds, and methods to avoid these problems when starting out, but hindsight is a ...  

With the above idea i have, it would be quite flexible, and allow the full or partial migration from one location to another, however those locations are defined.  Emby would just be tasked with a migration job.  And the db structure wouldn't have to be modified.  It'd just rewrite the new locations within the DB.

As our media collections grow and grow, having to migrate media from one location to another is only going to be more and more common.  This could make it very flexible today.

Emby Server Library Migration Task:
Is this item still in the original location? Yes... Move on to the next...
Is this item still in the original location? No.  Is it in the new location?  Yes.  Rewrite the location. Move on to the next...

Technicalities:
Emby just has to be told not to scan normally when it starts or if it's running, otherwise if the migrated data has already been moved, or is being moved, it'll just trim out those 'deleted' items. Doh!
One way would be to just start emby with the location/disk offline, as emby handles that fine. it just sees it offline and does nothing.  The Migration task can then be defined and it'll wait for the 1st and 2nd locations to come online... Of possibly only the 2nd, if a full migration is needed.  It'll then start the migration scan.
Perhaps if locations can't be taken offline easily, or emby shut down, then the migration job might have to be started first, which will disable that library's scanning, but keep serving media to users (maintaining uptime)...  The data would then have to be physically moved (maybe a few hours), and then the migration job told that it's been moved and to proceed to the location scanning/rewriting.  Any media that emby was serving that gets physically moved will just have to error-out for the viewer...  'Media Offline'... A price to pay for an online migration job.  But it would otherwise be able to serve media from every other library, or the items that aren't being physically migrated.
After the migration scan is complete, emby will resume normal running and resume scanning of media.
If the migration was a full one, the '1st' location, that'd now be empty, could then be removed from the Library.
Can't think of any other technicalities right now...

[XcOM9876 9]

To be honest I think that would be a good option for a future update to assist with migrating to new drives or changing locations for users

[Luke 37054]

Yes it would be. Thanks for the feedback.

[laie_techie 12]

On 1/27/2021 at 4:02 PM, topbanana said:

To enable changing paths to libraries, relative paths will be needed???

Perhaps something to follow in a future DB upgrade?

According to the KB, each Library could have more than 1 file system path; relative paths won't work.

[Guest topbanana]

3 minutes ago, laie_techie said:

According to the KB, each Library could have more than 1 file system path; relative paths won't work.

I think i addressed that...? 

The migration task would check each entry to see if the media was still there or not, only changing the missing media paths. 

This then allows for full OR partial migrations...

For instance, I've just had to split my movie library in half, as they don't make HDDs big enough to hold it all! 

So the migration would leave half untouched, and use the new path for the missing... If it found it there. 

This would allow for any number of paths being used within a library. 

[Guest topbanana]

Just now, topbanana said:

This then allows for full OR partial migrations...

And perhaps the migration task could ask if it's a full or partial migration...

A full migration might not need any scanning of the media.. And just take the word of the Admin that the new path was correct for all of the media...

Though database integrity might dictate it would scan, perhaps. 

[kootain 1]

this code may help, change all '/mnt/tv','/mnt/movie' to '/mnt/data/TV', '/mnt/data/Movie' in emby database

 

# coding=utf-8

import sqlite3

conn = sqlite3.connect('library.db')
conn.row_factory = sqlite3.Row

def colunms_info(table_name):
    c = conn.cursor()
    cursor.execute()

def check_text(table_name, texts):
    cursor = conn.cursor()
    cursor.execute('select * from {};'.format(table_name))
    items = cursor.fetchall()
    cursor.close()
    c = conn.cursor()
    for item in items:
        new_line = {}
        for column in item.keys():
            value = item[column]
            if isinstance(value, str):
                for text, replace in texts.items():
                    if text in value:
                        new_line[column] = value.replace(text, replace, -1)
        if len(new_line):
            params = ''
            args = []
            for k, v in new_line.items():
                params += '{} = ?, '.format(k)
                args.append(v)
            params = params[:-2]
            sql = 'update {} set {} where Id = {};'.format(table_name, params, item['Id'])
            print(sql, tuple(args))
            c.execute(sql, tuple(args))

c = conn.cursor()
c.execute("select name from sqlite_master where type = 'table';")
tables = c.fetchall()
c.close()

for table in tables:
    print(table)
    check_text(table[0], {'/mnt/tv': '/mnt/data/TV','/mnt/movie': '/mnt/data/Movie'})

conn.commit()
conn.close()

[EyeShore 0]

No updates on this, right?  No built-in feature has been added since (in the last year+) and no one is aware of a better method?

[richt 73]

Just a question for possibly an alternative solution, but could you use a symlink to point to the new location for the data?

[Guest topbanana]

We gotta pray that the next database structure update includes this...  It really, really, really should.  Please?!

Hindsight is wonderful.
Having the 'Folders'/root paths for each library as variables, with all media items location relative to them, is just the most obvious way that this should be coded.  It just makes so much sense.  Having every item listed with absolute paths is just not how we should do anything like this.
Especially with our exact application, a database referencing massive media files and associated little files, it's a given that at some point, that data is gonna move... its path is gonna change.  Disk space runs out, HDDs are replaced with NASs, old crappy naming schemes are tidied up... Data is always gonna grow and move around.
Having the paths to each library as variables just simply allows us move stuff around to our heart's content.
I've only played with databases a bit many years ago, but i don't even think it'd be that much of a job to code the change and to migrate everyone's libraries over to the new structure... It's simply a find & replace job.

Just think how much smaller the database will be when those thousands and thousands of entries to the same path are replaced with a variable.
And it'll also take some strain off of our beloved metadata providers!  As we won't be hammering them after a needless library rebuild. (especially for us that do not store nfo files lol).
It's just a no brainer to do it.
It's a shame it wasn't implemented this way from the beginning, but hay ho, this is just how it is.  Hindsight again...

Hopefully it'll come soon.

And then moving media around on our systems then becomes a very trivial task.

Fingers crossed!