best boy 0 Posted December 23, 2018 Share Posted December 23, 2018 (edited) Hi, I'm trying to migrate my emby server instance to a new host and I don't seem to be able to migrate my users. I followed the migration instructions of the wiki and read this related thread. It's not working, tho. Here's what I did: // on old host # copy everything over to the new host scp -r /var/db/emby-server root@new-emby-host:/var/db/emby-server-old // login to new-emby-host cd /var/db # make sure we start from scratch rm -rf /var/db/emby-server # remove unnecessary metadata rm -rf emby-server-old/metadata/views/ rm -rf emby-server-old/metadata/library/ # first startup service emby-server start //check web-ui; wizard comes up - good service emby-server stop # check if we got a new working directory ls -la emby-server # restore backup files into working directory cp -r emby-server-old/config/ emby-server/config/ cp -r emby-server-old/plugins/ emby-server/plugins/ cp -r emby-server-old/data/playlists/ emby-server/data/playlists/ cp -r emby-server-old/data/collections/ emby-server/data/collections/ cp -r emby-server-old/data/devices emby-server/data/devices cp -r emby-server-old/metadata/ emby-server/metadata cp emby-server-old/data/users.db emby-server/data/ cp emby-server-old/data/userdata_v2.db emby-server/data/ cp emby-server-old/data/displaypreferences.db emby-server/data/ cp emby-server-old/data/devices.db emby-server/data/ # omitted for the sake of brevity: # 'userdatas' table of library.db has been merged via 'REPLACE INTO NewDB.userdatas SELECT * FROM userdatas' cp emby-server-old/data/library.db emby-server/data/ service emby-server start At this point the configuration is restored, but the users and devices are missing. I'm logged in with the only existing user called "emby". Now I can set the libraries up as they were with the same paths as on the old system, but this does not change a thing to the users. I already checked the content of the users.db file and all the users are located in the 'LocalUsersv2' and 'users' table. All users are local users btw. Any help would be appreciated. Edited December 23, 2018 by best boy Link to comment Share on other sites More sharing options...
mastrmind11 717 Posted December 23, 2018 Share Posted December 23, 2018 so the backup plugin didn't work? Link to comment Share on other sites More sharing options...
best boy 0 Posted December 23, 2018 Author Share Posted December 23, 2018 I don't know. It's hidden behind a paywall. Link to comment Share on other sites More sharing options...
mastrmind11 717 Posted December 23, 2018 Share Posted December 23, 2018 I don't know. It's hidden behind a paywall. $5 seems like a small price to pay to find out. Link to comment Share on other sites More sharing options...
Luke 37119 Posted December 23, 2018 Share Posted December 23, 2018 Hi, are you sure you migrated users.db? That's where the users will come from. Thanks. Link to comment Share on other sites More sharing options...
best boy 0 Posted December 23, 2018 Author Share Posted December 23, 2018 (edited) Yes, I'm positive. I copied it over from the old location cp emby-server-old/data/users.db emby-server/data/ When I check a copy of that file with the SQLite Browser, then I do have rows for all my users in the tables 'LocalUsersv2' and 'users'. The tables are kinda simple with just 3 respective 2 columns: An integer id field, a GUID field and a data field with the actual payload as JSON. It looks good to me. I just seems that emby is not reading that file... Edited December 23, 2018 by best boy Link to comment Share on other sites More sharing options...
Luke 37119 Posted December 23, 2018 Share Posted December 23, 2018 Are the old and new server versions different? That will probably explain this if that's the case. Link to comment Share on other sites More sharing options...
best boy 0 Posted December 23, 2018 Author Share Posted December 23, 2018 (edited) Nope, both - the new and the old instance - are version 3.5.3.0. The old one has been created a long time ago and has gradually been updated to this version. The new instance is a fresh install of this version. Edited December 23, 2018 by best boy Link to comment Share on other sites More sharing options...
Luke 37119 Posted December 23, 2018 Share Posted December 23, 2018 Can you attach a server log from the new installation? Thanks. Link to comment Share on other sites More sharing options...
best boy 0 Posted December 24, 2018 Author Share Posted December 24, 2018 (edited) Sure. Please find attached a server log file from the very first startup, and the first startup after the fact that the backuped files have been replaced embyserver.first.startup.txt embyserver.startup.after.replacement.txt Edited December 24, 2018 by best boy Link to comment Share on other sites More sharing options...
Luke 37119 Posted December 24, 2018 Share Posted December 24, 2018 This is where the new db files should be going: /var/db/emby-server/data Link to comment Share on other sites More sharing options...
best boy 0 Posted December 24, 2018 Author Share Posted December 24, 2018 yep, that's where it is at: root@emby:/var/db # ls -la emby-server/data/ total 45174 drwxr-xr-x 7 emby emby 30 Dec 24 09:49 . drwxr-xr-x 10 emby emby 10 Dec 24 09:48 .. -rw-r--r-- 1 emby emby 4096 Dec 24 09:44 activitylog.db -rw-r--r-- 1 emby emby 32768 Dec 24 09:49 activitylog.db-shm -rw-r--r-- 1 emby emby 8272 Dec 24 09:44 activitylog.db-wal -rw-r--r-- 1 emby emby 4096 Dec 24 09:44 authentication.db -rw-r--r-- 1 emby emby 32768 Dec 24 09:49 authentication.db-shm -rw-r--r-- 1 emby emby 28872 Dec 24 09:44 authentication.db-wal drwxr-xr-x 6 emby emby 6 Dec 24 09:48 collections -rw-r--r-- 1 emby emby 243 Dec 24 09:44 connect.txt -rw-r--r-- 1 emby emby 35 Dec 24 09:44 device.txt drwxr-xr-x 48 emby emby 48 Dec 24 09:48 devices -rw-r--r-- 1 emby emby 360448 Dec 24 09:48 devices.db -rw-r--r-- 1 emby emby 32768 Dec 24 09:48 displaypreferences.db -rw-r--r-- 1 emby emby 32768 Dec 24 09:49 displaypreferences.db-shm -rw-r--r-- 1 emby emby 12392 Dec 24 09:44 displaypreferences.db-wal -rw-r--r-- 1 emby emby 121675776 Dec 24 09:48 library.db -rw-r--r-- 1 emby emby 32768 Dec 24 09:49 library.db-shm -rw-r--r-- 1 emby emby 251352 Dec 24 09:44 library.db-wal drwxr-xr-x 2 emby emby 3 Dec 24 09:48 playlists drwxr-xr-x 2 emby emby 4 Dec 24 09:44 ScheduledTasks -rw-r--r-- 1 emby emby 4096 Dec 24 09:44 sync14.db -rw-r--r-- 1 emby emby 32768 Dec 24 09:49 sync14.db-shm -rw-r--r-- 1 emby emby 32992 Dec 24 09:44 sync14.db-wal drwxr-xr-x 2 emby emby 5 Dec 24 09:49 SyncData -rw-r--r-- 1 emby emby 2166784 Dec 24 09:48 userdata_v2.db -rw-r--r-- 1 emby emby 28672 Dec 24 09:48 users.db -rw-r--r-- 1 emby emby 32768 Dec 24 09:49 users.db-shm -rw-r--r-- 1 emby emby 12392 Dec 24 09:44 users.db-wal -rw-r--r-- 1 emby emby 23 Dec 24 09:44 wan.dat root@emby:/var/db # od emby-server/data/users.db > new.as.txt root@emby:/var/db # od emby-server-old/data/users.db > old.as.txt root@emby:/var/db # diff -q new.as.txt old.as.txt root@emby:/var/db # diff -q new.as.txt /var/log/messages Files new.as.txt and /var/log/messages differ Link to comment Share on other sites More sharing options...
best boy 0 Posted December 25, 2018 Author Share Posted December 25, 2018 OK I finally managed to migrate my instance. I figured it out, when I took a look in a users.db file right after finishing the startup wizard and creating a test user. To my surprise the users.db file is empty then: no tables - no data - nada. So the issue is kind of that the instructions in the wiki are incomplete. It is mandatory to also copy the old users.db-shm and users.db-wal files over. Then the user database is accessible to the new instance. It is however important not to copy the .db-wal and .db-shm files of the other database files. Just the ones for the users.db or otherwise the new instance won't start up. I've attached my file restore shell script that I used for testing for future reference. It handles the copying part of the migration process in case anyone is interested (just remove the .txt file extension as .sh files are not allowed to be attached). restorefiles.sh.txt Link to comment Share on other sites More sharing options...
Luke 37119 Posted December 25, 2018 Share Posted December 25, 2018 Hi, actually no, it's not necessary to copy those files. The changes we written to the shm and wal files first, then get synced to the db later. But if you shutdown emby server first, then they get written and those two files are deleted. So the wiki assumes that Emby Server is not currently running. That's what we'll need to update to make clear. Thanks. Link to comment Share on other sites More sharing options...
best boy 0 Posted December 25, 2018 Author Share Posted December 25, 2018 The .shm and .wal files are never deleted when I stop an emby instance. Is this really supposed to happen? Link to comment Share on other sites More sharing options...
best boy 0 Posted December 25, 2018 Author Share Posted December 25, 2018 hmm.. looks like SQLITE_DEFAULT_WAL_AUTOCHECKPOINT is set to 1000 pages by default. If the application were not to trigger checkpoints manually, then I could see why the users.db was still empty. Creating the most simplest test user probably does not amount to 1000 dirty pages and thus does not trigger the WAL flushing via auto check points. Link to comment Share on other sites More sharing options...
Luke 37119 Posted December 26, 2018 Share Posted December 26, 2018 How are you stopping emby? Link to comment Share on other sites More sharing options...
best boy 0 Posted December 26, 2018 Author Share Posted December 26, 2018 I used to stop it from the command line withservice emby-server stop I now redid the installation and changed these things to my initial approach: I used the "shutdown" button from the dashboard to stop the instances. I did not copy the .db-shm and .db-wal files this time. I just copied the .db files as mentioned in the wiki. Contrary to what the wiki says, I finished the installation wizard on first startup and set up the libraries with it. As my "watched data" was missing in the first run, I assumed that there was an issue with the migration of the library.db file. So this time I changed the SQL query to use a case-sensitive table name like: REPLACE INTO NewDB.UserDatas SELECT * FROM UserDatas Now, everything seems to work fine. Link to comment Share on other sites More sharing options...
Luke 37119 Posted December 27, 2018 Share Posted December 27, 2018 Ok, stopping that way is not a graceful shutdown, so that's why that happened. If you were to instead stop using the server dashboard, then it would have been as i described. We'll look into intercepting service stop to handle this better. Thanks ! Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now