Manual migration issues (namely SSL and Library)

[Garbonzo17 26]

Ok, so this is kinda A LOT.

Sorry to do it this way, but all the searching I have done covered some of my issues, but not everything and I am frustrated.

Initially I was just doing an in-place Windows 10 Reset, so I cloned the bootdrive as a backup and then proceeded with the reset.  Generally I expect to have to reinstall at Win32/x64 programs and was surprised when everything in Emby still appeared to be in place.  The Library files and everything were there as well as the users and everything worked fine locally so I was happy, a few days later I realized that Port 8920 wasnt open, so I assumed it was something with the cert and IIS after the windows refresh, so I removed my cert in IIS created a replacement in Certyfy (default LetsEncrypt settings) and then re-exported the cert with password and updated it in EmbyServer settings, and rebooted the server.  Still no-joy, so I manually closed embyserver and went to re-launch from short-cut in programs and it said the shortcut was no longer valid (seemed weird, because all of the files were in the same locating afaik...) But I thought, OK, I will just re-run the server setup and it should do an inplace update and fix things/keep my data in tact... it didn't. I got a fresh install and the setup wizard... (looking back I get that now, but I hadn't run the downloaded emby setup manually in a LONG time....)

So now with the fresh install I put in my same SSL settings (domain, pfx file, password) and port 8920 is open and I can connect... so I just need to move my data from the cloned hard drive (I assume)... copying the whole programdata folder is apparently no the way to go... the server would just hang on startup... so I followed the info on https://support.emby.media/support/solutions/articles/44001159936-backup.. and on a restart the previous libraries appear, but NO folders connected to them, and when I try to add one I get a message at the bottom of the page "....error adding the media path. Please ensure the path is valid and the Emby Server process has access to that location..." (the locating that hasnt changed since the restore, yet even after I browse to in successfully I get same error. I don't mind recreating these, thats trivial as I have the metadata/images saved to the media folders on the drive pool  where they are located. The USERS and the SSL are all I REALLY need, but the watched data would be helpful....

So did emby use to store the pfx and an iis token file in the SSL folder in programdata?  I have that folder on the cloned drive and I am not sure if that is a legacy artifact or something I created to keep my files all in one place previously... regardless, I have it working on the FRESH install with my new PFX file generated on the refreshed IIS, its only when I try to copy the /ProgramData/config , plugins , data/displaypreferences.db , data/users.db and library.db that I seem to encounter this issue with SSL not working.

I assume the part of the log that is relevant is:

2021-02-11 16:03:41.532 Error App: Error loading cert from Z:\Documents\LetsEncryptEmby.pfx
	*** Error Report ***
	Version: 4.5.4.0
	Command line: C:\Users\PK\AppData\Roaming\Emby-Server\system\EmbyServer.dll -noautorunwebapp
	Operating system: Microsoft Windows 10.0.19042
	Framework: .NET Core 3.1.9
	OS/Process: x64/x64
	Runtime: C:/Users/PK/AppData/Roaming/Emby-Server/system/System.Private.CoreLib.dll
	Processor count: 4
	Data path: C:\Users\PK\AppData\Roaming\Emby-Server\programdata
	Application path: C:\Users\PK\AppData\Roaming\Emby-Server\system
	Internal.Cryptography.CryptoThrowHelper+WindowsCryptographicException: Internal.Cryptography.CryptoThrowHelper+WindowsCryptographicException: Access denied.
	   at Internal.Cryptography.Pal.CertificatePal.FilterPFXStore(Byte[] rawData, SafePasswordHandle password, PfxCertStoreFlags pfxCertStoreFlags)
	   at Internal.Cryptography.Pal.CertificatePal.FromBlobOrFile(Byte[] rawData, String fileName, SafePasswordHandle password, X509KeyStorageFlags keyStorageFlags)
	   at System.Security.Cryptography.X509Certificates.X509Certificate..ctor(String fileName, String password, X509KeyStorageFlags keyStorageFlags)
	   at System.Security.Cryptography.X509Certificates.X509Certificate2..ctor(String fileName, String password)
	   at Emby.Server.Implementations.ApplicationHost.GetCertificate(CertificateInfo info)
	Source: System.Security.Cryptography.X509Certificates
	TargetSite: Internal.Cryptography.Pal.Native.SafeCertContextHandle FilterPFXStore(Byte[], Microsoft.Win32.SafeHandles.SafePasswordHandle, Internal.Cryptography.Pal.Native.PfxCertStoreFlags)
	

But I obviously am not sure, else I could probably get it working.

Also I am adding these images, the library shows up, and I can play the files from here:

Yet when I look at the libraries under settings I just get this:

and the above error message if I try to add the previously existing folders.

("....error adding the media path. Please ensure the path is valid and the Emby Server process has access to that location...")

 

So, sorry for the ramble, but I was trying to be thorough... you guys really helped me last year when I lost ALOT of data (and got me on board with Sonarr/Radarr to find my missing stuff, so BIG THANKS for that!)

TIA

-G

Edited February 11, 2021 by Garbonzo17
added some images

[Carlo 4561]

Hi, Your SSL cert isn't setup correctly as you're getting errors with that.  Leave SSL to later as you likely want to concentrate on getting your libraries back first so you have something to use.

You need to check each library and make any adjustments to the mount points setup.  Also make sure each library has given permission to the user you are now using to run Emby.

Let us know how you make out.

Edited February 11, 2021 by cayars

[Garbonzo17 26]

I just renamed did a fresh install, and setup the same certs and it works as it did prior to be re-installing and then moving my .db files and config folder (I think it something in that config folder that is the hangup, but if I look at the system.xml file the location and password for the pfx is identical... so not sure what else there might be the issue.

BUT, after re-reading the https://support.emby.media/support/solutions/articles/44001159936-backup link I see that manually re-creating the libraries are required, so again fresh install and started with the SSL first, and that's all working, so I moved on to manually adding the libraries, that is happening now, when it's complete I am hoping to be able to move my user.db and metadata folder (if needed) and I should be done.... So I am really wondering if the solution at the above link needs to be updated for manually doing this.   I have been using Emby for quite a few years, and originally when the backup-sync plugin wasn't premier only I used it (and it continued to work for a year or two after it went premier but eventually stopped working) but back then the procedure was pretty much backup and restore the programdata folder and all was well (to my memory at least in 2.x-3.1.x versions) but then the things got more complex. I am still not opposed to paying for emby premier, but its just been a hard pill to swallow because I have been doing everything manually for so many years, and all of my family have already bought the apps on their relevant platforms, so really just this backup feature is my main benefit (though there may be a couple other options I have seen that I'd tinker with too).

Side note, any idea if EmbyConnect would help resolve the issue I am having still with FireTV devices connecting using the LetsEncrpt cert?  That actually may not be a thing anymore as I read that LetsEncrypt changed some things this year, I haven't tried since I just made this new cert yesterday, but I WAS still having to keep my server as "preferred" not "required" because of ONE user on a FireTV device, everyone else is Roku/iOS/Android/Web/Samsung/LG have no problem on 8920, just that one damn FireTV4k..

[Carlo 4561]

No don't do that.  If you're rebuilding your library it's going to recreate a new metadata folder so you don't want to replace all the scanning work you just did (depending on settings).

You either want to copy it all over first or just don't do it at all.  You need to be careful the way you're doing this because the old user.db file won't have the support directories it needs.

Personally I'd start over. 

Follow the KB Backup/Restore article for the folders to back (same as what you need to copy).  After coping everything over, install the same version of Emby smack on top of what you just installed.  Now try starting Emby.

If you hit a snag, post back here for help as you likely just need to make a small adjustment.

[Garbonzo17 26]

After all I have done I will just remake the users... There is only about 20 and most are cut/past setting (same adjustments), I read some thing about manually editing/merging with some sql commands, but honestly I have spent all day with it and this just seems easier.

I had already tried copying just the config and plugin folders, plus the displaypreferences.db, user.db, and library.db only, on-top of a FRESH install exited before the setup-wizard. And like the images showed in the OP my media shows up in the web player and actually play, but when I looked in Library settings ALL of the libraries show 0 folders... so I didn't trust that going forward. While I am waiting for the importing to finish I really want to finsh setting up my AutoOrganize settings, but I am to chicken to restart the server until the media-scan completes... I know it "should" be ok, but people can just deal with it being down tonight... I am watching "his dark materials" on hbomax tonight anyway...  Is there a plain text list of the usernames anywhere? in the old install config/users there are 21 folders (I assume each representing a user). Or can I open the old user.db in sql and query the names?  I can handle the passwords in a standard/reset type situation, but I didn't do a screen-cap of the old users page.

The frustrating part is that after doing the Windows 10 refresh/restore PC thing, everything was working except the new SSL.  I assume that has something to do with it "technically" being a new install/IIS implementation.  That seems reasonable.  But redoing all of the Certify/IIS/export pfx steps didn't resolve it.  But putting that SAME pfx file and password in place on a clean install and the SSL works as it should. Its frustrating not being able to figure out the actual problem (that is more frustrating to me that having to essentially start from scratch, that I can deal with.... I hate not figuring out the cause/fix)

 

** also, its only like 20% through the PEOPLE, am I safe canceling that and copying the old /ProgramData/metadata ?? that was what was recomended.  The Scan Media is only at like 91% but from what I remember last year the ~88%-100% took FOREVER.

Edited February 12, 2021 by Garbonzo17
question about "People"

[Luke 42089]

You can stop the refresh people task. That may help the library scan finish quicker.

[Happy2Play 9785]

Assuming you are using the database you copied over, you will run into People issues if said person is not in the /people or /library folder depending on what path is in the database for the person image.

As for users the users.db would have all the users and the \Emby-Server\programdata\config\users would have the user configs.  But this would have be a good use of Server Configuration Backup plugin.

[Carlo 4561]

Question for you.  Do you have a normal backup where you can just restore from a specific path down (Emby Server paths)?

Is all you media on other drives or on paths with the same drive letters or UNC paths?

If the answer to both are YES I can help you get this re-setup pretty quickly just as it was previously.
Not worried about the SSL cert as we would strip and redo this so it would work.

I know you're probably close to getting done scanning but if you find errors or things not working correct send me a PM and I can remote in and help you with this from scratch.

 

 

[Garbonzo17 26]

2 hours ago, cayars said:

Question for you.  Do you have a normal backup where you can just restore from a specific path down (Emby Server paths)?

Is all you media on other drives or on paths with the same drive letters or UNC paths?

If the answer to both are YES I can help you get this re-setup pretty quickly just as it was previously.
Not worried about the SSL cert as we would strip and redo this so it would work.

I know you're probably close to getting done scanning but if you find errors or things not working correct send me a PM and I can remote in and help you with this from scratch.

 

Yes to both. The Library files are/were mapped to a drive pool listed a Z:\

And I have a CLONE of the original C:\ drive (which contained the EmbyServer install so therefore the %appdata%\EmbyServer)

Stripping the SSL was the main problem I was having. If I just deleted the path to the pfx and deleted the password (and the domain info) and selected "disabled" for the requirements of ssl, it would save and seem fine, but after rebooting the server eveything was back... I suspect I could have just removed those lines in the XML file, but I didn't really notice that until I was comparing the system.xml files from the restored files (ssl not working) to the clean install (ssl working) and saw the same path/password in plain text. 

Edited February 12, 2021 by Garbonzo17