Emby 4.5.4 ---> 4.6.4 wiping libraries inconsistently

[Edrock200 53]

Hello,

I'm trying to figure out what is happening, and if this is intended behavior. When Emby 4.6 came out, I ran into the db locking issues. 4.6.4 seemed to resolve this on my test box. On my test box, the upgrade went smooth to 4.6.3 with libraries in tact, then from 4.6.3 to 4.6.4 successfully.

I then attempted the upgrade on my production box from  and although the upgrade appeared to go through fine, no db locks and such, every library was reset to 0 items, wiped essentially. I know a rescan will bring it back, but is this intentional behavior? Because it's not happening on my test box so I'm at a loss as to why the different behaviors. These are docker containers running on ubuntu 18.04. I've tried both the official emby docker as well as the linuxserver.io docker. Thanks in advance for any help/advice.

-Ed

Edited July 8, 2021 by Edrock200
Test box was upgraded to 4.6.3 successfully, then to 4.6.4

[Luke 42078]

HI, it's not intentional, but due to the scope of the changes in 4.6 there could be some cases of this happening. What are the content types of the libraries?

[Carlo 4561]

Are any of your libraries now set to mixed content that were not before?

[Edrock200 53]

49 minutes ago, Luke said:

HI, it's not intentional, but due to the scope of the changes in 4.6 there could be some cases of this happening. What are the content types of the libraries?

The library types are static, e.g. one set is movies and the other set is tv

17 minutes ago, cayars said:

Are any of your libraries now set to mixed content that were not before?

In this upgrade case no. But I did run into this once with 4.6.3 and one library switching to mixed and 2 folders vs movies and 7 folders. In that case I was able to update the xml library file in the root folder to correct per @Luke's direction. But this time, it's just wiping all libraries, and the library.db file is still 2gb after. I restored 4x and attempted 2x with embyserver/emby and 2x with linuxserver.io/emby with the same result. I confirmed the db wasn't locked via logs and ensured the /config folder remap ownership and permissions were correct and matched that of the docker pgid/puid env variables. 

I haven't tried going to 4.6.3 first, then 4.6.4, as my last test to 4.6.3 worked, less the one library switching to mixed, but was getting random db lock errors on restarts. Would this be a worthwhile or helpful test?

The only other difference with the 4.6.3 test was I deleted connect.txt and device.txt, and modified the custom url in system.xml as this was a test box and I didnt want it taking over the emby connect id of the production box I cloned it from.

 

One other question. Is the "lastversion.txt" file just for reference or does emby take different actions based on the version listed in this file at startup?

Thanks in advance again for any help/guidance.

Ed

Edited July 8, 2021 by Edrock200

[Edrock200 53]

I just realized my original post had a typo. "On my test box, the upgrade went smooth to 4.6.4 with libraries in tact" should have read 4.6.3, then subsequently upgraded to 4.5.4 successfully. My apologies.

Edited July 8, 2021 by Edrock200
4.x not 5.x

[Luke 42078]

Quote

One other question. Is the "lastversion.txt" file just for reference or does emby take different actions based on the version listed in this file at startup?

It's just for reference.

[Edrock200 53]

gotcha, thanks. any thoughts on how to upgrade without wiping libraries?

[Edrock200 53]

Any advice?

[Carlo 4561]

You should be able to just install the latest version, then check to make sure all libraries kept their proper type.

[Edrock200 53]

They do, it's just they are all empty afterwards.

[Carlo 4561]

That sounds like you didn't let it do a full scan after the upgrade which is needed.

[Edrock200 53]

OK. But, the fact that my other test server upgraded with libraries in tact, and this one won't, theres no explanation for that?

[Carlo 4561]

The explanation could be as simple as the Full Library Scan hasn't ran yet.

Have you done this?  That should bring the content of the libraries back pretty quickly as it does/finishes the migration steps it needs to do.

We have seen some people do a restart after installing the new version because it also updated plugins and shows a restart needed but this would stop the initial scan it does itself.  So it's best to just run a full scan manually.

If you still have empty libraries after doing the full library scan then we have something we need to look at.

[Luke 42078]

Were you able to run a full scan?

[Edrock200 53]

No. Not yet. I will need to clone it to a test box and give it a new ID as I would never hear the end of it from friends and fam. I use cloud storage and it takes a good 2 days to do a fresh scan on empty libraries. However, I'm guessing this is something with my setup. So my plan is to try and get to 4.6.4 in an environment that will not cause my family to drive me nuts first, ensure all is kosher, then rinse/repeat on main one.

[Carlo 4561]

Hi, normally a full scan runs every day.  It only picks up things that have changed since the last run.

Did you turn this off?

If you go to Scheduled Tasks and look at the Scan Media Task when was the last time it run and how long did it run for?

[Edrock200 53]

No I didn't. but as I mentioned my libraries went from having thousands of items to zero, which I thought meant, it will have to do a fresh scan as though its a fresh install again. Or are you saying despite it showing zero items, somehow this scan isn't the same as a "from scratch" scan? The daily "catch-up" scans aren't bad once the base libraries are scanned in for the frist time.

[Carlo 4561]

It's a requirement to run a full scan after the upgrade.

Run the full scan and let us know how things look after doing this.

[Edrock200 53]

Ok thanks. I'm in the process of migrating all cloud media to local storage. I'll try 4.6.4 again once that's complete.

[Carlo 4561]

Sounds good. let us know you make out.

[Edrock200 53]

So finally upgraded my embys. Just in case anyone else comes across this - I was under the impression that the upgrade wipes the libraries and requires everything to be rescanned in. This is NOT the case. 

As soon as you do the upgrade, emby will come up fairly quickly and initiates a full scan. The libraries will initially say no items. Don't panic. It appears emby is doing some sort of db upgrade on the first scan. You'll notice the scan progress stays at zero percent for a while. I have a large library and it took a good 15-20m before it progressed from 0% on the scan. As soon as it did progress beyond zero, all library items were back. 

[Happy2Play 9780]

38 minutes ago, Edrock200 said:

So finally upgraded my embys. Just in case anyone else comes across this - I was under the impression that the upgrade wipes the libraries and requires everything to be rescanned in. This is NOT the case. 

As soon as you do the upgrade, emby will come up fairly quickly and initiates a full scan. The libraries will initially say no items. Don't panic. It appears emby is doing some sort of db upgrade on the first scan. You'll notice the scan progress stays at zero percent for a while. I have a large library and it took a good 15-20m before it progressed from 0% on the scan. As soon as it did progress beyond zero, all library items were back. 

Yes that sounds correct.  Can you check one thing and verify your libraries came back as the correct content types.

[Luke 42078]

@Edrock200 ?

[Edrock200 53]

Yes all libraries have the correct content types. Prior to 4.6.4, I had one test upgrade change one to "Mixed." but with the 4 servers I updated to 4.6.4, all content libraries carried over correctly.

[Luke 42078]

Thanks for the feedback.