Home Videos and Photos Library nearly unusable with large libraries..

[cd01 8]

Not sure what changed but something with the sql queries is really bad with latest emby.

When you have a share that has setting "Home Videos and Photos" previously i could have over 1.000.000 photos in that share and the library would open nearly instantly.

Trying to open the same library in version 4.6.4.0 emby server more or less hangs and will not render the webui. If i open a share that has less items the problem is exposed, what previously was just opened as "folders" is now for whatever reason first sorted by pictures/videos/folders, so if you have over a million objects in a share the query must first look for videos and pictures in all sub folders, this alone will take a long time, many minutes, if not hours? then once this is complete the sorting will commence, now it will sort all this data by name or whatever. Once this is done the data can be shown to the end user, who has fallen a sleep.

Why was this terrible change introduced rendering the " Home Videos and Photos " library setting nearly useless if you have lots of items in a share?

Please fix this, roll it back to what it was before, no one needs emby to sort mixed files for you, i rather have my mixed content sorted on a filesystem level then depend on emby having my files sorted anyway because the filesystem level sorting trancends all systems while emby is just emby-

I see this has been discussed several of times before, i think i even reported this a couple of months ago, but for whatever reason it has not been fixed and there has been several of updates since this started happening! I guess not many users have massive libraries of files, but please don't neglect the few that have, emby was really good because it could handle massive libraries of mixed content.

Edited August 7, 2021 by cd01

[Luke 42077]

Hi there, please attach the emby server log from when you thought it was slow. Thanks.

[cd01 8]

On 08/08/2021 at 03:09, Luke said:

Hi there, please attach the emby server log from when you thought it was slow. Thanks.

Hmm not quiet sure what i am loooking at here but here's a log opening a semi large share that has mainly pictures.

Quote

http://media:8096/emby/Items/11487422/Images/Primary?maxHeight=211&maxWidth=375&tag=a8ce0dde685f93f379da8d645f4b0a11&quality=90. UserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:90.0) Gecko/20100101 Firefox/90.0
2021-08-09 09:43:16.592 Info Server: http/1.1 Response 200 to 1.1.1.2. Time: 1ms. http://media:8096/emby/Items/11487420/Images/Primary?maxHeight=211&maxWidth=375&tag=e8d34f6bffddfdfa2921ea96b69768e0&quality=90
2021-08-09 09:43:16.592 Info Server: http/1.1 GET http://media:8096/emby/Items/11487431/Images/Primary?maxHeight=211&maxWidth=375&tag=72e5861f3dbeeb5e284198d0c6e85d14&quality=90. UserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:90.0) Gecko/20100101 Firefox/90.0
2021-08-09 09:43:16.593 Info Server: http/1.1 Response 200 to 1.1.1.2. Time: 2ms. http://media:8096/emby/Items/11487421/Images/Primary?maxHeight=211&maxWidth=375&tag=39470db84c2179c8e99f7518d78ebc06&quality=90
2021-08-09 09:43:16.593 Info Server: http/1.1 GET http://media:8096/emby/Items/11487429/Images/Primary?maxHeight=211&maxWidth=375&tag=48d13dec0ecfb1650db77648acce65e5&quality=90. UserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:90.0) Gecko/20100101 Firefox/90.0
2021-08-09 09:43:16.593 Info Server: http/1.1 Response 200 to 1.1.1.2. Time: 1ms. http://media:8096/emby/Items/11487424/Images/Primary?maxHeight=211&maxWidth=375&tag=a8e8c8a729cffd266e550180f7476d9a&quality=90
2021-08-09 09:43:16.594 Info Server: http/1.1 Response 200 to 1.1.1.2. Time: 1ms. http://media:8096/emby/Items/11487431/Images/Primary?maxHeight=211&maxWidth=375&tag=72e5861f3dbeeb5e284198d0c6e85d14&quality=90
2021-08-09 09:43:16.594 Info Server: http/1.1 Response 200 to 1.1.1.2. Time: 2ms. http://media:8096/emby/Items/11487422/Images/Primary?maxHeight=211&maxWidth=375&tag=a8ce0dde685f93f379da8d645f4b0a11&quality=90
2021-08-09 09:43:16.594 Info Server: http/1.1 GET http://media:8096/emby/Items/11487179/Images/Primary?maxHeight=211&maxWidth=375&tag=91274241a9f19a6a047b660e02b6badf&quality=90. UserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:90.0) Gecko/20100101 Firefox/90.0
2021-08-09 09:43:16.595 Info Server: http/1.1 GET http://media:8096/emby/Items/11487432/Images/Primary?maxHeight=211&maxWidth=375&tag=be40745c9783b24d7422e139ab1ca9aa&quality=90. UserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:90.0) Gecko/20100101 Firefox/90.0
2021-08-09 09:43:16.595 Info Server: http/1.1 GET http://media:8096/emby/Items/11487435/Images/Primary?maxHeight=211&maxWidth=375&tag=05399ee64cfb9af54b4458197e9d00b3&quality=90. UserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:90.0) Gecko/20100101 Firefox/90.0
2021-08-09 09:43:16.595 Info Server: http/1.1 GET http://media:8096/emby/Items/11487439/Images/Primary?maxHeight=211&maxWidth=375&tag=aa196d4aa8fe221c5db3c29ab63e160d&quality=90. UserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:90.0) Gecko/20100101 Firefox/90.0
2021-08-09 09:43:16.595 Info Server: http/1.1 GET http://media:8096/emby/Items/11487440/Images/Primary?maxHeight=211&maxWidth=375&tag=7e817773702bfabb1f9f99fba8cbeac3&quality=90. UserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:90.0) Gecko/20100101 Firefox/90.0
2021-08-09 09:43:16.597 Info Server: http/1.1 Response 200 to 1.1.1.2. Time: 3ms. http://media:8096/emby/Items/11487179/Images/Primary?maxHeight=211&maxWidth=375&tag=91274241a9f19a6a047b660e02b6badf&quality=90
2021-08-09 09:43:16.597 Info Server: http/1.1 Response 200 to 1.1.1.2. Time: 4ms. http://media:8096/emby/Items/11487429/Images/Primary?maxHeight=211&maxWidth=375&tag=48d13dec0ecfb1650db77648acce65e5&quality=90
2021-08-09 09:43:16.597 Info Server: http/1.1 Response 200 to 1.1.1.2. Time: 3ms. http://media:8096/emby/Items/11487432/Images/Primary?maxHeight=211&maxWidth=375&tag=be40745c9783b24d7422e139ab1ca9aa&quality=90
2021-08-09 09:43:16.597 Info Server: http/1.1 Response 200 to 1.1.1.2. Time: 2ms. http://media:8096/emby/Items/11487440/Images/Primary?maxHeight=211&maxWidth=375&tag=7e817773702bfabb1f9f99fba8cbeac3&quality=90
2021-08-09 09:43:16.598 Info Server: http/1.1 Response 200 to 1.1.1.2. Time: 3ms. http://media:8096/emby/Items/11487435/Images/Primary?maxHeight=211&maxWidth=375&tag=05399ee64cfb9af54b4458197e9d00b3&quality=90
2021-08-09 09:43:16.598 Info Server: http/1.1 Response 200 to 1.1.1.2. Time: 3ms. http://media:8096/emby/Items/11487439/Images/Primary?maxHeight=211&maxWidth=375&tag=aa196d4aa8fe221c5db3c29ab63e160d&quality=90
2021-08-09 09:43:16.598 Info Server: http/1.1 GET http://media:8096/emby/Items/11481943/Images/Primary?maxHeight=211&maxWidth=375&tag=6ee43d726d72fb6c8ad1cec36e3bc4a4&quality=90. UserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:90.0) Gecko/20100101 Firefox/90.0
2021-08-09 09:43:16.598 Info Server: http/1.1 GET http://media:8096/emby/Items/8849559/Images/Primary?maxHeight=211&maxWidth=375&tag=e7aa52185d8e2451532fd6719f4b069b&quality=90. UserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:90.0) Gecko/20100101 Firefox/90.0
2021-08-09 09:43:16.598 Info Server: http/1.1 GET http://media:8096/emby/Items/21761/Images/Primary?maxHeight=211&maxWidth=375&tag=d445e473581d3a7dc39c04c43d4490ff&quality=90. UserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:90.0) Gecko/20100101 Firefox/90.0
2021-08-09 09:43:16.598 Info Server: http/1.1 POST http://media:8096/emby/Sessions/Capabilities/Full?X-Emby-Client=Emby Web&X-Emby-Device-Name=Firefox&X-Emby-Device-Id=240aa623-eff8-4fe2-a83e-d26424b888cf&X-Emby-Client-Version=4.6.4.0. UserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:90.0) Gecko/20100101 Firefox/90.0
2021-08-09 09:43:16.599 Info Server: http/1.1 Response 200 to 1.1.1.2. Time: 1ms. http://media:8096/emby/Items/21761/Images/Primary?maxHeight=211&maxWidth=375&tag=d445e473581d3a7dc39c04c43d4490ff&quality=90
2021-08-09 09:43:16.599 Info Server: http/1.1 Response 200 to 1.1.1.2. Time: 2ms. http://media:8096/emby/Items/11481943/Images/Primary?maxHeight=211&maxWidth=375&tag=6ee43d726d72fb6c8ad1cec36e3bc4a4&quality=90
2021-08-09 09:43:16.600 Info Server: http/1.1 Response 200 to 1.1.1.2. Time: 2ms. http://media:8096/emby/Items/8849559/Images/Primary?maxHeight=211&maxWidth=375&tag=e7aa52185d8e2451532fd6719f4b069b&quality=90
2021-08-09 09:43:16.601 Info Server: http/1.1 Response 204 to 1.1.1.2. Time: 2ms. http://media:8096/emby/Sessions/Capabilities/Full?X-Emby-Client=Emby Web&X-Emby-Device-Name=Firefox&X-Emby-Device-Id=240aa623-eff8-4fe2-a83e-d26424b888cf&X-Emby-Client-Version=4.6.4.0
2021-08-09 09:43:21.707 Info Server: http/1.1 GET http://media:8096/emby/Users/ca2edc0b5e604b658a75fcbcc71c93a3/Items/8659da281dbb01d5a371ffa1a700bc9a?X-Emby-Client=Emby Web&X-Emby-Device-Name=Firefox&X-Emby-Device-Id=240aa623-eff8-4fe2-a83e-d26424b888cf&X-Emby-Client-Version=4.6.4.0. UserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:90.0) Gecko/20100101 Firefox/90.0
2021-08-09 09:44:14.437 Info Server: http/1.1 Response 200 to 1.1.1.2. Time: 52730ms. http://media:8096/emby/Users/ca2edc0b5e604b658a75fcbcc71c93a3/Items/8659da281dbb01d5a371ffa1a700bc9a?X-Emby-Client=Emby Web&X-Emby-Device-Name=Firefox&X-Emby-Device-Id=240aa623-eff8-4fe2-a83e-d26424b888cf&X-Emby-Client-Version=4.6.4.0
2021-08-09 09:44:14.450 Info Server: http/1.1 GET http://media:8096/emby/Users/ca2edc0b5e604b658a75fcbcc71c93a3/Items?SortBy=SortName&SortOrder=Ascending&IncludeItemTypes=Video&Recursive=true&Fields=BasicSyncInfo,CanDelete,PrimaryImageAspectRatio,ProductionYear&ImageTypeLimit=1&EnableImageTypes=Primary,Backdrop,Thumb&StartIndex=0&ParentId=8659da281dbb01d5a371ffa1a700bc9a&Limit=50&X-Emby-Client=Emby Web&X-Emby-Device-Name=Firefox&X-Emby-Device-Id=240aa623-eff8-4fe2-a83e-d26424b888cf&X-Emby-Client-Version=4.6.4.0. UserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:90.0) Gecko/20100101 Firefox/90.0
2021-08-09 09:44:14.519 Info Server: http/1.1 Response 200 to 1.1.1.2. Time: 69ms. http://media:8096/emby/Users/ca2edc0b5e604b658a75fcbcc71c93a3/Items?SortBy=SortName&SortOrder=Ascending&IncludeItemTypes=Video&Recursive=true&Fields=BasicSyncInfo,CanDelete,PrimaryImageAspectRatio,ProductionYear&ImageTypeLimit=1&EnableImageTypes=Primary,Backdrop,Thumb&StartIndex=0&ParentId=8659da281dbb01d5a371ffa1a700bc9a&Limit=50&X-Emby-Client=Emby Web&X-Emby-Device-Name=Firefox&X-Emby-Device-Id=240aa623-eff8-4fe2-a83e-d26424b888cf&X-Emby-Client-Version=4.6.4.0
2021-08-09 09:44:34.184 Info Server: http/1.1 GET http://media:8096/emby/Items/8265689/Images/Primary?maxHeight=211&maxWidth=375&tag=30151d74d8893903f8152360431ff3d8&quality=90. UserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:90.0) Gecko/20100101 Firefox/90.0
2021-08-09 09:44:34.185 Info Server: http/1.1 GET http://media:8096/emby/Items/6240511/Images/Primary?maxHeight=211&maxWidth=375&tag=b169541e773bca425d323d2a3d30b9ce&quality=90. UserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:90.0) Gecko/20100101 Firefox/90.0
2021-08-09 09:44:34.185 Info Server: http/1.1 GET http://media:8096/emby/Items/6240512/Images/Primary?maxHeight=211&maxWidth=375&tag=e480cf1a1ce88f6124de3ed71eb116bd&quality=90. UserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:90.0) Gecko/20100101 Firefox/90.0
2021-08-09 09:44:34.186 Info Server: http/1.1 GET http://media:8096/emby/Items/6240513/Images/Primary?maxHeight=211&maxWidth=375&tag=5f0ad7adb8cde5fc9b9aedb43956828a&quality=90. UserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:90.0) Gecko/20100101 Firefox/90.0
2021-08-09 09:44:34.187 Info Server: http/1.1 GET http://media:8096/emby/Items/7603172/Images/Primary?maxHeight=211&maxWidth=375&tag=f7087361acc4321c78d2633182329ec0&quality=90. UserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:90.0) Gecko/20100101 Firefox/90.0
2021-08-09 09:44:34.187 Info Server: http/1.1 GET http://media:8096/emby/Items/7603173/Images/Primary?maxHeight=211&maxWidth=375&tag=99bfdee15560bf34d830a2a465a33d62&quality=90. UserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:90.0) Gecko/20100101 Firefox/90.0
2021-08-09 09:44:34.193 Info Server: http/1.1 Response 200 to 1.1.1.2. Time: 8ms. http://media:8096/emby/Items/6240512/Images/Primary?maxHeight=211&maxWidth=375&tag=e480cf1a1ce88f6124de3ed71eb116bd&quality=90
2021-08-09 09:44:34.195 Info Server: http/1.1 Response 200 to 1.1.1.2. Time: 9ms. http://media:8096/emby/Items/6240513/Images/Primary?maxHeight=211&maxWidth=375&tag=5f0ad7adb8cde5fc9b9aedb43956828a&quality=90
2021-08-09 09:44:34.195 Info Server: http/1.1 Response 200 to 1.1.1.2. Time: 9ms. http://media:8096/emby/Items/7603172/Images/Primary?maxHeight=211&maxWidth=375&tag=f7087361acc4321c78d2633182329ec0&quality=90
2021-08-09 09:44:34.196 Info Server: http/1.1 Response 200 to 1.1.1.2. Time: 11ms. http://media:8096/emby/Items/6240511/Images/Primary?maxHeight=211&maxWidth=375&tag=b169541e773bca425d323d2a3d30b9ce&quality=90
2021-08-09 09:44:34.196 Info Server: http/1.1 Response 200 to 1.1.1.2. Time: 10ms. http://media:8096/emby/Items/7603173/Images/Primary?maxHeight=211&maxWidth=375&tag=99bfdee15560bf34d830a2a465a33d62&quality=90
2021-08-09 09:44:34.199 Info Server: http/1.1 Response 200 to 1.1.1.2. Time: 15ms. http://media:8096/emby/Items/8265689/Images/Primary?maxHeight=211&maxWidth=375&tag=30151d74d8893903f8152360431ff3d8&quality=90

 

This query in particular stands out from what i gather?

Quote

2021-08-09 09:44:14.437 Info Server: http/1.1 Response 200 to 1.1.1.2. Time: 52730ms. http://media:8096/emby/Users/ca2edc0b5e604b658a75fcbcc71c93a3/Items/8659da281dbb01d5a371ffa1a700bc9a?X-Emby-Client=Emby Web&X-Emby-Device-Name=Firefox&X-Emby-Device-Id=240aa623-eff8-4fe2-a83e-d26424b888cf&X-Emby-Client-Version=4.6.4.0

52730 ms is not great when opening a share. previous builds of emby would instantly open the share even if there was a million objects in the share but it would go to folders.

im not an expert but now it seems that whenever you open a share from home videos & pictures the query has to look through the entire share (million of files in my case) then sort the files by pictures/videos then present the data. previously it would simply just look for the next level of content, it being only folders it would open it as folders and it would be instantly rendered until it got to a point where the next level had pictures it would render folders with thumbnail previews.

This was a great system giving me access to millions of files in my library at an instant, now its nearly impossible to navigate due to these extremely heavy queries having to sort through everything before it is presented.

Edited August 9, 2021 by cd01

[cd01 8]

if you need more logs/information let me know, i am willing to do testing to resovle this, i do realize that most people probably do not have an envoirment like mine with millions of files so its hard to test these things.

cheers and keep up the good work!

[Luke 42077]

Hi, can you please attach the complete emby server log file, rather than a snippet? thanks.

[cd01 8]

37 minutes ago, Luke said:

Hi, can you please attach the complete emby server log file, rather than a snippet? thanks.

Is it embyserver.txt you are looking for?

embyserver.txt

Would appreciate if you could remove the log file after you have checked it, its leaking a lot of filenames perhaps other privacy concerning stuff... thx.

Edited August 11, 2021 by cd01

[cd01 8]

I see in another thread you gave the tip to set the default view per share, i had missed this, but i tried it and it did not change the fact that emby is sorting all the data before displaying the folders, this is new behavior from emby and makes emby unusable for users with large shares.

Here's my reply in the other thread:

My suggestion is:

1. fix that each "default view" setting has their own optimized queries. If you set default view to folders, there's no reason why emby should run a query that looks for videos and pictures then sorting them by date then displaying the folders, instead just simply display the folders in the next level like emby was previously doing.

2. update the library settings panel to have setting to hard set the default order for each share. you should also include in the same setting panel a way to disable the other view buttons per share basis giving the admin control over their own server and removing any chance that a user is clicking "pictures" view in a share that has millions of pictures resulting in a query that nearly crashes the server.

 

 

Edited August 14, 2021 by cd01

[Luke 42077]

Quote

1. fix that each "default view" setting has their own optimized queries. If you set default view to folders, there's no reason why emby should run a query that looks for videos and pictures then sorting them by date then displaying the folders, instead just simply display the folders in the next level like emby was previously doing.

@cd01 why do you think this is happening?

[Luke 42077]

But regarding the original issue, when was this database originally built, do you know?

[cd01 8]

On 01/09/2021 at 17:48, Luke said:

@cd01 why do you think this is happening?

Because switching between the view modes makes the query run exactly as long as with the default view mode.

If i set the view mode to go to folders directly it takes 1 minute to load, then once loaded if i click "photos" or "videos" they instantly come up meaning they are already cached in memory from the query, meaning the query was doing all that work sorting the stuff even though all i wanted to do was view the folders.

if the quey was modified based on view mode it would take a long time to open photos or videos after opening folders now wouldn't it?

On 01/09/2021 at 17:50, Luke said:

But regarding the original issue, when was this database originally built, do you know?

Hmm, not sure what you mean by this, i have had the database for about 2 year or so, so whatever emby version was then. Can i view versions somewhere or provide you with relevant logs?

Edited September 2, 2021 by cd01

[Luke 42077]

Quote

If i set the view mode to go to folders directly it takes 1 minute to load, then once loaded if i click "photos" or "videos" they instantly come up meaning they are already cached in memory from the query, meaning the query was doing all that work sorting the stuff even though all i wanted to do was view the folders.

if the quey was modified based on view mode it would take a long time to open photos or videos after opening folders now wouldn't it?

The query is different depending on the view. The folders tab does not do all of the work of the other tabs, it only queries for what it needs.

Quote

Hmm, not sure what you mean by this, i have had the database for about 2 year or so, so whatever emby version was then. Can i view versions somewhere or provide you with relevant logs?

If you're willing to rebuild the database, a new library.db will probably be about half the size and should perform better as a result of that.

[cd01 8]

On 9/2/2021 at 3:46 PM, Luke said:

The query is different depending on the view. The folders tab does not do all of the work of the other tabs, it only queries for what it needs.

If you're willing to rebuild the database, a new library.db will probably be about half the size and should perform better as a result of that.

How would i trigger such a rebuild? I don't think a rebuild would be that bad because the thing that takes the most time is generation of thumbnails and such which is already done.

@cayars

Can you instruct me in how to trigger a proper rebuild of the database index but keep the thumbnails from being "regenerated" ?

[Carlo 4561]

If you're referring to the index files (bif) files and they are stored with your media then these will not need to be rebuilt as Emby Server will use the existing bif files already present.
In that can just shut down Emby Server and rename your library.db to library.old.
That will cause Emby Server to create a new library.db upon restart of the Server.  Your library definitions should still be there so all that's needed is a full library scan.

Before doing this I would make sure to configure and run the config backup plugin. This way you can restore your user play states with a partial backup restore after scanning of the new library is complete.

[cd01 8]

Is there anyone actually looking into this? I still do not understand why opening a folder that has 50 folders but the sub folders of these 50 folders may be over a million, this should not matter. Why is emby doing deep query dives into directories? Well to answer my own question its because emby has been updated to to be able to present to you 4 sections:

Videos, Photos, Tags, Folders

Even though you have specifically set a library to be "folders" you still have queries running through EVERYTHING to be able to sort by Videos, Photos, Tags, Folders.

If you have a large library like me with millions of files, this query takes over a minute to load. So essentially i have to nearly a minute to view 50 folders, i NEVER EVER want to view this library by any of the other filters.

I do not understand why emby has been so unoptimized, 2 years ago when i bought emby this was not a problem, all my libraries would load instantly, no matter if it was 5 folders with 50 files inside each, or 60 folders with over a million files in the deeper levels, now emby is unusable with large libraries for no reason what so ever.

Please give us an option to remove this bloat queries so users with large libraries can use emby again.

Quote

2021-12-16 08:35:51.048 Info Server: http/1.1 Response 200 to 1.1.1.2. Time: 56031ms. http://media:8096/emby/Users/f434dfb93bf8478280adc00b4195490c/Items/8659da281dbb01d5a371ffa1a700bc9a?X-Emby-Client=Emby Web&X-Emby-Device-Name=Firefox&X-Emby-Device-Id=6cea02f4-ddb1-4dcb-a600-67c656d5509c&X-Emby-Client-Version=4.6.7.0

 

56031ms, thats 56 seconds to open a folder

Edited December 16, 2021 by cd01

[cd01 8]

Oh and i know these queries are being ran on opening of the folder, because after you have waited 56 seconds for the folder to open, you can then click on Videos, Photos or Tags, and these sections will INSTANTLY load meaning they have already been queried and stored in memory. Please fix your folder query that it only does this awful sorting if you actually click on a filter that requires sorting.

Edited December 16, 2021 by cd01

[Luke 42077]

Have you updated to Emby Server 4.6.7?

[cd01 8]

On 16/12/2021 at 09:24, Luke said:

Have you updated to Emby Server 4.6.7?

I finally went ahead and deleted my library.db and let it rebuild as that was a solution. No change at all, running: 4.7.0.60 right now.

Opening a share that has a large amount of pictures nested in deeper folders but only listing the first root of the folder takes nearly a minute. But opening the same folder in windows explorer takes 0.001 second. Because it only looks for files in the root folder. I am once more sure that emby is looking for more information when you open a folder than just querying the actual root of the folder resulting in this massive slowness.

Can you please update and let us users who actually have shares that have pictures in them use a new library model called pictures? And make this model very simple, only look for pictures, and only list whatever is in the root folder and perhaps use one thumbnails if a picture is available inside the new folder on the next level. This way it will work exactly like explorer and will be fast even if you have 2+ million pictures in the share.

Emby is useless for any share with pictures if you have a lot of pictures, it takes forever to open the share. This was not a thing with old emby, like 3-4 years ago so something changed. It is annoying to me that it has not been fixed yet, seems like such an easy thing to fix.

If you need logs or any more data please let me know, i want this fixed, it can't be that hard to fix, just look at the query opening a folder for home videos & pictures setting, it should only look for content in the root not query everything inside the share at once.

EDIT: Sorry i feel bad for going so hard, i actually love emby, it is much better than plex, i just wish it worked with pictures still.

 

Edited May 19, 2022 by cd01

[Luke 42077]

Have you taken a look at the new database options? You should be able to see some improvement by increasing the default values.

[cd01 8]

9 minutes ago, Luke said:

Have you taken a look at the new database options? You should be able to see some improvement by increasing the default values.

Thanks for the quick reply, i added an exuse at the end of my post cause i felt bad, i actually love emby and i really appreciate the work you guys do and allowing us to have some privacy compared to plex.

Anyway, where can i find these new database options? Thanks!

EDIT: Never mind, found the new database options, will make some tests and report back. Thanks

Edited May 19, 2022 by cd01

[cd01 8]

So it finally finished to rebuild the library.db, it only saved me 500 mb compared to what my previous database where. I dont remove a lot of data as you can tell so not much "vacuming needed".

For whatever reason it doesn't seem that emby is logging access into folders anymore, it only logs when a user is accessing the root of the server so i cannot tell exactly how slow the queries are but they are faster, perhaps 50% faster but still hovering around 30 seconds to just open a share that only has 30 folders in it but deeper into these folders there are in the end over a million images. I don't know if this is a limitation of sqllite works and how the queries are made and how it reads the database but it seems very strange that it would take so long time to just open a folder when non of the data beneath said folder is being touched.

Anyway, i think emby just is at its limits i really wish it would work to have my photo library in emby to which would have been awesome, for whatever reason it worked with the older versions of emby much better, but something changed around 2 years ago that made just opening the main folder take forever. It doesn't seem like it can be fixed but perhaps tweaked?

What values do you recommend for a massive emby setup like mine in the database configuration, right now i am using:

Database Cache Size: 1024

Analysis Row Limit: 800

Thanks