Scans so damn slow

[macgyvertv 4]

On 4/2/2023 at 12:58 PM, Happy2Play said:

All I see is a user that is hammering provider apis and will almost guarantee debug logging will show throttling required for TMDB. 

So is it about importing media or presenting it with metadata as you cannot have both having to get from online providers as they do impose request/speed limits.

Not hammering. i kill the one instance i had and started up the beta to see if any thing change. It hasnt. still slow scans and the NFO files are local. I see the API v4 being hit and working, response are slow, but how is it the other guy can get metadata lot quicker and not be throttle? Can we use our own API keys in the plugins, maybe it will help instead of using one universal one. 

[Happy2Play 9780]

Have no idea how one can bypass restrictions while the other cannot as it has been proven in the past when user were getting blacklisted/429 errors from provider.

As for api keys I really don't see there being a difference being provided vs individual provided as the plug code will be exactly the same.

 

From an import of existing metadata standpoint, I disable providers so internet queries are irrelevant.  There are major speed differences of importing and requiring online metadata and providing existing but does require library config options to be adjusted.

[yyhemby 2]

On the other hand, when scanning files, it is just like plex.

When you click to scan a single file, it only scans the metadata of a single file, instead of directly refreshing the entire video media library.

When there are multiple libraries in the server, and when one of the media libraries is clicked, only the selected media library will be refreshed instead of all of them.

Also, like my answer above, set up a separate page to display the whole scanning process, so that users can perceive what content has been scanned, instead of just showing a progress.

All in all, regarding the optimization of the new version of media library scanning, I suggest you download a plex server to experience it, and see what optimizations they have made, so that the scanning effect and experience of plex are very good. As a user, I don’t think emby The scanning is poor, but it is not transparent enough. The function of scanning the media library is too monotonous, and users should be given more space to operate.

Edited April 4, 2023 by yyhemby

[macgyvertv 4]

It just absolutely sucks compared to other software.  Not trying to downplay this.. or be a D about it. 

[macgyvertv 4]

On 4/3/2023 at 7:31 PM, Happy2Play said:

Have no idea how one can bypass restrictions while the other cannot as it has been proven in the past when user were getting blacklisted/429 errors from provider.

As for api keys I really don't see there being a difference being provided vs individual provided as the plug code will be exactly the same.

 

From an import of existing metadata standpoint, I disable providers so internet queries are irrelevant.  There are major speed differences of importing and requiring online metadata and providing existing but does require library config options to be adjusted.

I have seen api get throttled, think it was the open movie DB that did it for a while, and changed up API to help speed up the process. 

[macgyvertv 4]

way to just limit amount of data needed to scan.. seems that what its trying to do. scan the media, checks its file size, type, all that.. when it just needs to grab the poster. 

[macgyvertv 4]

even the AI chat knows its slow .lol

[macgyvertv 4]

their a reason why i needs to probe everything mulit times. 

[y29shi 8]

On 28/03/2023 at 08:41, macgyvertv said:

Running basic setup, no addons, nothing on a 24 core 10gps server. 

1.2PB of tv and movies. 

1 Folder for TV is 300TB and yet takes nearly a week to scan in, vs Jellyfin took 2 days and completed without missing a single folder. 

Please tell me how is this possible when one is free and other i have to pay for services that dosnt work. 

BTw files are local storage, NFO files are with the media and on a sep SSD. 

No errors in the logs, other then timeouts with these random metadata api's. 

wow really? I only have a about 14TB of movies/tvs, emby took around a whole day to do the first scan, it missed a few but the manual scans after that were pretty quick (finishes within 30 sec unless I suddenly add a large folder into the library).

but I just tried jellyfin as well and it took 2 days to finish the first scan and it missed quite a lot of folders. then I did manual scans multiple times after that, the second scan still took a half day, the third took about 4 hours. after that all the scans take about 5 mins.

[macgyvertv 4]

Yep.. move to strm files and works faster. So not sure why..

[Luke 42077]

On 4/9/2023 at 1:41 AM, macgyvertv said:

their a reason why i needs to probe everything mulit times. 

Hi, please attach the complete emby server log file. Thanks.