Concurrent Library Scan Threads

[RedChili 1]

Hello,

I just switched over from plex to emby because I got fed up with photo libraries. I thought to give emby a shot for photos only.
I'm just performing an initial scan of my library (4500 videos, 72k photos). I noticed that emby is rather slow on this initial scan and it's mostly performing sequential video scan via ffmpeg.

Is there any reason why this is not done in multiple parallel threads to speed up things? My server has 6 cores but is rather bored CPU-wise and also not exhausted I/O-wise. I believe it could handle 2-4 more parallel ffmpeg calls.

kr 

[RedChili 1]

OK this has been discussed before (just didn't find it under library scan).

Still think having multiple ffmpeg processes would be fine and make the initial setup nicer but this seems to not getting added:

 

[Luke 37020]

Hi, the upcoming 4.8 server release will have improved initial library scan time.

[RedChili 1]

Thanks, looking forward to it!

[jiggity 21]

Given my feature request about the music library scanning got closed and I was told to post here, even though library scanning of music and FFMPEG multi thread usage are not the same issue, i will still reiterate the problem.

When you have multiple libraries for music and you add files to one of them emby will scan all of them.
This is unwieldy and not a good use of resources when you have a large music collection.
The discogs API throws non-stop errors and that slows the scanning to a crawl. If there a a way to scan a singular library that was updated, or to scan multiple libraries in a parallel fashion, with a smart task control to time out the task that throw continuous error and not bog or prevent other scanning form occurring that would be just swell.

The non stop errors on scanning means it takes over an hour to ingest a single album, and even then the embedded metadata is not ingested fully by emby.

Also, as I pointed out @Happy2Playhad identified that if music libraries are not all configured the same that will cause discogs errors as well.  So I was asking further questions about why separate libraries need to be config'd the same. And to what extent they must be similar, metadata setting, folder structure etc

TL;DR - Library parallel scanning is not the same as FFMPEG multithreading

[Happy2Play 8250]

2 minutes ago, jiggity said:

Given my feature request about the music library scanning got closed and I was told to post here, even though library scanning of music and FFMPEG multi thread usage are not the same issue, i will still reiterate the problem.

When you have multiple libraries for music and you add files to one of them emby will scan all of them.
This is unwieldy and not a good use of resources when you have a large music collection.
The discogs API throws non-stop errors and that slows the scanning to a crawl. If there a a way to scan a singular library that was updated, or to scan multiple libraries in a parallel fashion, with a smart task control to time out the task that throw continuous error and not bog or prevent other scanning form occurring that would be just swell.

The non stop errors on scanning means it takes over an hour to ingest a single album, and even then the embedded metadata is not ingested fully by emby.

Also, as I pointed out @Happy2Playhad identified that if music libraries are not all configured the same that will cause discogs errors as well.  So I was asking further questions about why separate libraries need to be config'd the same. And to what extent they must be similar, metadata setting, folder structure etc

TL;DR - Library parallel scanning is not the same as FFMPEG multithreading

From a Music aspect Emby currently isn't designed for multiple libraries as things get sort of get merged behind the scenes if they happen to match like wanting say a Flac and mp3 libraries separate but same content.