Scan stuck at 90%

[Lessaj 467]

1 hour ago, Sammy said:

What did you use to create this graph?

I have 17TB of Movies and Shows and my scans are usually pretty quick and complete without artificially hanging at 90%.

I took the ffprobe lines from the log provided (the lines that indicate what file it was looking at, and the lines indicating when it completed) and put them into excel, split the data into columns, added a column to calculate the time difference between between each log entry to get durations, then created a graph against the time and duration. If your log doesn't print these lines during your scan you wouldn't see this data, which I would expect for an already established library but this was a fresh import into a newly created library. I think I only see ffprobe entires in my logs when it's refreshing the data at the set intervals - the library option "after initial import, automatically refresh metadata from the internet".

I have about 35,000 items in my libraries and it takes about 3 minutes to complete a full scan on already fully established libraries, but I generally don't see any ffprobe entries in my log during that scan or just a few if they've fallen into that 30 day refresh that I set in the library options when the scan runs. Before running a scan just now to verify that number I saw the scan that ran earlier took over 20 minutes but I looked at my logs an there were a lot of probes occuring, so I would expect that to take longer.

[Neminem 1519]

I think I remember op telling us in other threads that he does not use a SSD in his server.

That is also a big factor to look at with many DB actions, when building a library from scratch.

[Lessaj 467]

If using a hard drive for the OS yes that can certainly have an impact on the performance of the DB as other parts of the OS fight for disk access at the same time, though I don't believe that's the case here since based on what I looked at the majority of the time is against probing the media files, which from what I recall in other topics are held within a large number of hard drives connected via USB and using Drive Pool. While this setup may work, in my opinion this has many performance implications which I expressed my thoughts in another topic about.

[rbjtech 5284]

11 minutes ago, Lessaj said:

If using a hard drive for the OS yes that can certainly have an impact on the performance of the DB as other parts of the OS fight for disk access at the same time, though I don't believe that's the case here since based on what I looked at the majority of the time is against probing the media files, which from what I recall in other topics are held within a large number of hard drives connected via USB and using Drive Pool. While this setup may work, in my opinion this has many performance implications which I expressed my thoughts in another topic about.

Agreed - and I mentioned this in my response also.

Drivepool is not the problem, as I use it myself, but USB connected drives (even USB 3.x) are simply not designed for high I/O

@ just over 30K items, my scan takes 47 seconds with zero new items to add.  That's using Drivepool with local SATA disks and Emby/DB on NVME.

A decent log viewer will read the emby logs, parse them and allow you to filter - I so wish emby would include one in the core product.    Emby have one already in their own 'Emby Support Tools' so really not sure what is stopping them.     This may help people find and attempt to resolve their own problems.