Jump to content


Photo

Library scan stuck at 86.4% forever

scan library 86.4 stuck

  • Please log in to reply
51 replies to this topic

#41 Malmgren OFFLINE  

Malmgren

    Advanced Member

  • Members
  • 41 posts
  • Local time: 01:01 PM

Posted 26 November 2019 - 04:27 PM

Anyone got anything out of reading my log?

 

The scanning of my movies actually have finished now, took about a week. So I started scanning my tv series, according to my calculations I should be able to actually start using Emby somewhere in late january.

 

/Daniel



#42 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 146173 posts
  • Local time: 07:01 AM

Posted 26 November 2019 - 04:37 PM

It's probing the media and making http requests to internet metadata providers. Can you attach a log from the TV scanning? Thanks.



#43 Malmgren OFFLINE  

Malmgren

    Advanced Member

  • Members
  • 41 posts
  • Local time: 01:01 PM

Posted 27 November 2019 - 01:50 AM

Attaching yesterdays log.

 

It seems like that for some periods it's really doing nothing at all for very long periods. And then suddenly things happen very fast, like all the mr Robot episodes that it traverses around 21:30.

 

/Daniel

Attached Files



#44 Malmgren OFFLINE  

Malmgren

    Advanced Member

  • Members
  • 41 posts
  • Local time: 01:01 PM

Posted 27 November 2019 - 08:47 AM

Great breakthrough!

 

First I tried installing Emby on a complete different machine. Got one running Ubuntu on an Intel i5 (as opposed to the first one with Debian on arm). Got the exact same problem, after three hours scanning where it had reached 86.5% I aborted. Then I thought about what the two machines had in common, and that was they were using the same movie library, nfs mounted.

 

So I tried mounting the exact same folder (shared from my d-link nas over both nfs and cifs) using cifs instead, wiped the installation and started over. And 15 minutes later my entire movie library is fully scanned! Happy happy joy joy!

 

So obviously there's something going on between Emby and nfs. I've been searching though and others seem to enjoy it just fine with nfs mounted movies so it doesn't seem to be general.

 

For me it isn't a problem, I can simply use cifs instead. But maybe it's worth digging into?

 

/Daniel

 

edit: And also on the original (arm) server I have now remapped the movie library to using cifs and it did the charm there as well. Didn't even remove the library, just stopped emby, unmounted and mounted, started emby and it continued scanning but now MUCH faster. So this is highly reproducable, at least on my servers.


Edited by Malmgren, 27 November 2019 - 09:25 AM.


#45 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 146173 posts
  • Local time: 07:01 AM

Posted 27 November 2019 - 09:49 AM

Thanks for the feedback.

#46 SNDS OFFLINE  

SNDS

    Newbie

  • Members
  • 4 posts
  • Local time: 04:01 AM

Posted 05 January 2020 - 01:02 AM

Hey folks,

 

I'm new on Emby coming from Plex. I'm having this same issue. My server is on an AMD Threadripper 1920x with 128gb of ram, direct SATAIII connections via a controller card to 6 12TB WD RED NAS drives. Plex has never had the kinds of problems that Emby is giving me when it comes to metadata gathering, media matching, etc. Even if I use the base defaults it goes quickly to 86.4% then just sticks there.

 

I really think you guys need to make a more comprehensive status system for library matching, thumbnail grabbing, subtitle grabbing, etc. In Plex, you're able to see what the state of the scan is using toast alerts in the bottom right. It shows which series is being processed, what element is being processed (imagery, thumbnails, etc). Additionally, you can still generally platy media files we even if they aren't matched yet, largely due to either my numerous threads (12-core, 24-threads) or due to hardware acceleration which I have enabled and connected to my Nvidia Quadro p2000.

 

I'm going to let my library sit for a few days and come back to it, but really right now, the processes running on my CPU are barely hammering it, and ram usage is low for the docker. Perhaps something is bugged? I'll get a log file to you in a few days to make sure I give the system a fair shake. 

 

Note: I am using the Emby supplied Docker via Unraid -> Community Applications Plugin.



#47 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 146173 posts
  • Local time: 07:01 AM

Posted 05 January 2020 - 01:03 AM

Hi, we'd have to look at the server log file, but most likely it's internet metadata downloads that are taking the longest.



#48 SNDS OFFLINE  

SNDS

    Newbie

  • Members
  • 4 posts
  • Local time: 04:01 AM

Posted 05 January 2020 - 01:06 AM

Hi, we'd have to look at the server log file, but most likely it's internet metadata downloads that are taking the longest.

Yep I do know this. As I said, I'll get you logs soon once it runs for a bit so you have some "meat" to chew on data-wise. The fact remains, however, none of the services Plex uses take this long. There's almost always immediate user interface feedback, the metadata and imagery shows up as soon as a match is found, then additional imagery and content is grabbed as a secondary priority. Perhaps the service that grabs metadata from the web tries to grab the whole kit and caboodle? 



#49 SNDS OFFLINE  

SNDS

    Newbie

  • Members
  • 4 posts
  • Local time: 04:01 AM

Posted 05 January 2020 - 01:08 AM

FWIW, I've attached the logs for today so far. I'll follow up with additional logs if needed.

Attached Files



#50 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 146173 posts
  • Local time: 07:01 AM

Posted 05 January 2020 - 02:16 AM

I notice you installed the anime plugin. This adds several metadata providers if I'm not mistaken so that will increase scanning times for tv libraries.



#51 SNDS OFFLINE  

SNDS

    Newbie

  • Members
  • 4 posts
  • Local time: 04:01 AM

Posted 05 January 2020 - 02:59 PM

Luke, I have to ask, are you skipping over ALL of the rest of my comments and only looking at the log file? I know I have additional plugins in there and according to the thread this process takes a while. My comments are to charge you guys with improving this experience and to try and replicate what your competition can do. I don't need you telling me it's going to be slow and that more plugins makes it slower, that's already a given.

 

Comparatively, Emby is about 10x slower than than the competition in this kind of functionality. 



#52 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 146173 posts
  • Local time: 07:01 AM

Posted 05 January 2020 - 03:03 PM

Yes we are always looking at improving it. Thanks for the feedback.


  • crunchyemby likes this





Also tagged with one or more of these keywords: scan, library, 86.4, stuck

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users