Jump to content


Photo

Library scan stuck at 86.4% forever

scan library 86.4 stuck

  • Please log in to reply
35 replies to this topic

#21 MRobi OFFLINE  

MRobi

    Advanced Member

  • Members
  • 271 posts
  • Local time: 02:52 AM

Posted 26 May 2019 - 10:46 AM

I don't know if it's a library size thing or just something invalid causing a halt somewhere. I've got a rather large library myself and the only time I've seen something stuck is when the database gets corrupted or a file somewhere is corrupt and Emby isn't handling an error correctly. SQLite isn't the best DB handling crashes or spontaneous reboots/crashes. Certainly not saying this is the issue but you should perform a database check.

At a minimum I'd shut Emby down and try a "sqlite3 library.db "PRAGMA integrity_check"
You can google the Pragma integrity_check for directions.

Another thing you could try to narrow down the problem to a particular library is to restart Emby then do a library refresh one by one (start with smallest and work your way up). You can "cat" your log file to watch progress and see it working vs just looking at the status display as well.

I'd offer to TeamView or remote in and give you a hand but I'm not familiar enough with your OS and wouldn't feel comfortable enough to help in this regard.

BTW, have you thought about starting to replace your 4 TB drives with bigger ones yet? 12 TB drives are pretty reasonable these days and really help with density.

I'm not the one having a library issue, I was simply offering that large libraries will take time to scan in.

12tb drives are still $500+ and with my setup I need to swap 4 at a time. I've still got around 10tb of usable space plus 4 slots available for more drives. When I fill this case I'll have to start looking at other options. But this is all off topic and not helping the OP so I'll back out of this thread now.

Sent from my LYA-L0C using Tapatalk

#22 cryobyte33 OFFLINE  

cryobyte33

    Member

  • Members
  • 11 posts
  • Local time: 01:52 AM

Posted 26 May 2019 - 10:55 AM

@cryobyte33 can you please attach a new emby server log? thanks!

Sure, here's the one from yesterday :) I haven't touched this since the 16th, so I'm not sure if it's really doing anything right now.

 

Edit: Just as an attempt to fix this, I shutdown the emby service, ran "sqlite3 com.plexapp.plugins.library.db "PRAGMA integrity_check"" and started the service back up. I'll post if that fixes this issue.

Attached Files


Edited by cryobyte33, 26 May 2019 - 11:00 AM.


#23 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 140179 posts
  • Local time: 01:52 AM

Posted 26 May 2019 - 01:12 PM

what kind of storage is your media contained on?



#24 cryobyte33 OFFLINE  

cryobyte33

    Member

  • Members
  • 11 posts
  • Local time: 01:52 AM

Posted 26 May 2019 - 01:15 PM

what kind of storage is your media contained on?

 

NFS over a 10G connection. I've never had a problem with it before, though.



#25 Malmgren OFFLINE  

Malmgren

    Member

  • Members
  • 10 posts
  • Local time: 07:52 AM

Posted 15 November 2019 - 06:22 AM

@cryobyte33, did you ever reach any conclusion here? I had the same problem when first setting up Emby (thread about that: https://emby.media/c...pping-at-864/).Kind of gave up Emby then, but I thought I'd give it a shot again. So I did a completely fresh install on a freshly installed server. And got the exact same problem. The progress reaches 86.4% in a matter of minutes and after that it's progressing VERY slowly. Been waiting four hours now and it's reached 86.6%.

 

From what I can get out of the log it seems that at this point it starts running ffprobe. This doesn't seem to be a good idea. On my system (a quad-core ARM Cortex-A53) it processes roughly one movie every 15 minutes. I have a quite small movie library, so I guess it will get through it in a few days, but it will have to work for quite some weeks on my tv series library.



#26 Sammy OFFLINE  

Sammy

    Advanced Member

  • Members
  • 2862 posts
  • Local time: 10:52 PM

Posted 15 November 2019 - 07:09 AM

This is the initial Thumbnail Extraction scan which is very processor intensive. Once done it won't be an issue going forward. Mine took 3 days on an i5 4690k Canyon Lake with about 1000 movies and several thousand TV Shows. Cortex ARM will take longer per file.

Sent from my SM-G960U1 using Tapatalk

#27 Happy2Play OFFLINE  

Happy2Play

    Trial and Error

  • Moderators
  • 16198 posts
  • Local time: 10:52 PM
  • LocationWashington State

Posted 15 November 2019 - 07:12 AM

Yes if you have thumbnail extraction enabled during library scan this can take awhile.  But that is a user configuration per library as it has its own task with a time limit, run nightly.  So we would need more information to determine what is actually happening.



#28 Malmgren OFFLINE  

Malmgren

    Member

  • Members
  • 10 posts
  • Local time: 07:52 AM

Posted 15 November 2019 - 09:26 AM

Yes if you have thumbnail extraction enabled during library scan this can take awhile.  But that is a user configuration per library as it has its own task with a time limit, run nightly.  So we would need more information to determine what is actually happening.

 

This is what it looks like in the log with thumbnail extraction disabled:

2019-11-15 13:39:24.021 Info MediaEncoder: ProcessRun 'ffprobe' Execute: /opt/emby-server/bin/ffprobe -i file:"/mnt/nas/public/Filmer/Star Wars Episode I - The Phantom Menace (1999)/Starwars episode I - the Phantom Menace.avi" -threads 0 -v info -print_format json -show_streams -show_chapters -show_format -show_data
2019-11-15 13:39:24.065 Info MediaEncoder: ProcessRun 'ffprobe' Started.
2019-11-15 13:39:24.959 Info MediaEncoder: ProcessRun 'ffprobe' Process exited with code 0
2019-11-15 13:48:24.091 Info MediaEncoder: ProcessRun 'ffprobe' Execute: /opt/emby-server/bin/ffprobe -i file:"/mnt/nas/public/Filmer/Snowden (2016)/Snowden.2016.720p.WEBRip.x264.AAC-ETRG.mp4" -threads 0 -v info -print_format json -show_streams -show_chapters -show_format -show_data
2019-11-15 13:48:24.137 Info MediaEncoder: ProcessRun 'ffprobe' Started.
2019-11-15 13:48:24.906 Info MediaEncoder: ProcessRun 'ffprobe' Process exited with code 0
2019-11-15 14:01:53.998 Info MediaEncoder: ProcessRun 'ffprobe' Execute: /opt/emby-server/bin/ffprobe -i file:"/mnt/nas/public/Filmer/Harry Potter och Halvblodsprinsen (2009)/Harry_Potter_and_the_Half_Blood_Prince_2009.mkv" -threads 0 -v info -print_format json -show_streams -show_chapters -show_format -show_data
2019-11-15 14:01:54.045 Info MediaEncoder: ProcessRun 'ffprobe' Started.
2019-11-15 14:01:54.525 Info MediaEncoder: ProcessRun 'ffprobe' Process exited with code 0
2019-11-15 14:22:24.092 Info MediaEncoder: ProcessRun 'ffprobe' Execute: /opt/emby-server/bin/ffprobe -i file:"/mnt/nas/public/Filmer/Sällskapsresan II - Snowroller (1985)/Snowroller.1985.swedish.DVDRip.XviD.CD2-Buddha.avi" -threads 0 -v info -print_format json -show_streams -show_chapters -show_format -show_data
2019-11-15 14:22:24.136 Info MediaEncoder: ProcessRun 'ffprobe' Started.
2019-11-15 14:22:24.755 Info MediaEncoder: ProcessRun 'ffprobe' Process exited with code 0

I realize now that it's not the ffprobe that takes time but rather something happening after that, not sure what?



#29 Happy2Play OFFLINE  

Happy2Play

    Trial and Error

  • Moderators
  • 16198 posts
  • Local time: 10:52 PM
  • LocationWashington State

Posted 15 November 2019 - 09:33 AM

Pretty sure that is just the process to get the MediaInfo for each item.  Has nothing to do with thumbnail extraction.  But that log snippet is normal for new items/initial library scan.



#30 Sammy OFFLINE  

Sammy

    Advanced Member

  • Members
  • 2862 posts
  • Local time: 10:52 PM

Posted 15 November 2019 - 09:39 AM

Post your whole log including transcode log.

Sent from my SM-G960U1 using Tapatalk

#31 cryobyte33 OFFLINE  

cryobyte33

    Member

  • Members
  • 11 posts
  • Local time: 01:52 AM

Posted 15 November 2019 - 10:00 AM

@cryobyte33, did you ever reach any conclusion here? I had the same problem when first setting up Emby (thread about that: https://emby.media/c...pping-at-864/).Kind of gave up Emby then, but I thought I'd give it a shot again. So I did a completely fresh install on a freshly installed server. And got the exact same problem. The progress reaches 86.4% in a matter of minutes and after that it's progressing VERY slowly. Been waiting four hours now and it's reached 86.6%.

 

From what I can get out of the log it seems that at this point it starts running ffprobe. This doesn't seem to be a good idea. On my system (a quad-core ARM Cortex-A53) it processes roughly one movie every 15 minutes. I have a quite small movie library, so I guess it will get through it in a few days, but it will have to work for quite some weeks on my tv series library.

 

Heya, I never actually found what the problem was, I waited for the scan for over 3 weeks and it never progressed any further, put any more entries into the log, nothing. Rebooted and same issue. I ended up trying a reinstall which was unsuccessful, then hail mary'd and tried Docker, which worked!

 

Not sure why Docker worked when several installs across 3 Linux distros wouldn't, but I just wanted my media back lol


  • Malmgren likes this

#32 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 140179 posts
  • Local time: 01:52 AM

Posted 15 November 2019 - 12:02 PM

I would just be patient and allow it to finish. Thanks.

#33 wondermouse OFFLINE  

wondermouse

    Newbie

  • Members
  • 4 posts
  • Local time: 07:52 AM

Posted Yesterday, 12:49 PM

I have a similar problem. The scan of my "TV Shows" library seemingly gets stuck after reaching 90%. This has not happened previously, and it still does not happen when scanning the "Movies" library.

I'm not aware of receiving a SW upgrade that could have caused this, and I'm not able to read much from the logs (although I have admittedly not spent much time trying to figure them out).

 

Setup

---------------------------------------------------------------
Mac mini (Mid 2011)
CPU: 2.3GHz dual-core Intel Core i5
GPU: Intel HD Graphics 3000
RAM: 16 GB
HD: Samsung SSD 860 EVO 250GB

OS: macOS High Sierra (10.13)
Emby Server Version 4.2.1.0

Media storage:
WD My Book 8TB
WD My Book 10TB

---------------------------------------------------------------

 

Hope someone here can be of help.

 

Cheers


Edited by wondermouse, Yesterday, 01:14 PM.


#34 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 140179 posts
  • Local time: 01:52 AM

Posted Yesterday, 12:53 PM

I have a similar problem. The scan of my "TV Shows" library seemingly gets stuck after reaching 90%. This has not happened previously, and it still does not happen when scanning the "Movies" library.

I'm not aware of receiving a SW upgrade that could have caused this, and I'm not able to read much from the logs (although I have admittedly not spent much time trying to figure them out).

 

Setup

---------------------------------------------------------------
Mac mini (Mid 2011)
CPU: 2.3GHz dual-core Intel Core i5
GPU: Intel HD Graphics 3000
RAM: 16 GB
HD: Samsung SSD 860 EVO 250GB

OS: macOS High Sierra (10.13)
Emby Server Version 4.2.1.0

Media storage:
WD My Book 8TB
WD My Book 10TB

---------------------------------------------------------------

 

Logs attached: emby_tv_scan_fail_20191117.7z

 

 

 

Hope someone here can be of some help.

 

Cheers

Hi there, did you forget to attach the log? But as stated above, the first time scan can take a little longer. I would just be patient and allow it to finish. Thanks.



#35 wondermouse OFFLINE  

wondermouse

    Newbie

  • Members
  • 4 posts
  • Local time: 07:52 AM

Posted Yesterday, 01:23 PM

Hi Luke,

 

Sorry, wasn't clear on how to post/reply. Attaching the logs.

 

This is not a first time scan. It's been working OK for quite some time, but for some reason it started to fail today.

It does seem to have gotten unstuck now, but I see that it has started adding existing media again, causing a horrible mess in the process :(

Attached Files


Edited by wondermouse, Yesterday, 01:25 PM.


#36 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 140179 posts
  • Local time: 01:52 AM

Posted Today, 01:49 AM

Can you please provide more detail? What does "horrible mess" mean?







Also tagged with one or more of these keywords: scan, library, 86.4, stuck

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users