Jump to content

scan speed, if you have some more files


newby

Recommended Posts

newby

sorry for my bad english.

 

i have over 4000 movies and over 100000 mp3 files and over 20000 photos on my storage. it is so slow to scan my library. it takes one week.

 

and every scheduled scan it takes a week again...

 

i have a fast computer with ssd´s and raid storage. computer is not the problem.

 

please make a update to scan the database more times faster..... give me a tip on how I can speed up search all :D

 

 

regards newby

 

 

 

 

Link to comment
Share on other sites

Hi, thanks for the info. We are always looking to improve the performance of the library scan.

 

Do you mind doing a test? Can you see how long it takes without photos? Thanks.

Link to comment
Share on other sites

newby

Hi,

 

thank you for your answer.

 

without photos at all, or delete the photo-library?

 

Please give me a chance. i am new here, but i have install emby with standard-configuration. not with more pictures to search.

 

Please tell me what i must do...  

 

i am a emby premiere user.

 

regards

Edited by newby
Link to comment
Share on other sites

AdrianW

Just remove the library from Emby media configuration and then scan again.

Link to comment
Share on other sites

newby

Thank you for your answer.

 

i have removed photots from my library, But scan work at this time at 71 Percent (12 hours). scan works on mp3 files. scan search for every artist a photo and write nfo. i have over 25000 artists in several folders. emby can not handle so much files.

 

whow.... this is so slow.

 

problem is rescan. every 12 hours. computer has never a break.

Link to comment
Share on other sites

newby

Hello,

 

now.... scan has 73,9 percent. music-library scan.

 

in 5 hours 2,9 percent more. 

Edited by newby
Link to comment
Share on other sites

you can change the schedule of the library scan under scheduled tasks

Link to comment
Share on other sites

porkbone

I would try removing your music from the library also, then doing a scan.

 

Music slows my scans down, but not like when I first started. My initial scan with MediaBrowser 3 took 3.5 days to complete and around an hour every scan after that and was all primarily due to my music library layout and tagging. I went through all my music and retagged with MusicBrainz Picard and changed my folder layout to artists, soundtracks etc...

Today, without music in my library, my scan is under 2 minutes. with my music is around 15 to 25 minutes.

 

I have a large library too.

Link to comment
Share on other sites

newby

Hello , and thank you for your tip . I have now turned off MusicBrain and it goes a little bit faster . I think that Emby collects a lot of data . However, the internet site relatively slowly responding to the requests . Other Media-Server scan my library in 2 hours, but with less data.

 

Photo-Library is not the problem.

 

A request to the developer . Please reconsider the database structure . Faster database and possibly other suppliers of information can be helpful already .

 

 

Hello , and thank you for your tip . I have now turned off Music Brain and it goes a little bit faster . I think that Emby collects a lot of data . However, the internet site relatively slowly responding to the requests . Of course I want to and get to see all the information on the Internet for Emby . To this end, I have to bite the bullet and let end the scan well .

Link to comment
Share on other sites

TheShanMan

In my case I'm having trouble with photos. A year or two ago I tried adding photos to my library and had a horrible experience. Scans would take forever and as far as I could tell, would simply hang at some percentage.

 

About a week ago I figured I'd give it a try again. Same experience. It hangs at some percentage and after a day I finally give up. Additionally when I check Process Explorer, Emby is typically consuming 10's of gigabytes of ram and it has 1 cpu pegged.

 

If I remove photos from my library, it all returns to normal (emby becomes more responsive, it doesn't consume a ton of memory, and scans take ~ 2 minutes). FWIW I have ~43000 photos on a network share.

 

I assume posting a server log is the first step I should take, but how long should I capture logs before I give up and post them?

Link to comment
Share on other sites

aspdend

On my Raid server, I have over 4,000 video files and over 10,000 TV shows/music videos/home videos and over 70,000 mp3 and FLAC files and my scan takes an hour at the most. There were times previously when I have had issues that was causing problems and I would constantly receive Error messages and scans that stuck at 73% and similar very long scan times. My issues mainly boiled down to bad metadata on music files and mis-filing some TV series episodes as films and films as TV series etc.

 

In the first instance, I would suggest looking at your logs and trying to find out where the scanning is having issues and check the relevant folders. For example I have a lot of one off BBC documentaries that I had filed under a named folder, but not in a Season 1 sub-folder. So Emby treats it as a film and this caused issues with my scans. Adding a season 1 sub-folder and moving the video file into the sub-folder resolved a lot of issues for me.

 

TL:DR - make sure your files use the correct folder structure

Link to comment
Share on other sites

newby

Hello,

 

okay i have make some tests. 

 

My database is now large. 700000 kb  or 700 MB.
When I select the Artists view , it takes a long time, or there is no information .

Would it be possible that you divide the database into several smaller databases ? One for video , one for photos and one for music

It is even better to have a separate database for each letter.

I can imagine that this solution for the speed is very beneficial .

 

Unfortunately I do not know much about databases . But I know from other systems or media servers that the databases are always divided .

I many cases, I only see the circle in the center of the screen .

 

i have many devices in my house, but i can not use it.

Link to comment
Share on other sites

eblue

Since about two previous updates ago, my scan times have increased exponentially from about 1.5 hours to over 12 hours. I also have a rather large library of about 3500 videos, 85000 music files and about 12500 photos. I am not aware of anything that has changed in my library.

Link to comment
Share on other sites

Since about two previous updates ago, my scan times have increased exponentially from about 1.5 hours to over 12 hours. I also have a rather large library of about 3500 videos, 85000 music files and about 12500 photos. I am not aware of anything that has changed in my library.

 

Hi, welcome. Please make sure to see how to report a problem. Thanks!

Link to comment
Share on other sites

TheShanMan

In my case I'm having trouble with photos. A year or two ago I tried adding photos to my library and had a horrible experience. Scans would take forever and as far as I could tell, would simply hang at some percentage.

 

About a week ago I figured I'd give it a try again. Same experience. It hangs at some percentage and after a day I finally give up. Additionally when I check Process Explorer, Emby is typically consuming 10's of gigabytes of ram and it has 1 cpu pegged.

 

If I remove photos from my library, it all returns to normal (emby becomes more responsive, it doesn't consume a ton of memory, and scans take ~ 2 minutes). FWIW I have ~43000 photos on a network share.

 

I assume posting a server log is the first step I should take, but how long should I capture logs before I give up and post them?

 

Luke, could I draw your attention to the question at the end of my post?

Link to comment
Share on other sites

newby

700mb is tiny for a database, I would not worry about it.

 

 

It takes a long time until the data is displayed on the screen . Particularly , when the menu is selected with interpreters.

The Menu Video or Photo is relatively fast . It could be a little faster . But MP3 songs take an eternity .

 

 

I 'm not worried about the size of the database . I 'm worried that the whole process takes a long time .

It takes a long time until the data is displayed on the screen . Particularly , when the menu is selected with interpreters .

 

Can it be that the data is cached ? in the cache directory ?

And are then written in the meta-database ? Can it take so long .. ? Cache management .

Edited by newby
Link to comment
Share on other sites

Taurec

Hello Newby

you will see, a library scan with your amount of media files, will take a couple of hours.

So long there is no possibility to scan the library seperatly you have to work with this.

 

@ Luke

I have post nearly the same probs with scan a couple of mounth ago.

and again a suggestion::

- implement a seperate scan of the library (e.g. as option at the admin library page)

or

- scan the library and do nothing with the knowed files, only wih recently added files (like Plex)

 

Please think about the proposal.

 

cu Taurec

P.S. a library (4) scan with prox 5000 video and 35k episode take around 8 hours.

(at Plex, new files are available after prox 5min.)

(both running on QNap TVS871T with 16GB Ram and Intel 4790s  CPU)

Link to comment
Share on other sites

- implement a seperate scan of the library (e.g. as option at the admin library page)

We already have that. Are you not able to find it?

Link to comment
Share on other sites

Taurec

Hi Luke,

sorry no, i did not find it.

 

If you mean at the metadata manager, selected Library, under the option (close to the save button) the point actualisation then:

I'm sorry, Emby starts to scan the library (every file again), but new media files are not added.

 

hope you mean a another point :-)

 

cu Taurec

Link to comment
Share on other sites

In the server dashboard under Library settings you can start a scan.

Link to comment
Share on other sites

dcook

You can also start a scan manually by clicking the arrow next to the "Scan Media Library" under "Scheduled Tasks" section of the dashboard

Link to comment
Share on other sites

Taurec

Sorry guys,

there's a misunderstanding.

 

I talk about a separate start for the individual libraries.

But thanks for your info. (Both are used otherwise I would have no files in the libraries) ;)

 

cu Taurec

Link to comment
Share on other sites

TheShanMan

In my case I'm having trouble with photos. A year or two ago I tried adding photos to my library and had a horrible experience. Scans would take forever and as far as I could tell, would simply hang at some percentage.

 

About a week ago I figured I'd give it a try again. Same experience. It hangs at some percentage and after a day I finally give up. Additionally when I check Process Explorer, Emby is typically consuming 10's of gigabytes of ram and it has 1 cpu pegged.

 

If I remove photos from my library, it all returns to normal (emby becomes more responsive, it doesn't consume a ton of memory, and scans take ~ 2 minutes). FWIW I have ~43000 photos on a network share.

 

I assume posting a server log is the first step I should take, but how long should I capture logs before I give up and post them?

 

I was hoping to get some feedback here. I don't want to post a massive log that'll get ignored because it's too hard to look through nor do I want to grab the log file too early. If I just let it go for around an hour to make sure it captures everything necessary, will that work and will you have a look?

 

Sorry guys,

there's a misunderstanding.

 

I talk about a separate start for the individual libraries.

But thanks for your info. (Both are used otherwise I would have no files in the libraries) ;)

 

cu Taurec

 

Having the ability to separately schedule the individual collections would be SO nice. My movie/tv show collection changes way more often than my photos and music collections do. I'd love to have a lightning fast movie/tv show scan and only scan the other stuff at like 2 AM when I'd be much more tolerant of longer scan times.

 

Being able to set multiple scan schedules and being able to configure each one to either scan the whole library or individually choose collections would be awesome.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...