Jump to content

Will the embarassingly slow initial scan ever get faster?


popeye246810

Recommended Posts

Weird. I have a mount on gdrive with around 1200 movies. Brand new emby instance with the defaults selected and the initial scan took...52 minutes.

John

Link to comment
Share on other sites

3 minutes ago, popeye246810 said:

Would you be willing to share you mount config, so I can give it a try please

Pretty basic config, actually - here's the vfs config I use. Everything else is just log location.

 --vfs-cache-mode full --vfs-cache-max-size 50G --vfs-cache-max-age 168h

 

Link to comment
Share on other sites

rbjtech
5 hours ago, popeye246810 said:

.. of the 20K lines in the log, 6.2K of them are file errors trying to retrieve the file.

So I think you need to fix your mounts first to get reliable file access and then try again.

If Plex/JF use the same mounts without issue - then over to the Dev's why this would be ..

image.thumb.png.2a022681e76727ac8cd4f5fac30d94e1.png

  • Agree 1
Link to comment
Share on other sites

1 hour ago, popeye246810 said:

Both Plex and Jellyfin are fine with the mounts exactly like this, it's only ever an issue with Emby

Do you run the emby service with the same user account as JF and/or plex?

 

 

Link to comment
Share on other sites

  • 2 weeks later...

Well, something is different because emby can't access the file paths when you're performing a media scan.

Are you hitting the gdrive API limit with rclone? Can you run the mount with rclone debug turned on?

Link to comment
Share on other sites

popeye246810

I'm at work at the minute but not hitting api limits, that's only ever happened a handful of times on over a year and most were when I first got it before I understood it.

 

You see I just redid my Plex library in order to tidy things up etc that took 3 days, Gabe up on Emby until someone figures out why it's so slow, until then I suppose I'm never gonna use it

 

 

 

Link to comment
Share on other sites

It's slow because it can't access the file path - it's nothing inherent with emby. Start by double-checking the file permissions for the user that emby is running as....

System.IO.DirectoryNotFoundException: System.IO.DirectoryNotFoundException: Could not find a part of the path 'P:\4K\Flash Gordon (1980)'

As a test, you could use the '--allow-other' flag on the rclone mount command - that should allow users other than the user running the rclone mount to access the files.

Nevermind that - '--allow-other' doesn't work on windows. You need to run the rclone mount as 'system user', if you're not already doing that.

Edited by jad3675
Link to comment
Share on other sites

popeye246810

Already running as system user, but what I don't understand is that Plex and Jellyfin don't have issue with this setup.  Literally nothing is different.

 

The only difference is I think Emby is using a custom version of ffmprobe? Maybe that's the issue? 

 

 

Link to comment
Share on other sites

2 minutes ago, popeye246810 said:

Already running as system user, but what I don't understand is that Plex and Jellyfin don't have issue with this setup.  Literally nothing is different.

 

The only difference is I think Emby is using a custom version of ffmprobe? Maybe that's the issue? 

 

 

Long shot, but could there be some weird hidden characters in the file path? Can you drill down through the library setup (where you select the root file path) and verify that emby can see those folders?

Link to comment
Share on other sites

  • 4 weeks later...
popeye246810

Sorry totally gave up after 9 days of scanning and still not finished, I switched to jellyfin, scanned the whole lot in 22 hours.  The issue is not my setup and is definitely Emby side, I re-setup Plex whilst I was at it and the took just over 2 days.  Just don't have the time to fix a setup that clearly works well with others but Emby.

 

I think you need to look into the scanning of new libraries, maybe dump the custom ffmpeg? I'm not exactly sure what the issue is, as I don't really know what goes on under the hood.  But as it stands Emby just is not an option for anyone with a cloud related setup

Link to comment
Share on other sites

18 minutes ago, popeye246810 said:

Sorry totally gave up after 9 days of scanning and still not finished, I switched to jellyfin, scanned the whole lot in 22 hours.  The issue is not my setup and is definitely Emby side, I re-setup Plex whilst I was at it and the took just over 2 days.  Just don't have the time to fix a setup that clearly works well with others but Emby.

 

I think you need to look into the scanning of new libraries, maybe dump the custom ffmpeg? I'm not exactly sure what the issue is, as I don't really know what goes on under the hood.  But as it stands Emby just is not an option for anyone with a cloud related setup

HI, there are improvements in the latest 4.8 release that should help with this. Thanks.

Link to comment
Share on other sites

popeye246810

I will give it a shot and let you know but unless it starts scanning considerably faster it's going to be a no go From me.

 

I want it to work so bad so I can test out the new features and maybe migrate from Plex

Link to comment
Share on other sites

1 hour ago, popeye246810 said:

I will give it a shot and let you know but unless it starts scanning considerably faster it's going to be a no go From me.

 

I want it to work so bad so I can test out the new features and maybe migrate from Plex

Previous builds were trying to probe subtitle streams individually and that has been removed. So that's pretty substantial.

  • Thanks 1
Link to comment
Share on other sites

popeye246810

This has fixed the issue.  Scans are flying now thank you very much for this cant wait to actually give it a real good go now

  • Like 1
Link to comment
Share on other sites

popeye246810

if we can move on now to the music section?  My issue here is plex will scan all artists and show me all album artists, Emby does the same but also shows all other tagged artists.  And leaves me with a huge job of trying to find pictures for all artists

 

is there a way to not have Emby list every single tagged artist and instead just list the album artists, of which i have all metadata for?

Link to comment
Share on other sites

1 hour ago, popeye246810 said:

if we can move on now to the music section?  My issue here is plex will scan all artists and show me all album artists, Emby does the same but also shows all other tagged artists.  And leaves me with a huge job of trying to find pictures for all artists

 

is there a way to not have Emby list every single tagged artist and instead just list the album artists, of which i have all metadata for?

Hi, where are you referring to? You could just use the Album Artists tab in your music library, right?

Link to comment
Share on other sites

popeye246810

Yes I could, but the issue is that where it says similar artists it will show me all the artists it has picked up and most are without pictures.  For instance I have about 1000 artists all tagged perfectly.  In Plex it will list 1000 artists, in Emby it will list about 4000+ artists.  

 

Could Emby Not just list the artists I have in folders like Plex does? I don't want to see all the other artists, only the ones I have in folders.

Link to comment
Share on other sites

22 minutes ago, popeye246810 said:

Yes I could, but the issue is that where it says similar artists it will show me all the artists it has picked up and most are without pictures.  For instance I have about 1000 artists all tagged perfectly.  In Plex it will list 1000 artists, in Emby it will list about 4000+ artists.  

 

Could Emby Not just list the artists I have in folders like Plex does? I don't want to see all the other artists, only the ones I have in folders.

We display the artists tagged in your audio files, but of course more options to control this are always possible for future updates.

Link to comment
Share on other sites

popeye246810

Just want to illustrate my point about what i mean

 

as you can see thats just 56 Album Artists but over 500 Artists i only actually have the 56 on my PC 

 

Emby shows me every single artist that is tagged and i mean everyone, where is Plex is far more intelligent and will only show the actual 56 that i have.

 

The issue here is i have the metadata for the 56 but not the 500+ so i end up with a load of mess on the server.  Emby is even worse if you add a compilation album.  For instance if you add now thats what i call music to plex it will show it as a compilation album under various Artists, and next to each track list each artist on that specific track.  This is perfect and how it should be done.

 

Emby will do the same, but it will also create an artist entry for each and every artist that appears on that album, this is not perfect at all.

 

I understand that Plex is quite superior at dealing with music than any other media server, and with their excellent standalone app which makes it harder to switch, but im really looking to do the switch, and originally it was scanning time but that now has been fixed, now its just the music section that finally needs fixing, by the time its finished with scanning of music it will leave me with 6000+ artists but i only have just over 1000 actual artists the rest to me is just garbage.

 

I understand that i can go and tag all my music so that i delete the extra artists that are on there and only have the main album artist, but this will not preserve the correct tagging

 

Hopefully someone will look at this and fix this age old problem

artist.JPG

album artists.JPG

Link to comment
Share on other sites

popeye246810

Maybe there could be an option added to not list all the artists and instead only list the album artists?  But the key here is to keep my correct tags.  so i want to see that the now thats what i call music will only be listed as various artists/compilations, but each track to also list other artists that contributed or featured

 

Heres an example of how Plex does it perfectly

 

What Plex does here is correct it adds the album as a various artist then it seperates it to a compilation then it has the artist for each song next to each song title, it will not create an artist entry for every song as that is the wrong way, emby will add an artist entry for each and every artist found in that compilation which no one wants but if i search for an artist i know is on the album it will still show up.

 

The only way to get emby to add this correclty is to remove all the artist tags and replace with Various Artists but that then destroys all the perfect tagging 

various.JPG

compilations.JPG

songs.JPG

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...