sesipod 0 Posted April 4, 2020 Share Posted April 4, 2020 I am new to emby I have added a library with around 1,000 episode of tv. The scan process is taking forever it's been going for Nealy 2+hrs now. Should it take this long? Before emby I was using Plex and the files would add immediately and be available to watch. What I am seeing here is that the files are being scanned but not appearing in the app or web view of emby for a long time. I have the new discord webhook bot and it's saying the episodes have been added. But they do not show up for streaming for quite some time after. As of this moment it seems that the media had been scanned completely so I don't understand what the delay is for it to be ready for streaming. ---- Nas permissions are set correctly emby:emby images are being added to where the episodes media folders are and subtitles are being download to the same folder. -- Server : Debian 10 CPU : 6 core 2.5ghz RAM : 8GB Storage : SSD for OS / nas for media. CPU usage is around 2% and ram usage is around 900mb - 1.2gb embyserver.txt Link to comment Share on other sites More sharing options...
Happy2Play 8252 Posted April 4, 2020 Share Posted April 4, 2020 Everything looks normal so far. Emby does take awhile to import content. Emby has to process/probe every item to get its media info. Without knowing the items media info Emby is unable to determine how to the stream the item. If you are still having issues after the initial library scan, we should be able to provide more information. Link to comment Share on other sites More sharing options...
sesipod 0 Posted April 4, 2020 Author Share Posted April 4, 2020 (edited) So it seems that all of the media has been added based on the discord bot. So at this point you are saying the delay is it's trying to analyze the media information as far as the type 264 or 265 / resolution such things like that? It almost seems also like a image download delay??? Could that also be a cause? Is something enabled under my library settings that could be causing the delay ? Coming from Plex I am able to add this same content and it's ready to stream and have all of media information and images in under 30min. Edited April 4, 2020 by sesipod Link to comment Share on other sites More sharing options...
Luke 37022 Posted April 4, 2020 Share Posted April 4, 2020 It all depends on the feature you enable. Please keep in mind everything you enable comes with a cost. I see you enabled MovieDb metadata for episodes, this will be considerably slower than tvdb due to the way their api works. And you also turned on some notification service based on what i see in your second screenshot. That's going to be sending out notifications during the library scan as new content is found, so that's not free either. Link to comment Share on other sites More sharing options...
sesipod 0 Posted April 4, 2020 Author Share Posted April 4, 2020 It all depends on the feature you enable. Please keep in mind everything you enable comes with a cost. I see you enabled MovieDb metadata for episodes, this will be considerably slower than tvdb due to the way their api works. And you also turned on some notification service based on what i see in your second screenshot. That's going to be sending out notifications during the library scan as new content is found, so that's not free either. The notifications have stopped at this point as it's notifyed me thats its added all of the media that I have in the folders. So should I stop the process and switch MovieDB to tvdb? Would I see a great improvement over the other? I'm just bothered by the delay for this small library as I have another one to add that is 4x as large. Then I have some 1,000+ movies to add next. Link to comment Share on other sites More sharing options...
sesipod 0 Posted April 4, 2020 Author Share Posted April 4, 2020 (edited) Looks like the MovieDB is set to the lowest priority already? Shouldn't it use the above options first and then lastly use that for everything it can't find with the others ? Edited April 4, 2020 by sesipod Link to comment Share on other sites More sharing options...
Luke 37022 Posted April 4, 2020 Share Posted April 4, 2020 Shouldn't it use the above options first and then lastly use that for everything it can't find with the others ? Yes it does, but it still runs the lower ones as well because they can be used to supplement with data that is not provided or supported by the higher ones. Link to comment Share on other sites More sharing options...
Happy2Play 8252 Posted April 4, 2020 Share Posted April 4, 2020 Well currently TVDB is having their own issues so they are currently hit and miss also. API downtime 4/4/20 Everyone's setup is different. With existing metadata and images my setup takes 24+ hours to import depending on initial settings. Just like it takes a couple days when I enable "Enable thumbnail image creation". Link to comment Share on other sites More sharing options...
sesipod 0 Posted April 4, 2020 Author Share Posted April 4, 2020 Yes it does, but it still runs the lower ones as well because they can be used to supplement with data that is not provided or supported by the higher ones.So if I'm following disable MovieDB and only enable it if the scan can't find all of the information? --- Another question I have is what if I delete the library and re-add it to a new server will the pre-existing downloaded images/subtitles in the media folders immediately be used? Or will they be replaced ? -- I also noticed I have download images in advance turned on. What is the average load speed for the images when requested by an app when a person clicks is there a noticeable delay? Link to comment Share on other sites More sharing options...
Luke 37022 Posted April 4, 2020 Share Posted April 4, 2020 So if I'm following disable MovieDB and only enable it if the scan can't find all of the information? Well it's really up to you to decide. If you're not OK with the time it adds to the library scan, then I would turn it off. Link to comment Share on other sites More sharing options...
Luke 37022 Posted April 4, 2020 Share Posted April 4, 2020 What is the average load speed for the images when requested by an app when a person clicks is there a noticeable delay? I don't have a number for you but I've always felt like it's pretty quick and it's only done once. After it's downloaded, the downloaded copy will be used from there on out. Link to comment Share on other sites More sharing options...
sesipod 0 Posted April 4, 2020 Author Share Posted April 4, 2020 Well currently TVDB is having their own issues so they are currently hit and miss also. API downtime 4/4/20 Everyone's setup is different. With existing metadata and images my setup takes 24+ hours to import depending on initial settings. Just like it takes a couple days when I enable "Enable thumbnail image creation". Ahh geez :/ So what I am hearing is set this up on what is going to be a long standing server and make backups of the server (VM). How about local drive space Currently I have 30GB allocated to the vm. Seeing as I'm telling emby to dump images and subtitles into the media folders will I need much more than 30GB? I can give more to the vm. Just would like to prepare for the future now. Well it's really up to you to decide. If you're not OK with the time it adds to the library scan, then I would turn it off. Okay so is there a way to tell it hey you didn't find everything now look at MovieDB for the rest? Would I just retick MovieDB and run a scan or ? Link to comment Share on other sites More sharing options...
Happy2Play 8252 Posted April 4, 2020 Share Posted April 4, 2020 Ahh geez :/ So what I am hearing is set this up on what is going to be a long standing server and make backups of the server (VM). How about local drive space Currently I have 30GB allocated to the vm. Seeing as I'm telling emby to dump images and subtitles into the media folders will I need much more than 30GB? I can give more to the vm. Just would like to prepare for the future now. Okay so is there a way to tell it hey you didn't find everything now look at MovieDB for the rest? Would I just retick MovieDB and run a scan or ? That is what the Provider priority already does. As for space 30Gb will vary. It really depends on how large your media library is. But the Metadata, Cache and Transcoding-temp folders are configurable. Primarily the Transcode-temp folder could eat all space quickly depending on transcoded media and how many transcodes at once. Emby maintains the entire media transcode until the item is stopped properly in the client, then clears that transcode from the folder. Link to comment Share on other sites More sharing options...
sesipod 0 Posted April 4, 2020 Author Share Posted April 4, 2020 (edited) That is what the Provider priority already does. As for space 30Gb will vary. It really depends on how large your media library is. But the Metadata, Cache and Transcoding-temp folders are configurable. Primarily the Transcode-temp folder could eat all space quickly depending on transcoded media and how many transcodes at once. Emby maintains the entire media transcode until the item is stopped properly in the client, then clears that transcode from the folder. Okay thanks --- I have the cache under setting location moved to my nas. I also have the metadata under library settings set to my nas as well. Also I have transcode and remix wothout rencode disabled for all of my users. All of my media is 100% h265 and all of my users use a phone app or roku / fire stick to play. So with these settings I assume I would be okay to have a smaller drive ? Edited April 4, 2020 by sesipod Link to comment Share on other sites More sharing options...
Happy2Play 8252 Posted April 4, 2020 Share Posted April 4, 2020 Okay thanks --- I have the cache under setting location moved to my nas. I also have the metadata under library settings set to my nas as well. Also I have transcode and remix wothout rencode disabled for all of my users. All of my media is 100% h265 and all of my users use a phone app or roku / fire stick to play. So with these settings I assume I would be okay to have a smaller drive ? These are not folders that both programs are going fight over are they? You can go with that on users setting but that is a big assumption all your media is playable/compatible on all devices. Link to comment Share on other sites More sharing options...
sesipod 0 Posted April 4, 2020 Author Share Posted April 4, 2020 These are not folders that both programs are going fight over are they? You can go with that on users setting but that is a big assumption all your media is playable/compatible on all devices. Plex is not currently running nor have I ever given Plex write access to the nas. It's always had to store it's files on the VM directly. Emby is the only thing running / has access to read and write to this folder structure. --- So far all of my users have been perfectly fine with streaming this way. I have around 10 people and they are happy family and a friend overseas really struggling to find good old American content where he is. I did have emby setup on windowsbut tore it all apart to start using Linux OS and the nas to hold the images / metadata / cache / subtitles. Link to comment Share on other sites More sharing options...
sesipod 0 Posted April 4, 2020 Author Share Posted April 4, 2020 (edited) Is Linux or Windows variant of emby quicker or slower? Or does it not really matter? So what I'm gathering here is this is down to the image providers / metadata providers that's causing the slowdown? Would a more beefier CPU and RAM make this library scan process faster? I have a i9-9900k at my disposal with 16gb of ram and a Samsung 500gb m.2 nvme Not doing much at all. If I use that it would be a baremetal install with windows 10. Not a VM install. As now I'm using a VM Debian based. Edited April 4, 2020 by sesipod Link to comment Share on other sites More sharing options...
Happy2Play 8252 Posted April 4, 2020 Share Posted April 4, 2020 It really shouldn't matter which OS, it is just a matter of getting the initial build done. It will take time no matter how powerful/fast the machine/drives are. Link to comment Share on other sites More sharing options...
sesipod 0 Posted April 4, 2020 Author Share Posted April 4, 2020 It really shouldn't matter which OS, it is just a matter of getting the initial build done. It will take time no matter how powerful/fast the machine/drives are. So from your post here it seems like an emby software limitation? Because me throwing an i9-9900k into the mix and you saying that won't make a difference and it will still take a long time points to an limitation of software or download providers for media info / metadata. I would expect a faster system to help over a slower system. Link to comment Share on other sites More sharing options...
Happy2Play 8252 Posted April 4, 2020 Share Posted April 4, 2020 (edited) I would not say limitation as I do not expect my media to be instantly available. I know Emby needs to Scan every item, probe every item and get metadata for every item. I see close to the same results in my X3450 and I7-8700 as the horsepower is irrelevant in this process. Edited April 4, 2020 by Happy2Play Link to comment Share on other sites More sharing options...
sesipod 0 Posted April 4, 2020 Author Share Posted April 4, 2020 (edited) Still does not make any since. Scanning a file should be faster on a faster system. I can understand if it's the data being downloaded that is causing a bottle neck because of API / provider limitations. But the file scan and reading of those files should 100% benefit from a faster system. It seems like the emby media scan process has a limitation. --- This same lib of data on Plex was scanned and images downloaded and ready to stream in under 30 min. I am on hr 5 now with emby and only about half of my media is ready to go now. ---- So since I have the metadata and everything being saved to my nas. If I delete the library and rebuild emby will re-add be faster because it already has the downloadable content saved and available? Edited April 4, 2020 by sesipod Link to comment Share on other sites More sharing options...
Happy2Play 8252 Posted April 4, 2020 Share Posted April 4, 2020 (edited) Still does not make any since. Scanning a file should be faster on a faster system. I can understand if it's the data being downloaded that is causing a bottle neck because of API / provider limitations. But the file scan and reading of those files should 100% benefit from a faster system. It seems like the emby media scan process has a limitation. --- This same lib of data on Plex was scanned and images downloaded and ready to stream in under 30 min. I am on hr 5 now with emby and only about half of my media is ready to go now. ---- So since I have the metadata and everything being saved to my nas. If I delete the library and rebuild emby will re-add be faster because it already has the downloadable content saved and available? I don't know all the work flow, but is does not matter how fast the system probes a item you are still throttle by how many items can get online metadata. So your processor burns through 10 items and has to wait on provider data to move on. There are many topics about being slow in the admins eye, but once it is done there is zero issue. And Plex has there own servers (mirrors) to get info from where Emby gets it directly from online source. Edited April 4, 2020 by Happy2Play Link to comment Share on other sites More sharing options...
sesipod 0 Posted April 4, 2020 Author Share Posted April 4, 2020 Thank you for the above answer - seems like something that should be corrected at some point as this is one of the main items people will do is add media to the server. Especially if they have a pre-existing library that's huge. Can you please provide some information to this question.? So since I have the metadata and everything being saved to my nas. If I delete the library and rebuild emby will the re-add be faster because it already has the downloadable content saved and available? Link to comment Share on other sites More sharing options...
Happy2Play 8252 Posted April 4, 2020 Share Posted April 4, 2020 Thank you for the above answer - seems like something that should be corrected at some point as this is one of the main items people will do is add media to the server. Especially if they have a pre-existing library that's huge. Can you please provide some information to this question.? So since I have the metadata and everything being saved to my nas. If I delete the library and rebuild emby will the re-add be faster because it already has the downloadable content saved and available? Are you talking about metadata/images saved along side media or the central metadata location? If the central location, then no as that is by databaseid and when you remove a library all databaseids are removed and new ones are generated when re-added. If with media then yes as Emby will prioritize local metadata/images first. Personally if I have metadata and images with media, I disable all library options during initial import. Then enable options and do a scan for missing later. Link to comment Share on other sites More sharing options...
sesipod 0 Posted April 5, 2020 Author Share Posted April 5, 2020 So I installed Emby directly on my windows 10 i9-9900K box and the Media added in about 30-40 min compared to using Emby on Linux. Switched back to Linux and I am seeing the same slow speed that I was when I made this topic. Windows 10 installed emby - Pointed to my network shares for cache / Lib metadata and the Media - Added the Folder form nas to emby and within about 30 - 40 min Everyone was able to see my media and start streaming. I rebuilt the Emby server on Linux and setup all of the shares again just like windows and added the Media to the lib - I am now on HR 2 and still do not have more than 50 episodes ready to stream.Very odd behavior here. Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now