Dizzy49 99 Posted October 20, 2024 Posted October 20, 2024 I am running Emby Server 4.8.7.0 on TrueNAS Scale from TrueCharts (Yes, I know TrueCharts is dead). I am trying to get my library cleaned up. I used Sonarr to rename all the files in my Anime folder, and then all the folders (Approx 450 folders, and 10k episodes). I expected the scan to take a while, I let it run while I was gone for a week and it was done when I got back. However, I found about 10 more folders that I needed to clean up. I did that and the scan it taking HOURS to run. When I use the browser, I see the progress wheel and % on the Anime library. It looks like it's scanning over and over and over again. I don't know if that's once for each folder or what, but only 10 of the folders changed, so I don't know why it would need to do a full scan like that on every folder. Additionally, it looked like it was done and moved on to the other libraries. Once they were done I couldn't see any more progress wheels, but it was stick at 94%, for over 3 hours. I went to Scheduled Tasks and stopped it. I had to stop it 3 more times before it quit trying to run. I don't know if it's stuck in some kind of loop, or what, but I need to do my TV Series and Movies, and while it's scanning, it's not picking up new stuff from other areas, and I don't have a month to wait for the scan to complete. Before I started this cleanup my scans would just take a few minutes. For resources, I have allocated 10000m cpu, 64Gb ram, and the nvidia 4060 to Emby. Would increasing the cpu and/or ram help? I have real time monitoring enabled. Is it trying to initiate a scan for every file/folder change or something? Ideas, thoughts, suggestions?
Dizzy49 99 Posted October 20, 2024 Author Posted October 20, 2024 FYI, I started a scan before I went to bed, we are coming up on 10hrs, and it's at 90.6%
Luke 42077 Posted October 20, 2024 Posted October 20, 2024 Hi there, let's look at an example of your issue: How to Report a Problem Thanks !
Dizzy49 99 Posted October 20, 2024 Author Posted October 20, 2024 (edited) embyserver.txt @Lukehere's the log. Edited October 20, 2024 by Dizzy49
Carlo 4560 Posted October 20, 2024 Posted October 20, 2024 The progress bar is deceiving in what's actually going on in the backend when adding new media. This is especially true when you have things like Download images in advance Video Preview Thumbnails enabled with as a scheduled task and when media is added Subtitle Downloaders You can turn those off for the initial scan then turn them on after the initial scan for new media added. You can also run jobs from scheduled tasks that do things like generate the thumbnails/Video Preview
Dizzy49 99 Posted October 21, 2024 Author Posted October 21, 2024 6 hours ago, Carlo said: The progress bar is deceiving in what's actually going on in the backend when adding new media. This is especially true when you have things like Download images in advance Video Preview Thumbnails enabled with as a scheduled task and when media is added Subtitle Downloaders You can turn those off for the initial scan then turn them on after the initial scan for new media added. You can also run jobs from scheduled tasks that do things like generate the thumbnails/Video Preview Yes, when I first ran it I saw that SubKiller was running through every episode and I turned that off. Later I went through every Library and changed the options to Run as Scheduled Task only. I stopped the task that was running, and when it started up again I let it run. Within an hour it had picked up the new episodes I had just added. It's now 7hrs later and again stuck at 90.4% Right now there is a whole bunch of: 2024-10-20 20:00:58.696 Debug Statistics: CalculateMovieCodecs-episode Almost Family: Related AF 1080p 2024-10-20 20:00:58.696 Debug Statistics: CalculateMovieCodecs-episode Futurama: Related to Items You've Viewed 4K 2024-10-20 20:00:58.696 Debug Statistics: CalculateMovieCodecs-episode Teachers (2016): Relationslut 1080p 2024-10-20 20:00:58.697 Debug Statistics: CalculateMovieCodecs-episode Manifest: Relative Bearing 1080p 2024-10-20 20:00:58.697 Debug Statistics: CalculateMovieCodecs-episode black-ish: Relatively Grown Man 1080p 2024-10-20 20:00:58.697 Debug Statistics: CalculateMovieCodecs-episode Farscape: Relativity 1080p 2024-10-20 20:00:58.698 Debug Statistics: CalculateMovieCodecs-episode Degrassi: Relax SDembyserver (1).txt 2024-10-20 20:00:58.698 Debug Statistics: CalculateMovieCodecs-episode Tokyo Revengers: Releap 1080p 2024-10-20 20:00:58.698 Debug Statistics: CalculateMovieCodecs-episode Angel: Release SD 2024-10-20 20:00:58.698 Debug Statistics: CalculateMovieCodecs-episode The X-Files: Release 1080p 2024-10-20 20:00:58.698 Debug Statistics: CalculateMovieCodecs-episode Law & Order: Release SD 2024-10-20 20:00:58.699 Debug Statistics: CalculateMovieCodecs-episode Black Clover: Release from Misfortune 1080p I uploaded another log file since I enabled logging for this one.
Happy2Play 9780 Posted October 21, 2024 Posted October 21, 2024 18 minutes ago, Dizzy49 said: Right now there is a whole bunch of: 2024-10-20 20:00:58.696 Debug Statistics: CalculateMovieCodecs-episode Almost Family: Related AF 1080p 2024-10-20 20:00:58.696 Debug Statistics: CalculateMovieCodecs-episode Futurama: Related to Items You've Viewed 4K 2024-10-20 20:00:58.696 Debug Statistics: CalculateMovieCodecs-episode Teachers (2016): Relationslut 1080p 2024-10-20 20:00:58.697 Debug Statistics: CalculateMovieCodecs-episode Manifest: Relative Bearing 1080p 2024-10-20 20:00:58.697 Debug Statistics: CalculateMovieCodecs-episode black-ish: Relatively Grown Man 1080p 2024-10-20 20:00:58.697 Debug Statistics: CalculateMovieCodecs-episode Farscape: Relativity 1080p 2024-10-20 20:00:58.698 Debug Statistics: CalculateMovieCodecs-episode Degrassi: Relax SDembyserver (1).txt 2024-10-20 20:00:58.698 Debug Statistics: CalculateMovieCodecs-episode Tokyo Revengers: Releap 1080p 2024-10-20 20:00:58.698 Debug Statistics: CalculateMovieCodecs-episode Angel: Release SD 2024-10-20 20:00:58.698 Debug Statistics: CalculateMovieCodecs-episode The X-Files: Release 1080p 2024-10-20 20:00:58.698 Debug Statistics: CalculateMovieCodecs-episode Law & Order: Release SD 2024-10-20 20:00:58.699 Debug Statistics: CalculateMovieCodecs-episode Black Clover: Release from Misfortune 1080p Just the statistics plugin doin its thing via its schedule task. But should not stop library scan but may just slow it down just like any other task.
Dizzy49 99 Posted October 21, 2024 Author Posted October 21, 2024 I am at 92.9% right now. It's 8:19am right now. It's either been running since around 2pm yesterday, or it restarted when the log rolled over and has been running since midnight. embyserver.txt
Happy2Play 9780 Posted October 21, 2024 Posted October 21, 2024 Yep as initial scans can take time and all scheduled task still happen making it even longer. Search "taskmanager" (42 hits in 1 file of 1 searched) C:\Users\Media\Downloads\embyserver - 2024-10-21T072640.110.txt (42 hits) Line 49: 2024-10-21 00:00:00.002 Debug TaskManager: Rotate log file Completed after 0 minute(s) and 0 seconds Line 2130: 2024-10-21 00:04:59.998 Debug TaskManager: DailyTrigger fired for task: Trim Db Line 2131: 2024-10-21 00:04:59.998 Debug TaskManager: Queueing task TaskCleanDb Line 2132: 2024-10-21 00:04:59.998 Debug TaskManager: Executing Trim Db Line 2135: 2024-10-21 00:05:00.023 Debug TaskManager: Trim Db Completed after 0 minute(s) and 0 seconds Line 8606: 2024-10-21 00:19:59.996 Debug TaskManager: DailyTrigger fired for task: User Activity Notification Line 8607: 2024-10-21 00:19:59.996 Debug TaskManager: Queueing task TaskNotifictionUserReport Line 8608: 2024-10-21 00:19:59.996 Debug TaskManager: Executing User Activity Notification Line 8612: 2024-10-21 00:20:00.000 Debug TaskManager: User Activity Notification Completed after 0 minute(s) and 0 seconds Line 16489: 2024-10-21 00:38:04.174 Debug TaskManager: IntervalTrigger fired for task: Refresh Internet Channels Line 16490: 2024-10-21 00:38:04.174 Debug TaskManager: Queueing task RefreshChannelsScheduledTask Line 16491: 2024-10-21 00:38:04.174 Debug TaskManager: Executing Refresh Internet Channels Line 16492: 2024-10-21 00:38:04.380 Debug TaskManager: Refresh Internet Channels Completed after 0 minute(s) and 0 seconds Line 31442: 2024-10-21 01:08:28.953 Debug TaskManager: IntervalTrigger fired for task: Refresh Emby Connect Data Line 31443: 2024-10-21 01:08:28.953 Debug TaskManager: Queueing task RefreshAuthorizationsScheduledTask Line 31444: 2024-10-21 01:08:28.953 Debug TaskManager: Executing Refresh Emby Connect Data Line 31445: 2024-10-21 01:08:28.954 Debug TaskManager: Refresh Emby Connect Data Completed after 0 minute(s) and 0 seconds Line 53251: 2024-10-21 01:59:59.997 Debug TaskManager: DailyTrigger fired for task: Thumbnail image extraction Line 53252: 2024-10-21 01:59:59.998 Debug TaskManager: Queueing task ChapterImagesTask Line 53253: 2024-10-21 01:59:59.998 Debug TaskManager: Executing Thumbnail image extraction Line 53254: 2024-10-21 01:59:59.998 Debug TaskManager: Scheduled task Thumbnail image extraction will run with a time limit of 240 minutes Line 56743: 2024-10-21 02:01:44.593 Debug TaskManager: Thumbnail image extraction Completed after 1 minute(s) and 44 seconds Line 81641: 2024-10-21 02:59:59.997 Debug TaskManager: DailyTrigger fired for task: Create Playlists Line 81642: 2024-10-21 02:59:59.997 Debug TaskManager: DailyTrigger fired for task: Refresh Recently Aired Episodes Line 81643: 2024-10-21 02:59:59.997 Debug TaskManager: Queueing task TaskRefresh Line 81644: 2024-10-21 02:59:59.997 Debug TaskManager: Queueing task TaskCreatePlaylists Line 81645: 2024-10-21 02:59:59.997 Debug TaskManager: Executing Refresh Recently Aired Episodes Line 81646: 2024-10-21 02:59:59.997 Debug TaskManager: Executing Create Playlists Line 81648: 2024-10-21 02:59:59.997 Debug TaskManager: Create Playlists Completed after 0 minute(s) and 0 seconds Line 82717: 2024-10-21 03:00:03.084 Debug TaskManager: Refresh Recently Aired Episodes Completed after 0 minute(s) and 3 seconds Line 167524: 2024-10-21 04:59:59.998 Debug TaskManager: DailyTrigger fired for task: Change episode date added to original airdate Line 167525: 2024-10-21 04:59:59.998 Debug TaskManager: Queueing task TLTVScheduledTask Line 167526: 2024-10-21 04:59:59.998 Debug TaskManager: Executing Change episode date added to original airdate Line 171956: 2024-10-21 05:00:06.327 Debug TaskManager: Change episode date added to original airdate Completed after 0 minute(s) and 6 seconds Line 217177: 2024-10-21 06:00:00.001 Debug TaskManager: DailyTrigger fired for task: New Media Notification Line 217178: 2024-10-21 06:00:00.001 Debug TaskManager: Queueing task TaskNotifictionMediaReport Line 217179: 2024-10-21 06:00:00.001 Debug TaskManager: Executing New Media Notification Line 217198: 2024-10-21 06:00:04.436 Debug TaskManager: New Media Notification Completed after 0 minute(s) and 4 seconds Line 246787: 2024-10-21 07:08:28.957 Debug TaskManager: IntervalTrigger fired for task: Refresh Emby Connect Data Line 246788: 2024-10-21 07:08:28.957 Debug TaskManager: Queueing task RefreshAuthorizationsScheduledTask Line 246789: 2024-10-21 07:08:28.957 Debug TaskManager: Executing Refresh Emby Connect Data Line 246790: 2024-10-21 07:08:28.957 Debug TaskManager: Refresh Emby Connect Data Completed after 0 minute(s) and 0 seconds But in the end the percentage displayed is really deceiving as yes all the small tasks complete quickly but that last 8-10% is where all the heavy lifting is done per item.
Carlo 4560 Posted October 21, 2024 Posted October 21, 2024 Just be patient and let it finish. There is a lot of tasks being performed when you initially scan your libraries the first time.
Luke 42077 Posted October 21, 2024 Posted October 21, 2024 Hi, please try removing these plugins: Anime 1.5.6.0 Episode Refresh 1.0.0.1 Genre Cleaner 1.0.25.0 Media Info 1.0.1.20 Playback Reporting 2.1.0.5 Statistics 3.3.1.1 Studio Cleaner 3.2.8.0 TimeLordTV 19.8.3.1105 Then restart the server and see how things compare. Thanks.
Dizzy49 99 Posted October 22, 2024 Author Posted October 22, 2024 First off, I've been at 90% for over 12hrs now. When I check the log it IS doing something, it looks like a lot of the "fingerprinting" ? Also, I am seeing a lot of lines for the stuff in /media/Sort/ Quote 2024-10-22 16:43:00.553 Info FfmpegManager: ProcessRun 'CreateTitleFingerprint' Process exited with code 0 - Succeeded 2024-10-22 16:43:00.553 Info App: CreateTitleFingerprint completed for /media/Video/TV Series/Halo (2022) [tvdb-366524]/Season 01/Halo (2022) - S01E07 - Inheritance [HDTV-2160p][DV HDR10][EAC3 Atmos 5.1][x265]-DV.mkv to /config/cache/temp/9a3c4863d2154ef685cc317c18f64487 2024-10-22 16:43:00.553 Info App: Moving fingerprint file from /config/cache/temp/9a3c4863d2154ef685cc317c18f64487 to /config/metadata/library/40/408e0cd952692f09e44439349a3e74fc/title_10_637873272000000000.fp 2024-10-22 16:43:00.555 Info FfmpegManager: ProcessRun 'CreateTitleFingerprint' Execute: /app/emby/ffmpeg -i file:"/media/Video/TV Series/Halo (2022) [tvdb-366524]/Season 01/Halo (2022) - S01E08 - Allegiance [HDTV-2160p][DV HDR10][EAC3 Atmos 5.1][x265]-DV.mkv" -t 00:10:00.000 -ac 1 -acodec pcm_s16le -ar 16000 -vn -sn -f chromaprint -fp_format raw "/config/cache/temp/7f07312102034a60956cfaaa05f2fe64" 2024-10-22 16:43:04.413 Info FfmpegManager: ProcessRun 'CreateTitleFingerprint' Process exited with code 0 - Succeeded
Solution Luke 42077 Posted October 23, 2024 Solution Posted October 23, 2024 Fingerprinting is the into detection feature. If that is taking too long for your comfort level then I would consider disabling it. 1
Dizzy49 99 Posted October 28, 2024 Author Posted October 28, 2024 On 10/23/2024 at 8:30 AM, Luke said: Fingerprinting is the into detection feature. If that is taking too long for your comfort level then I would consider disabling it. That certainly helped, odd though since I could have sworn I had disabled that already. 1
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now