TV library constantly scanning in a loop

[CharleyVarrick 283]

Log included

embyserver(1).txt

Edited March 30, 2022 by CharleyVarrick

[Luke 42078]

Hi there, why do you think it's constantly scanning?

[CharleyVarrick 283]

7 hours ago, Luke said:

Hi there, why do you think it's constantly scanning?

That little thing...

[ebr 16176]

Hi.  Do you have real time monitoring enabled and possibly some system updating files in that library's folders?

[seanbuff 1316]

I'm going to piggy-back off this topic because I have also noticed the same thing, major difference being OP is on Stable, and I am running latest Beta Server (4.7.0.32)

Seeing the progress circle appears to be continuously looping and scanning my TV Shows library outside of a typical scheduled scan.

When i checked the logs, it seems to be throwing continuous errors for both TVDB and TMDB, for two particular series only.

TVDB Error:

2022-03-31 16:34:46.293 Info HttpClient: GET https://api4.thetvdb.com/v4/series/84006/episodes/official/
  ?page=0
2022-03-31 16:34:46.575 Info HttpClient: GET https://api4.thetvdb.com/v4/episodes/6245785/extended
2022-03-31 16:34:46.877 Error App: Error in TheTVDB
	*** Error Report ***
	Version: 4.7.0.32
	Command line: C:\Users\<username_obfuscated>\AppData\Roaming\Emby-Server\system\EmbyServer.dll -noautorunwebapp
	Operating system: Microsoft Windows 10.0.19044
	Framework: .NET 6.0.2
	OS/Process: x64/x64
	Runtime: C:/Users/<username_obfuscated>/AppData/Roaming/Emby-Server/system/System.Private.CoreLib.dll
	Processor count: 16
	Data path: C:\Users\<username_obfuscated>\AppData\Roaming\Emby-Server\programdata
	Application path: C:\Users\<username_obfuscated>\AppData\Roaming\Emby-Server\system
	System.NullReferenceException: System.NullReferenceException: Object reference not set to an instance of an object.
	   at Tvdb.TvdbEpisodeProvider.GetMetadata(RemoteMetadataFetchOptions`1 options, CancellationToken cancellationToken)
	   at MediaBrowser.Providers.Manager.MetadataService`2.ExecuteRemoteProviders(MetadataResult`1 temp, LibraryOptions libraryOptions, String logName, TIdType id, IEnumerable`1 providers, MetadataRefreshOptions options, CancellationToken cancellationToken)
	Source: Tvdb
	TargetSite: Void MoveNext()

 

TMDB Error for the same series:

2022-03-31 16:34:45.072 Info HttpClient: GET https://api.themoviedb.org/3/tv/8519/season/1?api_key=x_secret1_x&append_to_response=images,keywords,external_ids,credits,videos&language=
  &include_image_language=
  ,null,en
2022-03-31 16:34:45.311 Error App: Error in TheMovieDb
	*** Error Report ***
	Version: 4.7.0.32
	Command line: C:\Users\<username_obfuscated>\AppData\Roaming\Emby-Server\system\EmbyServer.dll -noautorunwebapp
	Operating system: Microsoft Windows 10.0.19044
	Framework: .NET 6.0.2
	OS/Process: x64/x64
	Runtime: C:/Users/<username_obfuscated>/AppData/Roaming/Emby-Server/system/System.Private.CoreLib.dll
	Processor count: 16
	Data path: C:\Users\<username_obfuscated>\AppData\Roaming\Emby-Server\programdata
	Application path: C:\Users\<username_obfuscated>\AppData\Roaming\Emby-Server\system
	System.IO.IOException: System.IO.IOException: The filename, directory name, or volume label syntax is incorrect. : 'C:\Users\<username_obfuscated>\AppData\Roaming\Emby-Server\programdata\cache\tmdb-tv\8519\season-1-
  .json'
	   at Microsoft.Win32.SafeHandles.SafeFileHandle.CreateFile(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options)
	   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize)
	   at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize)
	   at System.IO.Strategies.FileStreamHelpers.ChooseStrategy(FileStream fileStream, String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, Int64 preallocationSize)
	   at System.IO.FileStream..ctor(String path, FileStreamOptions options)
	   at Emby.Server.Implementations.IO.ManagedFileSystem.GetFileStream(String path, FileOpenMode mode, FileAccessMode access, FileShareMode share, Int32 bufferSize, FileOpenOptions fileOpenOptions, Int64 preAllocationSize)
	   at Emby.Server.Implementations.IO.ManagedFileSystem.GetFileStream(String path, FileOpenMode mode, FileAccessMode access, FileOpenOptions fileOpenOptions)
	   at Emby.Common.Implementations.Serialization.JsonSerializer.SerializeToFile(Object obj, String file)
	   at MovieDb.MovieDbSeasonProvider.EnsureSeasonInfo(String tmdbId, Int32 seasonNumber, String language, String preferredMetadataCountry, IDirectoryService directoryService, CancellationToken cancellationToken)
	   at MovieDb.MovieDbSeasonProvider.GetMetadata(RemoteMetadataFetchOptions`1 options, CancellationToken cancellationToken)
	   at MediaBrowser.Providers.Manager.MetadataService`2.ExecuteRemoteProviders(MetadataResult`1 temp, LibraryOptions libraryOptions, String logName, TIdType id, IEnumerable`1 providers, MetadataRefreshOptions options, CancellationToken cancellationToken)
	Source: System.Private.CoreLib
	TargetSite: Microsoft.Win32.SafeHandles.SafeFileHandle CreateFile(System.String, System.IO.FileMode, System.IO.FileAccess, System.IO.FileShare, System.IO.FileOptions)

 

You can then see ffprobe continuously interrogating the same files over and over again.

2022-03-31 16:33:22.803 Info MediaProbeManager: ProcessRun 'ffprobe' Execute: C:\Users\<username_obfuscated>\AppData\Roaming\Emby-Server\system\ffprobe.exe -i file:"\\MediaNAS\TV Shows\Man v. Food\Season 5\Man v. Food - S05E01 - New York City, NY HDTV-720p.mkv" -threads 0 -v info -print_format json -show_streams -show_chapters -show_format -show_data
2022-03-31 16:33:22.859 Info MediaProbeManager: ProcessRun 'ffprobe' Process exited with code 0 - Succeeded

This goes on for hours. No activity internally or externally touching those files.

Happy to split this into a separate topic if you feel my issue is not the same as OPs.

embyserver.zip

[CharleyVarrick 283]

12 hours ago, ebr said:

Hi.  Do you have real time monitoring enabled and possibly some system updating files in that library's folders?

Hi,

real time monitoring enabled, always been.

Unsure what is meant by "...some system updating..." Can you give me an example ?

[ebr 16176]

1 hour ago, CharleyVarrick said:

Unsure what is meant by "...some system updating..." Can you give me an example ?

Anything that would be editing or writing files to the folders in that library.

[CharleyVarrick 283]

2 hours ago, ebr said:

Anything that would be editing or writing files to the folders in that library.

Nothing I am aware of.

I use Sonarr, but only the Calendar feature. In my limited understanding, that would not edit/write, just read (?)

Yesterday, Emby was scanning my TV show libray 24/24 non stop, but oddly enough, it is leaving it alone today.

EDIT: endless tv library scanning resumed.

Edited March 31, 2022 by CharleyVarrick

[ebr 16176]

If you turn off real time monitoring, does it stop?

Do you have more than one Emby server pointing to the same locations?

[CharleyVarrick 283]

21 hours ago, ebr said:

If you turn off real time monitoring, does it stop?

Do you have more than one Emby server pointing to the same locations?

I turned off rtm 30 minutes ago, and once the spinning circle s-l-o-w-l-y reached 100%, it then restarted a new scan.

Do you recommend I restart Emby (or the computer hosting it)

Only one Emby server.

This happened to me some years ago; Back then, from the log, we were able to find Emby querying for a deleted tmdb ID.

 

Edited April 2, 2022 by CharleyVarrick

[Luke 42078]

Yes please try restarting the server after making that change. Thanks.

[CharleyVarrick 283]

16 hours ago, Luke said:

Yes please try restarting the server after making that change. Thanks.

Restarting Emby via dashboard did not stop constant scanning.

Restarting the host pc seemed to have stopped it, at least for now.

Time will tell, but even if it works, having real time monitor turned off is not ideal.

 

[Spaceboy 2573]

i have had to turn off rtm on all my libraries apart from recorded tv, else i never get to see recently recorded programs added to that library which is my priority. i have quite alot of stuff added to my library on a daily basis (partly media and seperately subtitles) and it just seemed that emby was performing constant scans of those libraries and actually not really picking up anything.

part of the issue is the lack of visibility of what rtm or library scanning is actually doing to be able to narrow down what the problem is.

I've now just set it up so i have one daily addition of media, followed by tiny media manager adding the metadata, followed by emby doing a full library scan. it works well enough

[CharleyVarrick 283]

13 minutes ago, Spaceboy said:

i have had to turn off rtm on all my libraries apart from recorded tv, else i never get to see recently recorded programs added to that library which is my priority. i have quite alot of stuff added to my library on a daily basis (partly media and seperately subtitles) and it just seemed that emby was performing constant scans of those libraries and actually not really picking up anything.

part of the issue is the lack of visibility of what rtm or library scanning is actually doing to be able to narrow down what the problem is.

I've now just set it up so i have one daily addition of media, followed by tiny media manager adding the metadata, followed by emby doing a full library scan. it works well enough

An important part of my morning routine is adding yesterday's tv episodes to Emby, and then a minute later, seeing them added under Latest TV Shows.

I was convinced this was made possible by RTM, but lo and behold, it still works as intended, with RTM switched off.

Now I'm really confused about the role of RTM.

Edited April 3, 2022 by CharleyVarrick

[Luke 42078]

14 hours ago, CharleyVarrick said:

An important part of my morning routine is adding yesterday's tv episodes to Emby, and then a minute later, seeing them added under Latest TV Shows.

I was convinced this was made possible by RTM, but lo and behold, it still works as intended, with RTM switched off.

Now I'm really confused about the role of RTM.

You must have triggered a library scan manually or just gotten lucky and had the scheduled scan run shortly thereafter, or perhaps did not successfully disable the realtime monitor on the particular library in question.

[CharleyVarrick 283]

14 hours ago, Luke said:

You must have triggered a library scan manually or just gotten lucky and had the scheduled scan run shortly thereafter, or perhaps did not successfully disable the realtime monitor on the particular library in question.

Within the usual 1 minute or so, all new episodes added shows up under Latest TV Shows, this is happening hours before or after scheduled scan, at any time of the day.

So you're telling me it should not, with RTM apparently turned off ?

 

Edited April 4, 2022 by CharleyVarrick

[Happy2Play 9780]

2 hours ago, CharleyVarrick said:

So you're telling me it should not, with RTM apparently turned off ?

Correct, can you past a server log showing this.

[CharleyVarrick 283]

3 hours ago, Happy2Play said:

Correct, can you past a server log showing this.

give me 15 minutes or so, I will have 2 new episodes to add.

In the meantime...

[CharleyVarrick 283]

4 hours ago, Happy2Play said:

Correct, can you past a server log showing this.

I added 2 new episodes to Emby at 10:35, and they appeared under Latest TV Shows 3 minutes later

 

embyserver(2).txt

Edited April 4, 2022 by CharleyVarrick

[Luke 42078]

4 hours ago, CharleyVarrick said:

I added 2 new episodes to Emby at 10:35, and they appeared under Latest TV Shows 3 minutes later

 

embyserver(2).txt 43.52 kB · 0 downloads

It looks like you've still got the realtime monitor enabled on your TV library.

[CharleyVarrick 283]

2 hours ago, Luke said:

It looks like you've still got the realtime monitor enabled on your TV library.

and yet:

[Happy2Play 9780]

@Lukeis a restart required to stop RTM/LibraryMonitor or is there a delay for it to stop?

I unchecked RTM on the library and still see librarymonitor in the log.  Basically, the same events charleyvarrick showed.

2022-04-04 14:46:13.888 Debug LibraryMonitor: New file refresher created for C:\Users\Media\Desktop\Videos\TV Shows - nfo\Big Sky\Season 2\New Text Document.txt
2022-04-04 14:46:13.918 Debug LibraryMonitor: Resetting file refresher from C:\Users\Media\Desktop\Videos\TV Shows - nfo\Big Sky\Season 2\New Text Document.txt to C:\Users\Media\Desktop\Videos\TV Shows - nfo\Big Sky\Season 2
2022-04-04 14:47:20.431 Debug LibraryMonitor: Timer stopped.
2022-04-04 14:47:20.433 Info LibraryMonitor: Season 2 (C:\Users\Media\Desktop\Videos\TV Shows - nfo\Big Sky\Season 2) will be refreshed.
2022-04-04 14:47:20.503 Debug App: Running FFProbeProvider for C:\Users\Media\Desktop\Videos\TV Shows - nfo\Big Sky\Season 2\S02E03.strm
2022-04-04 14:47:20.504 Debug App: Running EpisodeXmlProvider for C:\Users\Media\Desktop\Videos\TV Shows - nfo\Big Sky\Season 2\S02E03.strm
2022-04-04 14:47:20.504 Debug App: EpisodeXmlProvider returned no metadata for C:\Users\Media\Desktop\Videos\TV Shows - nfo\Big Sky\Season 2\S02E03.strm
2022-04-04 14:47:20.504 Debug App: Running EpisodeNfoProvider for C:\Users\Media\Desktop\Videos\TV Shows - nfo\Big Sky\Season 2\S02E03.strm
2022-04-04 14:47:20.504 Debug App: EpisodeNfoProvider returned no metadata for C:\Users\Media\Desktop\Videos\TV Shows - nfo\Big Sky\Season 2\S02E03.strm
2022-04-04 14:47:20.504 Debug App: Running TvdbEpisodeProvider for C:\Users\Media\Desktop\Videos\TV Shows - nfo\Big Sky\Season 2\S02E03.strm
2022-04-04 14:47:20.505 Debug App: Running MovieDbEpisodeProvider for C:\Users\Media\Desktop\Videos\TV Shows - nfo\Big Sky\Season 2\S02E03.strm

Restarted Emby and retested and now no RTM/LibraryMonitor, no item added.  Ran library scan to add item.

Edited April 4, 2022 by Happy2Play

[Luke 42078]

Yes currently changing the realtime monitor only takes effect either after a restart, or a scan of all libraries (manual or scheduled). It's something that needs to be improved to make it more immediate.

[seanbuff 1316]

@Luke any insights into the information I posted above?

5 days later and the same series episodes are being continuously scanned non-stop.

2022-04-05 09:01:59.951 Info MediaProbeManager: ProcessRun 'ffprobe' Execute: C:\Users\<username_obfuscated>\AppData\Roaming\Emby-Server\system\ffprobe.exe -i file:"\\MediaNAS\TV Shows\Man v. Food\Season 5\Man v. Food - S05E01 - New York City, NY HDTV-720p.mkv" -threads 0 -v info -print_format json -show_streams -show_chapters -show_format -show_data
2022-04-05 09:02:00.079 Info MediaProbeManager: ProcessRun 'ffprobe' Process exited with code 0 - Succeeded
2022-04-05 09:02:00.098 Info HttpClient: GET https://api4.thetvdb.com/v4/series/84006/episodes/official/
  ?page=0
2022-04-05 09:02:00.241 Info HttpClient: GET https://api4.thetvdb.com/v4/episodes/6245785/extended
2022-04-05 09:02:00.542 Error App: Error in TheTVDB
	*** Error Report ***
	Version: 4.7.0.32
	Command line: C:\Users\<username_obfuscated>\AppData\Roaming\Emby-Server\system\EmbyServer.dll -noautorunwebapp
	Operating system: Microsoft Windows 10.0.19044
	Framework: .NET 6.0.2
	OS/Process: x64/x64
	Runtime: C:/Users/<username_obfuscated>/AppData/Roaming/Emby-Server/system/System.Private.CoreLib.dll
	Processor count: 16
	Data path: C:\Users\<username_obfuscated>\AppData\Roaming\Emby-Server\programdata
	Application path: C:\Users\<username_obfuscated>\AppData\Roaming\Emby-Server\system
	System.NullReferenceException: System.NullReferenceException: Object reference not set to an instance of an object.
	   at Tvdb.TvdbEpisodeProvider.GetMetadata(RemoteMetadataFetchOptions`1 options, CancellationToken cancellationToken)
	   at MediaBrowser.Providers.Manager.MetadataService`2.ExecuteRemoteProviders(MetadataResult`1 temp, LibraryOptions libraryOptions, String logName, TIdType id, IEnumerable`1 providers, MetadataRefreshOptions options, CancellationToken cancellationToken)
	Source: Tvdb
	TargetSite: Void MoveNext()

 

[Happy2Play 9780]

28 minutes ago, seanbuff said:

@Luke any insights into the information I posted above?

5 days later and the same series episodes are being continuously scanned non-stop.

2022-04-05 09:01:59.951 Info MediaProbeManager: ProcessRun 'ffprobe' Execute: C:\Users\<username_obfuscated>\AppData\Roaming\Emby-Server\system\ffprobe.exe -i file:"\\MediaNAS\TV Shows\Man v. Food\Season 5\Man v. Food - S05E01 - New York City, NY HDTV-720p.mkv" -threads 0 -v info -print_format json -show_streams -show_chapters -show_format -show_data
2022-04-05 09:02:00.079 Info MediaProbeManager: ProcessRun 'ffprobe' Process exited with code 0 - Succeeded
2022-04-05 09:02:00.098 Info HttpClient: GET https://api4.thetvdb.com/v4/series/84006/episodes/official/
  ?page=0
2022-04-05 09:02:00.241 Info HttpClient: GET https://api4.thetvdb.com/v4/episodes/6245785/extended
2022-04-05 09:02:00.542 Error App: Error in TheTVDB
	*** Error Report ***
	Version: 4.7.0.32
	Command line: C:\Users\<username_obfuscated>\AppData\Roaming\Emby-Server\system\EmbyServer.dll -noautorunwebapp
	Operating system: Microsoft Windows 10.0.19044
	Framework: .NET 6.0.2
	OS/Process: x64/x64
	Runtime: C:/Users/<username_obfuscated>/AppData/Roaming/Emby-Server/system/System.Private.CoreLib.dll
	Processor count: 16
	Data path: C:\Users\<username_obfuscated>\AppData\Roaming\Emby-Server\programdata
	Application path: C:\Users\<username_obfuscated>\AppData\Roaming\Emby-Server\system
	System.NullReferenceException: System.NullReferenceException: Object reference not set to an instance of an object.
	   at Tvdb.TvdbEpisodeProvider.GetMetadata(RemoteMetadataFetchOptions`1 options, CancellationToken cancellationToken)
	   at MediaBrowser.Providers.Manager.MetadataService`2.ExecuteRemoteProviders(MetadataResult`1 temp, LibraryOptions libraryOptions, String logName, TIdType id, IEnumerable`1 providers, MetadataRefreshOptions options, CancellationToken cancellationToken)
	Source: Tvdb
	TargetSite: Void MoveNext()

 

What plugin version do you have as there have been updates.  I can say I have none of these in my log and those id work directly in TVDB api.

Have you restarted Emby in these 5 days?

Edited April 4, 2022 by Happy2Play