Unresponsive Server during Live TV Guide Refresh

[ShadowKindjal 21]

I run a daily refresh of my guide data to obtain updates from my IPTV provider. This refresh is triggered at noon and it seems that pulling the guide data locks up the server and makes it unresponsive. I currently have 152 channels on my server and I'm pulling from six different guide sources. Allowing the refresh to complete doesn't seem to solve the issue. It doesn't seem to be resolved until I reboot the server.

In the logs I see the guide refresh trigger at noon and I see a bunch of guide data found. Once done I noticed the response times of http requests skyrocket. Any help would be appreciated. 

embyserver-63818454084.txt

[Luke 42077]

Hi there, please try removing these plugins:

2023-04-30 04:00:17.691 Info App: Loading Emby.Plugins.AniList, Version=1.0.2.0, Culture=neutral, PublicKeyToken=null from /config/plugins/Emby.Plugins.AniList.dll
2023-04-30 04:00:17.691 Info App: Loading EpMetaRefresh, Version=1.0.0.1, Culture=neutral, PublicKeyToken=null from /config/plugins/EpMetaRefresh.dll
2023-04-30 04:00:17.691 Info App: Loading MediaBrowser.Plugins.Anime, Version=1.5.4.0, Culture=neutral, PublicKeyToken=null from /config/plugins/MediaBrowser.Plugins.Anime.dll

Then restart the server and see how things compare.

[ShadowKindjal 21]

Will removing the Anilist and Anime plugins affect any of my anime metadata that's already loaded on my server?

[Luke 42077]

8 minutes ago, ShadowKindjal said:

Will removing the Anilist and Anime plugins affect any of my anime metadata that's already loaded on my server?

No.

[ShadowKindjal 21]

On 4/30/2023 at 12:59 PM, Luke said:

Hi there, please try removing these plugins:

2023-04-30 04:00:17.691 Info App: Loading Emby.Plugins.AniList, Version=1.0.2.0, Culture=neutral, PublicKeyToken=null from /config/plugins/Emby.Plugins.AniList.dll
2023-04-30 04:00:17.691 Info App: Loading EpMetaRefresh, Version=1.0.0.1, Culture=neutral, PublicKeyToken=null from /config/plugins/EpMetaRefresh.dll
2023-04-30 04:00:17.691 Info App: Loading MediaBrowser.Plugins.Anime, Version=1.5.4.0, Culture=neutral, PublicKeyToken=null from /config/plugins/MediaBrowser.Plugins.Anime.dll

Then restart the server and see how things compare.

Removing the plugins did not seem to affect this issue at all. 

embyserver (11).txt

[Luke 42077]

Another thing to try is disabling the realtime monitor for each of your libraries, then restart the server and see how things compare.

[ShadowKindjal 21]

Line 76744: 2023-06-07 10:44:14.836 Info Server: http/1.1 Response 206 to host21. Time: 3950534ms. http://emby_remote_ip/emby/videos/2565450/stream.mkv?DeviceId=6d0c26f4f7fc9c99&MediaSourceId=5e72cb3412ae747cde0984b61b3f0f82&Static=true&PlaySessionId=3eadadfa0ae6439b8fd6a52f565fba37
Line 77281: 2023-06-07 10:48:20.624 Info Server: http/1.1 Response 200 to host8. Time: 5052ms. http://emby_remote_ip/emby/Items/4283084/PlaybackInfo?format=json
Line 78186: 2023-06-07 10:52:37.419 Info Server: http/1.1 Response 206 to host20. Time: 3121357ms. http://emby_remote_ip/emby/videos/4939773/stream.mkv?DeviceId=30B0F886-3B52-4018-8613-1E341F8DBF43&MediaSourceId=a5c57bd30515418d7d5f1fbc64a54213&Static=true&PlaySessionId=de02cc45b2b5464f812a328b078b6641
Line 85168: 2023-06-07 11:05:46.740 Info Server: http/1.1 Response 500 to host1. Time: 25614ms. http://emby_remote_ip/emby/videos/2940944/hls1/main/0.ts?PlaySessionId=ea6fb98199a641b0bf02e6dbf6e287a3
Line 85183: 2023-06-07 11:05:53.434 Info Server: http/1.1 Response 200 to host1. Time: 11881ms. http://emby_remote_ip/emby/videos/2940944/hls1/main/0.ts?PlaySessionId=c5cccedebdba4cf48991ada188d9ee29
Line 85214: 2023-06-07 11:05:59.357 Info Server: http/1.1 Response 200 to host1. Time: 3001ms. http://emby_remote_ip/emby/videos/2940944/hls1/subs/3.vtt?PlaySessionId=c5cccedebdba4cf48991ada188d9ee29&VttTimestampMap=900000&CurrentSubtitleStreamIndex=2
Line 85220: 2023-06-07 11:06:02.905 Info Server: http/1.1 Response 200 to host1. Time: 3009ms. http://emby_remote_ip/emby/videos/2940944/hls1/subs/4.vtt?PlaySessionId=c5cccedebdba4cf48991ada188d9ee29&VttTimestampMap=900000&CurrentSubtitleStreamIndex=2
Line 85221: 2023-06-07 11:06:02.994 Info Server: http/1.1 Response 200 to host1. Time: 8698ms. http://emby_remote_ip/emby/videos/2940944/hls1/main/3.ts?PlaySessionId=c5cccedebdba4cf48991ada188d9ee29
Line 85234: 2023-06-07 11:06:06.440 Info Server: http/1.1 Response 200 to host1. Time: 3002ms. http://emby_remote_ip/emby/videos/2940944/hls1/subs/5.vtt?PlaySessionId=c5cccedebdba4cf48991ada188d9ee29&VttTimestampMap=900000&CurrentSubtitleStreamIndex=2
Line 85236: 2023-06-07 11:06:07.139 Info Server: http/1.1 Response 200 to host1. Time: 3803ms. http://emby_remote_ip/emby/videos/2940944/hls1/main/5.ts?PlaySessionId=c5cccedebdba4cf48991ada188d9ee29
Line 88697: 2023-06-07 11:24:59.389 Info Server: http/1.1 Response 204 to host1. Time: 1539ms. http://emby_remote_ip/emby/Sessions/Playing/Progress?X-Emby-Client=Emby Web&X-Emby-Device-Name=Firefox Windows&X-Emby-Device-Id=94dd02e9-ce33-4457-8a45-d0f9226a7c8c&X-Emby-Client-Version=4.7.12.0&X-Emby-Language=en-us&reqformat=json

I've disabled real-time monitoring on each of my libraries and I still seem to notice large latency spikes leading to slower performance. In this case, the library refresh occurred at 11 am.

embyserver (12).txt

[ShadowKindjal 21]

Is there anything else I can be doing to fix these long response times? The server seems to crawl to a slow at random intervals and then resolves itself before I can truly diagnose it or it's so slow that I'm forced to reboot docker to get it operational again.

Line  82761: 2023-07-18 10:53:32.866 Info Server: http/1.1 Response 204 to host4. Time: 2370ms. http://emby_remote_ip/emby/Sessions/Playing/Progress
Line  82762: 2023-07-18 10:53:32.867 Info Server: http/1.1 Response 204 to host31. Time: 2207ms. http://emby_remote_ip/emby/Sessions/Playing/Progress
Line  82764: 2023-07-18 10:53:32.868 Info Server: http/1.1 Response 200 to host29. Time: 1095ms. http://emby_remote_ip/emby/videos/2984540/hls1/main/83.ts?PlaySessionId=b653b3637b654d34887f5d028aaf735d
Line  85595: 2023-07-18 10:59:47.927 Info Server: http/1.1 Response 206 to host27. Time: 361508ms. http://emby_remote_ip/emby/Videos/4131623/stream.mkv?static=true&mediaSourceId=643e7982f83704e89a73476a1c5744c7
Line  90029: 2023-07-18 11:00:05.971 Info Server: http/1.1 Response 200 to host24. Time: 3527ms. http://emby_remote_ip/emby/LiveStreams/MediaInfo?liveStreamId=06044cf0e6f93cdae5f285c9ecfaaeb4_01413a525b3a9622ce6fdf19f7dde354_6c1c28ca006b9d5bd4bbe9eeda0e7d23
Line  92250: 2023-07-18 11:11:11.129 Info Server: http/1.1 Response 204 to host29. Time: 2628ms. http://emby_remote_ip/emby/Sessions/Playing/Progress?X-Emby-Client=Emby for Apple TV&X-Emby-Device-Name=Apple TV&X-Emby-Device-Id=08CA0ECF-1743-41E1-804D-BE8C1A77B00E&X-Emby-Client-Version=1.7.4 (2)&reqformat=json
Line  92251: 2023-07-18 11:11:11.130 Info Server: http/1.1 Response 204 to host24. Time: 2063ms. http://emby_remote_ip/emby/Sessions/Playing/Progress
Line  92252: 2023-07-18 11:11:11.132 Info Server: http/1.1 Response 200 to host29. Time: 2096ms. http://emby_remote_ip/emby/videos/2984540/hls1/main/435.ts?PlaySessionId=b653b3637b654d34887f5d028aaf735d
Line  92720: 2023-07-18 11:12:48.620 Info Server: http/1.1 Response 206 to host27. Time: 567258ms. http://emby_remote_ip/emby/Videos/4131624/stream.mkv?static=true&mediaSourceId=f592855dfd5bc85a3e95971cbb7d40de
Line  93101: 2023-07-18 11:14:32.345 Info Server: http/1.1 Response 206 to host4. Time: 1394913ms. http://emby_remote_ip/emby/videos/5063231/stream.mkv?DeviceId=36dfce3525a889cd&MediaSourceId=c296f6cde705f4b40a0385e78130a23f&Static=true&PlaySessionId=2ffa34cad3f544dab3f913fb7c1828d8
Line  94858: 2023-07-18 11:21:52.520 Info Server: http/1.1 Response 206 to host4. Time: 390904ms. http://emby_remote_ip/emby/videos/5063236/stream.mkv?DeviceId=36dfce3525a889cd&MediaSourceId=b7bee9d9a8d7f519db3dfe6b1834e0fc&Static=true&PlaySessionId=0a7b878f4d28497fb05e79aad92ede7e

As I mentioned in my previous message, real-time monitoring has been disabled for quite some time now.

embyserver (14).txt

[Luke 42077]

Which ones are you referring to?

[ShadowKindjal 21]

I'm not completely sure what is contributing to the slow load times. The snippet of code I pulled out of the log files shows load times from a couple of seconds to greater than five minutes. It seems to happen sporadically. I originally thought it was a live tv refresh issue but it doesn't seem to be connected to that.

[Luke 42077]

Which one shows five minutes?

[ShadowKindjal 21]

Line 92720 shows a response time of 567258ms which equates to 9.45 minutes.

[Luke 42077]

5 minutes ago, ShadowKindjal said:

Line 92720 shows a response time of 567258ms which equates to 9.45 minutes.

But that’s a video stream. I’m assuming you were watching for at least that long, right?

[ShadowKindjal 21]

I may be misunderstanding how the response times work in the log. I am still experiencing long load times sporadically and I thought those response times were all related. What lines should I look for when attempting to point out latency 

[Luke 42077]

What time in the log file did the problem occur?