XMLTV guide fails to start scanning - 4.6.1.0

[cochize1 40]

After update to 4.6.0.1 my XMLTV tv guide for m3u list I always used before fails to refresh. I already deleted both m3u list and xmltv link and reinstalled again. IPTV plays fine but the guide stays empty. Attached are the logs and here is the link to the guide:

https://epg.ovh/pltv.xml

Of course there are errors in the logs but I cannot decipher them.

embyserver.txt

[Luke 37007]

Have you configured a custom cache path for the server?

[cochize1 40]

Actually yes, I have a custom cache path. What should I do now?

[Luke 37007]

Try blanking it out and setting it back to default.

[cochize1 40]

ok, the guide is refreshing now. Does this mean that I cannot have a custom cache path or can I set it to custom again after the refresh?

[Luke 37007]

Well where is it pointing to? Whatever storage that is seems to be randomly throwing access denied errors. That's also causing your subtitle problem here:

 

[cochize1 40]

I redirected metadata, cache and  transcoding files all to external SSD drive plugged into my Synology NAS where I have 2 HDD drives. For the previous server version the server felt much snappier. I know that the current release could have fixed that problems but still, I would like to leave at least data and transcoding folder on the SSD. Is that wise?

[Happy2Play 8242]

Does this not suggest a permissions issue?

2021-05-26 00:49:28.719 Error LiveTvManager: Error refreshing channels for service
	*** Error Report ***
	Version: 4.6.1.0
	Command line: /volume1/@appstore/EmbyServer/system/EmbyServer.dll -programdata /volume1/Emby -ffdetect /var/packages/EmbyServer/target/bin/ffdetect -ffmpeg /var/packages/EmbyServer/target/bin/ffmpeg -ffprobe /var/packages/EmbyServer/target/bin/ffprobe -nolocalportconfig -ignore_vaapi_enabled_flag -updatepackage emby-server-synology_{version}_x86_64.spk -noautorunwebapp
	Operating system: Linux version 4.4.59+ (root@build3) (gcc version 4.9.3 20150311 (prerelease) (crosstool-NG 1.20.0) ) #25556 SMP PREEMPT Thu Mar 4 18:03:46 CST 2021
	Framework: .NET Core 3.1.13
	OS/Process: x64/x64
	Runtime: volume1/@appstore/EmbyServer/system/System.Private.CoreLib.dll
	Processor count: 2
	Data path: /volume1/Emby
	Application path: /volume1/@appstore/EmbyServer/system
	System.UnauthorizedAccessException: System.UnauthorizedAccessException: Access to the path is denied.
	 ---> System.IO.IOException: Operation not permitted
	   --- End of inner exception stack trace ---
	   at System.IO.FileSystem.CopyFile(String sourceFullPath, String destFullPath, Boolean overwrite)
	   at System.IO.File.Copy(String sourceFileName, String destFileName, Boolean overwrite)
	   at Emby.Server.Implementations.IO.ManagedFileSystem.CopyFile(String source, String target, Boolean overwrite)
	   at Emby.LiveTV.Listings.XmlTv.XmlTvProvider.UnzipIfNeeded(String compressedFile, String targetPath, Boolean isRemoteSource)
	   at Emby.LiveTV.Listings.XmlTv.XmlTvProvider.GetXml(ListingsProviderInfo info, String path, MediaProtocol protocol, CancellationToken cancellationToken)
	   at Emby.LiveTV.Listings.XmlTv.XmlTvProvider.GetChannels(ListingsProviderInfo info, CancellationToken cancellationToken)
	   at Emby.LiveTV.EmbyTV.GetEpgChannelData(IListingsProvider provider, ListingsProviderInfo info, CancellationToken cancellationToken)
	   at Emby.LiveTV.EmbyTV.QueryChannelsAsync(ChannelManagementQuery query, CancellationToken cancellationToken)
	   at Emby.LiveTV.EmbyTV.GetChannelsAsync(CancellationToken cancellationToken)
	   at Emby.LiveTV.LiveTvManager.RefreshChannelsInternal(ILiveTvService service, IProgress`1 progress, CancellationToken cancellationToken)
	   at Emby.LiveTV.LiveTvManager.RefreshChannelsInternal(IProgress`1 progress, CancellationToken cancellationToken)
	Source: System.IO.FileSystem
	TargetSite: Void CopyFile(System.String, System.String, Boolean)
	InnerException: System.IO.IOException: Operation not permitted
	Source: 
	TargetSite:

 

[cochize1 40]

Not sure if you ask me or @Luke and I know that Synology permissions are a bit tricky at a times but I think it looks ok from my side:

EDIT: usbshare1-2 is my SSD disc I mentioned above and I don't use docker at all

Edited May 25, 2021 by cochize1

[Luke 37007]

Yea it's not blocking all writes, just some, but I don't see why.

[Carlo 4330]

Hi, Did you create a new shared folder specifically to hold this data from Emby?

Example below could be folder "EmbyData".

Then make sure the Emby user is given proper access:

[cochize1 40]

I didn't create another Shared Folder as it was working flawlessly (and in fact still is apart from that cache bug). The setting is as follows:

[AboveUnrefined 7]

I'm hitting a similar situation running my emby server in a docker instance on my server...

I've tried removing the tuner, the XmlTv EPG source, both things, letting "Refresh Guide Data" go through in between each...

This is the only information I can get out from the logs:

 

2021-09-05 13:19:24.962 Error LiveTvManager: Error refreshing channels for service
	*** Error Report ***
	Version: 4.7.0.9
	Command line: /system/EmbyServer.dll -programdata /config -ffdetect /bin/ffdetect -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3
	Operating system: Linux version 5.10.28-Unraid (root@Develop) (gcc (GCC) 9.3.0, GNU ld version 2.33.1-slack15) #1 SMP Wed Apr 7 08:23:18 PDT 2021
	Framework: .NET Core 3.1.17
	OS/Process: x64/x64
	Runtime: system/System.Private.CoreLib.dll
	Processor count: 32
	Data path: /config
	Application path: /system
	MediaBrowser.Model.Net.HttpException: MediaBrowser.Model.Net.HttpException: InternalServerError
	   at Emby.Server.Implementations.HttpClientManager.CoreHttpClientManager.GetTempFileResponse(HttpRequestOptions options)
	   at Emby.Server.Implementations.HttpClientManager.CoreHttpClientManager.GetTempFile(HttpRequestOptions options)
	   at Emby.XmlTV.XmlTvProvider.GetXml(ListingsProviderInfo info, String path, MediaProtocol protocol, CancellationToken cancellationToken)
	   at Emby.XmlTV.XmlTvProvider.GetChannels(ListingsProviderInfo info, CancellationToken cancellationToken)
	   at Emby.LiveTV.EmbyTV.GetEpgChannelData(IListingsProvider provider, ListingsProviderInfo info, CancellationToken cancellationToken)
	   at Emby.LiveTV.EmbyTV.QueryChannelsAsync(ChannelManagementQuery query, CancellationToken cancellationToken)
	   at Emby.LiveTV.EmbyTV.GetChannelsAsync(CancellationToken cancellationToken)
	   at Emby.LiveTV.LiveTvManager.RefreshChannelsInternal(ILiveTvService service, IProgress`1 progress, CancellationToken cancellationToken)
	   at Emby.LiveTV.LiveTvManager.RefreshChannelsInternal(IProgress`1 progress, CancellationToken cancellationToken)
	Source: Emby.Server.Implementations
	TargetSite: Void MoveNext()

 I'm trying to go through and manually clear out caches and such, but no matter what I do nothing seems to work. I'm also not sure if I'm clearing out the right cache data, maybe I missed something?


What's even more vexing is that upon initially setting this all up, it worked. Now it just won't auto-update and no matter what I'm doing I can't get the bloody thing set back up to work again.

This bug is getting to be very mysterious given how the log only has this information in it. I guess we're suspecting some issue with the cache location, but I can't go changing that around, it's going to make a real messy situation happen.

[Luke 37007]

Hi there @AboveUnrefined please attach the complete emby server log. thanks.

[AboveUnrefined 7]

hey @Luke thanks for getting to me, here is the latest log after restarting and trying to configure the EPG - the same error occurs toward the end of the log.

Let me know if there's anything else I can do to try and zero in on this. It's been pretty annoying to set this up and I think what I'm doing is the way to get this done, but it's just failing on this last thing... I swear it worked when I initially set it up!

embyserver-aboveunrefined-202109051930.txt

[Luke 37007]

1 hour ago, AboveUnrefined said:

hey @Luke thanks for getting to me, here is the latest log after restarting and trying to configure the EPG - the same error occurs toward the end of the log.

Let me know if there's anything else I can do to try and zero in on this. It's been pretty annoying to set this up and I think what I'm doing is the way to get this done, but it's just failing on this last thing... I swear it worked when I initially set it up!

embyserver-aboveunrefined-202109051930.txt 448.69 kB · 1 download

That's your xml tv provider returning a 500 error code when we try to download xml data using the url you provided. 500 means internal server error, so based on that information it looks like they are having problems right now.

[AboveUnrefined 7]

@Lukethanks for your help, however, I get no 500 errors trying to utilize the epg links. I edited the log to prevent others from hijacking what I'm using...

if there is a 500 error coming to Emby server in my instance, I'd wonder why -- there's no sign of that in the log...

 

I'm able to use other variants of product that I will digress to mention, they are working (much more slowly) without an immediate failout like what's happening here. I can't tell yet if it's going to just exhibit the same behavior like what's going on here with emby until a day or so passes.

 

Also, like I've mentioned, it has worked when I did initially set this up. It IS behaving like there's something crazy going on with caching, but I can't put my finger on it either.... as if it just keeps trying to reuse what's been made available and it's not good, or something...

Either way, you COULD be on to something with a 500 coming from the EPG service if the server is doing something weird about the request... I doubt it though - I've even tried adding different user agent names (like emby-server) and still get same results

Edited September 6, 2021 by AboveUnrefined

[Luke 37007]

4 minutes ago, AboveUnrefined said:

@Lukethanks for your help, however, I get no 500 errors trying to utilize the epg links. I edited the log to prevent others from hijacking what I'm using...

if there is a 500 error coming to Emby server in my instance, I'd wonder why -- there's no sign of that in the log...

 

I'm able to use other variants of product that I will digress to mention, they are working (much more slowly) without an immediate failout like what's happening here.

 

Also, like I've mentioned, it has worked when I did initially set this up. It IS behaving like there's something crazy going on with caching, but I can't put my finger on it either.... as if it just keeps trying to reuse what's been made available and it's not good, or something...

Either way, you COULD be on to something with a 500 coming from the EPG service if the server is doing something weird about the request... I doubt it though - I've even tried adding different user agent names (like emby-server) and still get same results

It's possible they are intending to reject the request and sending back a 500 when they do that. That's why we offer some of the other options such as user agent configuration to allow you to try and disguise what software the request is coming from. So for example you could use that to make it look like it's coming from a browser.

[AboveUnrefined 7]

Just now, Luke said:

It's possible they are intending to reject the request and sending back a 500 when they do that. That's why we offer some of the other options such as user agent configuration to allow you to try and disguise what software the request is coming from. So for example you could use that to make it look like it's coming from a browser.

Let me try this out, you could be on to something -- what sucks is that it takes time to verify, I can at least confirm if it'll go through ok and chunk through it's thing (if it fails, it does so quickly)

[AboveUnrefined 7]

@LukeYou ARE right - it's so weird! I'm going to try to keep cracking at it tomorrow and NOT TOUCH the URL for the epg for a little while. I'm wondering if there's some crap going on with the server that prevents too many requests in a day because now I'm getting 500 errors like you've suggested when trying to hit the resource.

I might've just gotten lucky with the _other_ crap I mentioned because it was just enough time to have it load it or whatever... maybe the service for the EPG is just coming in/out on their end for whatever reason. HARD TO SAY!

anyway, thanks for your help and I'm going to explore trying to figure out how I can hook it up with an alternative EPG to stop this enigma from happening, I'm starting to think that this might just be something to do with throttling and yadda yadda yadda

[Carlo 4330]

Depending on how often you load your guide data you could adjust the schedule to load at 3am or similar and could configure a task outside of Emby to use CURL to download the EPG at 2:45am or so.  Then you could adjust Emby to load the EPG from disc instead of pulling it.

You shouldn't need to do that, but it's something you can keep in your back pocket just in case.

[AboveUnrefined 7]

11 minutes ago, cayars said:

Depending on how often you load your guide data you could adjust the schedule to load at 3am or similar and could configure a task outside of Emby to use CURL to download the EPG at 2:45am or so.  Then you could adjust Emby to load the EPG from disc instead of pulling it.

You shouldn't need to do that, but it's something you can keep in your back pocket just in case.

I've been trying stuff like this, but as Luke pointed out now I'm just getting 500 errors... I thought it was bizarre at first but it makes sense because if that's what's going on then I'm just dealing with an unreliable EPG - which sucks.
I'm thinking my best bet is to go through the pain of utilizing another epg service and mapping the channels accordingly, which is stupid and sucks but I think it's what I got to do if it's going to blank out with these 500s and random errors.
I'm not even sure if what I'm dealing with is the same sort of thing as this thread, so if I hijacked it - I apologize for that. something tells me it might be in the same vein of problems, not necessarily the exact same problem though. It _would_ be nice if we can get the error reason bubble up into the error log so we can see it... but I understand what the hold up on that might be with all the things going on.

It's only 180+ channels I'm dealing with so... it's doable. I think I'm just going to do that and hopefully I can have this put to bed for the next 10 months at least... hopefully more

Edited September 6, 2021 by AboveUnrefined

[Luke 37007]

Typically when you reject a request, the best practice error code would be something in 400-499 range. We've seen some examples of providers in the past that use 500 error codes because not only do they want to reject the request, but they also want to try and disguise the fact that they're doing that.

[Skyhigh69 7]

I had an issue if your making your own epg online, if the guide gets downloaded to many times in a day they will cut you off, my solution was to just use dropbox to host my epg and point kodi, emby or whoever else to the file, they dont care how many times it gets called since I have a premium dropbox acct, only downfall is the guide only holds so many days of data so you have to refresh on the guide site and upload that new guide data every 3 days or so, make sure the guide always has the same name when you overwrite so you dont screw with the guide addy. hope this helps, if not, no big deal lol.

All those IPTV guys have partial guides at best, thats why I dont use there epg addy and prefer my own.

Edited September 6, 2021 by Skyhigh69