manolodf 11 Posted March 15, 2018 Posted March 15, 2018 Hey guys, I was just trying out Emby to see if it fits my needs, one of the things that attracted me was the Live TV and being able to run IPTV on it. I run the emby docker in Unraid, so my experiences are in that. I got a warning my cache drive where app data is stored was gettin full so I was a bit shocked ( I normally use about 40gb out of 256gb) and I then realized Emby's /EmbyServer/cache/temp was taking up 140GB. Since the directory was owned by deamon:deamon I was not able to delete the temp files through network, so I had to SSH to do that. After testing I realized that the /temp folder started getting full again after adding a Live TV EPG XML file. I added the m3u file and nothing happened, but when I added the EPG via XML it started populating again, to the tune of 18GB on this EPG Refresh, I assume it was due to multiple refreshes without purging that it previously filled up so much in the past couple of hours. Not sure why it does that or how to prevent it either. Also if the folders could be owned by nobody:users like all other apps instead of deamon:deamon, it would also be easier to clean them off via network share, though obviously I would prefer it never blow up to that size honestly. Is there a way to prevent that? Anything I can provide to help diagnose or debug?
Luke 38528 Posted March 15, 2018 Posted March 15, 2018 Hi, can you dig deeper underneath and see what exactly is consuming the space? thanks.
manolodf 11 Posted March 15, 2018 Author Posted March 15, 2018 Sure, so the folder consists of tons and tons of these folders with weird character names allwith a 25.5mb p2.xml file, I am attaching a screenshot of it. Then just a few, not that many in comparison of those jpg files which just show like a distorted TV signal, also attaching one, then a couple of pngs that seem to be like multiple posters in one file.
Luke 38528 Posted March 15, 2018 Posted March 15, 2018 What is that xml file? is that your xmltv data?
manolodf 11 Posted March 15, 2018 Author Posted March 15, 2018 Hmm, it has to be correlated, my xml data is from a url and its called p2.xml.gz but if I am not mistaken its like a 200mb file uncompressed. But it literally repeats it 18gb worth, is it perhaps putting one up for every channel?
manolodf 11 Posted March 16, 2018 Author Posted March 16, 2018 Hi Luke, Just an update, the /EmbyServer/cache/temp folder went from 18gb to 53gb already. I almost have a feeling every time Emby processes the guide xml data it adds about 18gb to that folder. Adding hundreds of those folders with that 25mb file. The XML guide is entered as http://service.tv/p2.xml.gz Should I not enter the .gz if they have a non compressed one available? Is it a issue with the compressed xml? Let me know if you want me to monitor or send you something to help debug, maybe I can even trigger it by hitting "refresh guide"
Luke 38528 Posted March 16, 2018 Posted March 16, 2018 We're looking into it, thanks. I think it is downloading once per channel.
manolodf 11 Posted March 16, 2018 Author Posted March 16, 2018 I agree, I believe its one per channel per refresh, so after first refresh of about 18gb, it refreshed itself twice since then, each one being 18gb so thats where I get to the 53GB (~54gb) If you think I should enter the URL with just .xml instead of .gz let me know and I can try that and see what it does also. On the docker the files are owned deamon:deamon so I am not able to delete them easily, most other unraid dockers have the chown set to nobody:users
Luke 38528 Posted March 17, 2018 Posted March 17, 2018 Ok, please try again with the next release of emby server, thanks. @@denz, next beta should have less traffic to your xmltv provider.
denz 450 Posted March 17, 2018 Posted March 17, 2018 I used to have the direct url from my provider but there are times that i go to the guide just after a scheduled task has been done and there is nothing in the tv guide for the shows that are on at the time i have reported that couple of times so now i download to my pc and then into emby. It is all automatic using wget i rather not do that if it can be done.
Sammy 767 Posted March 17, 2018 Posted March 17, 2018 Related? https://emby.media/community/index.php?/topic/57113-Can-I-move-these? Sent from my SM-G930P using Tapatalk
manolodf 11 Posted March 19, 2018 Author Posted March 19, 2018 Hi @@Luke, So I changed the EPG URL to service.tv/p2.xml instead of the service.tv/p2.xml.gz and when I did that it did not add 18gb of duplicate xml files. It just creates the 700 or so folders in the cache/temp folder, but does not copy that 25-30mb file to each one, which is great. So I think it has to do with the .gz compression.
manolodf 11 Posted March 19, 2018 Author Posted March 19, 2018 Here is a Screenshot I think can help. So at 1:34pm is when I changed it from the .xml.gz to the .xml url (Which luckily works on my provider, may not be the case on others) and you can see it downloaded the 30.3mb file as a .tmp and then it just has empty folders for what I assume are one for each channel. You can also see the previous automatic EPG refresh it had automatically done at 12:50pm and each of those folders had that 30.3MB file inside each folder accumulating to over 18GB.
denz 450 Posted March 23, 2018 Posted March 23, 2018 I have set up url directly to my guide provider and again no guide data if the guide update tasks has just be done. Not noticing this if i use wget and set up scheduled task.
Luke 38528 Posted March 23, 2018 Posted March 23, 2018 I have set up url directly to my guide provider and again no guide data if the guide update tasks has just be done. Not noticing this if i use wget and set up scheduled task. Did you try refreshing the guide?
denz 450 Posted March 24, 2018 Posted March 24, 2018 When I seen that I had a look at the scheduled task and refresh task was done 11 minutes ago. I didn't run the task done. Is there anything that I can capture save so you can see what is causing this as this has happened numerous times and the only common thing is that the refresh task has been done very recently.
denz 450 Posted March 25, 2018 Posted March 25, 2018 Now it doesn't finish it gets stuck at 20.3% and stays like that for hours. Log.txt
denz 450 Posted March 26, 2018 Posted March 26, 2018 I just updated the server and the epg update worked it wasn't stuck now lets hope the blank guide doesn't happen again.
denz 450 Posted March 26, 2018 Posted March 26, 2018 Unfortunately i just happened to look at the guide now and the guide is blank it is full again around midnight. I did another guide refresh but no change.
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now