Practical permissible number of channels in an m3u file

[normanmurcheson 7]

I'm looking to bring channels into Emby from my IPTV services using an m3u.

It looks like I've been trying too many as I frequently reach a point where things just start bogging down royally. Channels are very slow to save at that point.

The channels and EPG info are ultimately there but the logos get all mixed up.

This is with about 150 channels.

Is there a recommended limit on the number of channels that should be attempted to bring in to keep things running well?

Thanks a lot in advance.

[Luke 37024]

Hi, what do you mean by very slow to save?

[normanmurcheson 7]

Thanks so much for the quick response.

I have brought in the full bunch of 150 channels or so. I have to manually map each one against the Emby Guide source which is okay. After I've done that for a while (about 60 channels or so) suddenly each item is very slow to save. The circle spins for up to about 30 seconds after updating each one before going back to the main channels screen. Every channel after that point has its channel icons mixed up in the server channels setup window. The channels done previously to that are fine.

Once that point has been reached, the channels list is also very slow to rebuild. It rebuilds in discrete chunks of about fifteen channels or so.

Not knowing this software that well yet (too many variables in the equation), this behavior seems to indicate to me that something is either getting corrupted or I am simply exceeding the design capacity of this m3u capability.

I do truly love the capability of being able to mix off air and IPTV channels in one place. I'm not aware of anything else that has that capability.

I much appreciate any light you can shed.

I'm going to try with a small m3u file with its own EPG (as against using yours) and see whether that makes any difference.

Just to give you heads-up. I could not get this reply box to stay open for more than a fraction of a second using either Vivaldi or Firefox. Edge did the trick.

Something weird...

Did the trial and it ran perfectly. Some 350 channels came in and look great not only on the server setup screen but the television as well. It only missed automapping on two. Very difficult to edit channel mapping after the fact with my own xml file (from disk by the way) because of all the extra information written in there. Very hard to see what belongs to what channel. Note to developers - a search function on the mapping screen would be great.

Scratching your head yet?

Edited August 1, 2021 by normanmurcheson
Cannot get the reply box to stay open. I can see a status saying waiting for Emby Media.

[Carlo 4330]

What may be happening when trying to manually map channels is that each time you edit a channel it may be trying to load the channel guide for it and depending on the number of days selected is slowing things down.

What you could try as a test is to go to Advanced->Number of days of guide data to download: and set this to 1 day.

Now when manually editing channels only 1 day of EPG guide is worked on.  This may allow you to work faster and map channels faster.  Then when complete set it back to auto or the number of days you want to import EPG guide data.  You can then run a guide refresh to fully load the guide.

Let us know how this works for you.

[emveepee 112]

How is this related to NextPVR?

 

Martin

[normanmurcheson 7]

@cayars Thank you so much for your response.

For learning sake I removed my m3u and brought it in again.

This time I used the Emby Guide as per your recommendation to bring in only one day of EPG

At this point I have done 130 channels. The system is starting to bog down considerably. As I try to edit a channel I at first see the screen I would see if I was adding a movie. It talks about release dates and the like. After about five seconds the channel loads. It does take several seconds to save the channel once I'm done. The channel list is slow to repaint.

It really looks to me like I've about reached the limit. Would you agree?

I'm guessing that when someone uses the Emby Guide there is a lot more going on in the background than when we use our own xml file, which we don't see.

Thanks again for steering me in the right direction.

 

[normanmurcheson 7]

@emveepeeI did post in the wrong place. Apologies.

[Carlo 4330]

No you haven't reached a limit as there isn't any.  I've had over 6K channels in testing and that does take a while to load but Emby handles it.

I've got something you can try.

What OS are you running on?

[normanmurcheson 7]

The server is on my Windows 10 machine. The client on my NVidia Shield.

 

[Carlo 4330]

OK try this.

Shut down Emby server then navigate in File Manager to
C:\Users\carlo\AppData\Roaming\Emby-Server\programdata\config

Change the username to your own where you have Emby installed.
In that folder is system.xml  
Open it in notepad and change these two lines
 

<VacuumDatabaseOnStartup>false</VacuumDatabaseOnStartup>
<DatabaseCacheSizeMB>96</DatabaseCacheSizeMB>

to

<VacuumDatabaseOnStartup>true</VacuumDatabaseOnStartup>
<DatabaseCacheSizeMB>256</DatabaseCacheSizeMB>

Save the file and restart Emby Server.  It may take a bit longer than normal to restart the first time.  These two changes are going to allow Emby to use a bit more memory with database operations and is going to reorganize the database to make it faster.

These two changes may help with guide loading speeds.  Can you give that a try and let us know if this helps or not?

[normanmurcheson 7]

@cayars Sorry I didn't get back to you sooner. Had to head out yesterday.

Your solution is exactly what the doctor ordered. I can't believe this is the same app.

Thank you so much. I can now continue to be blown away with what Emby can do. I'm visualizing my best IPTV experience ever.

[Carlo 4330]

Awesome, glad this worked for you!

[normanmurcheson 7]

The next trick I could use if available please. Is there any way in the channel edit window to select several channels at a time to move up and down in the list?

[Carlo 4330]

No not yet.  Right now it's done channel by channel.

Sorry about that.

[normanmurcheson 7]

I didn't think so. But always best to ask the one that knows.

That one is not the very biggest deal. For the little bit of extra work, the end result will be worth it. Thanks a lot.

[normanmurcheson 7]

Still having trouble I'm afraid.

I was merely editing the same 300 channels I had and mostly adjusting the channel numbers and resaving.

After the last one I did, the database started very badly bogging down again. Very same symptoms as previously described. I can't even get the full list of channels to load now. I just see a couple of clusters of them with lots of blanks in between. I would say it's out of memory again.

I do have 16 gigabytes of physical memory on this machine. I tried increasing the cache size to 512 from 256 as per your previous instructions. Nothing good happens.

Might you please have any other suggestions for me? I'd love to see this work.

One other thing if I may, please. There were 20 gigabytes of xml files in the cache/xml directory. I'm assuming it is safe to delete those. My SSD main drive only has 275 gigabytes; with all of those, I had less than 10 per cent capacity left.

Thanks very much.

 

[normanmurcheson 7]

I'm getting the feeling that the database itself got corrupted. The last two channels I renumbered and saved do not show as renumbered on the Android box on the television. I'm not able to get one thing from IPTV to play through Emby on the Android box. I am able to access all the IPTV channels through other software.

[Carlo 4330]

What might be happening is with the constant reloading of the channels your fragmenting the database with all the EPG rows being added, deleted, added, deleted over and over again.

That's where there Vacuum command helps as it tells Emby to organize the database on startup which gets rid of this fragmentation.  It's like optimizing a hard disc to make it faster.

That setting when set to "true" is only good for the next startup and Emby will set it back to false again.

While experimenting around with channel numbering/ordering you may want to set the Advanced->Number of days of guide data to download: to 1 day to import less guide data while playing.

[Luke 37024]

Can you please attach the emby server log from when you thought the database was bogging down? Thanks 

[emveepee 112]

Can a  moderator move this thread to the general sub-section?

Thanks,

Edit: thanks for moving it, I guess there is no forum delete option for this message.

Martin

Edited August 7, 2021 by emveepee

[normanmurcheson 7]

@Luke Thanks so much for your response. 

I've been working extensively with this. I set it up on other machines. I tried with different sized playlists. I eventually brought it back to this same machine and so far so good.

I have a feeling it was bogging down because of the way I was working with it. I'm not quite sure what it didn't like.

I'm not renumbering the channels this time and am just changing the EPG mapping for all channels from the IPTV provider to Emby data. I am also waiting for the EPG to fully load before doing any further editing. And I did follow @cayars advice as well. Fingers crossed...

Thanks a lot folks.

[Carlo 4330]

Let us know how you make out.

[normanmurcheson 7]

@cayarsI think this is finally working the way it is supposed to.

The difference this time is that I relocated the cache directory to another drive.

There were some 60 gigabytes of (mostly xml) files in there. That actually far exceeded the physical space available on my SSD C drive when I was running totally from there (40 gigabytes were available there). I was literally running out of disk space.

All is good now. I'm definitely heading toward the best IPTV viewing experience I've ever had. I just love the way Emby looks.

Thank you so much for your great support. You really know your stuff.

 

[Carlo 4330]

That's great news to hear.

Thanks for the update on your status!

[Happy2Play 8252]

That seems like a lot of cached data.  But believe everything in that folder is maintained for 30 days, as the cache clean up task removes anything older then 30 day.

But this does give a good troubleshooting point.