IPTV playback stable with Threadfin and hls-proxy

[bruor 35]

I've spent months evolving my Emby setup to a state where it is able to reliably play live TV streams from an IPTV provider.  Sharing my findings here to help the community and gather feedback in case I can make things even better in the future.  Up until I added hls-proxy to the mix I was attempting to use Threadfin as a buffer.  It helped stabilize things slightly but issues were still present.  Emby clients would randomly seem to drop their channel streams and fall back to the previous screen in the app, it was more prevalent with remote clients than with local, but all clients (web/android/roku) experienced the issues. Clients have no playback issues with local, or remote episode/movie media files.  

The IPTV service provider I use allows 5 concurrent connections to their service.  However, I have noticed that I cannot rely on Emby to enforce this limit.  If I set "simultaneous stream limit" to 5 in Emby it will eventually end up in a state where no channels can be played even though there are no active streams (verified by looking at the status page of hls-proxy).  I don't know why this happens and I can't seem to correlate it to anything specific.  After adding hls-proxy to my setup, I will sometimes see Emby make a secondary connection to the same stream even though a second user has not tuned to that channel, but I'm not sure if that has anything to do with the problem.  The workaround I've put in place for this is to set the "maxStreamsPerClientDefault" in hls-proxy to 5, and set "simultaneous stream limit" on the TV source in Emby to 0.  I've also configured a cron job to restart my Emby server early each morning to clear out any stuck tuners that might exist. 

When I set up hls-proxy I noticed a couple things.  My provider publishes 5 .ts files in the m3u playlist for a channel. If Emby is able to see a list of 5 .ts files, it will read them but start playback in the client at what I believe is the 3rd or 4th file in the list.  This means that Emby is only leaving a 1 or 2 file buffer between the playback position in the client and the stream coming from the provider.  If the provider experiences a delay in publishing the latest .ts files for the stream to their servers/CDN, the client will overrun the buffer and this may be why the client stops playback at seemingly random times.  I was able to configure hls-proxy to show Emby only 2 of the 5 files at the start of playback which seems to start the client playback at file 1 when starting a stream.  When it requests the playlist a 2nd time from hls-proxy the first 2, and next 3 pieces are in the playlist for Emby to fetch. At this point hls-proxy already has another 2-3 .ts files in the buffer that haven't been advertised to Emby yet, so not only does the client have a bit of a buffer to work with inside the Emby server, the Emby server has a 2-3 segment buffer in hls-proxy that it can draw from in the event that hls-proxy needs to retry fetching the stream due to a publishing delay or a temporary error on the server side.  The only time a stream drops now if if hls-proxy encounters a provider side error that it can't overcome in a reasonable amount of time.  Since Emby is still communicating with hls-proxy when this happens, the stream will freeze, and can recover if hls-proxy is able to retry the connection and fetch files before the retry limits or timeouts are consumed. 

Some notes on how I'm using these apps.  I've tested hls-proxy both in front of, and behind Threadfin with the same result on Linux and Windows. 
Threadfin - used currently with the buffer disabled so the m3u URLs from the provider are passed to the client.  I use it strictly to groom/curate the incoming M3U/XMLTV links from the provider.

hls-proxy - Used as a debugging platform and a buffer in-between Emby and the provider.  This allows me to get a peek at what is happening between the provider and the Emby server.  If there are any errors from the provider side I can see it in the logs.  You can view streams directly in their web interface. It also has options available to aggregate provider accounts based on the syntax of the URLs they use to serve content.  If you have many single line accounts with the same provider hls-proxy can aggregate them into a pooled resource. 

[Luke 39783]

Thanks for sharing.

[Nostradamus1973 23]

Good morning,

Threadfin user here looking to switch to HLS PRoxy.

With HLS Proxy I'm having some trouble with emby, I can view the m3u8 from looking at the raw link, it is also in the correct format as I'm able to import it into IPTVeditor and view/edit without any errors, and view links with VLC. So, here's my problem, when emby goes to load the m3u8, the progress bar flashes for a brief second(getting to approximately10%) and nothing, the channels sections shows nothing at all.

@bruorCould you point me to a resource regarding your Emby setup? Or, if anyone has experienced this, I'd love to hear how they solved this.

Thank you and have a nice day.

EDIT: I dumped the Docker Version and went with the baremetal version and it's working.

Edited October 30, 2023 by Nostradamus1973
Update

[jonaswerner 1]

@bruorMany thanks for your helpful information. 
I am currently testing StreamMaster (SenexCrenshaw/StreamMaster (github.com))  as an alternative to threadfin/xteve and hls-proxy. Very successful so far. Similar to hls-proxy, you can also see the current status of the connections. Docker containers are also available. 

Possibly also an alternative

[bruor 35]

How is it for cutting down the m3u channel list?  Would love to get down to 1 tool. 

[bruor 35]

Checked it out, seems like a fantastic replacement for xteve/threadfin.   I'll have to test it's buffer to see how well it works in a real world scenario.  It doesn't look like there is any visibility into how the buffer works under the hood, and no options to tune it if you need to get advanced.

 

Edited November 4, 2023 by bruor

[jonaswerner 1]

So far, I'm excited about StreamMaster.  Xteve has more features but StreamMaster is for my use case the better solution. 

The only thing missing for my use case so far is extended filter rules with include and exclude like it is possible with xteve (https://github.com/xteve-project/xTeVe-Documentation/blob/master/en/configuration.md#custom-filter). 

 

@bruorI'm looking forward to seeing your test results 
 

[codechurn 8]

@bruorThanks for sharing this information and starting this thread.

I have been fighting with xTeve for a while. I run xTeve as a service via nssm on a virtual machine as a streaming proxy with buffering. I've tried both the internal xTeve buffer, vlc and ffmpeg options with xTeve. All of the buffered options seem to work "ok", with some occasional drops when there is only one Emby client (ie a Roku tv) watching a channel. As soon as I start watching another channel on another client (like an Android phone) at the same time both streams crap out in a couple of seconds, If I disable buffering in xTeve and send the direct url to the Emby clients, everything works just fine. 

I'd give Streamaster a try, but it's only available as a containerized deployment.

[bruor 35]

@codechurnyou'll have to run a quick test to see if your provider is serving up HLS.  If not stream master might be a better option.  I got it running on windows using docker desktop without much fuss.  The only thing I had to fight with a little was the storage mapping configuration so that the configs were actually written to the windows filesystem somewhere.   Streammaster did seem to perform pretty well as a buffer and channel manager in testing but I found it had some issues when I tried to add additional playlists and select channels across more than one source, it would throw a weird error.  Also had weird glitches where saving a setting wouldn't be reflected in the UI unless I did a hard refresh. Will probably revisit it in the future to see how it develops over time. 

[Kimballslice1890 8]

On 10/25/2023 at 12:18 AM, bruor said:

When I set up hls-proxy I noticed a couple things.  My provider publishes 5 .ts files in the m3u playlist for a channel. If Emby is able to see a list of 5 .ts files, it will read them but start playback in the client at what I believe is the 3rd or 4th file in the list.  This means that Emby is only leaving a 1 or 2 file buffer between the playback position in the client and the stream coming from the provider.  If the provider experiences a delay in publishing the latest .ts files for the stream to their servers/CDN, the client will overrun the buffer and this may be why the client stops playback at seemingly random times.  I was able to configure hls-proxy to show Emby only 2 of the 5 files at the start of playback which seems to start the client playback at file 1 when starting a stream.  When it requests the playlist a 2nd time from hls-proxy the first 2, and next 3 pieces are in the playlist for Emby to fetch. At this point hls-proxy already has another 2-3 .ts files in the buffer that haven't been advertised to Emby yet, so not only does the client have a bit of a buffer to work with inside the Emby server, the Emby server has a 2-3 segment buffer in hls-proxy that it can draw from in the event that hls-proxy needs to retry fetching the stream due to a publishing delay or a temporary error on the server side.  The only time a stream drops now if if hls-proxy encounters a provider side error that it can't overcome in a reasonable amount of time.  Since Emby is still communicating with hls-proxy when this happens, the stream will freeze, and can recover if hls-proxy is able to retry the connection and fetch files before the retry limits or timeouts are consumed.

I came across this post and reworked my setup to be similar and the stability improvements have been amazing. 

 

Can you give some more details as to where in HLS proxy you were able to configure temporarily withholding the segments from emby? I run into this sometimes where emby only leaves a 1 segment buffer and can cause stutters. 

[bruor 35]

The parameter you're looking for in hls-proxy is called fastStartChunks and I set it to 2.

Also, I've since learned that threadfin is a pain if you have a provider that changes the channel lineup slightly from time to time because you have to manually check and enable new channels to fix the playlist. 

I learned that there was an enhanced fork of xteve by SenexCrenshaw on GitHub, and was going to use it as a replacement, but then noticed Senex is working on a new project called streammaster which I have up and running and it seems like an even better option to move forward with. 

[Kimballslice1890 8]

awesome thanks! I have it setup with provider-->Threadfin-->HLS-Proxy-->Emby. So threadfin connects to the iptv provider, managing the M3U and EPG. I setup filters for channels i want and manually remapped important channels for me and then pass through the M3U to HLS proxy. Point Emby to HLS Proxy for the M3U and to threadfin for the EPG. I'll take another look at streammaster. I had issues with it running it as a threadfin alternative acting as the buffer, but this was before I came across your post and put HLS Proxy into the mix.

[bruor 35]

I put HLS-proxy on the edge because I use it to merge IPTV accounts.   Using streammaster with buffer also didn't do much for me, I'm using it just as a playlist/guide manager now. 

[alexauran 0]

On 10/24/2023 at 11:18 PM, bruor said:

When I set up hls-proxy I noticed a couple things.  My provider publishes 5 .ts files in the m3u playlist for a channel. If Emby is able to see a list of 5 .ts files, it will read them but start playback in the client at what I believe is the 3rd or 4th file in the list.  This means that Emby is only leaving a 1 or 2 file buffer between the playback position in the client and the stream coming from the provider.  If the provider experiences a delay in publishing the latest .ts files for the stream to their servers/CDN, the client will overrun the buffer and this may be why the client stops playback at seemingly random times.  I was able to configure hls-proxy to show Emby only 2 of the 5 files at the start of playback which seems to start the client playback at file 1 when starting a stream.  When it requests the playlist a 2nd time from hls-proxy the first 2, and next 3 pieces are in the playlist for Emby to fetch. At this point hls-proxy already has another 2-3 .ts files in the buffer that haven't been advertised to Emby yet, so not only does the client have a bit of a buffer to work with inside the Emby server, the Emby server has a 2-3 segment buffer in hls-proxy that it can draw from in the event that hls-proxy needs to retry fetching the stream due to a publishing delay or a temporary error on the server side.  The only time a stream drops now if if hls-proxy encounters a provider side error that it can't overcome in a reasonable amount of time.  Since Emby is still communicating with hls-proxy when this happens, the stream will freeze, and can recover if hls-proxy is able to retry the connection and fetch files before the retry limits or timeouts are consumed.

Will you explain this a bit further? I'm new to this and am unsure where you can see that info or set it.  Thanks!

[bruor 35]

I don't have access to the server at the moment but it's the chunk options in hls-proxy, one controls how many are visible on first request from a client and the next controls how many it will download at a time. 

[Kimballslice1890 8]

On 4/4/2024 at 8:57 AM, bruor said:

I put HLS-proxy on the edge because I use it to merge IPTV accounts.   Using streammaster with buffer also didn't do much for me, I'm using it just as a playlist/guide manager now. 

I'm testing your setup to move to streammaster, threadfin has been annoying me by randomly disabling channels for not having an EPG associated with it upon scheduled refresh and looks like putting HLS proxy at the edge has proven promising results. I had kicked the tires on streammaster a year or so ago but the settings have changed on it with the new release. Do you know how to disable the restreamer on streammaster to just use it as a guide and playlist manager as you do? Currently it takes the HLS proxy M3U and then connects to it through VLC and then presents it to Emby which has to repackage it back to HLS.... Feels unnecessary.

 

Also have you had better luck giving emby the stream master m3u or the emulated HD Homerun?

[bruor 35]

I've not used the emulation option in any of the products.  I did find out that there's a good fork of xteve made by the guy that writes streammaster. This one has bulk channel re-enablement that does not currently work in threadfin.  https://github.com/SenexCrenshaw/xTeVe

For streammaster I'm not sure I disabled their buffer, I think I just used the default one with a small value but I don't remember or have the config anymore. 

 

My setup is a bit different these days.  I ended up writing some scripts for hls-proxy to filter out groups, duplicate channels, and fill in the gracenote ID so that emby can may its guide data to channels properly without issue or a full playlist refresh.

I've also written my own HLS proxy in Python so I can catch when my provider expires authentication on a stream so it can keep running, but that only works if the stream is identical.  Ideally I'd like to send some type of command to emby to tell it that the provider stream for a channel needs a full reconnect because it has changed but have to idea how to approach that. Not sure if I will develop it further to replace hls-proxy or try to migrate to streammaster. 

[Kimballslice1890 8]

Thanks for that! Played around with xteve fork and it's way better than threadfin. Bulk edit worked and the dummy guide data is much better. 

I think I found where the buffer should be disabled for streammaster but it wouldn't pass through the stream unless I had it enabled. I'm gonna stick with xteve for now to see if it improves and try streammaster if it doesn't.

 

Sounds neat. You're way more advanced than my capabilities. The issue im trying to overcome (not sure how to inspect and see what's going on) is for whatever reason, channels will stop working and I'll have to refresh the m3u on threadfin/hls proxy and then on emby for the channel to start working again. Not sure if that's authentication expiring or links changing.  I've tried to get ahead of this the best I can with scheduled refreshes every 4 hours but it's not bulletproof

[bruor 35]

You need to see if the provider changing their playlist URLs causes xteve to change the URLs it publishes to emby.  If not, then you should only have to tell xteve to refresh the playlist more frequently and proxy the streams to emby.  You don't want to pass the provider URLs to emby because it seems to cause issues with tuners getting stuck etc. 

[CharlesF 10]

On 10/25/2023 at 12:18 AM, bruor said:

I've spent months evolving my Emby setup to a state where it is able to reliably play live TV streams from an IPTV provider.  Sharing my findings here to help the community and gather feedback in case I can make things even better in the future.  Up until I added hls-proxy to the mix I was attempting to use Threadfin as a buffer.  It helped stabilize things slightly but issues were still present.  Emby clients would randomly seem to drop their channel streams and fall back to the previous screen in the app, it was more prevalent with remote clients than with local, but all clients (web/android/roku) experienced the issues. Clients have no playback issues with local, or remote episode/movie media files.  

The IPTV service provider I use allows 5 concurrent connections to their service.  However, I have noticed that I cannot rely on Emby to enforce this limit.  If I set "simultaneous stream limit" to 5 in Emby it will eventually end up in a state where no channels can be played even though there are no active streams (verified by looking at the status page of hls-proxy).  I don't know why this happens and I can't seem to correlate it to anything specific.  After adding hls-proxy to my setup, I will sometimes see Emby make a secondary connection to the same stream even though a second user has not tuned to that channel, but I'm not sure if that has anything to do with the problem.  The workaround I've put in place for this is to set the "maxStreamsPerClientDefault" in hls-proxy to 5, and set "simultaneous stream limit" on the TV source in Emby to 0.  I've also configured a cron job to restart my Emby server early each morning to clear out any stuck tuners that might exist. 

When I set up hls-proxy I noticed a couple things.  My provider publishes 5 .ts files in the m3u playlist for a channel. If Emby is able to see a list of 5 .ts files, it will read them but start playback in the client at what I believe is the 3rd or 4th file in the list.  This means that Emby is only leaving a 1 or 2 file buffer between the playback position in the client and the stream coming from the provider.  If the provider experiences a delay in publishing the latest .ts files for the stream to their servers/CDN, the client will overrun the buffer and this may be why the client stops playback at seemingly random times.  I was able to configure hls-proxy to show Emby only 2 of the 5 files at the start of playback which seems to start the client playback at file 1 when starting a stream.  When it requests the playlist a 2nd time from hls-proxy the first 2, and next 3 pieces are in the playlist for Emby to fetch. At this point hls-proxy already has another 2-3 .ts files in the buffer that haven't been advertised to Emby yet, so not only does the client have a bit of a buffer to work with inside the Emby server, the Emby server has a 2-3 segment buffer in hls-proxy that it can draw from in the event that hls-proxy needs to retry fetching the stream due to a publishing delay or a temporary error on the server side.  The only time a stream drops now if if hls-proxy encounters a provider side error that it can't overcome in a reasonable amount of time.  Since Emby is still communicating with hls-proxy when this happens, the stream will freeze, and can recover if hls-proxy is able to retry the connection and fetch files before the retry limits or timeouts are consumed. 

Some notes on how I'm using these apps.  I've tested hls-proxy both in front of, and behind Threadfin with the same result on Linux and Windows. 
Threadfin - used currently with the buffer disabled so the m3u URLs from the provider are passed to the client.  I use it strictly to groom/curate the incoming M3U/XMLTV links from the provider.

hls-proxy - Used as a debugging platform and a buffer in-between Emby and the provider.  This allows me to get a peek at what is happening between the provider and the Emby server.  If there are any errors from the provider side I can see it in the logs.  You can view streams directly in their web interface. It also has options available to aggregate provider accounts based on the syntax of the URLs they use to serve content.  If you have many single line accounts with the same provider hls-proxy can aggregate them into a pooled resource. 

Thank you very much for positing this.  While it seems to me this shouldn't be required and Emby should fix this innately.

[bruor 35]

There has been development around it but I haven't bothered to re-test things after finding a working setup. 

[Nonnos 2]

Is HLS proxy the key for non interrupted recording?

[bruor 35]

It depends on the provider and why the connection in dropping,  but it does help speed up how quickly the channel loads and helps in situations where congestion/bandwidth might be a problem

[Carlo 4550]

None of these proxy fixes will be needed in TV Next.

[bruor 35]

What is TV Next?