Live TV streaming from *.m3u source

[skivy71 5]

Hi,

 

I've just been coding my own dvb backend system, which merges physical dvb-s and iptv source and it works well in VLC.

 

I've added the automatically generated playlist file and emby parses it as expected, gets channel numbers etc... so all is well (up to this point).

 

My playlist is in the format <server>/channel?number=xxx and when my backend see's this request, it does all its look ups and and sets the stream up, redirecting the client to the appropriate <server>:<port>/bysid=<sid>

 

As I said, VLC handles this ok... but emby doesn't see to follow the redirect and attempts to start parsing the stream for info...

 

2017-06-07 23:44:10.0111 Info App: Live stream 02338a16f29447e7a1ce433144aeb8b47249570c08fc4430558c1a41b95181c9 consumer count is now 2
2017-06-07 23:44:10.0111 Info MediaEncoder: /var/lib/emby-server/ffmpeg/20170308/ffprobe -analyzeduration 2000000 -i "http://server:3142/channel?number=103"-threads 0 -v info -print_format json -show_streams -show_format
2017-06-07 23:44:12.7106 Info LiveTvMediaSourceProvider: Live tv media info probe took 2.699181 seconds
2017-06-07 23:44:12.7110 Info LiveTvMediaSourceProvider: Live stream info: {"Protocol":"Http","Id":"4d2f7018723248cab7d47dba37cca8b7_7249570c08fc4430558c1a41b95181c9","Path":"http://server:3142/channel?number=103","Type":"Default","Container":"mpegts","IsRemote":true,"ReadAtNativeFramerate":false,"IgnoreDts":false,"IgnoreIndex":false,"GenPtsInput":false,"SupportsTranscoding":true,"SupportsDirectStream":true,"SupportsDirectPlay":true,"IsInfiniteStream":true,"RequiresOpening":true,"RequiresClosing":true,"SupportsProbing":true,"LiveStreamId":"09efa0d56b934a82adec00a87b837fb0_4d2f7018723248cab7d47dba37cca8b7_7249570c08fc4430558c1a41b95181c9","RequiresLooping":true,"MediaStreams":[{"Codec":"h264","TimeBase":"1/90000","CodecTimeBase":"1/50","NalLengthSize":"0","IsInterlaced":true,"BitRate":20000000,"BitDepth":8,"RefFrames":1,"IsDefault":false,"IsForced":false,"Height":1080,"Width":1920,"AverageFrameRate":25,"RealFrameRate":25,"Profile":"Main","Type":"Video","AspectRatio":"16:9","Index":0,"IsExternal":false,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"PixelFormat":"yuv420p","Level":40,"IsAnamorphic":false},{"Codec":"mp2","Language":"NAR","TimeBase":"1/90000","CodecTimeBase":"1/48000","DisplayTitle":"NARMP2 stereo","IsInterlaced":false,"ChannelLayout":"stereo","BitRate":160000,"Channels":2,"SampleRate":48000,"IsDefault":false,"IsForced":false,"Type":"Audio","Index":1,"IsExternal":false,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Level":0},{"Codec":"dvbsub","Language":"eng","TimeBase":"1/90000","CodecTimeBase":"0/1","DisplayTitle":"Eng","IsInterlaced":false,"IsDefault":false,"IsForced":false,"Type":"Subtitle","Index":2,"IsExternal":false,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Level":0},{"Codec":"dvb_teletext","Language":"eng","TimeBase":"1/90000","DisplayTitle":"Eng","IsInterlaced":false,"IsDefault":false,"IsForced":false,"Type":"Subtitle","Index":3,"IsExternal":false,"IsTextSubtitleStream":true,"SupportsExternalStream":false,"Level":0},{"Codec":"ac3","Language":"eng","TimeBase":"1/90000","CodecTimeBase":"1/0","DisplayTitle":"Eng Dolby Digital 0 ch","IsInterlaced":false,"Channels":0,"SampleRate":0,"IsDefault":false,"IsForced":false,"Type":"Audio","Index":4,"IsExternal":false,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Level":0}],"PlayableStreamFileNames":[],"Formats":[],"Bitrate":20160000,"RequiredHttpHeaders":{},"AnalyzeDurationMs":2000,"DefaultAudioStreamIndex":1}
2017-06-07 23:44:12.7110 Info App: Profile: Unknown Profile, Path: http://server:3142/channel?number=103, isEligibleForDirectPlay: True, isEligibleForDirectStream: True
2017-06-07 23:44:12.7125 Info App: Profile: Unknown Profile, Path: http://server:3142/channel?number=103, isEligibleForDirectPlay: True, isEligibleForDirectStream: True

 

I cannot hardcode the actual stream path into the *.m3u as I only have 2 physical tuners and stream parameters (port and card etc). are determine upon connection.

 

Would it be possible to Emby to follow the redirected link and then analyse / stream etc?

 

Thanks

 

Neil

[Luke 37028]

Where exactly is the problem? That all looks fine to me but there are many places we might feed that URL into, and now you're asking can we make sure they all support redirects. That's why I would suggest keeping it simple and finding another way generate the m3u.

[skivy71 5]

I have it working now, but it seems the nature of how emby connects forces the first stream request to fail, and then it retries and gets it on the subsequent try.

 

I currently allow up to 10 requests, with 500ms delay before retry.

 

As I said, it works, but causes a delay in stream startup.

 

Just strange as VLC just plays straight away.

 

Each incoming port is mapped to the available tuner and this is not known until the connection request comes in.

[Luke 37028]

Thanks for the feedback.

[skivy71 5]

Hi Luke,

 

I've just finished some coding to my backend so that there is no longer any direction and the incoming request is forwarded and as a result, VLC does open quicker.

 

However, I've just tried the same back in emby and its still taking 2 attempts before it starts streaming....

 

Please find the log attached.

 

2017-06-10 00:04:56.0349 Info LiveTvMediaSourceProvider: Live stream info: {"Protocol":"Http","Id":"7a063aa572a4465e9c054bc2d9e9a2b7_d829aa99ef2ac57cacfe2c6a3eade6b9","Path":"http://server:31420/channel?number=103","Type":"Default","Container":"mpegts","IsRemote":true,"ReadAtNativeFramerate":false,"IgnoreDts":false,"IgnoreIndex":false,"GenPtsInput":false,"SupportsTranscoding":true,"SupportsDirectStream":true,"SupportsDirectPlay":true,"IsInfiniteStream":true,"RequiresOpening":true,"RequiresClosing":true,"SupportsProbing":true,"LiveStreamId":"09efa0d56b934a82adec00a87b837fb0_7a063aa572a4465e9c054bc2d9e9a2b7_d829aa99ef2ac57cacfe2c6a3eade6b9","RequiresLooping":true,"MediaStreams":[{"Codec":"h264","TimeBase":"1/90000","CodecTimeBase":"1/50","NalLengthSize":"0","IsInterlaced":true,"BitRate":20000000,"BitDepth":8,"RefFrames":1,"IsDefault":false,"IsForced":false,"Height":1080,"Width":1920,"AverageFrameRate":25,"RealFrameRate":25,"Profile":"Main","Type":"Video","AspectRatio":"16:9","Index":0,"IsExternal":false,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"PixelFormat":"yuv420p","Level":40,"IsAnamorphic":false},{"Codec":"mp2","Language":"NAR","TimeBase":"1/90000","CodecTimeBase":"1/48000","DisplayTitle":"NARMP2 stereo","IsInterlaced":false,"ChannelLayout":"stereo","BitRate":160000,"Channels":2,"SampleRate":48000,"IsDefault":false,"IsForced":false,"Type":"Audio","Index":1,"IsExternal":false,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Level":0},{"Codec":"dvbsub","Language":"eng","TimeBase":"1/90000","CodecTimeBase":"0/1","DisplayTitle":"Eng","IsInterlaced":false,"IsDefault":false,"IsForced":false,"Type":"Subtitle","Index":2,"IsExternal":false,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Level":0},{"Codec":"dvb_teletext","Language":"eng","TimeBase":"1/90000","DisplayTitle":"Eng","IsInterlaced":false,"IsDefault":false,"IsForced":false,"Type":"Subtitle","Index":3,"IsExternal":false,"IsTextSubtitleStream":true,"SupportsExternalStream":false,"Level":0},{"Codec":"ac3","Language":"eng","TimeBase":"1/90000","CodecTimeBase":"1/0","DisplayTitle":"Eng Dolby Digital 0 ch","IsInterlaced":false,"Channels":0,"SampleRate":0,"IsDefault":false,"IsForced":false,"Type":"Audio","Index":4,"IsExternal":false,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Level":0}],"PlayableStreamFileNames":[],"Formats":[],"Bitrate":20160000,"RequiredHttpHeaders":{},"AnalyzeDurationMs":2000,"DefaultAudioStreamIndex":1}
2017-06-10 00:04:56.0349 Debug MediaSourceManager: Live stream opened: {"Protocol":"Http","Id":"7a063aa572a4465e9c054bc2d9e9a2b7_d829aa99ef2ac57cacfe2c6a3eade6b9","Path":"http://server:31420/channel?number=103","Type":"Default","Container":"mpegts","IsRemote":true,"ReadAtNativeFramerate":false,"IgnoreDts":false,"IgnoreIndex":false,"GenPtsInput":false,"SupportsTranscoding":true,"SupportsDirectStream":true,"SupportsDirectPlay":true,"IsInfiniteStream":true,"RequiresOpening":true,"RequiresClosing":true,"SupportsProbing":true,"LiveStreamId":"a17c75760a04e99b68cf766e11316e1c_09efa0d56b934a82adec00a87b837fb0_7a063aa572a4465e9c054bc2d9e9a2b7_d829aa99ef2ac57cacfe2c6a3eade6b9","RequiresLooping":true,"MediaStreams":[{"Codec":"h264","TimeBase":"1/90000","CodecTimeBase":"1/50","NalLengthSize":"0","IsInterlaced":true,"BitRate":20000000,"BitDepth":8,"RefFrames":1,"IsDefault":false,"IsForced":false,"Height":1080,"Width":1920,"AverageFrameRate":25,"RealFrameRate":25,"Profile":"Main","Type":"Video","AspectRatio":"16:9","Index":0,"IsExternal":false,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"PixelFormat":"yuv420p","Level":40,"IsAnamorphic":false},{"Codec":"mp2","Language":"NAR","TimeBase":"1/90000","CodecTimeBase":"1/48000","DisplayTitle":"NARMP2 stereo","IsInterlaced":false,"ChannelLayout":"stereo","BitRate":160000,"Channels":2,"SampleRate":48000,"IsDefault":false,"IsForced":false,"Type":"Audio","Index":1,"IsExternal":false,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Level":0},{"Codec":"dvbsub","Language":"eng","TimeBase":"1/90000","CodecTimeBase":"0/1","DisplayTitle":"Eng","IsInterlaced":false,"IsDefault":false,"IsForced":false,"Type":"Subtitle","Index":2,"IsExternal":false,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Level":0},{"Codec":"dvb_teletext","Language":"eng","TimeBase":"1/90000","DisplayTitle":"Eng","IsInterlaced":false,"IsDefault":false,"IsForced":false,"Type":"Subtitle","Index":3,"IsExternal":false,"IsTextSubtitleStream":true,"SupportsExternalStream":false,"Level":0},{"Codec":"ac3","Language":"eng","TimeBase":"1/90000","CodecTimeBase":"1/0","DisplayTitle":"Eng Dolby Digital 0 ch","IsInterlaced":false,"Channels":0,"SampleRate":0,"IsDefault":false,"IsForced":false,"Type":"Audio","Index":4,"IsExternal":false,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Level":0}],"PlayableStreamFileNames":[],"Formats":[],"Bitrate":20160000,"RequiredHttpHeaders":{},"AnalyzeDurationMs":2000,"DefaultAudioStreamIndex":1}
2017-06-10 00:04:56.0349 Info App: Profile: Unknown Profile, Path: http://server:31420/channel?number=103, isEligibleForDirectPlay: True, isEligibleForDirectStream: True
2017-06-10 00:04:56.0359 Info App: Profile: Unknown Profile, Path: http://server:31420/channel?number=103, isEligibleForDirectPlay: True, isEligibleForDirectStream: True

2017-06-10 00:04:56.0370 Info HttpServer: HTTP Response 200 to 192.168.11.107. Time: 2735ms (slow). http://server:8096/emby/LiveStreams/Open?UserId=f7ed82ac399b4cf6a789763611235909&StartTimeTicks=0&ItemId=39c155ec8cb4910b100eab10dcf06e24&PlaySessionId=d115a2eb98df4e0a83295e8cf24af205&MaxStreamingBitrate=140000000 

[Luke 37028]

that's because we probe it first for media info, then ffprobe disconnects, and then the video player or ffmpeg will connect for playback.

[Luke 37028]

If you're dependent on single connections, that's not a great idea either. Let's say we direct play the m3u directly into a video player. We can't control whether or not the video player might make multiple requests to the stream url, and some players will do that.

[skivy71 5]

Oh ok...

 

The server that listens for incoming connections, finds the channel. start's the tuner (after finding a free one) and then a server side client picks up the local dvb stream and pipes it back down the same socket to the client.

 

I can see how the probing makes it make 2 requests.

 

If the request comes from the LAN and emby theatre and will ultimately be streamed direct, could that check / probe be bypassed?

[Luke 37028]

Eventually, probably, yes. But not at the moment.

[skivy71 5]

Ok, thats cool.

 

Do you know what would be responsible for de-interlacing of the live tv rendered within theatre?

 

I used to use Windows Media Center and I think the graphics card handled it.

 

Quite noticeable interlacing (as seen inside VLC - but with post processing off of course), inherent to the source

[88irocvert 0]

So, basically, if I'm understanding this correctly, the reason I can record two shows at a time even though I have three authorized streams is that two are gone with the first show and the second show only has one stream left which is used up during the probe.

[Luke 37028]

No, the probe stream is only open for a few seconds, at least from Emby's side. The question would be how long does your provider make you wait before you can use that slot again.

[88irocvert 0]

If I understand correctly, what I've explained above would call for four streams because two shows trying to start at once would each need one ffprobe and another stream. But that really doesn't matter. I've verified with my provider. They claim that the stream is dropped immediately upon end of use. They also claim I can request all three streams simultaneously. However, I can't record two programs at once even when I start a second recording minutes after the first one is going. This SHOULD'VE dropped the ffprobe and left two streams but no go. 

Edited June 23, 2017 by 88irocvert

[88irocvert 0]

Must apologize... Looks as though a lot (if not all) of the issues I've been having are due to the fact that some of my IPTV channels aren't working.

[Luke 37028]

Thanks for the feedback. I am thinking pretty soon we will drop the probe anyway, that will help reduce connections @@maegibbons

[CBers 6766]

No, the probe stream is only open for a few seconds, at least from Emby's side. The question would be how long does your provider make you wait before you can use that slot again.

So could that cause problems with consecutive recordings of IPTV channels to fail occasionally?

 

I posted here about an issue (still need to supply logs).

 

I only have 2 connections with my IPTV provider.

[Luke 37028]

No, it shouldn't but depends on this

The question would be how long does your provider make you wait before you can use that slot again.

[CBers 6766]

OK, I'll try and find that out.

 

Quick question, but what does the message "consumer count is now 2" mean?

 

I've seen it in my logs set to 0.

[Luke 37028]

That's the servers internal stream sharing