To complain about lacking TVnext progress

[softworkz 3335]

4 hours ago, neik said:

Additionally, will it also support communication with OScam for those that share their subscription within their home?

It supports tuners with CI modules, like Digital Devices, DvbSky and others. Besides that it works like other DVB applications, so it might pick up installed DirectShow filters (on Windows), but there's no built-in softcam support, if that's what you're asking.

[softworkz 3335]

3 minutes ago, cncb said:

So, we are waiting for the UI to be updated in all the clients to handle the new DVR features?

Yes and no. It will be done in a way that the new backend provides a compatibility layer that allows legacy (no-yet-updated) clients to continue working and new DVR features can be introduced in the clients subsequently without pressure on client development. 

[lightsout 144]

2 hours ago, softworkz said:

Yes and no. It will be done in a way that the new backend provides a compatibility layer that allows legacy (no-yet-updated) clients to continue working and new DVR features can be introduced in the clients subsequently without pressure on client development. 

Is the hesitancy from Emby to implement a fear of all the complaints and support issues that will come up?

[emveepee 112]

13 hours ago, Spaceboy said:

and i should remind you that nextpvr doesnt even support my tv card https://forums.nextpvr.com/showthread.php?tid=63931&pid=572447#pid572447 , tv next did 

That thread on an older version of NextPVR and sub never finished his review.  I expect tvnext is using BDA drivers so it seems like a parsing error on the scan that may need further analysis.

[softworkz 3335]

4 hours ago, lightsout said:

Is the hesitancy from Emby to implement a fear of all the complaints and support issues that will come up?

Considering how well and successful the initial phase has been completed, this can't actually be the reason. But it's one of the few things I don't know an answer to.

[softworkz 3335]

55 minutes ago, emveepee said:

14 hours ago, Spaceboy said:

and i should remind you that nextpvr doesnt even support my tv card https://forums.nextpvr.com/showthread.php?tid=63931&pid=572447#pid572447 , tv next did 

That thread on an older version of NextPVR and sub never finished his review.  I expect tvnext is using BDA drivers so it seems like a parsing error on the scan that may need further analysis.

TVnext uses BDA drivers on Windows but it takes the raw stream that comes from the tuner and doesn't rely on other DirectShow components which are typically used (like TIF and MP2 Demux). All stream processing, table parsing, descriptor parsing and demuxing is handled by TVnext itself for both channel scanning and actual reception.

[emveepee 112]

7 hours ago, softworkz said:

TVnext uses BDA drivers on Windows but it takes the raw stream that comes from the tuner and doesn't rely on other DirectShow components which are typically used (like TIF and MP2 Demux). All stream processing, table parsing, descriptor parsing and demuxing is handled by TVnext itself for both channel scanning and actual reception.

The NextPVR graph contains many custom DS filters too, but it is closed source so cannot use any of the logic from FFmpeg or VLC libraries, I don't know if it is "All" processing or not.  It is one of these filters that probably the issue. Debugging this can often result in a full mux capture being required, so that would be something to plan on.

[Guest CodeCat5]

@Luke@ebrPlease let me know if you need more beta testers once this is ready to move forward. I've been watching the other thread for months now and would love to see these improvements.

[softworkz 3335]

3 hours ago, emveepee said:

The NextPVR graph contains many custom DS filters too

There's no "too"  

We have a single custom DS filter to get the data (always full mux) out of the graph. Similar happens for Linux. The full mux is read from the tuner and pumped into Emby Server where it's being processed. 

One of the problems with DirectShow is that it's not possible to modify a filter graph while it's running. This makes it difficult to add or remove outputs without restarting the graph or using multiple graphs with some inter-com between, and at that point, things get ugly.

Anyway, BDA is just one source out of many others (Linux, HLS, RTSP, HDHR Native API, etc.), also DirectShow is limited to Windows, so in turn, developing this with "custom DS filters" has never even been a considerable option (even though I know DS pretty well from earlier works).
It was clear that only a custom implementation of "everything" can provide the flexibility that is required when you're up to shipping the best possible product and I think it was worth the effort. TVnext can dynamically add/remove programs to receive from a mux without restarting, which means that while a mux reception is in progress, recordings can start or end and users can start and stop watching channels from that mux while it keeps just running (until it's no longer used or the tuner is required to do something else (of higher priority) or forced by a user with higher privileges than the other consumers (= recording or live watching).

Not only muxes can be shared, it also works at the program level, when there are multiple consumers for the same channel, the channel is demuxed only once and also used to serve the second consumer. In that case, both will share the same time-shift buffer, even when one or both are getting it delivered with transcoding. At that level, things are handled dynamically as well: a user can switch between transcoding/non-transcoding  or between different transcoding qualities while watching a channel. This will create a new HLS streaming session between Emby server and the client, but the server won't restart the tuner reception. Also, the user won't "lose" the time-shift buffer. 

3 hours ago, emveepee said:

many custom DS filters too, but it is closed source so cannot use any of the logic from FFmpeg or VLC libraries, 

What is closed source? And what does it have to do with FFmpeg?

3 hours ago, emveepee said:

Debugging this can often result in a full mux capture being required, so that would be something to plan on.

TVnext is always working with full muxes. As mentioned above it does the stream processing on it's own without any library. It decodes every single bit of the TS packets on its own with  hand-written code. There's also a GUI application which decodes and analyzes muxes and displays the results in many different views and in all detail. Here's the main decoding view as an example:

 

The test application actually contains the TVnext backend and it allows to do exactly the same things that TVnext is doing when included in Emby Server. That allows to analyze and replicate most user issues very easily. When a user posts FFmpeg or Server logs I just need to click a single button in the browser, and our support application opens with the logs decoded, visualized.  and analyzed.
The same will happen when a user posts a full (mux) or filtered. mux: the (private/internal) TV test app will analyze and decode the mux. It also allows to emulate the original playback and streaming situation, not only for browser client but also for a number of clients Apps. etc etc.

3 hours ago, emveepee said:

full mux capture being required, so that would be something to plan on.

Do you still think that there's something you could tell me which I should be planning for?  

Edited January 8, 2023 by softworkz

[emveepee 112]

Remembering that I don't know how the NextPVR code works and this comes more from supporting it and what I see in the logs and users problems, the "too" meant the separate module (I guess you could call it engine in your terms).  Some things are done in common netcore, some in the engine and some by BDA/DVB API.  The engine is responsible for the full mux so multiple streams can by played from one mux, EIT processing for OTA EPG, PAT for the names but some things are done.     By doing everything in your engine there are many intrinsic advantages over NextPVR that I can think of, sharing IPTV streams,  EIT parsing from the HDHR in http mode come immediately to mind.

Thanks for the explanation, and it look like you are well prepared.  Hopefully the impatient Emby users can understand the complexity required preparing for this change.  Most of the alternatives have been doing this for a decade.  No commercial products have succeeded in this space and I wish you prolonged success, even if the market is small.

SAT>IP should be there day one though.

Martin

 

 

[softworkz 3335]

2 hours ago, emveepee said:

can think of, sharing IPTV streams

Implemented.

2 hours ago, emveepee said:

EIT parsing

Implemented for both, DVB and ATSC.

2 hours ago, emveepee said:

HDHR in http mode

BDA is an option and working on Windows, of course, but TVnext is not using the http mode. It is using the low-level API instead, which allows to get the full mux - on all platforms (Linux etc.). I suppose that NextPVR can't do that either, right?
It also implements dynamic filter updating: While the stream is being received and everything running, TVnext tells the HDHR which streams it wants to have. This allows to receive whatever is needed from the mux, but no more, which saves network bandwidth in the LAN and avoids unnecessary processing.

2 hours ago, emveepee said:

By doing everything in your engine there are many intrinsic advantages over NextPVR that I can think of,

Yes indeed. Besides the ones you mentioned, it also provides independency from any 3rd parties for both, fixing issues and supporting new features like new stream descriptors or updates to broadcast technologies (like recently for DVB-T2) and so many more things...

2 hours ago, emveepee said:

Hopefully the impatient Emby users can understand the complexity required preparing for this change.

If that would be true, at least...  It has been sleeping for a while a while now.

2 hours ago, emveepee said:

Most of the alternatives have been doing this for a decade.  No commercial products have succeeded in this space and I wish you prolonged success, even if the market is small.

It only makes sense in the context of Emby where it will be one important component and replace an area where Emby has been pretty weak and limited in the past. By now, hardly anybody is using Emby TV without any 3rd party software, be it xTeve, NextPVR, TvHeadend or one of the many others. I think there's quite a potential of users who will switch to Emby when it has proper TV support, but what counts most is the full package and the integration of all kinds of media consumption in a single application. As a standalone product, I agree to what you're saying, but Emby is an awesome product already and lifting up the one weak area it has to a top level will surely make a difference. 

Edited January 8, 2023 by softworkz

[neik 837]

6 hours ago, CodeCat5 said:

@Luke@ebrPlease let me know if you need more beta testers once this is ready to move forward. I've been watching the other thread for months now and would love to see these improvements.

If it only was beta... Unfortunately, it's not even at that stage yet.

Up to now only a selected group of testers have seen and tested it.

[arrbee99 1559]

Just to repeat - 'It has been sleeping for a while a while now.'

[emveepee 112]

2 hours ago, softworkz said:

BDA is an option and working on Windows, of course, but TVnext is not using the http mode. It is using the low-level API instead, which allows to get the full mux - on all platforms (Linux etc.). I suppose that NextPVR can't do that either, right?

Locally I don't have ATSC sub channels, but one day I will check with my DekTek to confirm, I suspect not.  Since legacy HDHR devices and DVB OTA EPG are supported on all devices it is a hybrid model of http and rtp so it is a possibility now and in the future.  Curious you spent effort on ATSC EPG since Emby has Gracenote support and North American data is so bad, NextPVR didn't both porting that for non-Windows users.

[softworkz 3335]

4 minutes ago, emveepee said:

Locally I don't have ATSC sub channels, but one day I will check with my DekTek to confirm, I suspect not.

Why do you need a different device? You could just go through the list on the HDHR and check whether there are multiple channels on a frequency.

19 minutes ago, emveepee said:

Since legacy HDHR devices and DVB OTA EPG are supported on all devices it is a hybrid model of http and rtp so it is a possibility now and in the future.

The low-level API is not legacy. It is what the BDA drivers are using internally. 
I don't understand what you mean by "hybrid"? The low-level API and the http API have nothing in common and cannot be used in a "hybrid" way. There are no connections points technically/logically. It's one or the other, there's nothing in-between.

23 minutes ago, emveepee said:

Curious you spent effort on ATSC EPG since Emby has Gracenote support and North American data is so bad,

Many reasons:

• When a channel can't be mapped you have at least the GUIDE for the next 24h 
  there's an FCC regulation that mandates this, sometimes it goes up to 3 days, but never 14 days like in other countries
• While the information is not overly comprehensive, you still get something important: Accurate Timing and up-to-date information
  Gracenote data is comprehensive, but it requires a lot of API calls to retrieve and (last time I looked at it) Emby is doing full data updates every few days only. This way, you'll surely miss late-breaking changes.
  TVnext implements incremental updates from Gracenote which can be run at least once or twice a day as it only updates the data within the next few hours. 
  But still, this doesn't give you precise information about when a certain broadcast really starts and ends.
  The in-band EPG data can be helpful in this regard (e.g. to avoid that a recording is stopped too early)
• The EPG data often includes useful information, like which streams will be broadcast, in which formats and which audio languages 
• I have implemented almost the full ATSC spec ("Program and Service Information") in for parsing, so that was just one part of it and it is quite similar to DVB EPG anyway.
• When you want to deliver the best possible implementation for something, you must not start making compromises. There are some exotic things in the spec, where it's clear that it will never be required under any circumstance, and of course I skipped those bits, but in this case, it's about really useful data even though it cannot replace an external guide (in case of ATSC).

[bungee91 98]

The details shared in this thread (many over my head) are fascinating. I just want to tip my hat to @softworkz for the prompt, open, and honest responses for this huge improvement to Emby Live Tv implementation. Too often we get silence, or the default "It's possible for future updates" boilerplate responses that only just annoy me more. Is it that hard to share with the community priorities, or some form of helpful informative response to set expecations?  

It sounds as if this is mostly all up side without much downside. Likely instantly resolve the frequent support posts complaining about tuner change time taking 5+ seconds, etc. let alone many many more. 

It is also obvious to me softworkz is proud of what he has developed, and seeing it being underutilized has to be crushing. For that, I am truly sorry and appreciate your efforts on this, and other enhancements you have been a part of recently.

So........ What exactly does this mean? Any takers?

On 1/6/2023 at 11:01 AM, Luke said:

We are looking at how best to integrate it. Thanks guys.

I want to think that given there hasn't been a beta server release in over a month that TvNext is the holdup, however I know that's far too optimistic. 

 

[emveepee 112]

Since all my local frequencies only have on sub channel,  how can I tell if I can receive two sub channels without using my modulator to send me a full stream which has this information?

I refer to legacy to refer to the pre-http devices just like SD.  Hybrid means using http for reliable streaming over wifi and low level (but not libhdhomerun) when required for EPG etc.

My experience is the majority of ATSC stations here have no EPG data a few have 8-12 hours and only two have multiple days.  Without the S/E information or even the title it is of little use for recording purposes and wouldn't want to keep updating the database since that used to be such a resource hog with the library.db, although I haven't used it with recent optimizations.  Using a free tuner for constant EPG + PSIP scanning adds risk when the user wants to use a tuner.

Martin

 

[softworkz 3335]

21 minutes ago, emveepee said:

Since all my local frequencies only have on sub channel,  how can I tell if I can receive two sub channels without using my modulator to send me a full stream which has this information?

Are you sure about always just one channel per frequency? Are these 4k channels or MPEG2Video in full-HD or how does that come?

25 minutes ago, emveepee said:

how can I tell if I can receive two sub channels

You mean with a HDHR and you want to verify whether the specs are true..?

26 minutes ago, emveepee said:

Hybrid means using http for reliable streaming over wifi and low level (but not libhdhomerun) when required for EPG etc.

What I meant to say above is that it's not easy (or better: there's no reliable way) to relate channels from the http API lineup to the channels from the low-level API. That API doesn't give you channels at all. You can only tune to frequencies and get the stream. With those streams, you need to do your own channel scanning - which is not that easy. But when you have managed to do that and you have all details from all channels parsed you still have a problem: You don't know which of the https API channels corresponds to the channels you have parsed from the low-level API because the http api doesn't give much information for matching. I had a talk with Nick from SD about that. Maybe it's improved by now, but I was just curious, at that time, I don't see a reasonable use case for working with both (from an app dev perspective - not as an owner).

38 minutes ago, emveepee said:

Without the S/E information or even the title it is of little use for recording purposes and wouldn't want to keep updating the database since that used to be such a resource hog with the library.db

TVnext uses a separate db (also SQLite). Guide data will be no longer stored in the same table as library items. 

41 minutes ago, emveepee said:

Using a free tuner for constant EPG + PSIP scanning adds risk when the user wants to use a tuner.

No it doesn't add any risk. It's exactly the same as when the user is watching a channel and switches to another channel (different freq, single tuner setup). That's something that can happen dozens of times within a short period and must be working reliably for normal usage already.

[emveepee 112]

I am quite sure.  Canada does not have the sub channel networks that they have in US, so I get 14 channels with just one virtual channel per physical physical channel.  There are a QAM frequencies with sub-channels but they are encrypted except for some radio

https://ipv4-api.hdhomerun.com/discover and https://ipv6-api.hdhomerun.com/discover are pretty reliable for getting the same channel lineups with the bonus that they work on Docker, but they will eventually disappear https://forum.silicondust.com/forum/viewtopic.php?t=77340 

Excellent news about leaving library.db but for Emby users, again that highlites how significant the change is when Live TV is just on function in Emby.

[bungee91 98]

How's it going, we get this out yet?  

I have a suspicion that the new Android app being reworked to replace our beloved ATV app doesn't allow for seeking in LiveTv due to this being incorporated into the server (#66 in the ET for Android and ET for Windows UI - Match Android TV App thread). I could be wrong, but it makes some sense from a development point of view. 

Anyhow, been a while since I complained.

[softworkz 3335]

1 hour ago, cncb said:

Watching recordings in progress yesterday (NFL games) was a total mess on my new Fire Cube.  I could not resume watching the recordings after stopping over an hour in.  I think this is due to the current transcoding necessary when watching in-progress recordings in Android.  Will TVNext improve this and if so, let's have it please?

I know very well how painful it is when you lose your time-shift buffer and you're unable to continue watching as expected. TVnext will be able to do a number of very cool things in this area which are on a quite different level, but there's something that even TVnext cannot do: It cannot read users' minds and intentions. 

The issue you encountered is not about any technical limitations - it's a decision-making problem. The will always get into situations where it needs to make certain decisions without actually knowing what the user are expecting it to do. In this case, the question is rather simple: Is the user still active and wants to continue watching or did she/he just leave while paused without hitting stop?
I can't say for sure what really happened in your case, but my latest state of knowledge was that paused transcodings are terminated after like an hour or so of inactivity. This would match your case and probably (I might be wrong) - that's what Luke meant by "it can be addressed". Now, the timeout could be simply increased - but that might affect other use cases. Or the timeout can be increased only for Live TV - maybe. Typically something like that will be done to "address" this.

TVnext will have the same kind of problems just at a different complexity level. For example: A server has 3 tuners, each of them currently in use. T1: UserA and UserB watching different Live channels (on the same frequency, so they can share T1). T2. An ongoing recording and UserC watching the same channel that is being recorded. T3: UserD - had started watching but has paused for 1h while the time-shift buffer is still being filled.

This is a situation that is working, but tuners are "overbooked" and decision-making can get quite difficult as soon as one of the following this would happen:

• UserB switches to a channel on a frequency that no tuner is currently receiving => What should happen?
• The recording on T2 is finished, another recording is scheduled  on a different frequency? => What should happen?
• Another user, UserE wants to watch a channel, on a different frequency => What should happen?

For all cases, the possible resolutions are almost the same;

• Cut off UserA from showing live tv (maybe due to lowest permission level
• Disallow UserB to switch to another channel
• Stop the recording
• Cut off UserC from live tv
• Cut off UserD from live tv and drop the time-shift buffer

Such situations cannot be addressed with timeouts.
TVnext has priority/permission levels for users and for recordings (depending on who scheduled it). But such permission levels do not help much to resolve those kind of conflict situations to everybody's satisfaction. They only help to determine who will ultimately win if he/she wants to win. But the one with the highest permission level will not always want to win. Maybe another family member is watching something and the one with the higher permission might just want to tune into a channel quickly  but wouldn't want the other family member's session to be ended due to it.
But nobody may even know from each other and that somebody else is watching or an important recording is going on that shouldn't be stopped. Neither can the server know.

The best way to handle such conflict situations with minimal collateral dissatisfaction is for the server to become a bit more "chatty" and talk to the active users, asking what they want to happen. WMC did something like that at a small scale. Many will probably remember the message "The channel is about to change to Chx in order to record XXX" and you could either confirm or cancel or do nothing and the channel will eventually be switched.
Showing such message ombination with something like "Are you still watching" and a bit more - that's the goal to achieve. Initially there will have to be a permission-based decision making as client need to be updated.

Doing this nicely is quite a challenge but eventually still much easier than trying to make automated decisions which would often be wrong and very annoying.

The same problem exists here at a very small scale? When to abandon a paused live tv session? No matter how you choose the timeout, it will always be wrong for somebody.

[kimmer2k 1]

@softworkzsorry for the hijack, but will TVNext have native support for Ceton InfiniTV tuners? I recently migrated my WMC box to Emby and have to go thru fHDHR, which takes forever to switch channels. So I bought a HDHR Prime which has a shorter delay, but it would be sweet if I could revert back to that Ceton card. Not sure which is the better tuner, but a PCIe card just feels better to me. Thanks in advance!

[softworkz 3335]

16 hours ago, kimmer2k said:

@softworkzsorry for the hijack, but will TVNext have native support for Ceton InfiniTV tuners? I recently migrated my WMC box to Emby and have to go thru fHDHR, which takes forever to switch channels. So I bought a HDHR Prime which has a shorter delay, but it would be sweet if I could revert back to that Ceton card. Not sure which is the better tuner, but a PCIe card just feels better to me. Thanks in advance!

We made some attempts to support Ceton devices but we didn't succeed when it comes to DRM protected CableCard channels. Are you talking about those kinds of channels? Can you watch them with fHDHR?
What we tried though, was accessing the devices with the BDA drivers on Windows. I didn't know that they also have an http API - which fHDHR is using. It doesn't appear to be overly complicated, so we can give it another try going this route.

Edited January 25, 2023 by softworkz

[kimmer2k 1]

7 hours ago, softworkz said:

Are you talking about those kinds of channels? Can you watch them with fHDHR?

No, just copy-free channels. The Ceton Linux drivers doesn’t support DRM afaik.

Thanks for the reply, and I look forward to the release of TVNext. Feel free to use me as a guinea pig.

[Dan_Austin 62]

The Ceton drivers do not appear to support DRM protected channels.  I got sick of waiting so Silicon Dust to release their 6 tuner, I tried the Windows based CetonProxy and even managed to get it working under Wine, but it was not pleasant to setup or use.  The fHDHR project was setup for plugin modules, and the code was easy enough to follow for this Python novice to understand.  I had a working basic module in a weekend.  It is at a state now that I tossed the 3 tuner SD HDHR in a drawer.  I use a PCI 6 tuner as my primary and an external 6 tuner as the roll-over. I do not subscribe to any premium channels, but I assume they may be problematic.

One interesting point that Kimmer2k raised.  Slow tuning.  I got rid of Rokus in my house since they tended to orphan tuners, and with only 3 tuners in a HDHR they were too high risk.  With 12, I allowed a couple back into the house.  They still orphan channels occasionally.  The last time it happened to be the channel I watch most.  Until I reset emby during an update I had sub-second tuning instead of the usual 7~8 seconds.

I too am looking forward to NextTV and hope it can be considered a priority in the next beta cycle...

 

 

Edited January 29, 2023 by Dan_Austin