Emby clients pausing/buffering too often - Clients need bigger buffers

[pwhodges 1530]

2 hours ago, RanmaCanada said:

The technical ignorance on the forums has been getting a little too strong lately with the ignorant touting their ignorance as a strength while telling those of us who are knowledgeable that we're wrong because they said so.

Be kind.  We all started ignorant; and the actions of streaming protocols are perhaps rather esoteric for those not directly concerned with them.

I myself am unclear about the details of modern streaming.  I have been working with computer software since 1970, and even wrote the networking layer of a commercial communications product around 1985; but things change, and it can be hard even for those relatively knowledgeable to keep up with things which are not one's own direct concern.

My concern and sadness is not for people's ignorance, but for their unwillingness to accept expert advice and to learn from those whose job is dealing with the problems they face.  It is necessary to accept (in all parts of life) that the answer is not always what you think it "should" be.

Paul

[RanmaCanada 345]

5 hours ago, pwhodges said:

Be kind.  We all started ignorant; and the actions of streaming protocols are perhaps rather esoteric for those not directly concerned with them.

I myself am unclear about the details of modern streaming.  I have been working with computer software since 1970, and even wrote the networking layer of a commercial communications product around 1985; but things change, and it can be hard even for those relatively knowledgeable to keep up with things which are not one's own direct concern.

My concern and sadness is not for people's ignorance, but for their unwillingness to accept expert advice and to learn from those whose job is dealing with the problems they face.  It is necessary to accept (in all parts of life) that the answer is not always what you think it "should" be.

Paul

I am being kind by not attacking them directly.  I am not going to hold their hand and tell them it's alright to be ignorant and ignore the explanations from experts just becase they believe diffrerently.  Covid has made it so people embrace their stupidity and to attack those who have knowledge.  I won't stand for it as Idiocracy should not be the prophecy it has turned out to be.

[visproduction 122]

Related to Starlink multi source satellite and TCP packets issue, video codec compression types and how that can affect playback with TCP transmission over networks with too many 'out of order' packets.  It seems like the video codec type, the bitrate and audio / video compression, number of I frames, etc., all can create strange effects when networks have jitter and other non-optimal delivery specs.  I think certain codecs with different encoding methods probably are more forgiving for lost and 'out of order' packets.  The right test would be to have the exact same original copy of test video content.  Also, somehow make sure that the first source sends the content with the same video and audio codec, bitrate that Emby does.  Just because you might actually start with the same test video, it won't be necessarily sent with the same codecs to the end user playback.

https://www.ietf.org/slides/slides-iepg-starlink-protocol-performance-00.pdf

Emby, I think often converts to .mp4 to allow the user to playback the content easily in a browser, but that could change, depending on the users' end hardware.  I have not dug into this with Emby transcoding choices.  Here is a paper that shows how .mp4 deals with 'out of order' packets.  Satellites delivery has the design issue of trying to handle 'out of order' packets and it seems like it takes a lot more effort to solve this, than cabled or fiber channel delivery. 

https://www.eecs.qmul.ac.uk/~tysong/files/MMCN09-QoE.pdf

Quote

Their research shows that the video quality remains constant until the PLR reaches certain value and that the higher the average bit rate, the lower the PLR after which the video quality drops. This is due to the constant packet size with which higher bit rates are achieved by more packet units. However, we believe that the bit-rate has no direct link to user experience concerning packet loss because video content is represented by a series of frames. Bit-rate only decides the balance between the frame rate and the quality of each frame.

(PLR is Packet Loss Ratio.)  With 'out of order' packets, the end user TCP controller, has a maximum number of packets it will accept, as being 'out of order' and then reject all the remaining packets since the first 'out of order' packet and re-request the entire group again... you get immediate buffering.  Each OS either in a workstation, mobile or TV has some system of getting an average needed number of 'out of order' packets it will accept before dumping the group.  TCP/IP measurement for this is RWIN.  I think the average was around 25 to 30 packets 1500 MTU of 1460 + 40 bit header, so the RWIN was set at around from 28,000 to 37,500.  That was wayback in the 90's, when the settings was adjustable.  I think all OS now does an average setting and fixes it to a typical Internet traffic needs with some auto adjustment that probably takes a long period of problem connections to slide upwards.  If the RWIN is conservative and lower, then the response time for any user click changes is faster.  I know when I had to fix connection to field offices in Africa, turning up the RWIN to a factor of 40 to 50 times the MTU would allow the main office in Boston to connect to African field offices.

1. Are these TCP settings automatically balanced for an average Internet video exchange and does that mean satellite delivery with higher packet loss issues has more playback issues? It seems this is probably true.
2. How does each user playback hardware automatically adjust upward to compensate for satellite 'out of order' packet issues? 
3. How quick is this auto adjustment for each test hardware? 
4. Does the TCP packet handling in each hardware need a lot of errors to slide up a little? 
5. Does the new RWIN acceptance for 'out of order' packets stay there or does it revert back to the average? 
6. Are newer codecs more forgiving for end user playback on packet problem networks, like satellite delivery? 
7. Is the test video from one source exactly the same data stream delivered by Emby, even if they both start with the same master video? 
8. Can an Emby buffer cache be automatically increased when buffering is experienced? 
9. Would the Emby buffer cache help out of order packets that are being discarded by the TCP settings in the user's hardware? 
10. Isn't that a hardware level that media server software cannot control?  The TCP packet controller would have to auto adjust. 
11. Emby can request to buffer more seconds, but if the TCP handling from the user's hardware just keeps discarding TCP packet groups because the 'out of order' packet limit is reached, say 25 packets where one was missing, so all 25 were dumped and rerequested? If that happens, then Emby buffering won't help, because the buffer will keep never see the discarded TCP packets that were dumped by the user's hardware TCP settings.
12. Don't you have to adjust the auto TCP policy of the hardware to fix this?

I think to get to the bottom of this, some of these questions need to be looked at and not just skipped over, as not important.  To be clear, I have not done anything with Emby's transcoding or video buffering. I just don't use it.  But I think getting to the bottom of these issues, requires first making sure end delivery of a test video that seems to work is exactly the same end copy of the data file in both delivery methods.  Youtube, Netflix and Emby need to be sending the exact same data to the end user and not two different audio / video codecs and compression rates.  Just starting with the exact same test video source is not proof that the end delivery media data from two different sources is the same.

I hope this helps.

Edited March 10 by visproduction

[est3ban129 0]

I have same problem, my parents never had problems and issue and from this weekend, only have buffering and buffering... i tried from more locations and always same. What happen ? Client app problem? 

[unisoft 272]

One thing to check when on WAN is embys automatic setting for video playback. It simply has never worked for me on cable broadband connection (1.2gbs down 105mbps up). I have to set Apple TV as 20mbps because that's the Apple specs of Apple TV 4K 2017 for max bit rate for HD video, and for LG tvs something less than the upstream of the server so 80mbps in my case. I usually only have 1 or 2 users streaming and most of my HD video is 11mbps for TV content and up to 35mbps for movies.

If I don't do this, and leave as automatic, I get auto detect doing a stupidly low rate which then transcodes and breaks up rather than direct play which the bandwidth is easily capable of.

Doing it this way for me has worked for several years. Reported to Luke a few times but nothing done to WAN speed detection.

Edited March 12 by unisoft