Jump to content

Need Transcoding help


sharrisct25@hotmail.com
 Share

Recommended Posts

sharrisct25@hotmail.com

So I have recently added a HDhomerun EXTEND to my Emby system.  The goal is to allow people to watch a few local stations in Emby that we do not get on other steaming services.   All of my TVs are Roku clients of different types.  Some hardwired some Wifi.  I use my Android client on my phone on occasion also.  I have 2 HDhomerun EXTEND units connected and my goal is to be able to support 4 users watching / recording at the same time without any major load on my Emby server.

 

The Server is virtualized running as a Windows 10 system in VMware 6.5.  It has 4 cores and 10GB or RAM.  All the storage is local SSD on a RAID 5 array.  The server is Emby Premier version 4.4.2.0 and the Roku clients are using Emby beta. 

 

Historically software transcoding has been fine for the movies and other videos I have in my library but with TV I knew the use would spike and more transcoding maybe needed.  This lead me to buy the HDhomerun Extend versions due to their capability to decode.  

 

Once setup I had a few issues with specific channels that I have learned are down to reception thanks to posts and help in other Emby forums.  Then I was still seeing ffmpeg activity going on, sometimes as a directstream where the audio was being converted, others full transcoding.  With more help from the Forum I purchased and installed a NVIDIA GTX 1030 card to do hardware decoding.  

 

The problem is I am still seeing ffmpeg activity for the Roku clients.  I now DO NOT see it for the Android but definitely for the Roku clients.  Can somebody help me sort what is going on here? Shouldn't the decoding all be happening in hardware now between the HDhomerun EXTEND and the NVIDIA card?  I see that the Processing Plan in the ffmpeg log still says False for all CanDoInHardware lines does that mean the card is not working correctly?

 

I have attached logs for more details.

hardware_detection-63724218955.txt

ffmpeg-directstream-0194bed2-a9ad-4aac-b654-03b1a909ac4e_1.txt

embyserver.txt

Link to comment
Share on other sites

Hi.  All that is happening in that playback is it is being re-packaged into a container the device can handle (stats for nerds or the server dashboard should have indicated this).  The Roku cannot take the stream straight as it is.

 

This should take very little resources on your server.

Link to comment
Share on other sites

sharrisct25@hotmail.com

Thank you ebr.  I will try stats for nerds going forward but it sounds like you do not see a concern here so that is good.  Is there a way to have it avoid any interaction with ffmpeg like it does for the Android client or is it unavoidable with a Roku?

Link to comment
Share on other sites

Thank you ebr.  I will try stats for nerds going forward but it sounds like you do not see a concern here so that is good.  Is there a way to have it avoid any interaction with ffmpeg like it does for the Android client or is it unavoidable with a Roku?

 

The general rule of thumb would be to play media that is already in a format that the device can direct play.

Link to comment
Share on other sites

  • 6 months later...
sharrisct25@hotmail.com

I think I am just not quick about the whole conversion stuff but I continue to struggle with constant transcoding when using live tv.  All my clients are Roku, it is just not practical to swap them over to Android.  I have upgraded a few of the older ones and they seem to be a bit faster in general but they still all transcode a bit.  This causes delays, caching, and issues with my local hard drive filling up with transcoding files that do not go away automatically.  

Attached is a zip with some prime examples of this.  It is a ffmpeg file for a user watching live TV in our Livingroom.  The client is a Roku 4k that is hard wired into the network.  The back end server has a NVIDIA GeForce 1030 card in it and the live TV is coming from a HDhomerun Extend so there should be as much hardware acceleration as possible.  Every once in awhile I get a ffmpeg-directstream file that gets REALLY big.  Earlier today there was one on my server that was over 3GB.  

What am I doing wrong here?  Is this really the best you can expect on a Roku client?  They are probably the most popular streaming device so I do not feel that should be the case.

My emby server works great minus the live tv.  The live tv works, it is just unreliable.

Any help you can provide would be great.  Thank you.

embyserverlogs.zip

Link to comment
Share on other sites

Happy2Play

As the log shows "TranscodeReasons=ContainerNotSupported" there really isn't anything else Emby can do if the device itself has this limitation.  That is why these streams are being repackages into a supported container.

Note GPU is irrelavant in DirectStreams as there is no video conversion.

17:36:55.800 Stream mapping:
17:36:55.800   Stream #0:0 -> #0:0 (copy)
17:36:55.800   Stream #0:1 -> #0:1 (copy)

 

Link to comment
Share on other sites

When the Roku receives a TS file for direct play it assumes this is a real-time broadcast. It will not allow seek. The only thing you can do is pause and let it buffer onto the device which can only hold pause for a very short duration. This is why we do not allow the Roku to direct play these TS containers and instead they are repackaged.

You will lose the ability to seek within the TS container if we direct play these. Is that okay?

We can look into improving this situation for users who sit through linear TV without touching the remote. For them seek is not done and direct play of TS could occur. For everyone else this would be worse. This is why it is the way it is. But there could be options perhaps if enough people use Emby this way with real-time TS they watch in a linear fashion without seeking. Entirely possible in the future.

When you repackage the copied video and audio streams from a TS into an m3u8 manifest takes very little resources from your CPU or FFMPEG. This can be done quickly and it is over. Hundreds of FPS per second since everything is copied. Once the device has the m3u8 it is served TS slices. These TS slices are what allow seek. Because the file is now fragmented and given a manifest of how the construction occurs the Roku can see the end. It can develop a runtime. When the Roku direct plays a TS file it does not know or show the runtime as it does not expect it to ever end. This is why you cannot seek a TS with direct play these do not show how much runtime there is and it is difficult to seek ahead accurately. This is why Roku has disabled seek when a TS direct plays. It isn't us being ridiculous or mean. It is just the way it is.

Edited by speechles
  • Like 1
Link to comment
Share on other sites

sharrisct25@hotmail.com

Thank you both, this is very helpful.  

If there could be some client side configuration to choose if live TV streams just directplay or cache so they can be paused etc I think that would be great.  My server has the resources needed to handle transcoding for a couple of users but after like 3 it starts to struggle.  A particular issue is the number of files created when watching live tv, they never seem to be cleaned up and it can suck up like 300GB of temp space in a few hours if a few people are doing it at the same time.

So if I could configure the bedroom TVs and the ones in the basement to just direct stream and leave the Livingroom with the ability to pause etc that should significantly improve the number of files created and resource requirements.

Could this be set via user or would it have to be done on each specific device?

Link to comment
Share on other sites

  • 2 weeks later...
  • 3 months later...
sharrisct25@hotmail.com

Have you guys put anymore thought into maybe making a enhancement to choose which devices or users transcode to provide rewind functionality?  

Link to comment
Share on other sites

37 minutes ago, sharrisct25@hotmail.com said:

Have you guys put anymore thought into maybe making a enhancement to choose which devices or users transcode to provide rewind functionality?  

What exactly do you mean by that?

Link to comment
Share on other sites

sharrisct25@hotmail.com

Earlier somebody had indicated that perhaps there could be some client side configuration to choose if live TV streams just directplay or cache so they can be paused etc.  I think that would be great if it is possible.

Link to comment
Share on other sites

sharrisct25@hotmail.com

Thanks Cayars that is great but for me with 10 Roku devices I really need to be able to do it on the Roku.   The caching really kills the live tv performance on my system despite having a hardware card and pretty fast disk.  

Link to comment
Share on other sites

2 hours ago, sharrisct25@hotmail.com said:

Thanks Cayars that is great but for me with 10 Roku devices I really need to be able to do it on the Roku.   The caching really kills the live tv performance on my system despite having a hardware card and pretty fast disk.  

I'm afraid the only way the Roku can play those live streams is the way we are delivering them now.

  • Like 1
Link to comment
Share on other sites

sharrisct25@hotmail.com

Oh so there is no option for direct play?  Speechless seemed to say if you did not want to be able to rewind etc it could be direct play.

Link to comment
Share on other sites

Just went back and read things and @speechles did mention this was possible so he would have to tell us if this has any movement.

But, let's step back a second and look at what's happening on your system presently.
There are basically 3 or 4 main ways Emby can play back media.

1) It can direct play where the media isn't touched in any way
2) The container is changed but the contents stay the same (think mkv vs mp4 or ts vs HLS)
3) The audio is TRANSCODED but video isn't touched at all.
4) The Video is TRANSCODED for some reason like a bitrate constraint, subtitles overlaid,etc

Any time you do 3 or 4 above a container change is likely to happen as well but the container change makes no different as the file has to be "re-written" anyway.  The container change of #2 is very easy on the CPU and is mostly just IO of reading/writing the new file on the fly in some manner (usually segments).  Also pretty easy on the CPU is an AUDIO TRANSCODE.

Just about any device from a Pi to an Android to a NAS can handle the first 3, but it's VIDEO TRANSCODING that is the demanding job.  This is the reason people add GPUs to their system to offload this processing from the CPU.

So for practical purposes we can say you direct play (1), remux (2 & 3) and Transcode (4).

When you play back Live TV on your present system to one of your LOCAL Roku clients you are likely getting a remux but not a transcode.  Your computer shouldn't have an issue with this as it's not heavy on the CPU/GPU resources.

Where things can/will get interesting is for REMOTE clients (any platform).  A typical OTA channel can easily be 15 Mb so that can use up your UPSTREAM bandwidth on the server.  As an example if you have Comcast internet you likely only have 10 to 17Mb of upstream bandwidth available regardless of your download rate.  So if you try to push a 15Mb stream through a 10Mb pipe it isn't going to work and will stutter on the client side as it will have to keep waiting to receive more data.  The solution to this is usually to TRANSCODE the video from mpeg2 to h.264 to make it much smaller and to fit the pipe.

All of the above is just a high level overview of what can take place.  Now let's dive a bit more into your specific situation.

How many of these Roku clients are local (same lan) and how many remote?
How many REMOTE clients (any client) do you expect to playback TV at the same time?
How many tuners do you have?

What is your UPLOAD bitrate if you run a test from speedtest.net from the Emby Server in a browser?
Do you have any bitrate limits set on the Emby Server?

Link to comment
Share on other sites

6 minutes ago, cayars said:

Just went back and read things and @speechles did mention this was possible so he would have to tell us if this has any movement.

The Roku supports TS. Can direct play TS all day 24/7 but only in a linear fashion. That means you can only pause and then gain a buffer. There will be no runtime known. The transport bar would only allow pause at first. Then as it played you would get rewind. As you rewound you get a fast forward. But it would always try to jump to the real time point everytime you fast forward. You have to watch linear. You can rewind. But once you try to fast forward the results get very unpredictable.

We disable the TS container. Because it has these problems. We could enable it through an option. That is possible. But it would need a huge disclaimer; flashing lights; a strobe light with disco ball over it. A huge neon sign. It would say "Use at your own risk. When you enable this option we do not troubleshoot issues.".

Is it a weakness not being able to direct play TS even though it will provide a less than stellar experience? Is it worth it? Would you use it?

Edited by speechles
Link to comment
Share on other sites

sharrisct25@hotmail.com

So honestly I am torn here.  Many people have posted that I should not have issues but I really do.  I have a GPU (GT 1030) in the system as still get issues.  I am running Emby in a virtual machine on a VMware server so I guess that is another layer of extraction but still the VM has 6vCPUs and 16GB or RAM and there is nothing really competing for those resources.   The VM also has local SSD for the system drive so anything it does write to disk is fast.

The tuners are 2 HDhomerun Extend boxes set to transcode to the mobile profile and advertise 30fps.  I disabled advertising native format and using the heavy profile based off feedback from other forum users troubleshooting this problem.  All of this in on the same 1GB switch as the Roku and the Emby server using the same /24 subnet.  A few of the Roku boxes are wired, the wireless ones connect via a Unifi WIFI access point and according to the monitoring on that there are no network problems to speak of.

Despite that hardware watching live tv via Emby is very haphazard for any Roku on our home network.  I doubt it is the network itself as there are no noticeable issues with other things running on it, nor do the Roku show any problem playing back stored media. Watching live TV even on our Roku 4 using a wired 1GB connection takes a minute to start and then will stop to cache again every 3 to 15 minutes.  

If I switch from Emby to the  HDhomerun app on the Roku for the same channel it does not seem to cache at all.

If this only happened on wireless clients I would think it is bandwidth but it happens just as much on wired ones.  

I would LOVE to make this work without having to sacrifice rewind capability but I really do not know how to do that.  Not sure what I am missing.  Any suggestions welcome

Link to comment
Share on other sites

Can you answer the questions I asked so we have more info to understand you setup?

Link to comment
Share on other sites

sharrisct25@hotmail.com

Sure here are my responses to the questions you posed, if I missed any please let me know.

How many of these Roku clients are local (same lan) and how many remote? There are 10 on the same lan.  2 are hard wired the rest are WiFI.  I have seen these issues even when only 1 hard wired Roku is being used.
How many REMOTE clients (any client) do you expect to playback TV at the same time?  I would like to allow 2 ideally with the ability for something to be recorded at the same time.  I would be happy with just 1 live and 1 recording.
How many tuners do you have? I have 2 HDHomerun Extend tuners.

What is your UPLOAD bitrate if you run a test from speedtest.net from the Emby Server in a browser?  Using googles speed test i get an average of 287Mbps down and 591Mbps down from the Emby Server.  The lowest test was 187Mbps down the highest was 412Mbps.  Upload was very consistent.
Do you have any bitrate limits set on the Emby Server?  I do not have any limit set for the internet streaming bitrate limit in the network settings for Emby.  Is there another setting I missed?

Link to comment
Share on other sites

Hmm, what is your upload speed as reported at speedtest.net ran from the Emby Server machine?

Play a Live TV channel for me via any roku device.  Now look at your logs files.  The very latest ffmpeg file should have a time stamp of playback (very it's the current one).

What is the name of the ffmpeg file?  Does it say transcode or remux in the file name?

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

×
×
  • Create New...