Jump to content

Massive transcoding temp files


andersoc27

Recommended Posts

Ronstang
34 minutes ago, Luke said:

That's kind of a vague question. Please ask instead in terms of something specific that you're looking for. Thanks.

Not really Luke.  He's asking when the new LiveTV is due to be released....in any form....beta or final.  We all have the same question.  It has been promised since last year.  I'd like to know too since all the problems I am having with emby should be solved by it's release.

  • Like 1
Link to comment
Share on other sites

jacotec
2 minutes ago, Ronstang said:

Not really Luke.  He's asking when the new LiveTV is due to be released....in any form....beta or final.  We all have the same question.  It has been promised since last year.  I'd like to know too since all the problems I am having with emby should be solved by it's release.

You brought it to the point 🙂

Link to comment
Share on other sites

So I am still having this issue and its become an every day occurrence. Emby doesn't report that anyone is watching live TV but still keeps tuners locked in the background and just eats up my SSD.

1330778353_whattheheck..thumb.PNG.48445bd9df30128730f40f2fc5706139.PNG

 I have to restart emby in order to unlock the tuner and get the file removed.

I have made sure that the 3 devices with access to live tv are direct playing it so I'm not sure why there is even such a huge file being made.  I have also been unable to find any ffmpeg logs related to any of the streams. 

@ebr or @Luke Is there any work being done to resolve this issue that can be reported / tested? 

Link to comment
Share on other sites

Ronstang

@lukeoslavia Are you exiting the channels you are watching correctly depending upon what client you are using to watch?  This is the big issue.  If you don't use the back button for instance on my Firestick then the tuner is not released.   If you just exit the application before exiting the channel then emby does not release the tuner....which is complete crap and I have no comprehension how anyone thought this was a good way to do it.   I have had to completely stop using LiveTV in my house except on my server for recording movies and show I am keeping in my library.  Let a woman or child use emby LiveTV and no matter how many times you explain how to exit a channel an hour later all your tuners are locked and can't be released without restarting something.

This is supposed to be fixed in the new LiveTV but no one here will even talk about "when".  It's been promised since last year but things move at a snail's pace.

  • Like 1
Link to comment
Share on other sites

lastwraith
On 8/22/2021 at 3:57 PM, Ronstang said:

Not really Luke.  He's asking when the new LiveTV is due to be released....in any form....beta or final.  We all have the same question.  It has been promised since last year.  I'd like to know too since all the problems I am having with emby should be solved by it's release.

+1 on this. I managed to make it through the Olympics without watching a recording in progress, but the wife is an NFL fan and there is no way I am going to avoid watching an in-progress recording once real games start. At least for our household, an updated LiveTV release that fixes the lock on tuner transcoding files would be well timed. Due to the large size (and disk hammering even with an SSD), it is also incredibly slow to FF an in-progress recording. I'm only using a 1TB SATA SSD but still.....

Otherwise, Emby is pretty great. It pairs well with Kodi also.

Link to comment
Share on other sites

4 minutes ago, Ronstang said:

@lukeoslavia Are you exiting the channels you are watching correctly depending upon what client you are using to watch?  This is the big issue.  If you don't use the back button for instance on my Firestick then the tuner is not released.   If you just exit the application before exiting the channel then emby does not release the tuner....which is complete crap and I have no comprehension how anyone thought this was a good way to do it.   I have had to completely stop using LiveTV in my house except on my server for recording movies and show I am keeping in my library.  Let a woman or child use emby LiveTV and no matter how many times you explain how to exit a channel an hour later all your tuners are locked and can't be released without restarting something.

This is supposed to be fixed in the new LiveTV but no one here will even talk about "when".  It's been promised since last year but things move at a snail's pace.

Typically yes, I personally don't use it all that much but all the involved devices are nvidia shields which are configured to power off with the tv they are being watched on. A lot of the time, powering the shield down works, but it also doesn't work a lot of the time. 

I have tried testing this over and over and have confirmed that even hitting the back button doesn't always release the tuner, but it works better than when just powering the shield down. The other problem with that is live TV often freezes and locks up and when that happens it nearly always keeps the tuners locked in.

For me its just becoming a really big pain as I have to be sure to check if I need to restart the emby server every day, and am now seeing smart test errors on the SSD that handles transcodes. 

Link to comment
Share on other sites

Ronstang

@lukeoslavia I'm with you.  I have even had problems with tuners being locked even when I exit "correctly" sometimes and like I said I had to stop using LiveTV  in the house on my Firesticks because of this.  Kind of disappointing.  I bought Premiere so I could dump all my expensive Cable TV rental equipment and almost a year later I still have it because LiveTV is not useable as a Cable TV equipment replacement yet.  I would pull what little hair I have left out if my girlfriend and her 4 children only had emby to watch TV.  They NEVER exit correctly so I took it away from them.

I love emby but I really need LiveTV to start working properly.  I have a bunch or other issues with it that should be resolved with a the new release but I'm not holding my breath.

Link to comment
Share on other sites

lastwraith
2 minutes ago, lukeoslavia said:

Typically yes, I personally don't use it all that much but all the involved devices are nvidia shields which are configured to power off with the tv they are being watched on. A lot of the time, powering the shield down works, but it also doesn't work a lot of the time. 

I have tried testing this over and over and have confirmed that even hitting the back button doesn't always release the tuner, but it works better than when just powering the shield down. The other problem with that is live TV often freezes and locks up and when that happens it nearly always keeps the tuners locked in.

For me its just becoming a really big pain as I have to be sure to check if I need to restart the emby server every day, and am now seeing smart test errors on the SSD that handles transcodes. 

Does the "freezing" mostly occur when trying to seek a recording-in-progress or is it freezing while you are literally watching it live? If the former, it may not actually be frozen, but the disk activity is insane on the server side for something like this (at least in my setup). Emby (server-side) was fine initially for me with an HDD for everything but when we started doing recordings and especially seeking those recordings, we thought the client had frozen because it took so long. Seeking an in-progress recordings (especially seeking far into the recording) brought it to an absolute crawl. Looking at the server disk IO during that time.....it was intense. Subbing in an SSD has definitely improved things by orders of magnitude (as expected, I was just trialing Emby initially with the HDD) but seeking recordings and esp in-progress ones is still not fast. The disk is still getting hammered (often with multiple files per recording) but obviously an SSD can handle a lot more random IO without falling down a queue-depth hole.

TL;DR - Parsing recordings is slow, parsing in-progress recordings is REALLY slow and it mostly seems due to disk IO on the server side.

Link to comment
Share on other sites

1 minute ago, lastwraith said:

Does the "freezing" mostly occur when trying to seek a recording-in-progress or is it freezing while you are literally watching it live? If the former, it may not actually be frozen, but the disk activity is insane on the server side for something like this (at least in my setup). Emby (server-side) was fine initially for me with an HDD for everything but when we started doing recordings and especially seeking those recordings, we thought the client had frozen because it took so long. Seeking an in-progress recordings (especially seeking far into the recording) brought it to an absolute crawl. Looking at the server disk IO during that time.....it was intense. Subbing in an SSD has definitely improved things by orders of magnitude (as expected, I was just trialing Emby initially with the HDD) but seeking recordings and esp in-progress ones is still not fast. The disk is still getting hammered (often with multiple files per recording) but obviously an SSD can handle a lot more random IO without falling down a queue-depth hole.

TL;DR - Parsing recordings is slow, parsing in-progress recordings is REALLY slow and it mostly seems due to disk IO on the server side.

I don't actually ever try to pause or scrub live tv because I know what happens when I do lol. I also refuse to watch anything I'm recording live in the event that it might get screwed. I personally at this point get annoyed by seeing commercials, so they all go bye bye before I watch any recordings. That started as a nice to have and became a necessity for me.

So typically when live tv is involved, its just straight Direct Play play back that doesn't get paused or scrubbed, and only on the rarest of occasions gets watched while recording. The last time I can think of that happening was during last years college football post-season. 

A complete re-do on live tv would be nice to see, but honestly for me, if you can fix the locked up tuners and massive files, id be stoked, so much so I'd bring back '50s surfer slang to show my excitement. 

Link to comment
Share on other sites

lastwraith
2 minutes ago, lukeoslavia said:

I don't actually ever try to pause or scrub live tv because I know what happens when I do lol. I also refuse to watch anything I'm recording live in the event that it might get screwed. I personally at this point get annoyed by seeing commercials, so they all go bye bye before I watch any recordings. That started as a nice to have and became a necessity for me.

So typically when live tv is involved, its just straight Direct Play play back that doesn't get paused or scrubbed, and only on the rarest of occasions gets watched while recording. The last time I can think of that happening was during last years college football post-season. 

A complete re-do on live tv would be nice to see, but honestly for me, if you can fix the locked up tuners and massive files, id be stoked, so much so I'd bring back '50s surfer slang to show my excitement. 

Fair enough, I've got nothing for that then.

I also volunteer to participate in '50s surfer slang if it helps us ride the wave of a new LiveTV release. Hawaii Five-O theme song is obv playing in my head right now (Jack Lord version of course and I'm not even a boomer).

The in-laws are heading towards needing a recording solution and I am leaning towards a native Tablo workflow even though they are already using Emby for local media. They won't be able to deal with the current LiveTV situation, so I can't in good conscience set them up like we have things in our place.

Link to comment
Share on other sites

1 hour ago, Ronstang said:

This is supposed to be fixed in the new LiveTV

I'm actually not sure that can be assumed because what you are experiencing shouldn't be happening with the current Live TV implementation so I don't think it has been diagnosed well enough for us to assume it is fixed anywhere.

Do you have a 100% repeatable scenario we could try out?

Link to comment
Share on other sites

Ronstang

 

1 hour ago, ebr said:

Do you have a 100% repeatable scenario we could try out?

Which problem are you referring to?  I am having so many different problems that I only use LiveTV to record movies I intend to keep and I manage these on my server....and it still causes me a number of headaches.  I NEVER use LiveTV to watch TV anymore because it won't release my tuners many times and many of the functions don't work properly like , FF, CCs....even scheduling a recording goes rogue on my a times.   

Link to comment
Share on other sites

3 minutes ago, Ronstang said:

 

Which problem are you referring to?  I am having so many different problems that I only use LiveTV to record movies I intend to keep and I manage these on my server....and it still causes me a number of headaches.  I NEVER use LiveTV to watch TV anymore because it won't release my tuners many times and many of the functions don't work properly like , FF, CCs....even scheduling a recording goes rogue on my a times.   

Please open separate topics for these and we're happy to help you resolve them. Thanks.

Link to comment
Share on other sites

2 hours ago, lukeoslavia said:

So I am still having this issue and its become an every day occurrence. Emby doesn't report that anyone is watching live TV but still keeps tuners locked in the background and just eats up my SSD.

1330778353_whattheheck..thumb.PNG.48445bd9df30128730f40f2fc5706139.PNG

 I have to restart emby in order to unlock the tuner and get the file removed.

I have made sure that the 3 devices with access to live tv are direct playing it so I'm not sure why there is even such a huge file being made.  I have also been unable to find any ffmpeg logs related to any of the streams. 

@ebr or @Luke Is there any work being done to resolve this issue that can be reported / tested? 

@lukeoslavia can we please look at a specific example along with the corresponding emby server log? thanks.

Link to comment
Share on other sites

1 hour ago, Luke said:

@lukeoslavia can we please look at a specific example along with the corresponding emby server log? thanks.

Well as of yet, and not for lack of trying, I have never been able to find a log related to these streams. I have no idea why they arent being created, but I would assume because the live tv streams are being direct played. I can only guess there.

Link to comment
Share on other sites

Happy2Play
14 minutes ago, lukeoslavia said:

Well as of yet, and not for lack of trying, I have never been able to find a log related to these streams. I have no idea why they arent being created, but I would assume because the live tv streams are being direct played. I can only guess there.

So you are saying that file does not appear in todays or yesterdays log?

Have you opened that file to see what it contains?

Link to comment
Share on other sites

Ok, I stand corrected. I just found out that my sister, who uses her Xbox, can watch live tv as well.

She just caused it to happen and I have a log for it! 

She backed out of the channel and went to play an episode from a tv show (not live tv) and the ffmpeg transcode just kept on keeping on. It's now 2 hours later and its still going but emby's activity log knows she exited the channel at 2 hours ago.

This is a slightly different behavior from what happens nearly every day with the shields as she doesn't often use live tv. I would guess I only have this log because the Xbox has to have the stream transcoded.

livetv.PNG

ffmpeg-transcode-c85d99e5-397b-40b7-b90d-6884b7c32d78_1.txt

Link to comment
Share on other sites

8 minutes ago, Happy2Play said:

So you are saying that file does not appear in todays or yesterdays log?

Have you opened that file to see what it contains?

Those files do not appear in logs from the current day or the day before, I spent a silly amount of time over the weekend trying to find a log file to see if I could figure anything out to no avail. 

I just opened up the file and it is a live stream from the channel she was playing.

Link to comment
Share on other sites

jad3675

I'm seeing something similar, but my setup is a bit...different.

I use Channels-DVR to feed my HDHomeRUN tuners into m3u streams. On my channels-DVR dashboard, I see that it is streaming 12.1 to my emby server.

image.png.d531e07c45abe392b587def175a95364.png

The only active device I see on my emby dashboard is for the Bravo stream (ch6058 above).

Looking at the emby logs....here's someone opening channel 12.1 on the 29th:

2021-08-29 16:22:27.725 Info HttpClient: GET http://192.168.1.30:8089/devices/1075AD49/channels/12.1/stream.mpg?filter=favorites&format=ts
2021-08-29 16:22:28.226 Info HttpClient: Http response 200 from http://192.168.1.30:8089/devices/1075AD49/channels/12.1/stream.mpg?filter=favorites&format=ts after 501ms. HeadersCache-Control=no-store, no-cache, Transfer-Encoding=identity, X-Channels-Dvr-Identifier=nUybNZzoSDjUI7MJnFi0gBu1ewE7WHuReGaVEAMCqkc=, X-HDHomeRun-Device=1075AD49, X-HDHomeRun-Resource=tuner0, X-Live-Delay=0.000000, Date=Sun, 29 Aug 2021 20:22:28 GMT, Connection=close
2021-08-29 16:22:28.227 Info SharedHttpPipelineSource: Beginning SharedHttpPipelineSource stream to /home/emby/transcoding-temp/33d1cb105f83474e8b9a5400526aabe4.ts

And here's the resulting transcode file....which is growing and has a current time/date stamp.

 ls -latrh /home/emby/transcoding-temp/33d1cb105f83474e8b9a5400526aabe4.ts
-rw-r--r-- 1 emby emby 231G Sep  1 11:00 /home/emby/transcoding-temp/33d1cb105f83474e8b9a5400526aabe4.ts

 

I do not see any transcode log files though for that session though to indicate what it is doing.

 

So yeah, something is a bit off.

 

John

 

Link to comment
Share on other sites

16 hours ago, lukeoslavia said:

Well as of yet, and not for lack of trying, I have never been able to find a log related to these streams. I have no idea why they arent being created, but I would assume because the live tv streams are being direct played. I can only guess there.

Hi.  There will always be a server log.  That is what Luke is asking for.  Please attach the server log along with a description of one of these instances and when it happened.  Thanks.

Link to comment
Share on other sites

1 hour ago, ebr said:

Hi.  There will always be a server log.  That is what Luke is asking for.  Please attach the server log along with a description of one of these instances and when it happened.  Thanks.

Oh I see. Well I don't really want to upload that log file, but I can tell you that emby logged this when it seems to think the stream had stopped. If I'm understanding that correctly, emby seems to think it had stopped the ffmpeg session and deleted that file, which as you can see from my previous image, was still there and growing 2 hours later.1906215384_logsnippit.thumb.PNG.736b0819fb61c90de526f35377cee407.PNG

Link to comment
Share on other sites

1 hour ago, lukeoslavia said:

Oh I see. Well I don't really want to upload that log file, but I can tell you that emby logged this when it seems to think the stream had stopped. If I'm understanding that correctly, emby seems to think it had stopped the ffmpeg session and deleted that file, which as you can see from my previous image, was still there and growing 2 hours later.1906215384_logsnippit.thumb.PNG.736b0819fb61c90de526f35377cee407.PNG

Hi.  No, that is not the transcoding file.  Can you PM the log to Luke?

Link to comment
Share on other sites

On 9/1/2021 at 2:33 PM, lukeoslavia said:

@ebr  @Luke I sent the log file.

Thanks. From what we saw in your log files, your issue should be resolved in the 4.7 server release. Thanks.

  • Like 1
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...