Jump to content
pir8radio

FFMPEG hung up

Recommended Posts

pir8radio

I had an issue yesterday that I have not seen in a while...  It looked like two roku users's ffmpeg was hung up.   Is that still an issue with roku if they quit in some weird way?

Share this post


Link to post
Share on other sites
Luke

Depending on how they exit the app it could take a few extra minutes to shut down, yes.

Share this post


Link to post
Share on other sites
speechles

If they pause the app and then press the home button the server might not notice to kill ffmpeg. I know if the person is playing an item and hits the home button the server knows since no more progress updates are incoming to kill the ffmpeg process. But if the user pauses and then presses home, is the server smart enough to still kill ffmpeg eventually? @@Luke

Edited by speechles

Share this post


Link to post
Share on other sites
ebr

If they pause the app and then press the home button the server might not notice to kill ffmpeg. I know if the person is playing an item and hits the home button the server knows since no more progress updates are incoming to kill the ffmpeg process. But if the user pauses and then presses home, is the server smart enough to still kill ffmpeg eventually? @@Luke

 

Yes because progress updates are still sent when paused.

Share this post


Link to post
Share on other sites
Luke

Yes it will eventually kill the process but it could take a few minutes.

Share this post


Link to post
Share on other sites
pir8radio

well i have been having intermittent ffmpeg hangs, that last until i force kill them...   had two yesterday, might not have any more for days...   Looks like im going to have to re-install process lasso and kill any tasks that have been running for longer than 6 hours..  had this on my last server setup, was hoping it was fixed...  not sure what is causing it to be honest..   can pm logs if needed...  but i think it will be hard to spot...

Share this post


Link to post
Share on other sites
Luke

Why do you feel they were hangs?

Share this post


Link to post
Share on other sites
pir8radio

Why do you feel they were hangs?

 

No users on server, no recordings, no streams, FFMPEG running and consuming 0% CPU, no scheduled tasks running, task stays running as long as emby is running.   

 

 

Below I have 3 legit streams going right now, look at ffmpeg instance #2, its been running for almost 6 hours, and only has 500ms of cpu time...  it has no average CPU usage.   Its like emby started it and never put it to use, but left it open...    Its also the only instance not using any of my drives. 

 

 

hungffmpeg.png

Edited by pir8radio

Share this post


Link to post
Share on other sites
Luke

How did they stop playback?

Share this post


Link to post
Share on other sites
speechles

Every time the video player exits normally a stop transcode api call (/Videos/ActiveEncodings DELETE) is made. One is also made every time an error causes playback fallback or the user uses playback correction.

 

The only thing I can think of is pausing. Are they pausing the stream and then while paused pressing home? The most important variable is as luke mentioned. How are they exit the app which leaves the ffmpeg process orphaned? Knowing this it would help tremendously in spotting a root cause here. Thanks. :)

Edited by speechles

Share this post


Link to post
Share on other sites
pir8radio

I don't think that is the case...     That ffmpeg instance only has 1/2 a second of CPU time..     no video actually played on that instance of ffmpeg.   Its like emby started it and didnt do anything with it.  

Share this post


Link to post
Share on other sites
speechles

If it happens again and you can isolate which ffmpeg log belongs to it that should give us an indication of what is happening. There should be something in the logs even if ffmpeg decided to orphan itself. It would be interesting to see a root cause for this as others have reported similar and we have never been able to spot it in the wild ourselves. Having the logs is like having the footprints of it. With those can put it to rest finally. :)

Edited by speechles

Share this post


Link to post
Share on other sites
pir8radio

Ill try to watch it.. the only pattern so far seems to be after i notice my only roku user has been watching live TV or a movie...   

Edited by pir8radio

Share this post


Link to post
Share on other sites
pir8radio

If it happens again and you can isolate which ffmpeg log belongs to it that should give us an indication of what is happening. There should be something in the logs even if ffmpeg decided to orphan itself. It would be interesting to see a root cause for this as others have reported similar and we have never been able to spot it in the wild ourselves. Having the logs is like having the footprints of it. With those can put it to rest finally. :)

 

Here is the ffmpeg log from the last hang.   I stripped the parameters off the beginning of the log, because i hate sanitizing it.   Looks like ffmpeg did its analyze just fine, then when it tried to play below it got a 403 for whatever reason... but ffmpeg just sits there?       Pretty sure this is the log from the frozen ffmpeg.  Im going to keep looking.

ffmpeg version 4.0.3-emby_2019_02_18-g5abbb1430d+380 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 8.2.1 (Rev1, Built by MSYS2 project) 20181214
  configuration:  --disable-autodetect --enable-amf --enable-bzlib --enable-cuda --enable-cuvid --enable-d3d11va --enable-dxva2 --enable-iconv --enable-lzma --enable-nvenc --enable-zlib --enable-sdl2 --disable-debug --enable-ffnvcodec --enable-nvdec --enable-gmp --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-fontconfig --enable-libass --enable-libbluray --enable-libfreetype --enable-libmfx --enable-libmysofa --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libwavpack --enable-libwebp --enable-libxml2 --enable-libzimg --enable-libshine --enable-gpl --enable-avisynth --enable-libxvid --enable-libaom --enable-version3 --enable-libzvbi --enable-gnutls --extra-cflags=-DLIBTWOLAME_STATIC --extra-libs=-lstdc++ --extra-cflags=-DLIBXML_STATIC --extra-libs=-liconv
  libavutil      56. 14.100 / 56. 14.100
  libavcodec     58. 18.100 / 58. 18.100
  libavformat    58. 12.100 / 58. 12.100
  libavdevice    58.  3.100 / 58.  3.100
  libavfilter     7. 16.100 /  7. 16.100
  libswscale      5.  1.100 /  5.  1.100
  libswresample   3.  1.100 /  3.  1.100
  libpostproc    55.  1.100 / 55.  1.100
[http @ 000002303a641700] HTTP error 403 Forbidden
http://iptv.com/play/2435.ts?token=eyJifQ==: Server returned 403 Forbidden (access denied)

Share this post


Link to post
Share on other sites
Luke

Your provider returned a forbidden response code. You may need to consult with them on why they did that.

Share this post


Link to post
Share on other sites
pir8radio

Your provider returned a forbidden response code. You may need to consult with them on why they did that.

 

 

Yea,  they do that some times if the token doesn't get refreshed in a timely manor,   but emby/ffmpeg should be able to handle that situation and not freeze up right?

Share this post


Link to post
Share on other sites
Luke

Yes we should be able to give you a graceful error message. For now, that at least answers why it didn't play.

  • Like 1

Share this post


Link to post
Share on other sites
pir8radio

did it again, not sure the issue this time,   I now have 11 ffmpegs running, emby restarted 3 or 4 times on its own. i have 4 tray icons running.    The 11 ffmpeg logs are all empty text files, not 1 character.   PMing @@Luke the server log. 

 

5c89a6038cbed_WTF.png

Share this post


Link to post
Share on other sites
pir8radio

the above was not related to ROKU to be clear.

Share this post


Link to post
Share on other sites
Luke

Please discuss these sessions. thanks.

Share this post


Link to post
Share on other sites
pir8radio

Please discuss these sessions. thanks.

 

what do you mean?     I dont know the specifics, people were trying to use emby...   probably watching live IPTV?   you can see in the log file that emby "restarted"  because it shows the new log header showing the server info, it happens like 4 times i think in that log. 

2019-03-09 10:08:50.664 Info Main: Application path: C:\Users\nam\AppData\Roaming\Emby-Server\System\EmbyServer.dll
2019-03-09 10:08:51.759 Info Main: Emby
	Command line: C:\Users\nam\AppData\Roaming\Emby-Server\System\EmbyServer.dll -noautorunwebapp
	Operating system: Microsoft Windows NT 6.2.9200.0
	64-Bit OS: True
	64-Bit Process: True
	User Interactive: True
	Processor count: 40
	Program data path: C:\Users\nam\AppData\Roaming\Emby-Server
	Application directory: C:\Users\nam\AppData\Roaming\Emby-Server\System
2019-03-09 10:08:51.760 Info Main: Tray icon path: C:\Users\nam\AppData\Roaming\Emby-Server\System\embytray
2019-03-09 10:08:51.929 Info App: Application version: 4.1.0.13
2019-03-09 10:08:51.931 Info App: Loading assemblies

the ffmpeg logs generated were totally empty so I cant post them. I PM'ed you the log. 

Edited by pir8radio

Share this post


Link to post
Share on other sites
pir8radio

IPTV acting up today, happened to notice this in the task bar, not sure how its related.  Logs only show starting the tray icon once at startup.

 

5c98467a2558f_Capture.png

Share this post


Link to post
Share on other sites
Luke

Are you still having an issue?

Share this post


Link to post
Share on other sites
pir8radio

Are you still having an issue?

 

have not seen any, But ill report back here if so.

Share this post


Link to post
Share on other sites
Luke

Thanks !

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×
×
  • Create New...