Jump to content

Emby isn't closing used network ports.


anderbytes

Recommended Posts

anderbytes

Knowing that works in HTTP but no HTTPS can be a good tip/info.

Thanks.

 

So... what we know for now:

- Emby Server in Linux

- Android App as Client

- HTTPS as connection port

 

Can anyone else confirm that the issue is not happening at 8096 (HTTP) ??

Link to comment
Share on other sites

pdoroba

Knowing that works in HTTP but no HTTPS can be a good tip/info.

Thanks.

 

So... what we know for now:

- Emby Server in Linux

- Android App as Client

- HTTPS as connection port

 

Can anyone else confirm that the issue is not happening at 8096 (HTTP) ??

I had the bug reappear just using firefox on android after having removed the emby app a few days earlier. I'm not sure its limited just to the android app but, maybe related to android itself? I have a samsung s7 kernel version 3.18.20

Edited by pdoroba
Link to comment
Share on other sites

runtimesandbox

Will based on Xenus report I'm not sure there's a problem here

 

Its good information but I don't think it rules out the problem entirely? 

 

Our servers must be doing that process fine 99% of the time. The issue isn't always happening, it is random and I can go weeks in between it happening.

Link to comment
Share on other sites

anderbytes

Its good information but I don't think it rules out the problem entirely?

 

Our servers must be doing that process fine 99% of the time. The issue isn't always happening, it is random and I can go weeks in between it happening.

That's easily explainable. As the issue needs some specific scenario, it won't happen all the time.

 

The ports become "zombie" only when the connection is improperly closed (as a sudden client network drop), so you can spend days without watching the issue.

 

Notice that you will see the ports opened forever after a day where you use the client app in a intensive basis. At least that's what I can see when I'm out of home and trying to use in my fragile 3G/4G

Link to comment
Share on other sites

anderbytes

Mono 4.4 went stable today. you can try playing with that but just be warned we haven't tested it yet. You'll need to revert back to 4.2.3 if you have any problems.

 

Mono 4.4.0 appeared just now and I updated. Now will begin my tests.

Good luck for me  :-))

Link to comment
Share on other sites

anderbytes

I added a 5 minute timeout to http connections with the last stable release. if there's still an issue then it's probably something else.

 

@@Luke I've seen something I believe it's important.

The 5-minutes timeout is working for most sockets... so they are closing. But ONE of them never closes, never obeys the timeout, or it's not implemented for it.

 

1 - Is the timeout implemented in a central connection manager or distributed along several connection creators over the code?

2 - Can you point me which change in github implemented the timeout?

 

 

Now some ideas:

- Can the timeout time be configurable? Maybe in an advanced tab... because some people like me think that 5 minutes is too long, other would prefer more...

- Can't the debug log show on which connection (by remote IP/port) the action is happening? That way we can dig better into network issues, and others..

 

Please answer 1 and 2 above

Thanks!

Link to comment
Share on other sites

anderbytes

STEP-BY-STEP

@@Luke and @@ebr, look at this.

 

Step 1: No connections. Emby for Android not started yet.

5762a5cf5bc1b_Tela2016061610h02m001.jpg

 

Step 2:  Emby for Android connected and in Home Screen. No navigation done. Executed 4G disconnect. 5-min clock started.

5762a5fead873_Tela2016061610h03m001.jpg

 

Step 3:    5 minutes later, Ports started closing process. Except one (37800 in list)

5762a63bb35cb_Tela2016061610h08m001.jpg

 

Step 4: Some minutes later. Only 37800 left. Forever.

5762a67fa5d5d_Tela2016061610h11m001.jpg

Edited by anderbytes
  • Like 1
Link to comment
Share on other sites

anderbytes

So it's just one per device?

 

One for each connection made .

 

Next time my 4G regains signal and reconnects... new connections are made, usually from another IP, and the step-by-step above happens all over again.

 

So... imagine I'm using Emby and operator network is unstable.... it will connect and reconnect from client to server several times. Each one (of the several) leaving one of the ports open forever... and using Emby along the day multiplies those ocurrences.

 

 

... and when it starts hanging... there are at least more than 50 zombie connections there. That hangs the server.

Edited by anderbytes
  • Like 1
Link to comment
Share on other sites

anderbytes

So it's just one per device?

 

Luke, please don't give up on this, this "just one" rapidly becomes a snowball effect while Emby is being used.

 

As several sockets are closed properly respecting the 5-min timeout, isn't that possible that somewhere in the code 1 connection created doesn't have that timeout set??

Link to comment
Share on other sites

runtimesandbox

I have stopped using the android app (around a month now) and have not experienced the issue since.

Link to comment
Share on other sites

anderbytes

I have stopped using the android app (around a month now) and have not experienced the issue since.

 

Hey @@Luke, this will eventually be recognized as the "official solution".

  • Like 1
Link to comment
Share on other sites

pdoroba

I have stopped using the android app (around a month now) and have not experienced the issue since.

I stopped using it as well but, still have the issue.  It doesn't occur as often as it did when I had the Android app installed but it still occurs once every 3 days or so.  It's very frustrating.  I like Emby and am a premier supporter.

Link to comment
Share on other sites

anderbytes

From what I've seen... any client that gets the connection abruptly closed makes possible for the event to happen, not only Android.

 

It occurs that browsers are much harder to make this condition happen.

Link to comment
Share on other sites

Banjo

From what I've seen... any client that gets the connection abruptly closed makes possible for the event to happen, not only Android.

 

It occurs that browsers are much harder to make this condition happen.

 

I just ran the same tests that you did run the Android client and ripping out the network from under it.  All of the network connections timed out after a few minutes.

 

Setup is:

 

Ubuntu Server 14.04

Docker 1.11.1, build 5604cbe

Docker image: emby/embyserver:latest
Image ID: e29933d87d82
 
Emby Server 3.0.5972.0
Emby Android Client 2.7.09
 
Access is through an Apache proxy handling SSL. The proxy runs in a docker container and binds the 443 port on the server interface.  It routes Emby traffic to the Emby container using a private docker network.
 
Would be interesting to know what causes such a difference in behaviour.
Link to comment
Share on other sites

anderbytes

@@Banjo, other users that use Docker also coudn't reproduce the issue... probably it creates it's self network layer that converses OK with Emby Server open sockets.

 

Why are you using Docker? Can you give me some tutorial and tell it's benefits??

 

Thanks!

Link to comment
Share on other sites

Banjo

I wonder if it might be a distro issue.  The Emby docker image is openSUSE based.  Which distro are you installed on?

 

I mainly use Docker because it means that if I need to reinstall the server then it's a lot easier.  Docker containers are also a lot lighter than full VMs which I was using before.  I guess if all you're doing is running Emby then it's less of a benefit but I'm running cloud, mail, git, jenkins, web, blog, vpn and emby servers.  Setting that lot up from scratch is a major hassle.  ;)

There's also the fact that it was a useful learning experience as we move to docker base CI at work.

Link to comment
Share on other sites

runtimesandbox

I wonder if it might be a distro issue.  The Emby docker image is openSUSE based.  Which distro are you installed on?

 

I mainly use Docker because it means that if I need to reinstall the server then it's a lot easier.  Docker containers are also a lot lighter than full VMs which I was using before.  I guess if all you're doing is running Emby then it's less of a benefit but I'm running cloud, mail, git, jenkins, web, blog, vpn and emby servers.  Setting that lot up from scratch is a major hassle.   ;)

There's also the fact that it was a useful learning experience as we move to docker base CI at work.

 

Agreed, VM's and containers are the way forward for running lots of apps. Much easier to reinstall one than everything!

 

So does this mean it appears to be a Debian based distribution issue? It has happened for me on both Debian 8, Ubuntu 14 and 16 in both VM and container form. I personally have not tried other distributions .

Link to comment
Share on other sites

If you guys could try the dev channel that would be helpful. Though I am still not convinced of the issue based on many people reporting everything closed, I may have found something. Thanks.

Link to comment
Share on other sites

anderbytes

If you guys could try the dev channel that would be helpful. Though I am still not convinced of the issue based on many people reporting everything closed, I may have found something. Thanks.

I have recently changed to using Emby inside Docker and since then havent been able to reproduce the issue.

 

Using Docker now I can install dev, beta... That I couldnt. Unfortunately (or not), inside a container I cant reproduce the socket issue.

Edited by anderbytes
Link to comment
Share on other sites

anderbytes

I've just seen the code changes in dev branch, well good luck to us.

 

About testing... I really wish I could help now, but I cant risk breaking my NAS by installing non-stable stuff outside Docker environment.

 

When it reaches stable, I commit to disable my Docker Emby, install the stable outside Docker, and confirm that the issue is gone.

Edited by anderbytes
Link to comment
Share on other sites

runtimesandbox

I've just seen the code changes in dev branch, well good luck to us.

 

About testing... I really wish I could help now, but I cant risk breaking my NAS by installing non-stable stuff outside Docker environment.

 

When it reaches stable, I commit to disable my Docker Emby, install the stable outside Docker, and confirm that the issue is gone.

 

Any indication what the code changes are...?

 

Happy to try dev or beta when I next have a free few hours

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...