3.2.7.0 -> 3.3.1.0 Upgrade External Connectivity Stopped Working (SSL)

[smokey7722 15]

I had 3.2.7.0 installed and working great and have had a certificate installed for years working well.  With this new upgrade, all external access is now failing.  I only allow SSL connectivity for external users so I can't tell at the moment if non SSL works.  The emby main screen still shows the correct URL for remote access as via SSL as https://emby.domain.net:8891/ (domain swapped)

 

I've verified that I can access that port via both wireshark and via a wget on a remote server (domain and ip swapped).

 

user@server:~ % wget https://emby.domain.net:8891
--2018-03-18 01:41:56--  https://emby.domain.net:8891/
Resolving emby.domain.net (emby.domain.net)... 1.1.1.1
Connecting to emby.domain.net (emby.domain.net)|1.1.1.1|:8891... connected.
HTTP request sent, awaiting response... 302 Found
Location: web/index.html [following]
--2018-03-18 01:41:57--  https://emby.domain.net:8891/web/index.html
Reusing existing connection to emby.domain.net:8891.
HTTP request sent, awaiting response... 200 OK
Length: 2643 (2.6K) [text/html]
Saving to: 'index.html'

index.html                100%[===================================>]   2.58K  --.-KB/s    in 0s

2018-03-18 01:41:57 (110 MB/s) - 'index.html' saved [2643/2643]

So there is connectivity.  Oddly enough theres just about NO log entries in Emby.  I do see a single request and response for the wget which is fine, its showing communication.  However any official client (or web browser) accessing remotely shows the similar as below - a single GET, followed by a Response 200 in those cases

2018-03-17 18:41:56.428 Info HttpServer: HTTP GET https://emby.domain.net:8891/. UserAgent: Wget/1.19.4 (freebsd11.1)
2018-03-17 18:41:56.428 Info HttpServer: HTTP Response 302 to 1.1.1.2. Time: 0ms. https://emby.domain.net:8891/ 

I went back into the emby config, verified the certificate, the port, domain, certificate path and password.  Allow remote connections is checked and it all looks fine.

 

Any ideas for what made this break with the 3.3.1.0 upgrade I did earlier today?

 

[Luke 42077]

Please attach the emby server log, thanks.

[smokey7722 15]

So just for kicks I deleted the server reference from my Android phone I was using to test as well, recreated it and it works.  However I have family members with Samsung TV's, Nvidia shields, etc. that shouldn't need the connection deleted/created again if thats what ultimately fixed it here.  Rather than me scrubbing the log to remove hostnames/ip's to post it to this thread - can I send it privately so you can see the entire log?  I don't mind posting scrubbed stuff.

 

Was there something changed in the newer server that may require a client create a new connection?  I'm not using Emby Connect, just a direct reference to the server.  

Edited March 18, 2018 by smokey7722

[Luke 42077]

You can send privately yes.

[smokey7722 15]

You can send privately yes.

Sent, thanks

[Luke 42077]

Have you tried a second device? there's nothing obvious here.

[smokey7722 15]

Yup thats my point.  The android phone didn't do a thing until I deleted/recreated the client entry.  A Samsung TV at a family members house has been trying for hours and that doesn't show up at all.  Unfortunately I can't delete/recreate their connection as I am not there (and they aren't the most technical of folk).  I did verify with them that the TV has internet (youtube app and streaming a random video).

Edited March 18, 2018 by smokey7722

[Luke 42077]

What do you mean doesn't show up at all?

[smokey7722 15]

Exactly that, I get no entry in the logs.  That Samsung TV has been trying for a few hours and theres no reference to it in either of the logs I sent you.  I can get over there tomorrow to look at the TV and recreate the connection if needed and physically get in front of it.  Thats very similar behavior though to the Android phone I tested with - I got no activity on the server until I created a new entry on the phones app to connect to the server - it couldn't use the old entry.

[Luke 42077]

Do you know what happens in the UI on the tv?

[smokey7722 15]

They see the Select Server view with my server as the option.  When they select it I am being told they get an error saying Connection Failure though I don't see an entry in the logs for any connection attempt

[Luke 42077]

does samsung or tizen appear at all in the log? it would suggest that the app is trying to use the wrong url.

[smokey7722 15]

does samsung or tizen appear at all in the log? it would suggest that the app is trying to use the wrong url.

That's the thing, no urls have changed or any configuration. The tv worked yesterday and this morning then stopped. My phone did the same until i deleted and recreated the connection. Very odd issue

 

Sent from my Pixel XL using Tapatalk

[smokey7722 15]

FYI this morning I got over there and had to do exactly what I did yesterday on my Android by re-entering the server details and it connected right away.  Another family member who is more technically capable did it at their house as well.  So I had three external clients so far that somehow needed to be re-entered.  No idea why but it worked.

[Luke 42077]

The only thing i can think of that would cause this is if your unique server id changed somehow. For example, let's say your apps have saved login info. If you were to uninstall and then reinstall your server, that's a brand new server installation. When you go back to one of the apps, if you click on the old server instance it will say failed to connect. Even though the connection actually succeeded, it's not the same server id that it was expecting so we treat it like a failure.

 

The server never clears this value on it's own but perhaps somehow it happened in your case due to a file getting deleted or corrupted. It's possible.

[smokey7722 15]

The only thing i can think of that would cause this is if your unique server id changed somehow. For example, let's say your apps have saved login info. If you were to uninstall and then reinstall your server, that's a brand new server installation. When you go back to one of the apps, if you click on the old server instance it will say failed to connect. Even though the connection actually succeeded, it's not the same server id that it was expecting so we treat it like a failure.

 

The server never clears this value on it's own but perhaps somehow it happened in your case due to a file getting deleted or corrupted. It's possible.

 

Certainly makes sense.  I know I was NOT able to initiate the upgrade via the Emby servers web interface - I had to download the installer manually and run it for some reason.  When I clicked the option to upgrade via the web interface and then restart, it just came back up as the old version.  Thats when I manually downloaded the installer and upgraded that way.  Not sure if that could have done it.