Problem getting lets encrypt ssl cert to work on embyserver dotnetcore

[Luke 37125]

We are still looking into it. It's definitely not affecting all systems. I wonder if the culprit is that we're using a newer .net core runtime version.

[Gerrit507 24]

Sadly I have issues with SSL too. With all versions after 2.2.70.0 the access over https just randomly gets unavailable and I get a time out. I use the same method to generate my certificate. Sometimes I even need to remove the cert and add it again in emby to get it working again...

[Luke 37125]

Sadly I have issues with SSL too. With all versions after 2.2.70.0 the access over https just randomly gets unavailable and I get a time out. I use the same method to generate my certificate. Sometimes I even need to remove the cert and add it again in emby to get it working again...

 

Please see how to report a problem. Thanks.

[night199uk 0]

Was there any progress on this one? I'm hitting the same issue with Emby 3.4.1.0 on Ubuntu from the official package. I spent some time debugging, and it looks to me as though Emby is not sending the full certificate chain or intermediate certificates. This is okay for macOS browsers & Safari on iOS, as you can include trust on the intermediate certificate, but it breaks the official iOS client which seems to want to verify all the way to the root.

 

Same symptoms as above; despite including the full chain in the .pfx, only the first certificate in the chain is sent to the client.

[Luke 37125]

Please try the new 3.5 release. Thanks.

[icsy7867 8]

I had some issues with emby not providing the full chain on 3.4 versions.  I switched to the beta and it was cleared up.  Now I am on 3.5 with no issues.

 

However, most web browsers could still handle the certificate fine.  If you open up emby on a computer browser instead of a phone does it open up normally?

Also have you tried:

openssl pkcs12 -inkey privkey.pem -in chain.pem -export -out emby.pfx

instead of with the fullchain?

[Luke 37125]

 

 

Now I am on 3.5 with no issues.

 

Thank you for reporting back on this.

[night199uk 0]

So I upgraded to 3.5.0.0, but still no dice. The server is only presenting the server certificate and not the intermediates or root:

 

openssl s_client -showcerts -connect emby.domain.com:8920

CONNECTED(00000005)

depth=0 C = GB, ST = Flux, L = Local, O = "SomeDomain, Inc.", CN = emby.domain.com

verify error:num=20:unable to get local issuer certificate

verify return:1

depth=0 C = GB, ST = Flux, L = Local, O = "SomeDomain, Inc.", CN = emby.domain.com

verify error:num=27:certificate not trusted

verify return:1

depth=0 C = GB, ST = Flux, L = Local, O = "SomeDomain, Inc.", CN = emby.domain.com

verify error:num=21:unable to verify the first certificate

verify return:1

---

Certificate chain

 0 s:/C=GB/ST=Flux/L=Local/O=SomeDomain, Inc./CN=emby.domain.com

   i:/C=GB/ST=Flux/O=SomeDomain, Inc./CN=SomeDomain, Inc. Intermediate Certification Authority/emailAddress=root@domain.com

-----BEGIN CERTIFICATE-----

…

-----END CERTIFICATE-----

---

Server certificate

subject=/C=GB/ST=Flux/L=Local/O=SomeDomain, Inc./CN=emby.domain.com

issuer=/C=GB/ST=Flux/O=SomeDomain, Inc./CN=SomeDomain, Inc. Intermediate Certification Authority/emailAddress=root@domain.com

---

No client certificate CA names sent

---

SSL handshake has read 2228 bytes and written 524 bytes

---

New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384

Server public key is 2048 bit

Secure Renegotiation IS supported

Compression: NONE

Expansion: NONE

No ALPN negotiated

SSL-Session:

    Protocol  : TLSv1.2

    Cipher    : ECDHE-RSA-AES256-GCM-SHA384

    Session-ID: D422040318DF37439000099BF2F8C474AE5478FEABD2DF516675E98E9CE3E48F

    Session-ID-ctx: 

    Master-Key: D6A4CDB39EA01A15EBFF0EBCFDC4D409E4EEF791AAB716D957C2F2E5D16CA307BC8EF2B369A4197B9D33528A00DB262F

    TLS session ticket lifetime hint: 300 (seconds)

    TLS session ticket:

    0000 - 64 67 6e c8 66 ee 05 e1-52 41 fa f2 3b 2f 62 c7   dgn.f...RA..;/b.

    0010 - 35 26 08 ec 4a f1 e5 ef-48 ac 81 88 6e 8c eb ae   5&..J...H...n...

    0020 - ba 2e 0d f5 8b 43 bf 16-10 f7 24 d6 0f 9a 4f 1b   .....C....$...O.

    0030 - 23 b3 84 99 eb 24 5a 76-90 ca bb 41 82 8e 66 cb   #....$Zv...A..f.

    0040 - 55 2d c2 ed 8e 4c 4b 72-e1 0a bd 6c 9c b4 81 8b   U-...LKr...l....

    0050 - 19 e2 a8 c0 69 8d 2b 66-8b 58 b6 64 e0 cf 42 a5   ....i.+f.X.d..B.

    0060 - 23 1f b5 3c 0e e7 79 7c-1f be f5 41 0b 12 39 57   #..<..y|...A..9W

    0070 - 6b 62 bd 58 61 b9 ed af-e5 ab c3 88 3a 19 ab 3a   kb.Xa.......:..:

    0080 - 4c 75 c9 0d 57 b5 89 e4-09 1e 71 71 7c d7 08 1d   Lu..W.....qq|...

    0090 - eb 7a 77 95 42 25 dd 79-c8 0b 16 c7 e5 6e 78 14   .zw.B%.y.....nx.

 

    Start Time: 1531919181

    Timeout   : 300 (sec)

    Verify return code: 21 (unable to verify the first certificate)

---

 

I tried two mechanisms to create the emby.pfx:

openssl pkcs12 -export -out emby.pfx -inkey key.pem -in cert.pem -password pass:

openssl pkcs12 -export -out emby.pfx -inkey key.pem -in emby.pem -certfile ca-chain.pem -password pass:

 

Where cert.pem contains the server cert, intermediate cert and root cert; key.pem contains the private key; emby.pem contains just the server cert and ca-chain.pem contains the intermediate and root certificates.

Both commands actually create the same PKCS#12 file in the end.

 

Whatever happens, I cannot get Emby to present the intermediate cert or full chain.

 

Can you show some output of the same openssl s_client command show the full chain and how you're building your PFX? Happy to believe this is something I'm doing wrong; but I can't spot it right now.

[icsy7867 8]

Here is mine:

(Also, I use a script I made to automate my PFX deployment

https://emby.media/community/index.php?/topic/59333-automate-pfx-certificate-deployment-from-letsencrypt/)

 

Make sure you are restarting emby after applying a new certificate.

[user@[member="Server"] ~]# openssl s_client -showcerts -connect domain.com:8920
CONNECTED(00000003)
depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
verify return:1
depth=0 CN = domain.com
verify return:1
---
Certificate chain
 0 s:/CN=domain.com
   i:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
-----BEGIN CERTIFICATE-----
-
-
-
-
-----END CERTIFICATE-----
 1 s:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
   i:/O=Digital Signature Trust Co./CN=DST Root CA X3
-----BEGIN CERTIFICATE-----
-
-
-
-
-----END CERTIFICATE-----
---
Server certificate
subject=/CN=domain.com
issuer=/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
---
No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 3380 bytes and written 415 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: 8F8F26EF5B5B395051F3EA538298E39F36CF82355030B2EB8F0E8220C6D533A5
    Session-ID-ctx:
    Master-Key: 5A859BCE952E2E24AA4B9DA98183C7F6D6039849EDD51EE0908D7AA55FA3FE4BAC7565634078A836D81F3A63E4DE1F4A
    Key-Arg   : None
    Krb5 Principal: None
    PSK identity: None
    PSK identity hint: None
    TLS session ticket lifetime hint: 300 (seconds)
    TLS session ticket:
    0000 - e8 5e 6e 0b 4a f6 0e 05-1f 37 f1 69 55 34 80 c9   .^n.J....7.iU4..
    0010 - 84 4a 2f 0a f9 c2 f9 7b-85 e9 f9 a7 e3 57 dc f0   .J/....{.....W..
    0020 - f5 7e 81 e5 57 96 35 32-7e 3c cb a3 2b 0c be e5   .~..W.52.?..+...
    0030 - 91 f1 3c be 64 29 0d d0-e3 e1 d0 31 54 ce 62 b2   ..<......1T.b.
    0040 - 2d 05 b7 a5 b9 fa d3 c2-c1 fa 2c 30 1b 5d 70 82   -....G....,0.]p.
    0050 - 65 c7 ef 96 eb 6b 03 bf-e4 bf 2d 41 31 34 4d 08   e....k....-A14M.
    0060 - 4c a0 1d 19 42 ab fe d8-68 21 0a be e8 8e 3a 14   L...X...h!....:.
    0070 - f5 4e 14 24 97 10 c7 41-f5 fb 8a 1b fd 9b c6 7a   ...$...A.......z
    0080 - 5a bc 23 aa e9 c5 a0 50-fe 84 01 35 e9 df a5 a1   Z.#....P...5....
    0090 - 99 39 7b dd cb 96 f6 75-7a 95 f6 06 5e 14 35 92   .9{....uz...^.5.

    Start Time: 1531923056
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)

[Handl3vogn 6]

Thank you @@Luke my SSL problems is now fixed with the new 3.5 great work  

[Luke 37125]

Thanks for the feedback.

[night199uk 0]

Thanks @@icsy7867. Real odd; you definitely see emby giving your cert & intermediate from the openssl s_client output, but whatever I do I cannot get emby to expose the intermediate cert.

 

I'm using exactly the same commands as you to build the pfx (see my post above); I'm restarting the server between tries. Mine is also scripted, as this server is part of a build of around 10-20 other servers, web servers etc with various certificates, all of which work (and send the intermediate certs) besides Emby - hence I'm *fairly* confident (as confident as you can ever be) that the basic CA setup is correct.

 

I wonder how else I can debug this? Turning on debug logging in Emby doesn't add any value.

[Luke 37125]

Are you both on Ubuntu?

[night199uk 0]

I'm on Ubuntu 18.04 here:

Linux emby 4.15.0-24-generic #26-Ubuntu SMP Wed Jun 13 08:44:47 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

 

AIUI Emby 3.4.x + is built with .Net Core so there's no Mono dependencies for me to check, right?

[Gerrit507 24]

 

I'm on Ubuntu 18.04 here:

Linux emby 4.15.0-24-generic #26-Ubuntu SMP Wed Jun 13 08:44:47 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

 

AIUI Emby 3.4.x + is built with .Net Core so there's no Mono dependencies for me to check, right?

 

The latest version is still available with mono too. If you have installed the .deb file it's .net core.

[Luke 37125]

Correct, and 3.5 is built on the newly released 2.12 netcore runtime.

[night199uk 0]

Can someone that has this working do this:

 

openssl pkcs12 -info -in emby.pfx -nodes

I get this:

MAC:sha1 Iteration 2048

PKCS7 Encrypted data: pbeWithSHA1And40BitRC2-CBC, Iteration 2048

Certificate bag

Bag Attributes

    localKeyID: 1D F7 0E 10 CD A2 FA BA 49 4C 2D A7 B8 9B 84 CE 99 86 6E E3 

subject=/C=GB/ST=Flux/L=Local/O=SomeDomain, Inc./CN=emby.domain.com

issuer=/C=GB/ST=Flux/O=SomeDomain, Inc./CN=SomeDomain, Inc. Intermediate Certification Authority/emailAddress=root@domain.com

-----BEGIN CERTIFICATE-----

…

-----END CERTIFICATE-----

Certificate bag

Bag Attributes: <No Attributes>

subject=/C=GB/ST=Flux/L=Local/O=SomeDomain, Inc./CN=SomeDomain, Inc. Root Certificate Authority/emailAddress=root@domain.com

issuer=/C=GB/ST=Flux/L=Local/O=SomeDomain, Inc./CN=SomeDomain, Inc. Root Certificate Authority/emailAddress=root@domain.com

-----BEGIN CERTIFICATE-----

…

-----END CERTIFICATE-----

Certificate bag

Bag Attributes: <No Attributes>

subject=/C=GB/ST=Flux/O=SomeDomain, Inc./CN=SomeDomain, Inc. Intermediate Certification Authority/emailAddress=root@domain.com

issuer=/C=GB/ST=Flux/L=Local/O=SomeDomain, Inc./CN=SomeDomain, Inc. Root Certificate Authority/emailAddress=root@domain.com

-----BEGIN CERTIFICATE-----

…

-----END CERTIFICATE-----

PKCS7 Data

Shrouded Keybag: pbeWithSHA1And3-KeyTripleDES-CBC, Iteration 2048

Bag Attributes

    localKeyID: 1D F7 0E 10 CD A2 FA BA 49 4C 2D A7 B8 9B 84 CE 99 86 6E E3 

Key Attributes: <No Attributes>

-----BEGIN PRIVATE KEY-----

…

 

-----END PRIVATE KEY-----

Edited July 18, 2018 by night199uk

[night199uk 0]

So I started looking at the code behind this, and what's odd is that Emby seems to use X509Certificate2. According to this page and the spec,

X509Certificate2 should only ever import the leaf certificate/key from a store.

https://stackoverflow.com/questions/9141198/how-to-programmatically-import-a-pfx-with-a-chain-of-certificates-into-the-certi

 

To import the full chain, it *seems* you'd need to use X509Certificate2Collection, or post process the X509Certificate2 with something like:

X509Chain chain = new X509Chain();

chain.Build(cert);

 

What I do not understand however is that if it was a code issue, why is it working for some users and not for others? Sadly, C# is about the only language I don't do anything in and I don't have time to investigate myself.

[Handl3vogn 6]

@@night199uk

 

I get this

openssl pkcs12 -info -in emby.pfx -nodes

Enter Import Password:

MAC Iteration 2048

MAC verified OK

PKCS7 Encrypted data: pbeWithSHA1And40BitRC2-CBC, Iteration 2048

Certificate bag

Bag Attributes

    localKeyID: 8A 1C B8 5F 85 B8 3E D4 24 C2 D4 AE 55 FB D7 ED 9C 4E 80 FD

subject=/CN=mydomain.org

issuer=/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3

-----BEGIN CERTIFICATE-----

-

-

-

-

-----END CERTIFICATE-----

Certificate bag

Bag Attributes: <No Attributes>

subject=/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3

issuer=/O=Digital Signature Trust Co./CN=DST Root CA X3

-----BEGIN CERTIFICATE-----

-

-

-

-

-----END CERTIFICATE-----

PKCS7 Data

Shrouded Keybag: pbeWithSHA1And3-KeyTripleDES-CBC, Iteration 2048

Bag Attributes

    localKeyID: 8A 1C B8 5F 85 B8 3E D4 24 C2 D4 AE 55 FB D7 ED 9C 4E 80 FD

Key Attributes: <No Attributes>

-----BEGIN PRIVATE KEY-----

-

-

-

-

-----END PRIVATE KEY-----

Edited July 18, 2018 by Handl3vogn

[night199uk 0]

Thanks, so the same as me. I've tried both including the root or not including the root in the .pfx with no success. In fact, I've tried pretty much every combination now.

 

Other things I've tried:

1) Using .p12 instead of .pfx extension (yeah, just in case)

2) Pretty much all combinations of openssl pkcs12 -export with key, cert, intermediate cert, root cert using CAfile and certfile, -chain or not.

3) Including the Intermediate & Root certificates in /usr/local/share/ca-certificates (with update-ca-certificates and server restart).

4) Using a password or not using a password on the .pfx.

 

Obviously restarting emby between each test. Whatever happens, Emby will only send the server cert from the .pfx. Reading back up the thread, it seems like this is localised to Ubuntu with dotnetcore; OP was able to get this working by switching to ArchLinux from the looks? I'm at a loss for figuring this out now.

Edited July 18, 2018 by night199uk

[Luke 37125]

It's strange because I think between the two we're even embedding the same openssl version.

[night199uk 0]

Thanks. I was looking at this as well:

https://bugzilla.xamarin.com/show_bug.cgi?id=46398

 

And some other links.

 

My suspicion right now is that X509Certificate2 does not actually import intermediates; perhaps only using Mono? I believe it might need to change to an X509Certificate2Collection. Is there any way I could get a test build using X509Certificate2Collection to see? I *think* from my rudimentary understanding of C# it looks like HttpListener would take an X509Certificate2Collection.

[Handl3vogn 6]

Strange that you cannot get it to work. The new 3.5 versjon fixed my problem and I now get the complete cert from my emby server.

Only tested on official docker versjon.

 

Here is my results getting cert from server

 

openssl s_client -showcerts -connect domain.org:8920

CONNECTED(00000003)

depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3

verify return:1

depth=1 C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3

verify return:1

depth=0 CN = domain.org

verify return:1

---

Certificate chain

 0 s:/CN=domain.org

   i:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3

-----BEGIN CERTIFICATE-----

-

-

-

-

-----END CERTIFICATE-----

 1 s:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3

   i:/O=Digital Signature Trust Co./CN=DST Root CA X3

-----BEGIN CERTIFICATE-----

-

-

-

-

-----END CERTIFICATE-----

---

Server certificate

subject=/CN=domain.org

issuer=/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3

---

No client certificate CA names sent

Peer signing digest: SHA512

Server Temp Key: ECDH, P-256, 256 bits

---

SSL handshake has read 3963 bytes and written 302 bytes

Verification: OK

---

New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384

Server public key is 4096 bit

Secure Renegotiation IS supported

Compression: NONE

Expansion: NONE

No ALPN negotiated

SSL-Session:

    Protocol  : TLSv1.2

    Cipher    : ECDHE-RSA-AES256-GCM-SHA384

    Session-ID: 9252C06E565CF0C909F5F60C50B67E9041D135175F63406B7200989E329C2F9B

    Session-ID-ctx: 

    Master-Key: 40D3BA14704B1F4C1D6B242AE207C8F744F9DF766315F31448DEFCF44A331E7BD92460E26899F3E82F5D0C16EB6E925D

    PSK identity: None

    PSK identity hint: None

    SRP username: None

    TLS session ticket lifetime hint: 300 (seconds)

    TLS session ticket:

    0000 - 73 49 da d2 14 6a 9e b9-f5 23 88 d7 40 23 a5 70   sI...j...#..@#.p

    0010 - e1 07 bc 76 45 5a 58 10-09 d4 e9 70 16 6f 7e aa   ...vEZX....p.o~.

    0020 - d3 0a 56 a3 d4 f5 02 57-77 29 44 d2 0f c2 9e 4c   ..V....Ww)D....L

    0030 - 5c db dd f0 2e 5a 00 17-52 b0 aa 41 57 46 91 ba   \....Z..R..AWF..

    0040 - c5 95 bc a3 91 61 4f fa-2e e1 1a b7 8f 8b 1e 38   .....aO........8

    0050 - c4 1d 40 7d 7c c3 9d c3-96 f6 b6 67 3b 7b b2 fe   ..@}|......g;{..

    0060 - bc b5 74 21 c8 65 80 48-fc 41 4e cc 90 58 3b 79   ..t!.e.H.AN..X;y

    0070 - 35 a9 55 f0 63 b3 ec bd-62 42 0f 03 f7 f1 be c1   5.U.c...bB......

    0080 - f6 23 12 32 f4 c3 0c e5-68 18 83 0b 7c 6b d4 a4   .#.2....h...|k..

    0090 - 80 33 d7 cd a1 70 bf 0d-b4 9f 28 34 a8 26 6f 27   .3...p....(4.&o'

 

    Start Time: 1531954320

    Timeout   : 7200 (sec)

    Verify return code: 0 (ok)

    Extended master secret: no

---

[Q-Droid 657]

Disregard...

Edited July 22, 2018 by Q-Droid