Thursday 20 August 2009

GMail's SSL Certificate Updated: Fixing STUNNEL

Today it appeared that Google Mail had updated the SSL certificate they use to secure access to the SMTPS service on smtp.gmail.com:587.   The change stopped me from sending personal email through a corporate HTTP proxy today, and I couldn't immediately remember how to load fresh certificates in my Stunnel server again...

Corporate proxies often block direct access to Gmail on port 587, but allow connections to arbitrary SSL services elsewhere. So I run two copies of the STUNNEL SSL proxy back-to-back on my home server. First, a STUNNEL SSL listener accepts incoming SSL client connections. After authenticating the client by checking for a valid client-side SSL certificate, the SSL listener decrypts my traffic and connects onward to a STUNNEL SSL client listening on the loopback address, 127.0.0.1. The STUNNEL SSL client then connects out to smtp.gmail.com:587, verifies GMail's SSL certificate, and encrypts my outbound mail before sending it on its way.
The outbound STUNNEL client was failing to connect to GMail, because their SSL certificate could not be verified. The STUNNEL syslog entries confirmed this (and restarting it didn't help). Things had probably been like that for a while, because it's only one of my work laptops that uses the STUNNEL proxies to send and receive mail (SMTPS and IMAPS): my other machines connect directly to GMail.

To try to see why it was failing, I checked the STUNNEL config file that controls my SMTP client connections to Google. The config file started with:
CAfile = Equifax_Secure_Certificate_Authority.pem
verify = 2
So I thought if I retrieved the current SSL certificate from smtp.gmail.com:587, I could just check the CA they'd used, then pull in the .PEM file for that CA.

I'm very rusty on all this. A handy reference is: http://www.madboa.com/geek/openssl/ .

OK, let's use the openSSL utility to connect to the Gmail SMTP service and see what's going on...
# openssl s_client -connect smtp.gmail.com:587 -starttls smtp
CONNECTED(00000003)
depth=1 /C=US/O=Google Inc/CN=Google Internet Authority
verify error:num=20:unable to get local issuer certificate
verify return:0
---
Certificate chain
0 s:/C=US/ST=California/L=Mountain View/O=Google Inc/CN=smtp.gmail.com
i:/C=US/O=Google Inc/CN=Google Internet Authority
1 s:/C=US/O=Google Inc/CN=Google Internet Authority
i:/C=US/O=Equifax/OU=Equifax Secure Certificate Authority
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIDWzCCAsSgAwIBAgIKYg1RaQADAAAJ6TANBgkqhkiG9w0BAQUFADBGMQswCQYD
VQQGEwJVUzETMBEGA1UEChMKR29vZ2xlIEluYzEiMCAGA1UEAxMZR29vZ2xlIElu
dGVybmV0IEF1dGhvcml0eTAeFw0wOTA3MTcxNzE3MzVaFw0xMDA3MTcxNzI3MzVa
MGgxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlhMRYwFAYDVQQHEw1N
b3VudGFpbiBWaWV3MRMwEQYDVQQKEwpHb29nbGUgSW5jMRcwFQYDVQQDEw5zbXRw
LmdtYWlsLmNvbTCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAyfUHCINEHBj0
nyzNX/elLmKptij/pO5jri0vjGFKdQ/iMmcaxVNMoHcXYHKAwXZ91FN7MSu5iqOc
moDnQVc2iHRQRdVNK4bT0Rtfp4CvaSQPtdZE6ECpc369d2K4jlblcMW84akKUjtP
A+Nlt5BEasFv44+IlMsWa7dKOvFSZN0CAwEAAaOCASwwggEoMB0GA1UdDgQWBBTw
isHdXiADnrSmIxV8tmr7cj5wLzAfBgNVHSMEGDAWgBS/wDDr9UMRPme6npH7/Gra
42sSJDBbBgNVHR8EVDBSMFCgTqBMhkpodHRwOi8vd3d3LmdzdGF0aWMuY29tL0dv
b2dsZUludGVybmV0QXV0aG9yaXR5L0dvb2dsZUludGVybmV0QXV0aG9yaXR5LmNy
bDBmBggrBgEFBQcBAQRaMFgwVgYIKwYBBQUHMAKGSmh0dHA6Ly93d3cuZ3N0YXRp
Yy5jb20vR29vZ2xlSW50ZXJuZXRBdXRob3JpdHkvR29vZ2xlSW50ZXJuZXRBdXRo
b3JpdHkuY3J0MCEGCSsGAQQBgjcUAgQUHhIAVwBlAGIAUwBlAHIAdgBlAHIwDQYJ
KoZIhvcNAQEFBQADgYEACr0GdO989vBLvdZVHEPN6eNMTjTpDifg3m0wJGLH3nWH
jhiN8akr7q6MyzJxU8RBrcsKrqI631zmKd8tp2e8wRdtnqxkG7EJ1UxIcez6ZQx0
5IN48+ygwLaOSll0qK1Z6qThTqwCTgq5LlHyd0kihfos5m537daAOp6o9zGNc9Y=
-----END CERTIFICATE-----
subject=/C=US/ST=California/L=Mountain View/O=Google Inc/CN=smtp.gmail.com
issuer=/C=US/O=Google Inc/CN=Google Internet Authority
---
No client certificate CA names sent
---
SSL handshake has read 1915 bytes and written 353 bytes
---
New, TLSv1/SSLv3, Cipher is RC4-MD5
Server public key is 1024 bit
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1
Cipher : RC4-MD5
Session-ID: 5DEDE00746F38BF291827E3755F2C26022F0F0DDE976BC1F74E4635AC9866E28
Session-ID-ctx:
Master-Key: 9C80B5DEE18ACCE28FEB3197B885B8C8FEC3B577462DFFF31DBD5472F23F16E2F5CD41392DD78851A5D0A3C9031F5B00
Key-Arg : None
Start Time: 1250808647
Timeout : 300 (sec)
Verify return code: 20 (unable to get local issuer certificate)
---
250 PIPELINING
^C

If we save this output to a file called 'stuff' and read that using another openssl command, we can get more information about dates, issuers, etc:-
# openssl s_client -connect smtp.gmail.com:587 -starttls smtp >stuff 2>&1
# openssl x509 -text -in stuff
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
62:0d:51:69:00:03:00:00:09:e9
Signature Algorithm: sha1WithRSAEncryption
Issuer: C=US, O=Google Inc, CN=Google Internet Authority
Validity
Not Before: Jul 17 17:17:35 2009 GMT
Not After : Jul 17 17:27:35 2010 GMT
Subject: C=US, ST=California, L=Mountain View, O=Google Inc, CN=smtp.gmail.com
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
RSA Public Key: (1024 bit)
Modulus (1024 bit):
00:c9:f5:07:08:83:44:1c:18:f4:9f:2c:cd:5f:f7:
a5:2e:62:a9:b6:28:ff:a4:ee:63:ae:2d:2f:8c:61:
4a:75:0f:e2:32:67:1a:c5:53:4c:a0:77:17:60:72:
80:c1:76:7d:d4:53:7b:31:2b:b9:8a:a3:9c:9a:80:
e7:41:57:36:88:74:50:45:d5:4d:2b:86:d3:d1:1b:
5f:a7:80:af:69:24:0f:b5:d6:44:e8:40:a9:73:7e:
bd:77:62:b8:8e:56:e5:70:c5:bc:e1:a9:0a:52:3b:
4f:03:e3:65:b7:90:44:6a:c1:6f:e3:8f:88:94:cb:
16:6b:b7:4a:3a:f1:52:64:dd
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Subject Key Identifier:
F0:8A:C1:DD:5E:20:03:9E:B4:A6:23:15:7C:B6:6A:FB:72:3E:70:2F
X509v3 Authority Key Identifier:
keyid:BF:C0:30:EB:F5:43:11:3E:67:BA:9E:91:FB:FC:6A:DA:E3:6B:12:24

X509v3 CRL Distribution Points:
URI:http://www.gstatic.com/GoogleInternetAuthority/GoogleInternetAuthority.crl

Authority Information Access:
CA Issuers - URI:http://www.gstatic.com/GoogleInternetAuthority/GoogleInternetAuthority.crt

1.3.6.1.4.1.311.20.2:
...W.e.b.S.e.r.v.e.r
Signature Algorithm: sha1WithRSAEncryption
0a:bd:06:74:ef:7c:f6:f0:4b:bd:d6:55:1c:43:cd:e9:e3:4c:
4e:34:e9:0e:27:e0:de:6d:30:24:62:c7:de:75:87:8e:18:8d:
f1:a9:2b:ee:ae:8c:cb:32:71:53:c4:41:ad:cb:0a:ae:a2:3a:
df:5c:e6:29:df:2d:a7:67:bc:c1:17:6d:9e:ac:64:1b:b1:09:
d5:4c:48:71:ec:fa:65:0c:74:e4:83:78:f3:ec:a0:c0:b6:8e:
4a:59:74:a8:ad:59:ea:a4:e1:4e:ac:02:4e:0a:b9:2e:51:f2:
77:49:22:85:fa:2c:e6:6e:77:ed:d6:80:3a:9e:a8:f7:31:8d:
73:d6
-----BEGIN CERTIFICATE-----
MIIDWzCCAsSgAwIBAgIKYg1RaQADAAAJ6TANBgkqhkiG9w0BAQUFADBGMQswCQYD
VQQGEwJVUzETMBEGA1UEChMKR29vZ2xlIEluYzEiMCAGA1UEAxMZR29vZ2xlIElu
dGVybmV0IEF1dGhvcml0eTAeFw0wOTA3MTcxNzE3MzVaFw0xMDA3MTcxNzI3MzVa
MGgxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlhMRYwFAYDVQQHEw1N
b3VudGFpbiBWaWV3MRMwEQYDVQQKEwpHb29nbGUgSW5jMRcwFQYDVQQDEw5zbXRw
LmdtYWlsLmNvbTCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAyfUHCINEHBj0
nyzNX/elLmKptij/pO5jri0vjGFKdQ/iMmcaxVNMoHcXYHKAwXZ91FN7MSu5iqOc
moDnQVc2iHRQRdVNK4bT0Rtfp4CvaSQPtdZE6ECpc369d2K4jlblcMW84akKUjtP
A+Nlt5BEasFv44+IlMsWa7dKOvFSZN0CAwEAAaOCASwwggEoMB0GA1UdDgQWBBTw
isHdXiADnrSmIxV8tmr7cj5wLzAfBgNVHSMEGDAWgBS/wDDr9UMRPme6npH7/Gra
42sSJDBbBgNVHR8EVDBSMFCgTqBMhkpodHRwOi8vd3d3LmdzdGF0aWMuY29tL0dv
b2dsZUludGVybmV0QXV0aG9yaXR5L0dvb2dsZUludGVybmV0QXV0aG9yaXR5LmNy
bDBmBggrBgEFBQcBAQRaMFgwVgYIKwYBBQUHMAKGSmh0dHA6Ly93d3cuZ3N0YXRp
Yy5jb20vR29vZ2xlSW50ZXJuZXRBdXRob3JpdHkvR29vZ2xlSW50ZXJuZXRBdXRo
b3JpdHkuY3J0MCEGCSsGAQQBgjcUAgQUHhIAVwBlAGIAUwBlAHIAdgBlAHIwDQYJ
KoZIhvcNAQEFBQADgYEACr0GdO989vBLvdZVHEPN6eNMTjTpDifg3m0wJGLH3nWH
jhiN8akr7q6MyzJxU8RBrcsKrqI631zmKd8tp2e8wRdtnqxkG7EJ1UxIcez6ZQx0
5IN48+ygwLaOSll0qK1Z6qThTqwCTgq5LlHyd0kihfos5m537daAOp6o9zGNc9Y=
-----END CERTIFICATE-----

OK so the SSL certificate itself is only valid for a year, but if I could persuade Stunnel to trust Equifax and Google, hopefully it should all be OK. Equifax's root cert can be found on the web, and there's a link to Google's root cert in the X509 data shown in the output of the previous command. Putting it all together:-

# wget http://www.gstatic.com/GoogleInternetAuthority/GoogleInternetAuthority.crt
# wget https://www.geotrust.com/resources/root_certificates/certificates/Equifax_Secure_Certificate_Authority.cer
# cat GoogleInternetAuthority.crt Equifax_Secure_Certificate_Authority.cer > Certs.cer
So the file Certs.cer can be referenced from the STUNNEL config file like so:-
CAfile = Certs.cer
verify = 2
This initially appeared to have fixed the problem - but then I realised that the equivalent IMAPS proxy was still working, without a similar change being made. I then became suspicious, and so backed out the change to my SMTPS proxy (restoring CAfile = Equifax_Secure_Certificate_Authority.pem in the Stunnel config file) and the service continued to work.

So I start to think the most likely explanation is that GMail re-generated their certificate while I was dicking around at my end, but I'm not entirely convinced. I was working through midnight, so maybe that had something to do with it. If anyone can shed any more light on this, I'd like to know!

In the longer term, it might be better if FreeBSD just knew about the normal root CA's, but I can't see how to set that up.

3 comments:

  1. You saved the day!!! Thank you :)

    ReplyDelete
  2. plz help me ,i'm recieving a message dat gstatic.com 443 security certificate while accessing google.co.in n the other sites too,it again showing a warning like untrusted connection eventhgh i'm using mozilla firefox browser-6,it is troubling me a lot,plz find any solution to overcome it

    ReplyDelete
  3. Rama, I just wrote a long reply, but the editor lost it!

    Firefox 6 is very old.

    So, how old is your operating system? Have you ever applied Service Packs? Do you apply updates e.g. from Microsoft Windows Update?

    If your operating system is VERY out of date, then it won't have up-to-date CA certificates, so it can't check website SSL certificates against the CA certificates.

    That's just a guess. If you give the exact error message, it may be possible to give a more accurate diagnosis.

    It is always possible that something more sinister is happening, but that's quite rare.

    ReplyDelete

Spammers: please stop wasting my time. All comments are moderated before publication.