GMail's SSL Certificate Updated: Fixing STUNNEL

Today it appeared that Google Mail had updated the SSL certificate they use to secure access to the SMTPS service on smtp.gmail.com:587.   The change stopped me from sending personal email through a corporate HTTP proxy today, and I couldn't immediately remember how to load fresh certificates in my Stunnel server again...

Corporate proxies often block direct access to Gmail on port 587, but allow connections to arbitrary SSL services elsewhere. So I run two copies of the STUNNEL SSL proxy back-to-back on my home server. First, a STUNNEL SSL listener accepts incoming SSL client connections. After authenticating the client by checking for a valid client-side SSL certificate, the SSL listener decrypts my traffic and connects onward to a STUNNEL SSL client listening on the loopback address, 127.0.0.1. The STUNNEL SSL client then connects out to smtp.gmail.com:587, verifies GMail's SSL certificate, and encrypts my outbound mail before sending it on its way.

The outbound STUNNEL client was failing to connect to GMail, because their SSL certificate could not be verified. The STUNNEL syslog entries confirmed this (and restarting it didn't help). Things had probably been like that for a while, because it's only one of my work laptops that uses the STUNNEL proxies to send and receive mail (SMTPS and IMAPS): my other machines connect directly to GMail.

To try to see why it was failing, I checked the STUNNEL config file that controls my SMTP client connections to Google. The config file started with:

CAfile = Equifax_Secure_Certificate_Authority.pem

verify = 2

So I thought if I retrieved the current SSL certificate from smtp.gmail.com:587, I could just check the CA they'd used, then pull in the .PEM file for that CA.

I'm very rusty on all this. A handy reference is: http://www.madboa.com/geek/openssl/ .

OK, let's use the openSSL utility to connect to the Gmail SMTP service and see what's going on...

# openssl s_client -connect smtp.gmail.com:587 -starttls smtp

CONNECTED(00000003)

depth=1 /C=US/O=Google Inc/CN=Google Internet Authority

verify error:num=20:unable to get local issuer certificate

verify return:0

---

Certificate chain

0 s:/C=US/ST=California/L=Mountain View/O=Google Inc/CN=smtp.gmail.com

i:/C=US/O=Google Inc/CN=Google Internet Authority

1 s:/C=US/O=Google Inc/CN=Google Internet Authority

i:/C=US/O=Equifax/OU=Equifax Secure Certificate Authority

---

Server certificate

-----BEGIN CERTIFICATE-----

MIIDWzCCAsSgAwIBAgIKYg1RaQADAAAJ6TANBgkqhkiG9w0BAQUFADBGMQswCQYD

VQQGEwJVUzETMBEGA1UEChMKR29vZ2xlIEluYzEiMCAGA1UEAxMZR29vZ2xlIElu

dGVybmV0IEF1dGhvcml0eTAeFw0wOTA3MTcxNzE3MzVaFw0xMDA3MTcxNzI3MzVa

MGgxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlhMRYwFAYDVQQHEw1N

b3VudGFpbiBWaWV3MRMwEQYDVQQKEwpHb29nbGUgSW5jMRcwFQYDVQQDEw5zbXRw

LmdtYWlsLmNvbTCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAyfUHCINEHBj0

nyzNX/elLmKptij/pO5jri0vjGFKdQ/iMmcaxVNMoHcXYHKAwXZ91FN7MSu5iqOc

moDnQVc2iHRQRdVNK4bT0Rtfp4CvaSQPtdZE6ECpc369d2K4jlblcMW84akKUjtP

A+Nlt5BEasFv44+IlMsWa7dKOvFSZN0CAwEAAaOCASwwggEoMB0GA1UdDgQWBBTw

isHdXiADnrSmIxV8tmr7cj5wLzAfBgNVHSMEGDAWgBS/wDDr9UMRPme6npH7/Gra

42sSJDBbBgNVHR8EVDBSMFCgTqBMhkpodHRwOi8vd3d3LmdzdGF0aWMuY29tL0dv

b2dsZUludGVybmV0QXV0aG9yaXR5L0dvb2dsZUludGVybmV0QXV0aG9yaXR5LmNy

bDBmBggrBgEFBQcBAQRaMFgwVgYIKwYBBQUHMAKGSmh0dHA6Ly93d3cuZ3N0YXRp

Yy5jb20vR29vZ2xlSW50ZXJuZXRBdXRob3JpdHkvR29vZ2xlSW50ZXJuZXRBdXRo

b3JpdHkuY3J0MCEGCSsGAQQBgjcUAgQUHhIAVwBlAGIAUwBlAHIAdgBlAHIwDQYJ

KoZIhvcNAQEFBQADgYEACr0GdO989vBLvdZVHEPN6eNMTjTpDifg3m0wJGLH3nWH

jhiN8akr7q6MyzJxU8RBrcsKrqI631zmKd8tp2e8wRdtnqxkG7EJ1UxIcez6ZQx0

5IN48+ygwLaOSll0qK1Z6qThTqwCTgq5LlHyd0kihfos5m537daAOp6o9zGNc9Y=

-----END CERTIFICATE-----

subject=/C=US/ST=California/L=Mountain View/O=Google Inc/CN=smtp.gmail.com

issuer=/C=US/O=Google Inc/CN=Google Internet Authority

---

No client certificate CA names sent

---

SSL handshake has read 1915 bytes and written 353 bytes

---

New, TLSv1/SSLv3, Cipher is RC4-MD5

Server public key is 1024 bit

Compression: NONE

Expansion: NONE

SSL-Session:

Protocol : TLSv1

Cipher : RC4-MD5

Session-ID: 5DEDE00746F38BF291827E3755F2C26022F0F0DDE976BC1F74E4635AC9866E28

Session-ID-ctx:

Master-Key: 9C80B5DEE18ACCE28FEB3197B885B8C8FEC3B577462DFFF31DBD5472F23F16E2F5CD41392DD78851A5D0A3C9031F5B00

Key-Arg : None

Start Time: 1250808647

Timeout : 300 (sec)

Verify return code: 20 (unable to get local issuer certificate)

---

250 PIPELINING

^C

If we save this output to a file called 'stuff' and read that using another openssl command, we can get more information about dates, issuers, etc:-

# openssl s_client -connect smtp.gmail.com:587 -starttls smtp >stuff 2>&1

# openssl x509 -text -in stuff

Certificate:

Data:

Version: 3 (0x2)

Serial Number:

62:0d:51:69:00:03:00:00:09:e9

Signature Algorithm: sha1WithRSAEncryption

Issuer: C=US, O=Google Inc, CN=Google Internet Authority

Validity

Not Before: Jul 17 17:17:35 2009 GMT

Not After : Jul 17 17:27:35 2010 GMT

Subject: C=US, ST=California, L=Mountain View, O=Google Inc, CN=smtp.gmail.com

Subject Public Key Info:

Public Key Algorithm: rsaEncryption

RSA Public Key: (1024 bit)

Modulus (1024 bit):

00:c9:f5:07:08:83:44:1c:18:f4:9f:2c:cd:5f:f7:

a5:2e:62:a9:b6:28:ff:a4:ee:63:ae:2d:2f:8c:61:

4a:75:0f:e2:32:67:1a:c5:53:4c:a0:77:17:60:72:

80:c1:76:7d:d4:53:7b:31:2b:b9:8a:a3:9c:9a:80:

e7:41:57:36:88:74:50:45:d5:4d:2b:86:d3:d1:1b:

5f:a7:80:af:69:24:0f:b5:d6:44:e8:40:a9:73:7e:

bd:77:62:b8:8e:56:e5:70:c5:bc:e1:a9:0a:52:3b:

4f:03:e3:65:b7:90:44:6a:c1:6f:e3:8f:88:94:cb:

16:6b:b7:4a:3a:f1:52:64:dd

Exponent: 65537 (0x10001)

X509v3 extensions:

X509v3 Subject Key Identifier:

F0:8A:C1:DD:5E:20:03:9E:B4:A6:23:15:7C:B6:6A:FB:72:3E:70:2F

X509v3 Authority Key Identifier:

keyid:BF:C0:30:EB:F5:43:11:3E:67:BA:9E:91:FB:FC:6A:DA:E3:6B:12:24

X509v3 CRL Distribution Points:

URI:http://www.gstatic.com/GoogleInternetAuthority/GoogleInternetAuthority.crl

Authority Information Access:

CA Issuers - URI:http://www.gstatic.com/GoogleInternetAuthority/GoogleInternetAuthority.crt

1.3.6.1.4.1.311.20.2:

...W.e.b.S.e.r.v.e.r

Signature Algorithm: sha1WithRSAEncryption

0a:bd:06:74:ef:7c:f6:f0:4b:bd:d6:55:1c:43:cd:e9:e3:4c:

4e:34:e9:0e:27:e0:de:6d:30:24:62:c7:de:75:87:8e:18:8d:

f1:a9:2b:ee:ae:8c:cb:32:71:53:c4:41:ad:cb:0a:ae:a2:3a:

df:5c:e6:29:df:2d:a7:67:bc:c1:17:6d:9e:ac:64:1b:b1:09:

d5:4c:48:71:ec:fa:65:0c:74:e4:83:78:f3:ec:a0:c0:b6:8e:

4a:59:74:a8:ad:59:ea:a4:e1:4e:ac:02:4e:0a:b9:2e:51:f2:

77:49:22:85:fa:2c:e6:6e:77:ed:d6:80:3a:9e:a8:f7:31:8d:

73:d6

-----BEGIN CERTIFICATE-----

MIIDWzCCAsSgAwIBAgIKYg1RaQADAAAJ6TANBgkqhkiG9w0BAQUFADBGMQswCQYD

VQQGEwJVUzETMBEGA1UEChMKR29vZ2xlIEluYzEiMCAGA1UEAxMZR29vZ2xlIElu

dGVybmV0IEF1dGhvcml0eTAeFw0wOTA3MTcxNzE3MzVaFw0xMDA3MTcxNzI3MzVa

MGgxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlhMRYwFAYDVQQHEw1N

b3VudGFpbiBWaWV3MRMwEQYDVQQKEwpHb29nbGUgSW5jMRcwFQYDVQQDEw5zbXRw

LmdtYWlsLmNvbTCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAyfUHCINEHBj0

nyzNX/elLmKptij/pO5jri0vjGFKdQ/iMmcaxVNMoHcXYHKAwXZ91FN7MSu5iqOc

moDnQVc2iHRQRdVNK4bT0Rtfp4CvaSQPtdZE6ECpc369d2K4jlblcMW84akKUjtP

A+Nlt5BEasFv44+IlMsWa7dKOvFSZN0CAwEAAaOCASwwggEoMB0GA1UdDgQWBBTw

isHdXiADnrSmIxV8tmr7cj5wLzAfBgNVHSMEGDAWgBS/wDDr9UMRPme6npH7/Gra

42sSJDBbBgNVHR8EVDBSMFCgTqBMhkpodHRwOi8vd3d3LmdzdGF0aWMuY29tL0dv

b2dsZUludGVybmV0QXV0aG9yaXR5L0dvb2dsZUludGVybmV0QXV0aG9yaXR5LmNy

bDBmBggrBgEFBQcBAQRaMFgwVgYIKwYBBQUHMAKGSmh0dHA6Ly93d3cuZ3N0YXRp

Yy5jb20vR29vZ2xlSW50ZXJuZXRBdXRob3JpdHkvR29vZ2xlSW50ZXJuZXRBdXRo

b3JpdHkuY3J0MCEGCSsGAQQBgjcUAgQUHhIAVwBlAGIAUwBlAHIAdgBlAHIwDQYJ

KoZIhvcNAQEFBQADgYEACr0GdO989vBLvdZVHEPN6eNMTjTpDifg3m0wJGLH3nWH

jhiN8akr7q6MyzJxU8RBrcsKrqI631zmKd8tp2e8wRdtnqxkG7EJ1UxIcez6ZQx0

5IN48+ygwLaOSll0qK1Z6qThTqwCTgq5LlHyd0kihfos5m537daAOp6o9zGNc9Y=

-----END CERTIFICATE-----

OK so the SSL certificate itself is only valid for a year, but if I could persuade Stunnel to trust Equifax and Google, hopefully it should all be OK. Equifax's root cert can be found on the web, and there's a link to Google's root cert in the X509 data shown in the output of the previous command. Putting it all together:-

# wget http://www.gstatic.com/GoogleInternetAuthority/GoogleInternetAuthority.crt

# wget https://www.geotrust.com/resources/root_certificates/certificates/Equifax_Secure_Certificate_Authority.cer

# cat GoogleInternetAuthority.crt Equifax_Secure_Certificate_Authority.cer > Certs.cer

So the file Certs.cer can be referenced from the STUNNEL config file like so:-

CAfile = Certs.cer

verify = 2

This initially appeared to have fixed the problem - but then I realised that the equivalent IMAPS proxy was still working, without a similar change being made. I then became suspicious, and so backed out the change to my SMTPS proxy (restoring CAfile = Equifax_Secure_Certificate_Authority.pem in the Stunnel config file) and the service continued to work.

So I start to think the most likely explanation is that GMail re-generated their certificate while I was dicking around at my end, but I'm not entirely convinced. I was working through midnight, so maybe that had something to do with it. If anyone can shed any more light on this, I'd like to know!

In the longer term, it might be better if FreeBSD just knew about the normal root CA's, but I can't see how to set that up.