Serverless connection outage due to mismanaged certificates

Incident Report for CockroachDB Cloud

Resolved

A routine process of rotating soon to expire SSL certificates for Cockroach Cloud clusters had an unintended consequence of pushing invalid certificates to Cockroach Cloud Serverless clusters. Because of a bug, the process installed certificates not compatible with the recently-introduced SNI connection string format. This caused applications connecting to these clusters using SNI connection strings to fail to connect.

We have fixed the bug in the certificate update process, and have replaced the non-compatible certificates unintentionally installed by the faulty process with the correct certificates

Posted Jan 21, 2023 - 19:00 UTC