How OpenAI Scales PostgreSQL to 800M Users

How OpenAI Scales PostgreSQL to Serve 800 Million ChatGPT Users with a single primary Azure PostgreSQL instance

• ~50 read replicas across multiple regions • Millions of queries per second • p99 latency: low double-digit milliseconds • 99.999% availability

Main points:

Strict Read/Write Separation Reads go to replicas. Only queries that are part of write transactions hit the primary.

PgBouncer for Connection Pooling Connection time dropped from 50ms to 5ms. Statement/transaction pooling mode to reuse connections efficiently.

Cache Locking Mechanism When multiple requests miss the same cache key, only one fetches from PostgreSQL. Others wait for the cache to update. This prevents cascading load spikes.

Workload Isolation Low-priority and high-priority requests route to separate instances. Noisy neighbors can't impact critical traffic.

Offload Write-Heavy Workloads Shardable writes moved to Azure CosmosDB. PostgreSQL handles what it does best: reads.

Aggressive Query Optimization They identified a 12-table join causing SEVs. Complex joins are now broken down and moved to the application layer.

Trade-Offs

• New tables no longer allowed in PostgreSQL, new features default to sharded systems • MVCC creates write amplification under heavy write loads • They're working on cascading replication to scale beyond 50 replicas

PostgreSQL can scale remarkably far for read-heavy workloads. The secret isn't exotic technology, it's disciplined engineering: connection pooling, query optimization, workload isolation, and knowing when to offload writes elsewhere.

Their only SEV-0 incident in 12 months was during the ChatGPT ImageGen launch, when 100M users signed up in a week.