Production notes.
Notes from running AI and messaging systems in production: what broke, the numbers, and what I changed.
5 min read AI in production 5 min read Reliability 7 min read Scaling
Pin your model IDs: when a “latest” alias tripled our price
A vendor moved a model alias and every AI feature changed model and price with no deploy. What we pin now, and why failover skips vendors without keys.
No lost turns: an AI reply pipeline that can’t silently drop a customer
Queues lose tasks and deploys kill workers. How an owed-reply marker, a 60-second sweeper and a sent marker make sure every customer gets an answer.
Congestive collapse: why our API got slower the harder we pushed it
Throughput fell from 157.6 to 55 req/s as load rose, with zero errors. How a latency-derived concurrency limit fixed it, and where it stops helping.