Loop Canary
Test cases fly the whole loop at dawn; a dead one names the stage.
The job
Tests the process that handles customer problems. Each morning it pushes labelled test cases through every stage, catches silent failures before customers do, counts the real customers affected and publishes how often each stage is working.
The moment
Every morning the app sends labelled test cases through each stage of the process that handles customer problems, like canaries sent into a mine.
At 06:04, test case 7 fails at the stage that sends customer updates. A WhatsApp message template was rejected at 14:00 the day before.
Since then, 212 real citizens have had no updates. The on-call owner gets an alert on their phone.
The Head of CX reads the morning bulletin before the first complaint arrives.
What it does
- Page the stage owner
- Re-run a canary
- Pause a dependent automation
- Close an incident
What you see
Canary board with a dead-canary incident page, monthly uptime report and a morning WhatsApp bulletin
What it moves
The minutes a stage was broken before anyone noticed, the real customers affected each month, and the percentage of time each stage is working.
Built for
- CX leaders & VoC
