Fanning Out Webhooks Without Tears

An internal routing service had one job: take an incoming WhatsApp event and forward it to every app that subscribed to it. The naive version — loop over subscribers and POST synchronously — works in a demo and falls over in production.

The slow-consumer problem

If you forward synchronously, the slowest subscriber sets the pace for everyone. One app having a bad day means delayed delivery for all of them, and eventually a backlog that takes the router down too.

Decouple with a queue

Accept the event, persist it, return 200 fast. A separate worker delivers to each subscriber independently, with retries and backoff per target. A failing subscriber only hurts its own queue.

• Acknowledge fast, deliver asynchronously.
• Isolate failure per subscriber — one bad target should not block the rest.
• Make delivery idempotent so retries are safe.