Webhooks & Events
A webhook is an HTTP callback: your system sends an event to a URL owned by a consumer. It replaces polling and couples you to consumer availability, which is why delivery needs careful design.
Delivery semantics
Assume at-least-once:
- you retry failures,
- consumers may see duplicates,
- ordering is not guaranteed unless you design for it.
Do not promise exactly-once. Promise effectively-once processing via event ids and idempotent consumers.
The request
POST /hooks/orders HTTP/1.1
Host: consumer.example.com
Content-Type: application/json
X-Event-Id: evt_01H...
X-Event-Type: order.shipped
X-Event-Timestamp: 1735689600
X-Signature: t=1735689600,v1=5f3a...
User-Agent: CodeCript-Webhooks/1.0
{"id":"evt_01H...","type":"order.shipped","createdAt":"2025-01-01T00:00:00Z","data":{"orderId":"ord_1"}}
Include:
- a unique
event id(idempotency), - a
typeand a schemaversion, - a timestamp,
- the signature headers,
- a minimal payload (ids + what changed), not the whole database row.
Signing
Consumers must verify the request came from you. HMAC over a canonical string:
const timestamp = Math.floor(Date.now() / 1000);
const body = JSON.stringify(event);
const signature = createHmac("sha256", secret)
.update(`${timestamp}.${body}`)
.digest("hex");
headers["X-Event-Timestamp"] = String(timestamp);
headers["X-Signature"] = `t=${timestamp},v1=${signature}`;
Verification:
function verify(rawBody: string, header: string, secret: string): boolean {
const parts = Object.fromEntries(header.split(",").map((p) => p.split("=")));
const timestamp = Number(parts.t);
if (Math.abs(Date.now() / 1000 - timestamp) > 300) return false; // replay window
const expected = createHmac("sha256", secret)
.update(`${timestamp}.${rawBody}`)
.digest("hex");
return timingSafeEqual(Buffer.from(parts.v1), Buffer.from(expected));
}
Rules:
- sign the raw body, not a re-serialised object;
- include a timestamp to prevent replay;
- use constant-time comparison;
- support secret rotation (multiple active signatures);
- never put the secret in the payload or URL.
Retries and delivery
| Policy | Recommendation |
|---|---|
| Timeout | 5–10 s per attempt |
| Retries | 5–10 attempts with exponential backoff + jitter |
| Retry on | timeouts, connection errors, 5xx, 429 |
| Do not retry | 4xx (except 408/429) |
| Success | any 2xx |
| Dead letter | after max attempts; expose in dashboard + replay |
Retries must be scheduled durably (queue/outbox), not in-memory. A deploy should not lose pending deliveries.
await deliveryQueue.add(
"deliver",
{ eventId, url, payload },
{ attempts: 8, backoff: { type: "exponential", delay: 1_000 } },
);
Ordering
HTTP delivery plus retries means events can arrive out of order:
- include a monotonically increasing
sequenceper subscription/stream; - consumers buffer or reject stale sequences;
- or make consumers state-based (
order.shippedsets status, not append).
Document that ordering is best-effort unless you provide a stream id + sequence and consumers enforce it.
Replay
Consumers will have outages. Provide:
- a replay endpoint/UI by event id or time range,
- an API to fetch missed events (
GET /events?since=...) for reconciliation, - retention long enough for the worst realistic outage.
Replay must be idempotent on the consumer side; the same event id is sent again.
Subscription management
- One or more endpoints per account/tenant.
- Verify endpoint ownership (challenge token or signed request).
- Store per-subscription secrets, event filters, and status.
- Auto-disable after sustained failure and notify the owner.
- Allow pause/resume and test delivery from the dashboard.
Security
- SSRF: never fetch arbitrary URLs without validation; block private IP ranges and metadata endpoints, disallow redirects to internal hosts.
- DNS rebinding: resolve and pin, or use egress controls.
- Redirects: do not follow them (or re-validate the target).
- TLS: require HTTPS; validate certificates.
- Payload secrecy: assume the consumer endpoint may leak; send minimal data.
- Rate limiting: cap deliveries per consumer to avoid weaponising your retries.
[!WARNING] Webhooks are outbound requests to untrusted URLs. Without SSRF protection, an attacker can make your server call internal services or cloud metadata endpoints.
Versioning
- Version the event schema (
v: 1), not the URL. - Add fields; never change types/meaning in place.
- Keep old versions for the deprecation window.
- Publish a changelog and a test suite consumers can run.
Testing
- Contract fixtures: sample payloads per event type/version.
- Signature tests: valid, expired, tampered, rotated secret.
- Retry tests: 500/timeout/429 then success.
- Replay tests: same event id delivered twice, consumer dedupes.
- Security tests: SSRF-blocked URL, redirect handling.
- Consumer simulator in the dashboard.
Failure modes
| Failure | Impact | Fix |
|---|---|---|
| No signature | forged events accepted | HMAC + timestamp |
| Retry storm | consumer DDoS after outage | backoff + jitter + caps |
| In-memory retries | lost on deploy | durable queue |
| No idempotency | duplicate side effects | event id dedupe |
| Unverified URL | SSRF/internal access | allowlist/egress controls |
| Silent failures | events lost | DLQ + alerting + auto-disable |
| Schema change | consumer crashes | versioned additive evolution |
Checklist
- Events carry a unique id, type, version, and timestamp.
- Requests are signed over the raw body with a replay window.
- Consumers can rotate secrets without downtime.
- Retries are durable, bounded, and jittered.
- A DLQ, alerting, and replay path exist.
- Endpoint ownership is verified; SSRF protections block private ranges.
- Ordering semantics are documented (or consumers are state-based).
- Event schemas evolve additively with a deprecation policy.
Next: API Security & OAuth 2.0 — identity, scopes, tokens, and the mistakes that become breaches.