Skip to content

Ops Runbook — Gateway Redis & Memory Stores

Shared gateway state (JTI replay, rate limits, SSE tickets, Streams, SSE pub/sub) must not silently become per-replica memory in multi-instance deploys.

Boot policy (fail-closed)

TopologyRequired envOn Redis connect failure
Production / hardened multi-replicaREDIS_URL set; prefer REQUIRE_REDIS=trueBoot refuses — process exits
Single-replica pilot (memory)ALLOW_MEMORY_STORE=true, REDIS_URL unsetN/A (memory by design)
Local / testunset RedisSoft-falls to memory with stderr warning
REQUIRE_REDIS=trueREDIS_URL requiredBoot refuses; ALLOW_MEMORY_STORE cannot override

Do not set both REDIS_URL and ALLOW_MEMORY_STORE expecting soft failover. If REDIS_URL is set and Redis is down, boot refuses (no silent demotion).

Health checks

bash
curl -sS "$GATEWAY_URL/health"
curl -sS "$GATEWAY_URL/healthz"   # alias

Payload fields:

FieldMeaning
statusok or degraded (postgres down or store down)
postgresup | down | skipped
store_backendredis | memory
storeRedis PINGup/down; memory → skipped

Load balancers should treat status=degraded or HTTP ≠ 200 as unhealthy when Redis is mandatory.

Incident: replay possible across replicas

Symptom: Same JWT/jti accepted on more than one gateway instance; rate limits appear ineffective.

Likely cause: Instances booted with memory stores (ALLOW_MEMORY_STORE or Redis never connected).

Actions:

  1. GET /health — confirm store_backend is redis and store is up on every replica.
  2. Ensure REDIS_URL is identical and reachable from all replicas.
  3. Set REQUIRE_REDIS=true and remove ALLOW_MEMORY_STORE for any multi-replica service.
  4. Rolling restart after Redis is healthy; spent JTIs in memory are not recovered — treat as a short replay window and rotate partner keys if abuse is suspected.

Provisioning reminders

bash
pnpm --filter @waspada/gateway-api provision

See also Phase B1–B4 runtime env table and RMIT mapping ops row.

Edge tools for citizens. Threat feeds for banks.