Skip to content

Phase B1–B4 — Runtime (shipped)

Builds on Phase B0 foundations.

PhaseWhat shippedComponents
B1202 ACCEPTED + event_id + schema_version: 2; durable audit_events; Redis Streams publish (waspada:events, MAXLEN~)Event bus, audit logger
B2In-process pipeline consumer; Redis Streams consumer when DATABASE_URL set (idempotent via processed_events)Pipeline consumer
B3Encrypted alerts + corroboration writers (per-row DEK + AAD)Alert repository, crypto envelope
B4delivery_outbox enqueue + poller; Redis pub/sub waspada:sse for cross-replica SSEOutbox poller, SSE publisher

Correctness notes

  • Atomic event claim (ADV-001): processed_events uses a lease-aware lifecycle — claim inserts status=in_progress, success flips to committed, caught failures delete the claim, and reclaimStaleEventClaims (each outbox poll tick, EVENT_CLAIM_LEASE_MS default 60s) frees hard-killed mid-pipeline claims so peers / future XAUTOCLAIM can retry. Inline ingest and the Redis Streams consumer share the same claim — only one wins. Corroboration report_count bumps only when a new alerts row is inserted. Outbox idempotency key is fms:${event_id}. alert_id is derived deterministically from event_id so SSE retries stay stable.
  • Outbox lease reaper (ADV-002): in_flight rows older than OUTBOX_IN_FLIGHT_LEASE_MS (default 60s) return to pending. Attempts increment on failure, not on claim.
  • Tenant-scoped SSE (ADV-004): ANALYST_KEYS_JSON (or legacy ANALYST_API_KEY + ANALYST_TENANT_IDS) binds analysts to tenants. Explicit ["*"] (or omitted tenantIds) means all tenants; empty tenantIds: [] is deny-all. Stream tickets carry the scope; local listeners and Redis {v:1,tenant_id,payload} envelopes both filter before send. Boot parse-validates ANALYST_KEYS_JSON. Durable disposition returns uniform 404 for unknown and cross-tenant alerts.
  • XACK after durable work when the Streams consumer runs (requires Postgres idempotency).
  • Without DATABASE_URL, ingest still runs the in-process pipeline (SSE + inline FMS) and returns 202.
  • Gateway PEM remains mandatory in production / hardened (ALLOW_EPHEMERAL_GATEWAY_KEYS opt-out only outside prod).

Env

VarRole
REDIS_URLJTI/rate-limit/tickets + Streams + SSE pub/sub
REQUIRE_REDISRefuse memory soft-fail (no JTI split-brain)
DATABASE_URL + ALERT_DATA_KEKDurable alerts / audit / outbox
EVENT_BUS=memoryForce in-process bus (tests)
FMS_WEBHOOK_URLOutbox delivery target
ANALYST_KEYS_JSONPreferred multi-tenant analyst keys: [{keyId, apiKey, tenantIds}]. Use ["*"] for all tenants; omit tenantIds for the same. Empty [] is deny-all (never default-open). Malformed JSON refuses boot.
ANALYST_TENANT_IDSLegacy single-key scope (ANALYST_API_KEY); unset/* = all; empty = deny-all
OUTBOX_POLL_MSOutbox poll interval (default 5000)
OUTBOX_IN_FLIGHT_LEASE_MSReclaim stale in_flight rows (default 60000)
EVENT_CLAIM_LEASE_MSReclaim stale processed_events in_progress claims (default 60000)
OUTBOX_CLAIM_LIMITRows claimed per poll (default 10)
OUTBOX_BACKOFF_BASE_MSRetry backoff base (default 60000; soak may use 2000)

Known gaps (B5+)

GapWhy it mattersPlan
No XAUTOCLAIM sweepIf a gateway replica crashes mid-handler, the message stays in the consumer-group PEL until claimed. Event in_progress / outbox in_flight leases are reaped so a future XAUTOCLAIM retry is not defeated.Periodic XAUTOCLAIM for stuck PEL entries — B5
Inline RDAP on ingest pathVariable latency under the event-claim lease (EVENT_CLAIM_LEASE_MS); move enrichment off the hot pathB2 enrichment worker
Audit writes are 1 INSERT per eventFine for pilot; a 1k ingest burst = 1k round-tripsBatch or tx-buffered audit inserts before harder load-test
VAPT / chaos reportExternal assurance still openVAPT engagement brief + hire; soak harness in load-test docs

Waspada AI: Enterprise infrastructure powering public defense.