Health indicators — classification audit
Every ctx.health.register(...) call in packages/*/src, classified. This is the M70c audit
deliverable (plan §3.9): it stops the “hardcoded up next to live data” pattern recurring by
making every indicator’s status source explicit, and it is enforced by
test/health-indicator-audit.test.ts, which fails if a
ctx.health.register site is added (or moved) without a matching row here.
The CLI schematic at packages/cli/src/schematics/health-indicator.ts mentions
ctx.health.register inside a template string and is not a registration site.
There are 32 registration sites. static-plugin registers the same static-files indicator
from two branches (the no-fs arm and the fs arm), so it contributes two rows; only one is active
at a time.
What each status means
The health plugin projects every indicator to { status, data } (M70c, fixing X3-7) and aggregates:
/ready is 200 only when every indicator is up; /health is 503 only when any is down
(degraded is 200). The gRPC bridge maps the aggregate to grpc.health.v1.Health/Check — since
M70c, degraded maps to NOT_SERVING, agreeing with /ready (X7-8).
| Classification | Meaning |
|---|---|
live-state | The status reflects a live fact about the backend or a live probe: a reachability probe (isHealthy()), a readiness flag that tracks a real connection, or an actual I/O check. The status can change at runtime as the backend changes. |
justified-literal | The status is a fixed literal, but the justification is documented and the literal is the correct signal for that plugin (e.g. a platform binding whose presence is known at registration). |
configuration-literal | The status is a fixed literal (or a lifecycle flag that only ever reports “started / stopped”) while the data payload is live. The status says nothing about the backend’s health. These are the X8-5 class of defect — the indicator’s status is decorative. |
The M70c in-scope packages (messaging, storage, mail, queue, service-discovery) are
live-state: each reports both signals — isReady() (lifecycle) and isHealthy() (reachability) —
and the indicator maps them (see the per-package READMEs for the status tables). M70c’s sixth,
realtime-backplane, is justified-literal since M113b, with auth, feature-flags,
worker-pool and the no-file-system static-files arm: each reports up and carries its condition
in data, because /ready withdraws a replica on anything but up and none of those conditions
stops the replica serving. M90b added cache and database; the audit, notification and
worker-pool rows followed, closing the last three H-70c defects this audit recorded, so no
ctx.health.register site outside the defensible in-process set is configuration-literal any
more.
The register
| # | Indicator | Package | Site | Classification | Status source and note |
|---|---|---|---|---|---|
| 1 | messaging (token) | messaging-plugin | packages/messaging-plugin/src/plugin/messaging-plugin.ts:656 | live-state | M70c: isReady() lifecycle + isHealthy() reachability; down when not ready or unreachable, degraded when ready but unreachable and a reconnect is in flight. M90b: every broker probe is cached 5 s and bounded 2 s; the Service Bus transport probes the namespace (getNamespaceProperties(), 401/403 counts as reachable). M95b: the Service Bus broker records data-plane publish outcomes the window reads first; the RabbitMQ probe opens a throwaway channel; every arm’s probe is bounded at this indicator (a hung broker settles reachable: 'unknown'). M101a: a retained Service Bus network failure answers false at once; the management probe only clears it, in the background, for the next read. |
| 2 | realtime-backplane | realtime-backplane-plugin | packages/realtime-backplane-plugin/src/plugin/realtime-backplane-plugin.ts:157 | justified-literal | M113b: status stays up; transport isHealthy() supplies data.reachable, including false on shared fan-out outages. Local request delivery still works. M119b (V10-20): the reachability check is cached 5 s and bounded 2 s; a check that does not answer reports reachable: 'unknown', and the status stays up. |
| 3 | storage | storage-plugin | packages/storage-plugin/src/plugin/storage-plugin.ts:212 | live-state | M70c: provider isHealthy() probe (HEAD/list) + lifecycle; down when unreachable. |
| 4 | mail | mail-plugin | packages/mail-plugin/src/plugin/mail-plugin.ts:165 | live-state | M70c: provider isHealthy() (SMTP verify?, SES isHealthy?) + lifecycle. H-70c-4: read through MailService.isHealthy() — now public on IMailer — so the indicator and any holder of the capability answer through one implementation. M119b (V10-49): registered outside readiness by default — an unreachable backend’s down stays on /health (503, alerting) while /ready ignores it; HealthPlugin({ readiness: { include: ['mail'] } }) restores the previous behaviour. |
| 5 | queue (token) | queue-plugin | packages/queue-plugin/src/plugin/queue-plugin.ts:350 | live-state | M70c: adapter isHealthy() (PING / connection-fault / GetQueueAttributes) + lifecycle; data.reachable distinguishes up/down/unknown. M70k (X8-4): data.queues adds per-name ready/processing/dead depths where the adapter can count them cheaply, omitted where it cannot. M101a: each Redis command is bounded by commandTimeoutMs, so a paused server fails the probe instead of hanging it. M119b (V10-22): the RabbitMQ probe opens and closes a throwaway channel instead of reading a supervisor flag; for every adapter, a check that does not answer within its 2 s bound reports up with reachable: 'unknown' and no depth counts, while a refused one still reads down. |
| 6 | service-discovery | service-discovery-plugin | packages/service-discovery-plugin/src/plugin/service-discovery-plugin.ts:122 | live-state | M70c: #everResolved + provider probe (Consul /v1/status/leader, Kubernetes limit=1 EndpointSlice LIST); never-resolved-and-unreachable is down (X10-3 fix). M113o: otherwise up, including warm-cache backend outages; data.degraded, reachable and everResolved retain visibility. |
| 7 | grpc | grpc-plugin | packages/grpc-plugin/src/plugin/grpc-plugin.ts:79 | configuration-literal | status: 'up' beside live data.available/serviceCount. The transport’s real health is the Health/Check RPC (mapped from the health plugin’s aggregate), so the plugin’s own indicator only reports that the plugin registered. |
| 8 | scheduler | scheduler-plugin | packages/scheduler-plugin/src/plugin/scheduler-plugin.ts:255 | configuration-literal | createHealthIndicator() reports #connected (lifecycle). M113b: data.lock.reachable reports the lock backend through IDistributedLock.isHealthy?(), cached 5 s and bounded 2 s ('unknown' when the lock cannot probe). Status stays lifecycle by design: a lock outage stops fires, not serving. |
| 9 | audit | audit-plugin | packages/audit-plugin/src/plugin/audit-plugin.ts:158 | live-state | CLOSES H-70c-1: isReady() lifecycle + a cached, bounded sink probe — database select on a sentinel key (never an insert, which would write a fabricated record into the trail), file stat plus the last append’s outcome; memory/log report lifecycle truth. |
| 10 | cache (token) | cache-plugin | packages/cache-plugin/src/plugin/cache-plugin.ts:171 | live-state | M90b (closes H-70c-2): isReady() lifecycle + a cached, bounded reachability probe — Redis ping(); memory/noop report lifecycle truth; no probe is reachable: 'unknown', never a false true. M101a: each cache Redis command is bounded by commandTimeoutMs; the indicator reads a paused server down through its own 2-second ping() bound. |
| 11 | database (token) | database-plugin | packages/database-plugin/src/plugin/database-plugin.ts:306 | live-state | M90b (closes H-70c-3): uncached lifecycle-only gate (DatabaseService.isClosed, reaching no adapter), then adapter readiness through a cached, bounded probe; a Drizzle poolStats callback publishes data.capacity — data, not a threshold. M95b: adapters carrying isHealthy?() publish reachable from the bounded probe — true/up, false/down, no answer inside the bound 'unknown'/degraded (never up); a probe-less adapter keeps the M90b path with reachable omitted. M101a: with poolStats showing a saturated pool (idle === 0, waiting > 0) the indicator is up with reachable: 'unknown' while queries keep completing through the adapter (none in 10 s is degraded; after the typed Drizzle seam hands out the native instance progress is unobservable and saturation stays up); a pool connection timeout is degraded, not down. M113o: optional schema readiness adds a separate 5 s cached, 2 s bounded ledger read: pending names → degraded, unreadable history → down, and current history preserves status even with unknown names. Already-down results skip the ledger; absent readiness preserves the existing output. |
| 12 | notification | notification-plugin | packages/notification-plugin/src/plugin/notification-plugin.ts:105 | live-state | CLOSES H-70c-4: per-channel reachability in data.reachable; one contacted-and-unreachable channel takes the indicator down. Email delegates to IMailer.isHealthy; the send-only transports report 'unknown', because probing them would deliver a real notification. |
| 13 | worker-pool | worker-pool-plugin | packages/worker-pool-plugin/src/plugin/worker-pool-plugin.ts:108 | justified-literal | M113b: status stays up without a worker host. data.available: false and data.reason expose unsupported task execution while /ready stays 200. data.exitDetection and pool counters remain data. |
| 14 | cloudflare | cloudflare-plugin | packages/cloudflare-plugin/src/plugin/cloudflare-plugin.ts:259 | justified-literal | Reports the billed binding read — which arms are present is a platform fact known at registration, not a backend to probe. The status is up/down on the presence of the bindings the arms need. |
| 15 | feature-flags | feature-flags-plugin | packages/feature-flags-plugin/src/plugin/feature-flags-plugin.ts:138 | justified-literal | M113b: status stays up when the provider is unhealthy. data.healthy: false and optional data.detail expose the condition while the last good snapshot serves. |
| 16 | session | session-plugin | packages/session-plugin/src/plugin/session-plugin.ts:140 | live-state | service.storeHealth() — a store that reports unhealthy is down (a session store failure is the one invisible from outside). |
| 17 | static-files | static-plugin | packages/static-plugin/src/plugin/static-plugin.ts:56 | justified-literal | M113b: the no-fs runtime arm reports up with data: { fs: false, detail: 'no file system on this runtime' } and registers no static route. |
| 18 | static-files | static-plugin | packages/static-plugin/src/plugin/static-plugin.ts:85 | live-state | The fs arm: a real fs.stat(root); down when the root is missing or not a directory. |
| 19 | sse | sse-plugin | packages/sse-plugin/src/plugin/sse-plugin.ts:115 | configuration-literal | status: 'up' beside live data.connections. The SSE service has no backend to probe; the status is decorative. |
| 20 | websocket | websocket-plugin | packages/websocket-plugin/src/plugin/websocket-plugin.ts:193 | configuration-literal | status: 'up' beside live data.available/connections/rooms/routes. No backend to probe. |
| 21 | cqrs | cqrs-plugin | packages/cqrs-plugin/src/plugin/cqrs-plugin.ts:149 | configuration-literal | status: 'up' beside live data.commands/queries handler counts. In-process bus, no backend. |
| 22 | events | events-plugin | packages/events-plugin/src/plugin/events-plugin.ts:201 | configuration-literal | status: 'up' beside live data.handlers. In-process bus, no backend. |
| 23 | graphql | graphql-plugin | packages/graphql-plugin/src/plugin/graphql-plugin.ts:211 | configuration-literal | status: 'up' beside live data.endpoint/cachedDocuments/subscriptions. The endpoint is in-process. |
| 24 | multi-tenancy | multi-tenancy-plugin | packages/multi-tenancy-plugin/src/plugin/multi-tenancy-plugin.ts:331 | configuration-literal | status: 'up' beside live data.resolver/strategy/store. In-process middleware, no backend. |
| 25 | react-router | react-router-plugin | packages/react-router-plugin/src/plugin/react-router-plugin.ts:235 | configuration-literal | status: 'up' beside live data.mode/serverBuildPath. Stateless handler (the source notes there is no socket, pool, timer, or subscription to close). |
| 26 | secrets | secrets-plugin | packages/secrets-plugin/src/plugin/secrets-plugin.ts:187 | live-state | M90b: isReady() lifecycle + a cached, bounded reachability probe — Vault /v1/sys/health (no secret read, no token); env reports lifecycle truth; a cloud facade without the optional isHealthy() member is reachable: 'unknown'. M101a: the Vault probe is bounded by requestTimeoutMs. |
| 27 | view | view-plugin | packages/view-plugin/src/plugin/view-plugin.ts:77 | justified-literal | M92: rendering is stateless and touches no backend — there is nothing to probe — so up is the correct signal, and the live data carries the real fact: the selected engine (hono-jsx, hono-html, or custom). |
| 28 | auth | auth-plugin | packages/auth-plugin/src/plugin/auth-plugin.ts:298 | justified-literal | M113b: registered for configured issuers. Status stays up; data.current reports whether every issuer key set is current, and data.issuers exposes each cached state. No probe I/O. |
| 29 | localization | localization-plugin | packages/localization-plugin/src/plugin/localization-plugin.ts:153 | justified-literal | M103: catalogues are validated and loaded during register() and nothing is held afterwards — there is no backend to probe — so up is the correct signal, and the live data carries the real facts: the supported locale count, the default locale, and the catalogue source (static or injected[:name]). |
| 30 | outbox (outboxToken) | messaging-plugin | packages/messaging-plugin/src/plugin/messaging-plugin.ts:785 | live-state | M107: lifecycle truth first (down, ready: false before the store is verified at onInit and once the outbox is closing); otherwise reads IOutboxStore.stats() from every store through createCachedProbe (TTL 5 s, bound 2 s) — down with reachable: false when the store does not answer. up with fixed data.reasons: cluster-wide from the STORE (oldest-pending-age past health.degradedAfterMs, failed-rows, failed-scan-cap when a store’s failed count reaches relay.maxFailedScan) and per-instance from this relay (blocked-key-cap, store-write-failing, relay-overlap) — a replica that never sweeps reports none of the per-instance reasons. data carries counts, ages and the last local sweep’s counts only; never a tenant id, key, topic or error text. |
| 31 | idempotency | idempotency-plugin | packages/idempotency-plugin/src/plugin/idempotency-plugin.ts:94 | live-state | M109a: lifecycle truth first for EVERY store arm — a store that is not connected is down — then, where the store exposes isHealthy (Redis ping()), a cached, bounded reachability probe; a probe-less store (memory, custom) reports up with reachable: 'unknown'. The status reflects a live fact, so a Redis outage or a closed store turns it down. |
| 32 | inbox (inboxToken) | messaging-plugin | packages/messaging-plugin/src/plugin/messaging-plugin.ts:865 | live-state | M108: lifecycle truth first (down, ready: false before the store is verified at onInit and once the inbox has closed); otherwise reads IInboxStore.stats() through createCachedProbe (TTL 5 s, bound 2 s) — down with reachable: false when the store rejects, up with reachable: 'unknown' when the read does not answer within the bound. up with the fixed data.reasons entry parked-rows when any delivery is parked. data carries the parked count only; never a consumer, topic, envelope id or error text. |
Out-of-scope defects recorded (not changed on this branch)
Per plan §3.9, the configuration-literal sites outside the six in-scope packages that
nonetheless hide a real backend behind a decorative status were recorded in the smoke register
(smoke/DEFECTS.md, which is deliberately not tracked in git — hence a plain reference rather than
a link) rather than fixed on that branch. All five are now closed, and the table is kept as the
record of what the audit found:
| Defect ID | Package | Issue |
|---|---|---|
| H-70c-1 | audit-plugin | CLOSED — the audit indicator now probes the sink (database sentinel read, file stat + last append outcome). |
| H-70c-2 | cache-plugin | CLOSED in M90b — the cache indicator now probes reachability (Redis ping(); unknown, never a false true). |
| H-70c-3 | database-plugin | CLOSED in M90b — the database indicator gates on uncached lifecycle behind a bounded probe and publishes capacity. |
| H-70c-4 | notification-plugin | CLOSED — per-channel reachability; email delegates to IMailer.isHealthy, send-only transports report 'unknown'. |
| H-70c-5 | worker-pool-plugin | M113b corrects the mapping: up with data.available: false and a reason, preserving readiness on Workers. |
The remaining configuration-literal rows (7, 8, 19–25) are defensible: the plugin has no
external backend — the status says “this in-process capability is registered”, and the live data
carries the useful part. They are recorded here so a future milestone can decide whether to
reclassify them, but they are not defects in the X8-5 sense (no backend is being hidden).