Security & Trust

Versus Sentinel

Self-healing at the code level — Sentinel watches the Versus control plane and every tenant and on-prem instance for glitches, diagnoses the root cause in Versus's own source, and generates a tested, scanned, reversible fix under governance you set per environment. Ships read-only.

Every other tool in Versus watches your network. Sentinel watches Versus itself — and, uniquely, it doesn't stop at noticing a problem. It diagnoses the root cause in Versus's own code and proposes a tested, reversible fix.

Versus runs a critical-infrastructure control plane and a fleet of tenant and on-prem instances. A glitch in any of them — an unhandled exception, a 5xx spike, a failed background task, a slow query, a broken migration — shouldn't have to wait for a human to notice. Sentinel collapses the time from a tenant hit a bug to a tested fix is proposed from days to minutes. It is the code-level evolution of Self-Healing: where that surfaces a detected issue and packages a support bundle, Sentinel goes on to diagnose and fix. (It is not Diagnostics AutoPilot — that fixes your managed network devices; Sentinel fixes Versus. They share the agentic and governance patterns, nothing else.)

Ships read-only. Sentinel defaults to detect, diagnose, and propose — it opens a pull request for humans to review. Auto-apply is opt-in and gated per environment, and production is never blind-merged: a fix reaches prod only after green tests, clean security scans, a passing staging verification, a canary with no recurrence, and human approval — regardless of the model's confidence.


The self-healing loop

Versus Sentinel self-healing loop: signals from across the whole estate — unhandled exceptions and 5xx, background-task failures, slow queries, JS and synthetic-flow failures, data-integrity anomalies, plus privacy-scrubbed beacons from every tenant and on-prem instance — are clustered into a Glitch by signature. An AI agent diagnoses the root cause in Versus's own code and localizes the suspect files, then generates a patch plus a regression test and iterates until green. The change enters a governed, reversible rollout: a CI gate runs tests and SAST/SCA, then an opened pull request where the flow stops by default (propose-only); only where tolerance allows does it continue to staging verify, canary on one tenant or a small percentage, and production with tenant code-update distribution, with auto-rollback on any regression. A governance rail keeps everything reversible, hash-chain audited, kill-switchable, and off-limits to auth, crypto, RLS, and migration code; a proven fix inoculates the rest of the fleet.

Detect: glitches across the whole estate

Sentinel captures malfunctions everywhere Versus runs — the SaaS control plane and every tenant slice and on-prem install — and clusters raw errors into a Glitch: a deduplicated signature (error class + code location + version) with first- and last-seen, affected tenants and versions, frequency, severity, and evidence. Signal sources include:

  • Unhandled exceptions and 5xx — captured with stack trace and request context, with rate and new-signature detection.
  • Background-task failures — stuck workflow steps, poller and scheduler errors, migrations that fail on boot.
  • Performance regressions — slow queries, p95 latency spikes, memory and CPU climb, via Observe.
  • Frontend runtime errors — a JavaScript error beacon from the console SPA.
  • Synthetic-flow failures — scheduled end-to-end checks of critical flows (login, device import, workflow run, diagnostics).
  • Data-integrity anomalies — invariant checks for orphaned rows, RLS leaks, tenant-stamp gaps, and vault errors.
  • Tenant beacons — each tenant and on-prem instance reports its own error signatures and version home, so a bug hitting one install is seen centrally — privacy-scrubbed the same way Collective Defense sanitizes signals.

Diagnose: the root cause in the code

For each Glitch, an AI agent reads the trace, the recently changed source, and the implicated files, then produces a diagnosis: the suspect file(s) and function(s), the change class (null-guard, off-by-one, missing migration, bad query, race, config), and a proposed fix strategy — with a confidence and a risk class. This is read-only: at this stage Sentinel has explained the bug in Versus's own code, not touched anything.

Fix: a tested, scanned, reversible change

When enabled to proceed, a code agent with least-privilege repo access and a sandboxed workspace produces a minimal patch plus a regression test that reproduces the glitch and passes only with the fix. It runs the targeted tests and the affected app's suite, iterating up to a bounded number of attempts until green, then runs SAST/SCA over the diff and blocks on any new finding. The result is an opened pull request carrying the diagnosis, the diff, the new test, the scan results, and the affected-instance list — and by default the flow stops there for human review. Where tolerance allows, the rollout continues as a reversible workflow: staging deploy and verify, then a canary on one tenant or a small percentage watched for signature recurrence, then production and distribution to on-prem and tenant installs through the existing code-update poll-and-apply channel (air-gapped installs receive signed bundles and never phone home). Every stage is a gate, and every stage can roll back.

Governance and safety

The hard part of fixing your own code is doing it safely — so the guarantees are structural, not a promise:

  • Read-only by default, with auto-apply opt-in per environment and production never blind-merged — green tests, clean scans, a passing staging verify, a clean canary, and human approval are all required, whatever the confidence.
  • Reversible everything — rollout is workflow-driven with governed back-out; a canary auto-rolls-back on regression; on-prem applies verify-then-commit with local rollback.
  • Blast-radius limits — a per-day cap on fixes, one-glitch-one-fix, and a hard allow-list: Sentinel never autonomously touches auth, crypto, RLS, or migration code.
  • Oscillation guard — a fix that doesn't clear its signature (or reopens it) is quarantined; Sentinel won't re-fix the same signature in a loop.
  • Isolation — a tenant's raw error data stays RLS-scoped; only de-identified signatures cluster across tenants.
  • Transparency — every glitch, diagnosis, fix, and rollout is a Versus Desk issue plus a hash-chained audit entry, and an operator can pause Sentinel instantly with a kill-switch.

How it's delivered

Phased, safest-first — each phase useful on its own:

  1. Detection and clustering — exception, 5xx, and task-failure capture plus tenant beacons, deduplicated into Glitches on a dashboard and as Desk issues. No code changes.
  2. AI diagnosis — root-cause-in-code, suspect paths, and a proposed strategy on each Glitch, read-only.
  3. Fix generation to PR — patch plus regression test through CI and scans, opened as a pull request for human merge.
  4. Governed auto-rollout to staging and canary with synthetic verify and auto-rollback; production stays approval-gated.
  5. Tenant and on-prem distribution through the code-update channel, staged and per-version gated, with Collective-Defense linkage so a fix proven on one glitch pre-empts the fleet.

Related