Versus Desk

Problem Management

How Versus Desk investigates recurring faults — the PRB ticket model, a declarative investigate-to-fix state machine, known errors and workarounds, PRBT investigation tasks, links to the incidents and change that fix it, and promotion to the knowledge base.

A problem in Versus Desk is the underlying cause behind one or more incidents — the recurring pool flap, the certificate that keeps expiring, the config drift that trips the same alert every week. Where an incident is about restoring service now, a problem is about finding root cause, publishing a workaround, and driving a permanent fix. Problems live in the same service desk as incidents, changes, and requests, share one ticket model and one hash-chained audit trail, and link directly to the incidents they explain, the devices they touch, and the change that resolves them.

This page covers what a problem is, the lifecycle it moves through, how known errors and workarounds are recorded, how investigation is broken into tasks, and how an operator works a problem from the API.


The problem record

Every ticket type — incident, problem, change, request — is a row on a shared Ticket table, with type-specific fields on a one-to-one extension. Problems carry a ProblemExt.

Core fields on the ticket:

FieldMeaning
numberHuman ID, PRB-000001, allocated monotonically per prefix inside the create transaction (gaps from rolled-back writes are acceptable, not backfilled).
short_description / descriptionThe problem statement and full detail.
stateWhere the problem is in its lifecycle. Validated by the state machine, never a database enum.
priority1 (highest) to 4, derived from impact (1–3) and urgency (1–3).
assignment_group / assigneeThe queue that owns the investigation and, once picked up, the person.
originHow it was created: manual, workflow, api, sync, and so on.
opened_at / resolved_at / closed_atLifecycle timestamps.

Problem-specific fields on ProblemExt:

  • known_error — set once the root cause is understood well enough to have a documented workaround.
  • workaround — the interim mitigation, recorded so it can be applied while the permanent fix is pending; workaround_applied_at stamps when it went out.
  • rca_summary — the root-cause analysis narrative; rca_confirmed_by records the user who confirmed it.
  • fix_change — a foreign key to the Change ticket that carries the permanent fix; fix_notes captures the fix detail.
  • major_problem — flags a high-severity problem for special handling.
  • throughline_analysis_id — a reference to a Throughline diagnostic analysis when the root cause was worked up from packet-level evidence.

Lifecycle

Problems move through a declarative state machine — one engine drives all four ticket types, and transitions are data, not hardcoded buttons. The problem path is:

New ──investigate──▶ Investigation ──mark_known_error──▶ Known Error ──start_fix──▶ Fix In Progress ──resolve──▶ Resolved ──close──▶ Closed
                          │                                                              ▲
                          └──────────────────────── start_fix ──────────────────────────┘

Each transition fires an event and may enforce a guard:

  • investigate (New → Investigation) opens the root-cause work.
  • mark_known_error (Investigation → Known Error) sets known_error = true and records the workaround supplied in the transition — you declare a known error precisely when you have a mitigation to hand out.
  • start_fix (Investigation or Known Error → Fix In Progress) requires a linked fix_change — you cannot start the fix without pointing at the change that carries it. You can jump straight from Investigation to a fix if the root cause is clear enough to skip the known-error step.
  • resolve (Fix In Progress → Resolved) marks the underlying cause corrected.
  • close (Resolved → Closed) finalizes it and stamps closed_at.

Transitions are concurrency-safe: the service layer takes a row lock on the ticket (select_for_update), re-reads the current state, re-checks the guard, applies the change, and commits — all in one transaction. An illegal transition (wrong state for the event, or a failed guard) is rejected with 409 Conflict and changes nothing.

The UI does not hardcode which buttons to show. It reads the ticket's available_events — the set of events legal from the current state — and renders exactly those actions.

Every transition writes two records: a TicketTransition row (the human-readable "what happened" log) and an entry on the append-only, hash-chained audit trail (ItsmAuditEvent), so the full history of a problem is tamper-evident.


Known errors and workarounds

The Known Error state is the heart of problem management. It says: we understand this well enough that anyone hitting it again can be given a workaround, even before the root cause is permanently removed.

Declaring a known error is a single transition. mark_known_error sets the known_error flag and stores the workaround text in one atomic step, so the mitigation and the flag are never out of sync. From there the workaround can be surfaced to agents working the incidents that link back to this problem, and later captured into the knowledge base (below).

Root-cause analysis lives alongside the known error: rca_summary holds the narrative and rca_confirmed_by records who signed off on it, so the analysis is attributable rather than anonymous.


Investigation tasks (PRBT)

A problem's work is broken into problem tasks — the problem-side analogue of a change task. Each is numbered PRBT-000001 via the same shared counter as ticket numbers, and carries a task_type:

Task typeTypical use
investigationGather evidence, reproduce, narrow the cause.
root_causeConfirm the underlying cause.
fixBuild or stage the corrective change.
testValidate the fix.
reviewPost-fix review / retrospective.

Tasks run their own small lifecycle — open → in_progress → one of closed_complete, closed_incomplete, or closed_skipped — independent of the parent problem's state machine; they are a checklist and assignment breakdown, not a gate on the problem itself. Closing a task stamps closed_at.

Problem tasks are their own endpoint:

GET  /api/itsm/problem-tasks/?ticket={problem_id}&state=open
POST /api/itsm/problem-tasks/
{
  "ticket": "…",
  "short_description": "Reproduce the pool flap in the lab",
  "task_type": "investigation",
  "assignment_group": "…"
}

Creating or editing a task requires the itsm.agent permission; reading requires itsm.view. The PRBT- number is allocated server-side on create.


Linking incidents, CIs, and the fix

A problem earns its keep by tying things together.

Related incidents. Incidents roll up to a problem through a problem_incident ticket link (and the problem foreign key on IncidentExt). That aggregation is what turns "the same alert fired six times this month" into one investigation with a shared root cause. Other link types are available too — caused_by, duplicate_of, related, and fixes.

Configuration items. A problem links to the infrastructure it concerns through CI links, which point directly into existing Versus inventory — there is no separate CMDB to keep in sync. A CI link references a device, vip (virtual server), certificate, rule, or service (application) by type and id. Because links are indexed by (ci_type, ci_id), Versus can answer "every ticket touching this device" instantly — the same index the change-management risk engine reads.

The fix change. When the permanent fix is ready, the problem points at the Change ticket that carries it via fix_change. That link is what the start_fix guard checks, and it closes the loop: the change goes through its own approval, risk scoring, and implementation lifecycle, and its outcome is traceable back to the problem it resolved. See Change Management.


Promoting to the knowledge base

Once a problem is understood, its workaround and root cause are worth keeping. A knowledge-base article can be captured from a known error, and the article records source_problem — a link back to the PRB it came from — so published knowledge is traceable to the investigation that produced it. Articles have their own draft → published → retired lifecycle and full-text search.

POST /api/itsm/kb/
{
  "title": "vs_app01 pool flap — cause and workaround",
  "body": "## Symptom …\n## Root cause …\n## Workaround …",
  "source_problem": "…",
  "tags": ["f5", "pool", "monitor"]
}

See Catalog & Knowledge for how the knowledge base fits the wider desk.


Working a problem from the API

Problems are the problem ticket type on the ticket endpoint, addressed by number. Reading requires the itsm.view permission; creating and editing require itsm.agent.

Create a problem:

POST /api/itsm/tickets/
Authorization: Bearer …
Content-Type: application/json

{
  "ticket_type": "problem",
  "short_description": "Recurring pool flap on vs_app01",
  "priority": 2,
  "impact": 2,
  "urgency": 2,
  "assignment_group_id": "…",
  "ci_links": [ { "ci_type": "vip", "ci_id": "…", "ci_label": "vs_app01" } ]
}

Link the incidents it explains:

POST /api/itsm/tickets/PRB-000001/links/
{ "to_ticket_id": "…", "link_type": "problem_incident" }

Drive it through its lifecycle with the transition action — the event must be legal from the current state, and guarded events carry their required data in fields:

POST /api/itsm/tickets/PRB-000001/transition/
{ "event": "investigate" }

POST /api/itsm/tickets/PRB-000001/transition/
{
  "event": "mark_known_error",
  "fields": { "workaround": "Re-enable the monitor and bounce the flapping members." }
}

POST /api/itsm/tickets/PRB-000001/transition/
{ "event": "start_fix", "fields": { "fix_change_id": "…" } }

POST /api/itsm/tickets/PRB-000001/transition/
{ "event": "resolve" }

Attempting start_fix without a fix_change (on the request or already set on the record) is rejected with 409 Conflict and the reason Fix In Progress requires a linked fix_change.

List and filter the queue with standard query parameters — ticket_type=problem, state, assignment_group, assignee, priority — plus desk shortcuts: q (text search), ci_type+ci_id (tickets touching one CI), and saved views view=mine|submitted|group|open|closed.


Where problems fit

Problems are the investigative corner of Versus Desk, and they connect in every direction:

  • Incidents feed them — recurring incidents roll up through problem_incident links for root-cause analysis, and a documented workaround flows back to the agents working those incidents (Incident Management).
  • Changes resolve them — the fix_change link ties a problem to the change that permanently corrects it (Change Management).
  • Throughline can supply the evidence — throughline_analysis_id references a packet-level diagnostic that pinned the root cause.
  • Knowledge outlives them — a known error becomes a KB article that records the source_problem it came from (Catalog & Knowledge).
  • Existing ServiceNow problem data can be brought across, and the desk can run alongside ServiceNow (ServiceNow Integration).

Every problem action — transition, task, known-error declaration, fix link — lands on the same tamper-evident audit chain, so a problem's whole life, from first investigation to permanent fix, is reconstructable after the fact.


Related