F5 Report Card

Overview

A Report Card is poller-writes / API-reads. A background evaluator runs on a timer, reads live telemetry, and writes an immutable snapshot. The widget just reads the newest snapshot for its card type — so the dashboard is always a cheap, instant read, never a live device call.

Each snapshot carries:

• Score — an integer 0–100, or N/A when there is nothing eligible to grade yet.
• Badge — Good (90–100), Warning (60–89), Critical (0–59), or N/A.
• Checks — the sub-checks that make up the score, each with objects eligible, objects deviating, a severity, a summary, and its penalty.
• Findings — the individual deviating objects (a VS, pool, device, or certificate) behind each check, reached from View Details.

The four cards

From Versus Platform → Dashboards, click Add Widget, choose F5 Report Card, pick which card to show (BIG-IP Metrics, Anomaly Detection, Application Performance, or Security), and optionally set a title. The widget begins polling immediately and refreshes every 30 seconds. Add it four times to see all four cards side by side — each widget stores only its card_type; the work has already happened server-side.

Architecture at a glance

The evaluator runs as a daemon thread in the application worker, started on boot when in-process schedulers are enabled. It is idempotent per process and safe to run redundantly — extra runs simply raise snapshot cadence, which is thinned away later.

Collection cadence

A single loop drives everything off a 60-second base tick:

Collectors always run before the evaluators in the same cycle, so each evaluation sees fresh data. The 5-minute cadence matches the underlying telemetry refresh, so baseline learning gets exactly one new sample per cycle. Every collector and evaluator swallows its own errors — one failing card can never poison another.

Collectors — what is read from the BIG-IP

Collectors talk to each managed BIG-IP (active/standby, with a credential) over iControl REST:

Because F5 reports HTTP stats per profile, not per virtual server, the HTTP collector sums the stats of every HTTP profile attached to a VS to synthesize per-VS counters; a VS with no HTTP profile produces no row and is excluded from grading. Base metrics — CPU, memory, disk, throughput, and per-VS connection counts — are not collected here; they come from the platform's existing stats poller, and the evaluators read those tables directly.

Scoring — the shared machinery

Bands

Every card maps its final score to a badge the same way, with a critical floor on top: a single Critical finding drags a would-be Good badge down to at least Warning, so a green badge never hides a critical object.

Weighted-penalty scoring (Anomaly & Application Performance)

• Checks with zero eligible objects are dropped entirely — not counted as a free, passing vote.
• The remaining weights renormalize to sum to 1, so the score reflects only what was actually measurable.
• If nothing was measurable, the score is N/A, not 100.

Penalty-sum scoring (Security)

Security sums fixed penalties: score = 100 − min(100, sum_of_penalties), with N/A when there are neither eligible certificates nor ASM-provisioned devices.

Healthy-ratio scoring (BIG-IP Metrics)

The simplest: score = round(healthy_devices / total_devices × 100), and N/A when there are no BIG-IPs in inventory.

Baselines — learning “normal”

Anomaly and Application Performance grade against a learned baseline, so “normal” is defined per object and per time-of-week:

• Each metric keeps a running mean and variance via Welford's online algorithm — no raw history is stored.
• Statistics are bucketed by hour-of-week (168 buckets, UTC), so Monday 9am is compared against previous Monday-9am traffic, absorbing daily/weekly seasonality.
• A sample is tested against the baseline as it existed before that sample was folded in — a value never inflates the baseline judging it.
• A baseline is eligible only after it has learned enough: ≥2 prior samples in the current hour-bucket and roughly a full day of samples overall (≈288 at the 5-minute cadence). Until then the metric contributes nothing and the score can read N/A.

Deviation tests

Hysteresis — no flapping

A single noisy sample never flips a badge. Each (object, check) keeps a hysteresis counter: an object must read over-threshold on 2 consecutive evaluations before it is marked deviating, and 5 consecutive under-threshold readings before it clears.

The four cards in detail

BIG-IP Metrics (device_health)

Runs every 60 seconds; a pure DB read. A BIG-IP is unhealthy if any of:

1. Status is not active or standby.
2. It has never reported a successful stats poll.
3. Its last stats poll is stale — older than 15 minutes (3× the 5-minute poll interval).
4. Its failover status is error/unknown/empty (only once a device-health signal exists).
5. Its sync status is not In Sync (same condition).

Score is the healthy fraction of the fleet; any unhealthy device makes the check Critical. CM checks are skipped until the first signal is collected, so devices aren't falsely flagged on a cold start.

Anomaly Detection (anomaly)

Application Performance (app_perf)

Error ratios use deltas between consecutive cumulative counters; an interval is skipped when a counter goes backwards (a reboot/failover reset) or when there was no traffic — so a reboot never manufactures a fake error spike.

Security (security)

Certificate expiry — for each certificate bound to a BIG-IP (resolved through the real client-SSL profile → certificate binding):

WAF events — for each ASM-provisioned device, count critical/error ASM events in the last 5 minutes; more than 30 in the window costs 30 points and raises a Critical finding. Devices without ASM provisioned are green here, not N/A. Final score = 100 − min(100, total penalty).

Findings — the drill-down

When a check has deviating objects, the widget shows View Details, listing each deviating object — device, virtual server, pool, or certificate — with its measured value, its baseline, and the threshold it crossed (e.g. > 3σ above baseline). Each finding links straight to that object's page in Versus.

API

The widget reads a small, read-only API (tenant-scoped by the caller's token where multi-tenancy is enabled). Snapshots are append-only; reads never touch a device.

Alerting

Each card's fleet score is wired into Versus Observe as an alert rule, created disabled so you opt in:

• A postgresql datasource points at the application database.
• An evaluation group runs every 5 minutes.
• Each rule selects the latest fleet score for its card and fires when score < 60 (the Critical band) after a 5-minute pending period.

Un-pause the rule in Observe and route it to any contact point. Re-running the seeder is safe — the rules are created idempotently.

Retention

Snapshots and stats are down-sampled (RRD-style) rather than kept forever:

• Snapshots — one per 15 min after an hour, hourly after a day, daily after a week, hard-deleted after 90 days.
• AppPerfStats — same thinning, hard-deleted after 30 days.
• WafEvent — hard-deleted after 30 days.

Deleting a snapshot cascades to its checks and findings.

Multi-tenancy

On single-tenant and on-prem deployments this is inert. Where multi-tenancy is enabled, every row carries a tenant_id, the evaluator runs one scoped pass per active tenant (stamping each snapshot), and the read API scopes queries to the caller's tenant claim. Baselines, hysteresis state, and device-health signals are regenerable, so a background command can safely purge any tenant-less rows left from the rollout.