How should teams set severity levels for AI visibility incidents?

by

Not every change in an AI-generated answer deserves the same response.

A missing citation on one exploratory prompt is different from an inaccurate claim repeated across high-intent questions. I think an AI-visibility severity model should combine four factors:

1. Scope — one prompt, one topic cluster, or the full benchmark?

2. Claim risk — cosmetic wording, commercial misinformation, or a factual/compliance problem?

3. Customer exposure — low-volume research prompt or a question close to purchase?

4. Persistence — one observation or a reproducible change across a defined window?

A simple operating model could be:

P0: A high-risk false claim, safety issue, or compliance problem. Preserve evidence and escalate immediately.

P1: Broad, persistent loss across important prompt families, or a trusted citation being replaced by a materially weaker source.

P2: Persistent change limited to one topic cluster, region, model, or citation family.

P3: An isolated observation that should be retained and monitored but not treated as an incident yet.

Collection health has to be a gate before any severity is assigned. If the crawler, provider, parser, region, or prompt set is incomplete, the correct status is “measurement degraded,” not “brand visibility incident.”

For each incident I would store the exact prompt, raw answer, model context, citations, timestamp, collection-health state, owner, and verification criteria. That keeps urgency tied to evidence instead of whoever noticed the dashboard first.

Corank is built around auditable AI-search measurement and action:

What dimensions would you add to this severity model—revenue exposure, legal risk, competitor movement, or something else?

1 view

Add a comment

Replies

Be the first to comment