AI confidence is becoming a security problem

One thing we’ve noticed while testing AI systems:

Confidence is often mistaken for correctness.

A system can:

• sound certain

• appear reliable

• produce professional outputs

…while still behaving incorrectly under certain inputs.

That becomes risky when these systems are trusted in real workflows.

Feels like we need better ways to evaluate not just outputs

but confidence and behaviour under pressure.