My tests passed, but the real model ignored my prompt rules. How do you test that?

I give a text generator six different voices (a dry friend, a drill sergeant, a corporate manager, and so on). Every prompt carries shared rules: word count, how many emojis, where they go, never invent a number. I tested it end to end against a fake model server: 127 checks that the exact prompt reaches the model for every voice, all passing. That proves the prompt is assembled correctly. It proves nothing about whether the model follows it.

First real run, one sample per voice:

1. Emojis clumped. One reply put all 12 emojis in a single block at the end. In the short replies, three voices ended with 4, 5, and 6 in a row. The rule said never more than two together.

2. Under target. Four of six roasts used fewer than the 8 emojis I asked for.

3. A copied example. Each voice had a tone example marked “never reuse it”. One voice reused its closing line in both runs.

4. An invented number. It produced a ratio that wasn’t in the input, under a rule that says use only numbers the user gave.

5. My own rules contradicted each other. “Never repeat an emoji” plus “use 8 to 12 emojis” can’t both hold when two palettes had 7 and 6 emojis. The fake model never complained. Reading real output did.

What I changed: concrete placement rules (end of a sentence, about one every second sentence, spread across the whole text), tone examples as short fragments instead of full sentences so there’s nothing to copy, a rule against computing new ratios, tighter length wording, and bigger palettes. I also wrote a preview script that checks eight rules on real output and prints ok or CHECK for each, using the same checking code my unit tests load. I broke each rule on purpose 42 ways, and every break failed a test.

Weak spot: that is one sample per voice from a free-plan model, so it’s thin evidence, and I’m re-running it to see whether the clumping is really gone.

Questions: how do you test that a model follows formatting rules, not just that the prompt is built correctly? Do you check output with code, grade it with a second model, or read samples? And do examples in your prompts leak into the output too?