AdaptGauge
p/adaptgauge
Detect when few-shot examples make your LLM worse
•0 reviews•1 follower
Start new thread
trending
Shuntaro Okuma•

20h ago

AdaptGauge - Detect when few-shot examples make your LLM worse

AdaptGauge detects when adding few-shot examples degrades LLM performance instead of improving it. Testing 8 models across 4 tasks revealed three failure patterns: • Peak regression — 64% at 4-shot, crashed to 33% at 8-shot • Ranking reversal — best zero-shot model dropped to third with examples • Selection collapse — TF-IDF examples broke a model from 50%+ to 35% Tracks learning curves, auto-detects collapse, classifies patterns, and compares example selection methods. Demo results included.