If your product picks the model for the user, it has to say why
We route across 30+ models and for a while the picker just swapped silently. Someone asks for a video, we choose what we think is right, they get a result. Clean, and wrong.
A user can't tell the difference between a deliberate routing decision and a model having a bad day. The output changes, the price changes, and nothing on screen explains either one, so the conclusion they reach is that the product got worse.
So the picker now names what it chose and why before it runs, in one line. It makes the thing look less magic, and I think that's the right trade. Silent routing is the same bet as a silent retry or a silent price change. It holds while the output is good, and the first time it isn't, you've taught someone to distrust the whole product instead of one model.
If you auto route anything, agent, model or tool, do you show the user what got picked, or does it happen behind the curtain?
Replies
I totally agree that transparency is the right move here, otherwise a single bad output ruins the reputation of the whole product.
@amanndaphillips The reputation bit is where I'd push slightly. It's not that one bad output sinks you, it's that with nothing on screen people can't tell a bad output from a different model, so both get filed under the product got worse. Naming the pick doesn't make the result any better, it just stops one bad run spreading to everything else.
@asadmalik901 Got it. It doesn't fix the bad output, but it stops them from thinking your whole product is just breaking down.
An agent routed a refactor to a smaller model and reported a clean build, even though it dropped the error handling. Bad code does not kill trust. Being told everything is fine right before finding broken logic does.
@konstantin_tikhaev That's the worse version of it, a silent swap with a confident all clear stacked on top. The thing reporting the build is clean is the same thing that wrote the code, so the report inherits whatever the work got wrong. Naming the model at least tells you which claim to be suspicious of, but in your case I'd want the build result coming from something that didn't write the refactor.
@asadmalik901 Separating verification from generation is the only real fix. Letting the model grade its own refactor just turns blind spots into false confidence.
I would also let people keep that model for the next run
Otherwise they cannot tell whether a better result came from changing the prompt or changing the model
The explanation helps more when there is a choice behind it
@al_kub That's the part we got wrong first. We named the pick but rerouted fresh on every run, so someone would tweak the prompt, the model moved underneath them, and there was no way to tell which change did the work. It's pinnable now, and the thing I didn't expect is how many people pin once and never go back to auto, which says something uncomfortable about how much the routing was worth in the first place.