How are you stopping users from tricking your AI into breaking its own rules?
by•
Writing system prompts feels like building a fortress out of cardboard sometimes. You spend days crafting strict guardrails telling the model "never reveal internal logic or offer unauthorized discounts," and then a curious user simply types a clever hypothetical scenario and the model folds immediately. You can't possibly predict every weird phrasing or social engineering trick people will throw at the chat box once it goes live.
How are you hardening your application against prompt injections and jailbreaks before creative users find the loopholes in production?
6 views
Replies
The most reliable guardrail I have is giving the model no power worth tricking it into using. In my song app, nothing a model says can grant credits, change a price or unlock a feature: credits are written only by the backend, and the expensive engine is checked against a server side flag on every request, because a switch the client merely draws is a suggestion. For the decisions a model does make, there are layers: plain rules first, then before a song can be released to streaming, a separate judge model with its own prompt, stricter than the screen every request passes. Prompts can always be talked around. Permissions that live outside the model cannot.