On-device, gateway, or model? Where AI security belongs

Hey PH

We are building in the AI security space right now, and there's one question I can't get a clean answer to: when you put guardrails around LLMs, where should they actually live?

Four options and the trade-offs:

• On the device: sees everything, but needs to be installed

• Gateway / proxy: easy to deploy, useless against employees pasting data into ChatGPT in their browser

• Model layer (Llama Guard, NeMo, provider filters): close to the action, locked to one provider

• App logic: contextual, but every team rebuilds it from scratch

A bank worried about data exfiltration through Copilot has a different problem than a dev team worried about prompt injection. So the answer probably isn't universal, but I'd love to hear what people actually shipping are betting on.

• Which layer are you running, and why?

• Running multiple? What's overlap vs. redundancy?

• Where should this sit 5 years from now, when every app has an LLM?