agents-cli
The CLI your coding agent uses to ship agents
170 followers
The CLI your coding agent uses to ship agents
170 followers
One command-line tool to scaffold, evaluate, and deploy AI agents on Google Cloud — built to be driven by your coding agent (e.g Antigravity, Claude Code, Codex). Scaffold a production-ready project, evaluate against a real signal, and ship to Agent Runtime, Cloud Run, or GKE or anywhere else!
Agent Starter Pack
the headless self-optimize loop is the part I'd want to understand before trusting it. if the coding agent is scoring its own agent against eval criteria and iterating unsupervised until the number goes up, what stops it from overfitting to the eval dataset or gaming the specific metric instead of actually improving behavior on real traffic? that's a known failure mode any time the thing being optimized also controls the optimization loop. do you have guardrails around eval set size/diversity or a human checkpoint before it ships to Cloud Run/GKE, or is it fully autonomous end to end
Agent Starter Pack
@galdayan Absolutely and great point!
Most of the guardrails live in the skills we load into the coding agent:
- You are always in control of the evaluation process, you can define how the dataset looks like and how varied it is. Coding agents and skills are there to support you with those best practices you mentioned.
- It never deploys without you saying yes. It runs the evals, asks "ok to deploy?", and waits.
Agreed that this won't save you from overfitting a tiny eval set, so it also nudges you to grow coverage once a case passes instead of tuning against a handful of examples.
@elia_secchi that's the piece that matters, explicit "ok to deploy" gate rather than a fully autonomous loop. the eval-coverage-growth-over-time nudge is a nice touch too, most tools just leave that as a docs recommendation nobody follows. makes sense now, thanks for walking through it
The 'driven by your coding agent' framing is the interesting bet here. When we let a coding agent run our deploy CLI unattended, the thing that bit us wasn't the deploy logic, it was output format. The agent would read a human-formatted stderr, miss that the deploy half-failed, and cheerfully report success. Are the CLI's outputs, eval scores, deploy status, errors, structured for a model to consume, like JSON with explicit exit states, or the same prose a person reads? That one choice decided whether our agent could actually close the loop.
Agent Starter Pack
@dipankar_sarkar For sure, we are actually continuously improving the output generated by the CLI to be Agent friendly. It's actually a core philosophy of the product: being a CLI optimized for agents first and allow human-friendly output when running in interactive mode
Toolport (formerly Conduit)
Congrats on the launch Elia! The "demo in an afternoon, production in weeks" framing is painfully accurate, and the detail that barely any of those weeks are the actual agent matches what I see too.
The headless part is the bit I'm most curious about: when Claude Code is driving the CLI end to end, how does auth work once the scaffolded agent needs real credentials for its tools and MCP servers? That's usually where the self-driving loop stalls and a human gets pulled back in. I live in that corner of the stack (launching an MCP thing myself today), so genuinely curious how you handle it.
To your question: I'd ship a changelog agent. It reads the PRs merged since the last tag, drafts release notes plus the announcement post, and files the result as a PR for review. Curious what an eval signal looks like for something that fuzzy, "good release notes" is a hard thing to score.
Agent Starter Pack
@tsouth2 Thanks Tyler, and good luck with the MCP launch today.
Auth is two layers. Claude Code will inherit how you're authed as locally.
The deployed agent's tool and MCP creds go through Secret Manager with a dedicated service account, not baked into the scaffold. And you're right that this is where a human comes back on purpose: the agent can wire up the secret reference, but adding the actual secret is a human step. Better it stalls there than have a coding agent minting prod credentials on its own.
On the changelog agent (fun one): "good release notes" is fuzzy, so I'd grade it with an LLM-as-judge rubric (covered every merged PR, categorized right, invented nothing, matched your tone) plus a couple of hard checks like every PR number present and links resolving. The goal should be for the rubric to bring the signal, and you tune it as you spot misses.
The “CLI built for coding agents” framing feels right. The part I’d want to see is how the tool keeps the agent inside the intended change boundary.
When a scaffold/eval/deploy flow touches infra files, tests, Docker, telemetry, and CI, a coding agent can accidentally turn a product task into a repo-wide cleanup. A small plan/diff contract before each step — what files may change, what success signal matters, and how to roll back — would make me trust the loop much more.
Curious if agents-cli exposes that kind of step-level boundary, or if it’s mostly handled through the coding agent’s prompt/skill layer today?
Agent Starter Pack
@grace_lee26 Honest answer: today that boundary is mostly the skill/prompt layer, not something the CLI enforces per step.
What the skills push the coding agent toward:
- Plan first: it lays out the intended change before editing, so you see the scope up front.
- Preserve everything outside the target. The skill is explicit that code, config, comments, and formatting outside the specific change stay identical, which is meant to stop exactly the "product task becomes repo-wide cleanup" drift you're describing.
- Human approval before deploy, and regressions are stop-the-line.
Structurally the CLI helps in two ways: everything it scaffolds or edits lands as plain files, so it's all a reviewable git diff and git is your rollback; and deploys are idempotent and preserve your existing spec, so re-running is safe.
But a hard, tool-enforced "these files may change, here's the success signal, here's rollback" contract per step isn't there yet. Good ask though, noting it.
How does the evaluation step actually work in practice? Like, does it spin up a real signal in the cloud or do I need to wire up my own eval set before it can tell me if my agent is any good?
Agent Starter Pack
@semiha327125118 great question - it works according to what you define as success criteria for the agent.
based on that it will generate synthetic conversations, start testing the agent and then based on the output converting those in metrics. Then fix code & prompts and iterate until satisfying your success criteria!
Very neat! Do you have plans to support cloudflare by any chance?
Agent Starter Pack
@cadell_falconer Thanks a lot! Not in the short term but the generated code is fully portable to any platform!