Promptic
Optimize GenAI applications for quality and cost
173 followers
Optimize GenAI applications for quality and cost
173 followers
Promptic is the optimization platform for GenAI applications, better quality at lower cost. Benchmark models, tune prompts and agents, and optimize tool use against your own data and business metrics. Every candidate is scored on the quality and cost you actually care about, so you ship the configuration that wins instead of the one that sounded right. Runs wherever you are, dashboard UI, your CI, or your coding agent.
Free Options
Launch Team / Built With
Render Sandboxes: a computer for every agentIsolated compute, persistent state and a real filesystem
Promoted
We built Promptic because GenAI teams should not have to choose prompts, models, or agent architectures by intuition. Promptic brings optimization to the GenAI era by turning your own examples, evaluations, and production traces into an optimization loop. Define what good means, compare candidates on quality, cost, and latency, and ship the best-value configuration.
Today we are opening the platform to everyone. We would love to hear what you are optimizing, where your current workflow falls short, and which tradeoffs matter most for your application.
@dominik_bura being able to use our own production traces for the optimization loop is a massive selling point.
@dominik_bura Awesome to see more tooling focused on post deployment ai observability.
Is there a way to export the benchmark reports to share with non-technical stakeholders? congrats for shipping
Thanks, @vikramp7470 ! Not yet, but that’s a great suggestion. We’ll add benchmark report sharing to our roadmap. Would you consider PDF export or a shareable link to be more useful?
@danielschroterthuem I think a PDF export would be great, especially for attaching to formal reports or sharing with clients. good luck for the launch🫡
cost + quality tracking in one place is usefull 👏 easy to plug into existing agents?
@petrkovacik Yes, super easy. We didn't want to create yet another agent framework, that you have to use. Ours is just a wrapper around your existing agent invocation, you can therefore use whatever agent framework you want. You just have to setup the benchmark (evaluation cases, evaluators) and integrate our SDK wrapper. In the wrapper, we provide you with the individual evaluation case input and you give us the prediction for the case back, everything else runs automatically. We also provide a skill for your coding agents, so integration is pretty easy and fast.
Seem interesting. Optimising prompts as well!
@parth_makwana07 Yes! Question is whether switching to a cheaper model, will lower your performance metric and then especially how much quality can you recover by optimizing the prompt for that model? Typically there is no time for manual prompt tuning of all sorts of model-prompt combinations. These type of questions we want to answer with promptic in order to finde the best cost-value fit.
Scoring quality and cost together is the right call. Most of our savings came from trimming what we sent the model, not switching models.
@salman_parvez Exactly. A cheaper model isn’t always the answer. Sometimes the biggest win is sending less context while keeping the quality your use case needs (prompt/context engineer).
the "benchmark against your own business metrics, not a generic eval" framing is the right one. we run a voice agent and the model that scores best on generic transcript quality benchmarks isn't always the one that keeps latency low enough for a live phone call, those two things trade off against each other in ways a leaderboard never shows. does Promptic let you weight latency as a hard constraint rather than just another scored dimension, or is it purely score-based ranking
@galdayan Very good question! In the Cost & Performance analysis view, you can set thresholds for allowed latency to find best cost-performance fit given that constraint. Yet we unfortunately do not support realtime voice models with all features. You could explore with the free plan if it fits your need. TTS and STT architectures should work fine.