Launching today
Overmind
Open-source platform for continuous learning
269 followers
Open-source platform for continuous learning
269 followers
Overmind is the model training platform for AI teams. Turn your production traces into specialised models you own, trained, benchmarked, and served.
Heyyy PH, Frey here, founding engineer at Overmind.
When most teams hear "build your own AI", they picture a research lab, a GPU budget and a year of hiring. We think it should feel more like flipping a switch. So we built one open-source platform that watches every agent you have in production, learns from your own data, trains models made for those agents and keeps making them better well after launch. All of it runs on your own infrastructure, so your data never leaves your hands. The traffic you see today turns into tomorrow's training set, and your agent gets a little better every day.
We kept one test in our heads the whole time: can someone who has never fine-tuned a model go from one prompt in their coding agent to their own model in production? Making the answer yes is what the whole team spent the last few months on.
It is all open source, so if something looks off, open an issue or come find me in the comments. Happy to go as deep as you want on the technical side!
Toodaloo 👋
@freyam7 toodaloo!
@freyam7 looks amazing. Great work team and congrats on big launch momentum already.
@hissingkitty Thanks!! The team can't wait to see everybody start "creating" AI rather than just consuming it!
@freyam7 @hissingkitty thanks Christine! just getting started :)
Hey team, this sounds crazy, I always had this mental friction in my head that training model is a tedious task, and needs some decent technical bandwidth. But this sounds like it'll remove that friction.
@priyansh_agrawal2 Thanks Priyansh! That friction is what we built Overmind to remove - your coding agent handles the data prep and training for you so don't need ML experience to get a trained, smaller (and very often better) model out. Anything you can think in Shadow where Overmind would help?
Hey @priyansh_agrawal2 ! That friction is exactly what we were chasing! Your coding agent does the ML team’s job: it turns your traces into evals, trains a small model and benchmarks it against what you run today. One prompt to start: "Set me up on Overmind https://console.overmindlab.ai/setup.md". Curious what you would train first for Shadow 🙌
Hey Product Hunt 👋
I'm Tyler, co-founder and CEO of Overmind.
Overmind is the open-source platform for AI agent training. What we're working towards is giving every product its own AI lab. 🚀
Most AI products run on the same few frontier models. A frontier lab works differently, collecting data, building evals, training, testing. Ship and repeat.
To get started: Connect our MCP, and paste one prompt into Cursor, Claude Code, Codex or OpenCode.
What you get:
🔸 Tracing: OpenTelemetry traces, scored as they arrive and matched to the part of your agent that produced them
🔸 Datasets: production traces turned into versioned eval and training sets
🔸 Evals: you define what "good" means for each capability, and Overmind measures it on live traffic
🔸 Optimisers: prompt, tool and code experiments run in your repo, and the winner comes back as a git diff
🔸 Fine-tuning: small specialised models trained on your data and benchmarked against the model you run today, with weights you can download and export
🔸 Inference: your own models and frontier models behind one OpenAI-compatible API
Why are we doing this? 💭
Today, millions of developers can build with AI, but creating and improving the models behind it still requires expertise concentrated in relatively few teams.
We believe that gap will close. Every application, business, device and individual has different needs. Meeting those needs will require more people to be able to create, test and run specialized models.
Computing became widespread as processors became cheap enough to build into almost anything, and development tools made them useful to far more people. We believe intelligence needs a similar expansion in who can build it and where it can run.
Overmind is our contribution to that: putting the work of an AI lab into tools any developer can use, so they can create, own and improve intelligence for their own tasks.
Our mission is to make intelligence as easy to create as it is to consume.
The team is here all day. What would you train first?
The humans at Overmind ❤️ 🌎
Follow you guys for some time and this one looks crazy! how do you know the small model's good enough to swap in for what I’m using right now? do I guess right you test it against the current one?
Spot on, Art! Every small model we train gets benchmarked against the model you run today, on the exact same frozen set of your own production traces, scored on the evals you defined for what "good" means. You see the before/after numbers side by side, so the swap only happens when the small model actually wins. And when it does not, Overmind tells you that too, instead of letting you ship a coin flip :D Easiest way to see it is on your own agent: one prompt in your coding agent, "Set me up on Overmind https://console.overmindlab.ai/setup.md".
Thank you for following along all this time 🙌
The versioned eval sets are a useful part of this. I'd want to be able to replay the same cases when a new model starts behaving differently.
Hey @oleksii_sekundant 👋 That's exactly what we built for. You turn your real production traffic into a test set and freeze it. Then when a new model comes along (a frontier one or one you fine-tuned with us), you replay the same cases on it next to your current model. You see what got better and what broke before your users do. We'd love for you to point Overmind at one of your agents and tell us how it feels.
@oleksii_sekundant adding to frey's answer: the eval set is pinned to a dataset version, so when you replay it on a new model you're comparing on the exact same rows, not "roughly the same traffic". we also split by conversation rather than by row, so a model can't score well just because it saw other turns of the same chat in training
hey PH, pritam here, one of the founding engineers at overmind 👋
the part i've been living in is fine-tuning and evals. the thing that surprised me most building it: the eval is what makes the training worth anything. we had a run recently where every model, big and small, scored ~67% on a yes/no task. no amount of tuning fixes that, the data had no signal. overmind showed it in the before/after numbers instead of letting us ship a "fine-tuned" model that was really a coin flip.
the happy path looks more like our support demo: qwen 3.5 0.8b went from 36.7% to 84.0% on messages it had never seen, in 23 minutes from launch to live.
if you've got an agent in prod making the same kind of call over and over (routing, classification, tool choice), that's where i'd start. ask me anything about how the evals work
@pritamsonihsr The eval wizard has spoken 🧙♂️🙌
The idea of turning production traces into a model for a specific task is interesting. Teams already have a lot to learn from how their agents perform in the real world. Congrats on the launch!
@marianna_tymchuk Thank you Marianna! Your agents are already telling you what good looks like, we just help you listen 😄
@marianna_tymchuk Thank you Marianna!! Yes how your agents behave in the real world is the most data you've got, treasure it, and use it to train more efficient smaller models! Simply start from your own data and one prompt in whatever coding agent you're using. Are you running any agents in Stripo atm?