The pinned dataset version detail stood out to me — replaying the exact same rows, split by conversation rather than by row, is what would make me trust a before/after comparison. The Qwen 0.8B example going from 36.7% to 84% on message classification is striking. How do you guard against the eval set going stale as the agent's tools or prompts change underneath it? Do you version the tool and prompt snapshot alongside the dataset, or flag cases that no longer match the current agent setup?
multiple years of AI experience told me turn any data into training data is very hard tbh. not data format itself, how to define the label, how to collect enough signal, which data is good to use through hundreads of thousands of table, also you need to have a really deep understanding for the business domain as well. Also curious how you handle entire mlops, how to minotor the drifting and model retraining, and infrerence cost
Happy launch team! Sounds like interesting project, I would like to check it out. But why not adding more information in the description field? Most people here don't really check the screenshots or videos ;)
As a marketer, I'm not new to using AI, but I am new to actually building and running AI agents. Overmind seems like it would be able to help me get there much quicker than I anticipated, and without hiring an entire ML team, too 😁 Best of luck with the launch!
Follow you guys for some time and this one looks crazy! how do you know the small model's good enough to swap in for what I’m using right now? do I guess right you test it against the current one?
The idea of turning production traces into a model for a specific task is interesting. Teams already have a lot to learn from how their agents perform in the real world. Congrats on the launch!
Hey team, this sounds crazy, I always had this mental friction in my head that training model is a tedious task, and needs some decent technical bandwidth. But this sounds like it'll remove that friction.
Fine tuning on the traces your agent already produces is the most honest training data there is, and shipping it as a Cursor plugin means I can try it without wiring up a whole new pipeline first.
hey PH, pritam here, one of the founding engineers at overmind 👋
the part i've been living in is fine-tuning and evals. the thing that surprised me most building it: the eval is what makes the training worth anything. we had a run recently where every model, big and small, scored ~67% on a yes/no task. no amount of tuning fixes that, the data had no signal. overmind showed it in the before/after numbers instead of letting us ship a "fine-tuned" model that was really a coin flip.
the happy path looks more like our support demo: qwen 3.5 0.8b went from 36.7% to 84.0% on messages it had never seen, in 23 minutes from launch to live.
if you've got an agent in prod making the same kind of call over and over (routing, classification, tool choice), that's where i'd start. ask me anything about how the evals work
The versioned eval sets are a useful part of this. I'd want to be able to replay the same cases when a new model starts behaving differently.
This is legitimately really cool! Agent training, evals, observations, etc is such a fragmented and clunky process and I loveee what Overmind has done to make so much simpler! 🚀
Heyyy PH, Frey here, founding engineer at Overmind.
When most teams hear "build your own AI", they picture a research lab, a GPU budget and a year of hiring. We think it should feel more like flipping a switch. So we built one open-source platform that watches every agent you have in production, learns from your own data, trains models made for those agents and keeps making them better well after launch. All of it runs on your own infrastructure, so your data never leaves your hands. The traffic you see today turns into tomorrow's training set, and your agent gets a little better every day.
We kept one test in our heads the whole time: can someone who has never fine-tuned a model go from one prompt in their coding agent to their own model in production? Making the answer yes is what the whole team spent the last few months on.
It is all open source, so if something looks off, open an issue or come find me in the comments. Happy to go as deep as you want on the technical side!
Toodaloo 👋
Super happy to be launching Overmind today 🔥 want to get started? Paste this onboarding prompt into the coding agent of your choosing and let it work:
Overmind is the open-source platform for AI agent training. What we're working towards is giving every product its own AI lab. 🚀
Most AI products run on the same few frontier models. A frontier lab works differently, collecting data, building evals, training, testing. Ship and repeat.
We wanted any team building an agent to be able to run that same loop on its own data, without hiring an ML team or building infrastructure.
To get started: Connect our MCP, and paste one prompt into Cursor, Claude Code, Codex or OpenCode.
What you get:
🔸 Tracing: OpenTelemetry traces, scored as they arrive and matched to the part of your agent that produced them
🔸 Datasets: production traces turned into versioned eval and training sets
🔸 Evals: you define what "good" means for each capability, and Overmind measures it on live traffic
🔸 Optimisers: prompt, tool and code experiments run in your repo, and the winner comes back as a git diff
🔸 Fine-tuning: small specialised models trained on your data and benchmarked against the model you run today, with weights you can download and export
🔸 Inference: your own models and frontier models behind one OpenAI-compatible API
Why are we doing this? 💭
Today, millions of developers can build with AI, but creating and improving the models behind it still requires expertise concentrated in relatively few teams.
We believe that gap will close. Every application, business, device and individual has different needs. Meeting those needs will require more people to be able to create, test and run specialized models.
Computing became widespread as processors became cheap enough to build into almost anything, and development tools made them useful to far more people. We believe intelligence needs a similar expansion in who can build it and where it can run.
Overmind is our contribution to that: putting the work of an AI lab into tools any developer can use, so they can create, own and improve intelligence for their own tasks.
Our mission is to make intelligence as easy to create as it is to consume.
The team is here all day. What would you train first?
The humans at Overmind ❤️ 🌎
About Overmind on Product Hunt
“Open-source platform for continuous learning”
Overmind launched on Product Hunt on October 27th, 2026 and earned 241 upvotes and 57 comments, earning #2 Product of the Day. Overmind is the model training platform for AI teams. Turn your production traces into specialised models you own, trained, benchmarked, and served.
Overmind was featured in Developer Tools (521.9k followers), Artificial Intelligence (482.2k followers) and GitHub (41.5k followers) on Product Hunt. Together, these topics include over 255.4k products, making this a competitive space to launch in.
Who hunted Overmind?
Overmind was hunted by fmerian. A “hunter” on Product Hunt is the community member who submits a product to the platform — uploading the images, the link, and tagging the makers behind it. Hunters typically write the first comment explaining why a product is worth attention, and their followers are notified the moment they post. Around 79% of featured launches on Product Hunt are self-hunted by their makers, but a well-known hunter still acts as a signal of quality to the rest of the community. See the full all-time top hunters leaderboard to discover who is shaping the Product Hunt ecosystem.
Want to see how Overmind stacked up against nearby launches in real time? Check out the live launch dashboard for upvote speed charts, proximity comparisons, and more analytics.
fmerian
The pinned dataset version detail stood out to me — replaying the exact same rows, split by conversation rather than by row, is what would make me trust a before/after comparison. The Qwen 0.8B example going from 36.7% to 84% on message classification is striking. How do you guard against the eval set going stale as the agent's tools or prompts change underneath it? Do you version the tool and prompt snapshot alongside the dataset, or flag cases that no longer match the current agent setup?