Evaluate your AI applications with Braintrust: the enterprise-grade stack for building high quality AI products. From experiment tracking, to prompt playground, to data management, we take uncertainty and tedium out of shipping AI.
Thanks so much for hunting us @rrhoover
We're excited to introduce Braintrust, a platform for running and tracking AI evaluations (“evals”) [1].
At my previous startup Impira and leading AI at Figma, we had this recurring problem where we never knew if changes we made to our products would improve or regress key user scenarios. We built some tooling to solve this problem and after talking to other developers learned that it was a widespread issue. Specifically, it’s challenging to establish a great dev loop that lets you systematically improve and ship high quality AI products.
We worked with the teams at Zapier, Coda, and Replit to refine Braintrust. We consistently heard that they were facing challenges with evaluation, so we built Braintrust to help them. Today we’re releasing the product for anyone to use — including a free plan [3]
There are a lot of LLM tooling products on the market. Here are a few ways Braintrust is different:
- Rather than showing eval metrics in your observability tools, Braintrust offers an “experiment tracking” workflow, meaning you can try out changes while developing them, and drill down into diffs between other experiments and git branches before you ship. Check out our docs [3] for more details.
- We believe strongly that you should own your data and support on-premises and private VPC deployments.
- We natively and equally support Typescript and Python.
- We have a flexible free plan for builders and an unlimited free plan for academic and non-commercial open source projects [3]. We will introduce a self-service paid (”Pro”) tier, hopefully with feedback from this community.
Our mission is to enable developers to build high quality, reliable AI products. We couldn’t be doing this without Elad Gil, who helped me incubate the initial idea and team, which today includes founding designer, Coleen Baik, and founding engineer, Manu Goyal. Also big thanks to David Song, from Elad’s team, who is also helping us.
We’re excited to launch today [4], but we know there’s a lot left to build and are excited to hear your feedback.
[1] https://www.braintrustdata.com
[2] http://www.braintrustdata.com/do...
[3] https://www.braintrustdata.com/p...
[4] https://www.braintrustdata.com/b...
About Braintrust on Product Hunt
“Rapidly ship AI without guesswork”
Braintrust launched on Product Hunt on September 29th, 2023 and earned 176 upvotes and 52 comments, placing #9 on the daily leaderboard. Evaluate your AI applications with Braintrust: the enterprise-grade stack for building high quality AI products. From experiment tracking, to prompt playground, to data management, we take uncertainty and tedium out of shipping AI.
On the analytics side, Braintrust competes within Developer Tools and Artificial Intelligence — topics that collectively have 980.9k followers on Product Hunt. The dashboard above tracks how Braintrust performed against the three products that launched closest to it on the same day.
Who hunted Braintrust?
Braintrust was hunted by Ryan Hoover. A “hunter” on Product Hunt is the community member who submits a product to the platform — uploading the images, the link, and tagging the makers behind it. Hunters typically write the first comment explaining why a product is worth attention, and their followers are notified the moment they post. Around 79% of featured launches on Product Hunt are self-hunted by their makers, but a well-known hunter still acts as a signal of quality to the rest of the community. See the full all-time top hunters leaderboard to discover who is shaping the Product Hunt ecosystem.
Ryan Hoover