Braintrust

Rapidly ship AI without guesswork

5.0•5 reviews•

213 followers

Rapidly ship AI without guesswork

5.0•5 reviews•

213 followers

Visit website

Testing and QA software

Evaluate your AI applications with Braintrust: the enterprise-grade stack for building high quality AI products. From experiment tracking, to prompt playground, to data management, we take uncertainty and tedium out of shipping AI.

Free

Launch tags:Developer Tools•Artificial Intelligence

Launch Team

Render Sandboxes: a computer for every agentIsolated compute, persistent state and a real filesystem

Promoted

Product Hunt

Hunter

📌

We're seeing a massive shift in how some software products are built, which of course introduces new challenges not covered by incumbent infra. Braintrust is worth watching, founded by @ankur_goyal, former Head of ML Platform at Figma (by way of acquisition).

Report

3yr ago

chatgpt.js

@ankur_goyal @rrhoover why does it say "Featured on September 27th, 2023" already? If you hunt it it's automatic? (And if so, why is it set that way)

Report

3yr ago

Braintrust

Maker

@rrhoover Thanks for hunting us!!

Report

3yr ago

Braintrust

Maker

📌

Thanks so much for hunting us @rrhoover We're excited to introduce Braintrust, a platform for running and tracking AI evaluations (“evals”) [1]. At my previous startup Impira and leading AI at Figma, we had this recurring problem where we never knew if changes we made to our products would improve or regress key user scenarios. We built some tooling to solve this problem and after talking to other developers learned that it was a widespread issue. Specifically, it’s challenging to establish a great dev loop that lets you systematically improve and ship high quality AI products. We worked with the teams at Zapier, Coda, and Replit to refine Braintrust. We consistently heard that they were facing challenges with evaluation, so we built Braintrust to help them. Today we’re releasing the product for anyone to use — including a free plan [3] There are a lot of LLM tooling products on the market. Here are a few ways Braintrust is different: - Rather than showing eval metrics in your observability tools, Braintrust offers an “experiment tracking” workflow, meaning you can try out changes while developing them, and drill down into diffs between other experiments and git branches before you ship. Check out our docs [3] for more details. - We believe strongly that you should own your data and support on-premises and private VPC deployments. - We natively and equally support Typescript and Python. - We have a flexible free plan for builders and an unlimited free plan for academic and non-commercial open source projects [3]. We will introduce a self-service paid (”Pro”) tier, hopefully with feedback from this community. Our mission is to enable developers to build high quality, reliable AI products. We couldn’t be doing this without Elad Gil, who helped me incubate the initial idea and team, which today includes founding designer, Coleen Baik, and founding engineer, Manu Goyal. Also big thanks to David Song, from Elad’s team, who is also helping us. We’re excited to launch today [4], but we know there’s a lot left to build and are excited to hear your feedback. [1] https://www.braintrustdata.com [2] http://www.braintrustdata.com/do... [3] https://www.braintrustdata.com/p... [4] https://www.braintrustdata.com/b...

Report

3yr ago

AVI by True AI

Congrats on the launch! Just wondering how you're looking into the relevancy of the responses generated w.r.t the prompts themselves?

Report

3yr ago

Braintrust

Maker

@dr_raja_m_suleman Thanks for the question! We have an open source library (https://github.com/braintrustdat...) that helps you evaluate the quality of a response. But in practice, defining a good scoring metric takes iteration and engineering, and so we encourage users to evolve and improving their scoring methods as they find new & interesting datapoints. Let me know if that makes sense!

Report

3yr ago

Airtable

Braintrust has quickly become an essential platform for engineers on my team that are working on AI features. Given how hard it is to know precisely what LLMs are capable of, tools that allow engineers to be easily data driven is critical for ensuring product quality and preventing regressions!

Report

3yr ago

Braintrust

Maker

@raghavsethi It's been a joy collaborating with your team so far! Airtable is such a great product, and it's a real honor to help out.

Report

3yr ago

Morph

I like this challenge. We use AI for our product, morphdb.io , but I have huge issue of evaluation of AI we’re building. I want to try it and we provide AI infrastructure with our client using Morph as well. Your product seems help us to deliver!

Report

3yr ago

StartOS: Plug-and-Play Startup System

I've built user facing interfaces with AI backends before and I see exactly what Braintrust is doing! Clear pain point here, I've faced it way too many times! Awesome idea :)

Report

3yr ago

Braintrust has significantly optimized Coda's AI operations! Initially, we started with a rather manual process for GenAI evals, and with Braintrust, we've implemented a robust workflow that facilitates scalable evaluations. Moreover, their datasets feature has streamlined the management of benchmark datasets, thereby minimizing the workload on our AI team. This efficiency enables us to concentrate more on feature and model development tasks.

Report

3yr ago

@siuheihk +1, using Braintrust at Coda has sped our team way up. We now have a single link we can send to understand how a feature's AI is performing, not to mention what's working and failing.

Report

3yr ago

Braintrust

Maker

@siuheihk Thanks for the kind words! We have learned so much from collaborating with you. It's great to work with AI innovators :)

Report

3yr ago

1 2 3 4

5.0

Based on 5 reviews

Review Braintrust?

Reviews

All Reviews

Most Informative