Airtrain.ai LLM Playground

Vibe-check many open-source and proprietary LLMs at once

407 followers

Vibe-check many open-source and proprietary LLMs at once

407 followers

Visit website

Data analysis tools

•

AI Coding Agents

•

AI Metrics and Evaluation

A no-code LLM playground to vibe-check and compare quality, performance, and cost at once across a wide selection of open-source and proprietary LLMs: Claude, Gemini, Mistral AI models, Open AI models, Llama 2, Phi-2, etc.

Airtrain.ai LLM Playground gallery image

Free Options

Launch tags:Developer Tools•Artificial Intelligence•Data Science

Launch Team

Framer AI AgentsDesign and publish professional sites with AI

Promoted

Airtrain.ai LLM Playground

Maker

📌

Hello Product Hunt community! 🚀 We're very proud to introduce the Airtrain.ai LLM Playground, a no-code tool to prompt many open-source and proprietary LLMs at once: Claude, Gemma, GPT-4, Llama 2, Gemini, Phi-2, Mistral models, and more. Compare quality, cost, and performance. We built this playground to help AI enthusiasts and practitioners of all stripes easily “vibe check” popular LLMs. Key features include: 📌 Prompt multiple models at once 📌 18 models supported (8 open-source, 10 proprietary) 📌 Inference metrics (i/o token counts, throughput, inference cost) 📌 Persisted sessions (review and resume previous chat sessions) We'd love for you to try it out and share your feedback with us. Feel free to ask any questions, and we'll be more than happy to answer them. Thanks so much for your support, and we hope you enjoy using the LLM Playground! ✨

Report

2yr ago

Emmanuel and team, congratulations on the launch! I have to say, it is so very cool to be able to query Claude 3, GPT-4, and Gemini all at once. What inspired you to build this?

Report

2yr ago

Airtrain.ai LLM Playground

Maker

@joy_larkin We heard from a lot of users that they wanted to try other models than the OpenAI ones but they did not know where to start. It starts with evaluation, and evaluation starts with a vibe check. Running a handful of prompts through various models to build an intuition about their relative quality, cost, and performance, before moving to batch evaluation across an entire test dataset (which we also offer :)

Report

2yr ago

LangWatch

That’s very interesting, I really like the concept of “vibe checking”. Is it possible to upload many samples I may have or run multiple times at once for a prompt? Since due to temperature responses vary right

Report

2yr ago

Airtrain.ai LLM Playground

Maker

@rchaves Yes, to do this, go back to the task menu (New task in the top bar) and select "Evaluate Models". You can upload a CSV or JSONL file of up to 10k examples and configure the models you want to test and the metrics you care about. See docs here: https://docs.airtrain.ai/docs/ba...

Report

2yr ago

Startup Digest

Congrats on the launch! Curious, what open source models are you seeing get the most usage on the platform?

Report

2yr ago

Airtrain.ai LLM Playground

Maker

@mccannatron Mistral and Llama 2 are neck to neck. Although Mistral includes 2 open-source and 3 proprietary.

Report

2yr ago

Super interesting! Gotta say I love the inference metrics -- makes it way easier to compare costs than what I've been doing. Claude 3 is so pricey!

Report

2yr ago

Airtrain.ai LLM Playground

Maker

@will_jacob Indeed Opus is quite expensive! Luckily Sonnet is still close to GPT-4 level but at a more reasonable price. It'll be interesting to have a chance to play with Haiku when Anthropic releases it, since that should be much more affordable.

Report

2yr ago

Airtrain.ai LLM Playground

Maker

@will_jacob yeah those jumbo proprietary models are luxury!

Report

2yr ago

This is awesome. Really like the broad coverage across both OS and closed LLMs. Good luck with the launch!

Report

2yr ago

Airtrain.ai LLM Playground

Maker

@alex_li_mvp thanks! We noticed that not a lot of other tools allow you to have both of those side-by-side

Report

2yr ago

Airtrain.ai LLM Playground

Maker

@alex_li_mvp Thank you Alex!

Report

2yr ago

Kniru: AI-Powered Finance

Hi @emmanuel_turlay I love Airtrain.ai's comprehensive playground for LLM comparison! It's a brilliant tool for developers and AI enthusiasts like me :) Have you considered integrating a community-driven feature where users can share their findings or best practices with specific models? I think this would help with the platform's utility and have shared insights.

Report

2yr ago

1 2 3 4

Reviews