The Problem: You're using ChatGPT because everyone does. But is it actually the best AI for YOUR specific task? Maybe Claude writes better code. Maybe Llama is 10x cheaper and just as good.
The Solution: PromptPerf lets you test your actual prompts across 100+ AI models simultaneously and shows you which one wins.
āØ What's New in 2.0:
ā¢ 100+ models
ā¢ Speed benchmarks
ā¢ Ready-to-use templates
ā¢ Save tests and re-run when models update
Thank you everyone for the support. I have received nearly 40 signups and 1 paid user which is massive for me as I am still on the early stages of validating the product. So thank you everyone. Next steps: Even though the signups are coming in I am tracking the usage of the app and I dont see many users running the evaluations and I need help. How should I get you to try and test the product. My current thoughts are: - User using their API keys means friction and would go away from the platform as can't test it immediately so perhaps allow free trial with my API key which involves unlimited runs with 3 test cases.
- Create a onboarding guide? Like the ones of enterprise softwares that says "Click here" "Next Steps": What tool can I use for this?
For both the options above will still need to inform them about the updates and hope they will signin again.
- Reach out to all 40 users for a 1-1 15 mins call and show them the product. Assuming 30% respond that 12 calls booked. Do you have any suggestions? Keen on feedback. This is critical as I need to solve these issues before building next features i.e (Adding more models and multi model runs).
PromptPerf lets you test a prompt across GPT-4o, GPT-4, and GPT-3.5 and compares results to your expected output using similarity scoring.
Models change fast. Prompts break. This helps you stay ahead.
Unlimited free runs. More models coming soon.
