Hi Product Hunters! 👋
I'm happy to show you our vision for the future of app development - GPT Pilot beta.
🤔 Why we decided to build GPT Pilot?
After talking to over 200 users of our previous project Pythagora, there was one feedback that kept popping up. That many developers are more prone to trusting the code that’s generated from tests than the test that’s generated from code.
When you look at it closely, an automated test is nothing more than a description of what needs to happen and since LLMs process natural language so good, we decided to create a tool that would create features from a developer’s prompt.
So, we sketched how could that work and the idea behind GPT Pilot is that it serves as a blueprint of how an app development process looks like. For example, when you start working on a feature, you need to first break down the feature into smaller steps, then write initial code, run it, look at the logs, debug the code, etc.
LLMs (eg. GPT-4) can write code amazingly well. However, writing code is only 1 part of creating an application. To fully build an app, LLMs need to be able to
- write code
- run it
- read the output
- add logs around the code to better understand an issue
- debug the code
- etc.
GPT pilot provides a way so that LLMs can interact with the machine resources like the ones listed above so that AI can, not only generate code, but do real work of app development.
🏗️ How GPT Pilot works?
Here are the steps GPT Pilot takes to create an app:
1. You enter the app name and the description
2. Product Owner agent asks a couple of questions to understand the requirements better
3. Product Owner agent writes user stories and asks you if they are all correct (this helps it create code later on)
4. Architect agent writes up technologies that will be used for the app
5. DevOps agent checks if all technologies are installed on the machine and installs them if they are not
6. Tech Lead agent writes up development tasks that Developer will need to implement. This is an important part because, for each step, Tech Lead needs to specify how the user (real world developer) can review if the task is done (e.g. open localhost:3000 and do something)
7. Developer agent takes each task and writes up what needs to be done to implement it. The description is in human-readable form.
8. Finally, Code Monkey agent takes the Developer's description and the existing file and implements the changes into it. We realized this works much better than giving it to Developer right away to implement changes.
🔮 Vision behind GPT Pilot
We see how LLMs can code amazingly well. However, they are not 100% perfect so developers will be needed until we get to AGI.
We think that now, developers can act as tech leads who are reviewing the code that LLMs generate instead of coding the entire day. With this, we're imagining a future in which developers can create full production-ready apps in days rather than in months.
📖 Additional resources
👨💻 [Github repo](https://github.com/Pythagora-io/...)
📝 [Technical blog post explaining how GPT Pilot works](https://blog.pythagora.ai/2023/0...) [part 1/3]
📝 [Technical blog post explaining how GPT Pilot coding works](https://blog.pythagora.ai/2023/0...) [part 2/3]
### Finally, since GPT Pilot is open source, we’d be so grateful if you could star GPT Pilot repo on Github - https://github.com/Pythagora-io/....
After a year of iterating on 4 different test automation products, I believe we finally hit the market need so I’m super excited to present to you Pythagora -
an open-source dev tool that creates automated integration tests by analyzing server activity without you having to write a single line of code!
@leon_ostrez now, GPT Pilot sounds very cool! I really dig how each 'agent' has its own role in simplifying the whole process, especially easing the devs' load. I'm going to chat about this with our team, bet they'll find it super interesting, especially considering the unique agent-based approach you’ve adopted. By the way, did Code Monkey ever surprise you by generating code in a way you didn’t initially anticipate?
Cheers on the launch, can’t wait to dive deeper into GPT Pilot!
@mocarg thank you! I'm really interested to hear what you will think about it. There is definitely a lot of work ahead of us but we are very impressed how it works so far. Well, most of the time it sticks to best practices, at least that is our goal :)
I'm genuinely impressed with GPT Pilot! After trying it out, I can't help but feel excited about its future developments and where it's headed.
Regarding the underlying LLMs, are they fixed, or do users have the option to choose between different models, such as ChatGPT3.5, ChatGPT4, or LLAMA?
Thank you so much @tvrtko_sternak! At the beginning we focused on GPT-4 and we were using function calling (feature specific to openai). Gpt3.5 had a lot worse performance so we definitely don't recommend it but if you want to try it out feel free to do so. Last week we removed function calling so that we can try any LLM but we just didn't have time and resources to start testing them... If you do try them please let us know how it worked for you!
Report
Wow! This project looks really impressive!
I have a question regarding the "plan for development" phase - do you create unit tests for each planning step?
Thank you @matijash ! I have to say this is by far most complex project I have worked on so there were many challenges... One on top of my head is how to make recursive conversations, for debugging for example, but keeping context small. But I think real challenges start now in finding more ways to improve and optimize gpt-pilot
Report
This is a really awesome project, and it will be helpful for creating micro apps at blazing speed. The good thing about this project is it includes a testing phase. I'm planning to try out some cool ideas on the weekends. Could you let us know about the token consumption of the apps you have developed?
Wow. I am interested in the potential time it could save for developers who are building projects to scratch their itch. Is there a specific programming language or tech stack that works best with GPT Pilot? @leon_ostrez
Very impressive!
I knew what Zvonimir and the team were working on and was still floored when I saw the demo.
The ability to basically write a tech spec from user requirements, then plan the project and then execute it step by step is a game changer, and a definitive difference from other "code-with-ai" tools out there.
I also like that it shows you exactly what it's doing along the way (so you know why it does what it does), and that it asks you to allow it to run commands (I'm wary of just giving some AI full control of my terminal).
Amazing work so far, looking forward to using this and seeing you grow the product!
Thank you @senko_rasic for kind words. Yeah, we are definitely trying to be different than "code-with-ai" tools that you mentioned. Our vision is much bigger and setting up architecture was big part of gpt-pilot, but still there are many things we plan to do to improve it.
Replies
Pythagora
Videobolt
Pythagora
Pythagora
Scade.pro
Pythagora
Pythagora
Pythagora
Open SaaS
Pythagora
Pythagora
Zintlr
Pythagora
Pythagora
A Web Whiteboard
Pythagora
Pythagora
Pythagora