Introducing Embedefy 🎉 Embeddings for everyone.
A couple of months ago, I was working on an AI side project aimed at finding relevant information in a PostgreSQL database based on user prompts. My goal was to combine this information with the user prompts and send it to a Language Model (LLM) for generating accurate and up-to-date responses. This technique is known as Retrieval-Augmented Generation (RAG).
After using OpenAI's text-embedding-ada-002 model for a few days, I quickly realized that, given the millions of records in my datasets, I would have to spend tens of thousands of dollars. Additionally, I learned that OpenAI doesn't maintain their models indefinitely, which means I would need to regenerate embeddings once they retire their models. Therefore, it wasn't a viable option for me.
After conducting some research, I decided to migrate everything to open-source embedding models using Hugging Face's Transformers library. However, I then faced another challenge: my app runs on a serverless platform with strict runtime limitations, which made generating a large number of embeddings impossible. To overcome this, I created an API service, enabling the creation of as many embeddings as needed. This service was built leveraging awesome open-source tools and cost-effective platforms, resulting in an extremely affordable API service for providing embeddings.
As a firm believer in the democratization of AI, I’m excited to share this service with the wider AI community. Whether you’re operating on a tight budget or managing an app on resource-limited systems, this offers a viable option for you. So, what are you waiting for? Go build something!
P.S. I must say that this project has been made possible by the awesome Transformer.js library by Xenova, the lightning-fast NATS platform by Synadia & NATS community, and the magical IPv6 network of Fly.io.
Replies
ReviewGPT