OpenSauced

Amazing Work @bdougieyo @nickytonline @bekahhw and team! I showed this to a few product managers at my company and they loved it. Opens new possibilities for non-tech folks to estimate productivity and maybe even hire through open source instead of the standard coding interviews!

A few months ago, @bdougieyo came to me with what sounded like a big, crazy, amazing idea 🤔 We have ALL this public unstructured GitHub data. We've ramped up our data-pipelines to operate at GitHub's incredibly huge scale. We've even invented nuanced and unique metrics that can give very good insights into open-source communities. But what if we enhanced those mechanisms to work with AI capabilities? ✨ StarSearch is the result of months of work experimenting, living on the bleeding edge of AI technologies, and bringing a huge scale AI system to production. Ask StarSearch about projects you care about, things you're trying to work on, the Lottery Factor, and see what sorts of unique discoveries you can make! 🔧 I built the backend and data pipelines that make all this possible: StarSearch is a complex, multi-agent RAG (retrieval-augmented-generation) that has access to droves of our pristine GitHub data and other services. Several micro-services working together make this all possible: the "pizza-time" service consumes realtime GitHub events from the public GitHub events API. This populates the metrics, histograms, and time-series charts you see today on OpenSauced. Next, we have something called the "star-search-embedder" that looks for rows of data in our database that match the criteria we've set. It takes batches of those rows and embeds them into our vector store using a text embedding model. But first, we generate an AI summary on this unstructured texts in order to scrub them of any "messy" metadata or stuff we don't really need to embed. This is accomplished at-cost using a few GPUs on Kubernetes, vLLM, and open-source AI models. Then, within our API, we use a multi-agent architecture for our LLMs to enable a powerful question and answer engine. The agents have access to several of our other services and can "call" each-other to get necessary information from one agent to the other. For example, our internal "Bing Search AI Agent" knows how to interface with the Azure Bing search service, can generate summaries on mass amounts of raw, unstructured search results, and can hand those back to a calling agent that need made the query. More importantly, these agents can perform a vector-search on all those vectors we've embedded through our data pipelines. Today, we have over 100m+ vectors that power this engine and enable StarSearch to search through pull requests and issues. 🚀 One of my favorite queries to use is a sort of "needle in the hay-stack" search. Let's say I'm working at a product team shipping a Kubernetes platform. And I need to upstream some changes to how persistent storage volumes are claimed in the upstream kubernetes/kubernetes project: that's a very, VERY heavy lift if you've never worked in that community before! Where to even start!? What groups are involved? Who would be best to reach out to for feedback in the community? Where is the prior art? Asking StarSearch questions like "Who has worked on persistent volume claims in kubernetes/kubernetes?" can start to give you an idea of who those people might be, unlocking you to gain community support on a huge undertaking!! 🙏 Would love to hear any and all feedback!! Thanks all and stay saucey!

Huge! Excited for what insights we can pull as well as searching for the right people when leveraging a new open source package/project 🙌

Congrats on the launch @bdougieyo ! I just queried for some of the top contributors for one repo I was interested in and got good insights. I like how it not just shows the list, but a glimpse of their contributions, and an insight at the end

Killer!!! No more manual comparison shopping for libraries ever again!!

I normally spend hours in my week hunting for guests on CodingCat.dev podcast. Now I can find people so much easier. For example I gave this prompt today: "Who can I interview on my podcast CodingCat.dev about typescript projects that are trending" And got back: Here are several contributors to trending TypeScript projects that would make excellent guests on your podcast: @anonrig: Project: getsentry/sentry Contribution: Added more TypeScript rules to improve type checking and code quality. Summary: Anonrig submitted a pull request to enforce stricter TypeScript rules in the Sentry codebase, adding the @types/jest and ts-jest packages, and creating a new test suite. @Lightning00Blade: Project: puppeteer/puppeteer Contribution: Improved test reliability and added TypeScript types. Summary: Lightning00Blade worked on reducing test flakiness and adding TypeScript types, improving the development experience for those using TypeScript with Puppeteer. @ScreamZ: Project: amannn/next-intl Contribution: Updated documentation for TypeScript usage. Summary: ScreamZ enhanced the next-intl project documentation, offering better guidance on how to use TypeScript within the project. @jakebailey: Projects: microsoft/TypeScript-Website microsoft/TypeScript Contributions: Forced TypeScript to emit .d.ts files and linked changes with Microsoft's monaco-editor. Updated dependencies, switching back to TypeScript's bundler from esbuild. Summary: Jake Bailey has been active in both updating dependencies and linking TypeScript changes, focusing on improving the TypeScript development environment and its tooling. You can reach out to these contributors to share their insights and experiences on working with TypeScript on trending repositories.

Starsearch is a pretty cool feature! I can definitely see myself using it in the future when I want to maintain my own projects. Congratulations on the launch @bdougieyo