Launched this week
The Grid
Spot market API for cheaper LLM inference
24 followers
Spot market API for cheaper LLM inference
24 followers
The Grid turns LLM inference into a live spot market where suppliers bid to serve requests in real time. Teams swap a few lines of code to buy model output at market price with tier-based quality guarantees.
Facts
From the description, I was under the impression it is formed around a dynamic pool of inference providers, but it looks like it's more focused on aliasing different classes of model capability under specific tier and bundling them.
I'm actually using inference APIs quite a lot and I'd be interested if there'd be an inverse product - specific model and matching the best price/quality inference for it (for open weights, of course).
Anyone else dealing with unpredictable LLM costs lately? This spot pricing idea feels like a different approach.