Introducing Hindsight™, a new approach to agent memory. Best in the world on benchmarks. Best in production for your agents.
This is the 5th launch from Vectorize. View more
Hindsight Models and Inference API
Launching today
Hosted models for the steps around your LLM: reranking, embeddings and typed decisions. Point your Cohere or OpenAI client at inference.hindsight.vectorize.io and keep your code. hs-rerank-1 and hs-embed-1 are the most accurate small open models on our benchmarks and run fast on a CPU; hs-decide-1-flash answers yes/no, pick-one and how-likely questions in 167 ms, instead of an LLM call. From $0.005 per 1M tokens, $5 free credit, open weights on Hugging Face.
Free Options
Launch Team
Framer AI AgentsDesign and publish professional sites with AI
Promoted
Hi Product Hunt 👋 Chris from Vectorize here.
We build Hindsight, an open-source memory system for AI agents. A memory system is full of small judgments: is this fact new? Which of these memories matter? Is this evidence enough to answer? We used to send every one of them to an LLM: seconds of latency and a bill for every token, just to get a yes or no back.
So we trained small models for those steps. Today we're releasing them as Hindsight Models, with open weights on Hugging Face and a hosted Hindsight Inference API:
• hs-rerank-1 (114M): a reranker, drop-in for Cohere Rerank. More accurate than gte-reranker-modernbert-base, and 6x faster on a CPU.
• hs-embed-1 (112M): embeddings, drop-in for OpenAI Embeddings. 16.5 ms per query on a CPU.
• hs-decide-1-flash (0.8B): a decision model. Ask it typed questions (pick one, how likely, where on a scale) and get probabilities back in about 167 ms. Rerank with it and it also drops the documents that don't help.
Inside Hindsight, the decision model made memory up to 2.5x cheaper, with better answers from six of the seven LLMs we tested. Every number and the method behind it is in the launch post and on benchmarks.hindsight.vectorize.io.
You pay for input tokens only, from $0.005 per 1M, and new accounts get $5 of free credit. Try every model in the Playground before you write any code, or run the open weights yourself.
What would you use a decision model for? We're here all day to answer questions.
Launch post: https://hindsight.vectorize.io/b...
Congrats! How well does hs-decide-1-flash generalize beyond agent memory—for example, to support ticket routing? Would a clear prompt and a few examples be enough to get started, or would it need fine-tuning?
@stan_xu1 It does generalize and scored competitively on benchmarks for its size. However, we mostly focused on training it for the decisions that Hindsight needs to make.
can imagine this'd be useful as my token consumption climbs every month
the typed decision model is the part i'd try first
there are plenty of small yes or no steps in a workflow where waiting for a full chat response feels excessive
@saad_el_gueddari Yes, this is exactly where you will want to use a decision model.