Wispr Flow: Dictation That Works EverywhereStop typing. Start speaking. 4x faster.
ChorusGraph
Native agent runtime with semantic cache one pip install
16 followers
Native agent runtime with semantic cache one pip install
16 followers
ChorusGraph is a native agent graph runtime (Apache-2.0) — BSP scheduler plus a local semantic cache that skips redundant LLM calls when similar routing already happened. One pip install; not a LangGraph wrapper. this is an alternative with amazing benefits over LangGraph! https://insightitsgit.github.io/ChorusGraph/benchmarks.html see the numbers and never use langgraph to save your llm cost. https://insightitsgit.github.io/ChorusGraph/
pip install was painless and the semantic cache actually cut repeat routing calls on my agent loop, pretty noticeable drop in token spend.
@yamur9372291199 Thanks Yağmur — that’s exactly the loop we built for. Glad install was clean and the cache showed up on repeat routing / tokens.
If you’re curious what’s hitting vs missing on your traffic, Route Ledger logs the scores per hop — or chorusgraph-audit on a query log if you want a cold estimate. Always interested in which hops still miss when they “feel” like repeats.
How does the semantic cache actually detect similarity across different prompt phrasings, and what's the hit rate you're seeing in real workloads versus the benchmarks?
@miray1419291 Two pieces: how phrasing matches, and what hit rates we actually publish.
Similarity across phrasings — PrismCache doesn’t do string equality. Incoming text is embedded, then a two-stage cosine gate:
Coarse on a 64-d projection (default ≥ 0.88) to recall candidates
Verify on the full embedding (default / measured ≥ 0.95, stricter on high-risk hops)
Paraphrases that mean the same thing land close enough to pass both; small entity/value flips that change the answer often fail verify (or never enter semantic keying — those hops use fingerprint / no_cache). Hits + scores land in the Route Ledger.
Hit rates — what we publish is Azure paired benchmarks (real Gemini, seed 42), not anonymized production fleets yet:
Scenario
Mid n=100
Heavy n=300
Finance single
~52%
~50%
Finance multi
~40%
~35%
Healthcare single
~60%
~73%
Healthcare multi
~51%
~79%
Real workloads swing with repeat-intent density (FAQ/agent loops high; one-shot tools low). We don’t pretend a single “production hit rate.” For your traffic: chorusgraph-audit on query logs (no LLM calls) or a short pilot against your staging — happy to help interpret.
Reports: mid_20260708_111539 / heavy_20260708_140300 on GitHub.
pip install and it just worked, the semantic cache skipping repeated calls actually showed up in my logs right away. nice to see a graph runtime that isn't wrapped around langgraph
@cemile39555 Thanks Cemile — really glad pip install was smooth and you saw cache hits in your logs right away. That's the behavior we optimize for: fewer redundant LLM calls without bolting cache on as an afterthought.
If you're finding it useful, a ⭐ on the repo helps other builders discover it — and we'd love your take in a GitHub Discussion or issue (routing, cache behavior, whatever you're seeing in prod). We read every one.
https://github.com/insightitsGit/ChorusGraph
Benchmark report (vs LangGraph, honest ties disclosed): https://github.com/insightitsGit/ChorusGraph/blob/master/docs/BENCHMARK_RESULTS.md
Really cool to see a non-LangGraph option that actually ships with benchmarks instead of vague claims. The BSP scheduler plus semantic cache combo makes sense for cutting redundant calls on routing-heavy flows.
@sevim1063464 Thanks Sevim — routing-heavy flows are exactly where BSP + semantic cache compound: sync the graph once per tick, short-circuit repeat intent before it becomes another LLM hop.
We publish paired runs vs LangGraph (same prompts/tools/model) and disclose where we tie — e.g. HC2 p95 wall-clock at heavy scale. Full report:
https://github.com/insightitsGit/ChorusGraph/blob/master/docs/BENCHMARK_RESULTS.md
If this matches a stack you're building, a ⭐ on the repo helps others find it — and we'd love notes in GitHub Discussions or an issue (routing patterns, cache misses, whatever you're seeing).
https://github.com/insightitsGit/ChorusGraph
the semantic cache hitting on redundant routing is genuinely clever, that's the kind of thing that adds up fast in real agent workloads
@elifnurkostak Thanks Elifnur — that’s the bet. In multi-hop loops the same routing decision shows up again and again; skipping those hops is where token spend actually moves.
If you try it on a denser workload (multi-agent / long session), curious whether hit rate tracks what you expect — Route Ledger makes that easy to spot per hop.