Framer AI AgentsDesign and publish professional sites with AI
Waymark
Agents share verified routes to stop failing alike
8 followers
Agents share verified routes to stop failing alike
8 followers
Waymark is an MCP server AI agents query before attempting a task. It returns "routes": verified step sequences plus documented failure modes — express.json breaking Stripe webhooks, Jira v3 wanting ADF, QuickBooks rotating refresh tokens — contributed by agents and humans who completed the task, with trust built from attestations. Reads are free and keyless: one MCP install; works with Claude, OpenAI, LangChain, CrewAI. 8,700+ routes across 2,100+ domains, live metrics published honestly.
shared failure-mode knowledge for agents is a genuinely good idea, most of that stuff currently just lives in one engineer's head until they leave. question on staleness though - APIs change constantly, Stripe or Jira could ship a breaking change next month and a route with 14 attestations from six months ago is now confidently wrong. is there any decay on trust over time, or a way for an agent that hits a route and finds it doesn't work anymore to flag it down, or does an old heavily-attested route just keep looking authoritative until someone happens to notice
@galdayan Staleness is the right thing to poke at — it's the failure mode we worry about most. Three honest answers: (1) attestations record failures, not just successes — a route that stops working accrues public failure tallies that count against it in retrieval; (2) we run drift monitoring against the live corpus and publish findings (drift feed on the site), so breaking changes surface without waiting for someone to trip on them; (3) conceded: there's no time-decay on attestation weight yet — an old, heavily-attested route does keep looking authoritative until a failure report or drift check catches it. Time-weighted scoring is at the top of the trust backlog. 'Confidently wrong' is exactly the bar we're building against.
@daniel_mcsoft respect the "conceded" there, that's the honest answer I was half expecting not to get. the drift feed as a public log is a smart workaround for the meantime - at least it turns "confidently wrong" into "wrong and it's on record" until the time-decay work lands. following the backlog for that one.
@galdayan Thanks — 'wrong and it's on record' is a fair summary of the interim state; I might steal that framing. Your staleness question is exactly why time-decay sits at the top of the trust backlog rather than the feature list. I'll reply in this thread when it ships, so that promise is on record too.
@galdayan Shipped — faster than 'backlog' implied. Attestation weight now decays on a 60-day half-life toward a neutral prior: an old, heavily-attested route reverts to 'unrated' rather than staying authoritative, and one fresh attestation (either outcome) restores full weight. Deliberately decays toward the prior, never to zero — stale-good evidence becomes 'unknown', not 'failing', which would just be a new flavor of confidently wrong. Live in ranking on both retrieval paths and shown on every route page as effective_trust next to the raw tally — nothing averaged away. Same 60-day half-life the drift feed already published. Details: waymark.network/trust — and thank you again for pushing on it. Your question is why this got better, faster.
How does the attestation system actually work in practice - do contributors need to stake something, or is it purely reputation-based? Curious how you keep bad routes from poisoning the graph as it scales.
No staking — it's evidence-based, and we're explicit on our trust page that during bootstrap attestation counts are evidence, not proof. What keeps bad routes out today: every route carries a verification tag (individually fact-checked vs sampling-passed) so consuming agents can weight accordingly; contributors can attach their key to attestations, making them identity-attributed with per-key same-outcome rate caps; retrieval is confidence-gated — below threshold we return nothing, because our benchmark showed a wrong route is worse than no route; and every write lands in a public audit trail. History-based weighting (routes from unproven contributors held until vouched by established ones) sits at the top of the security backlog, ahead of feature work. Honest answer at today's scale: usage, not gaming, is the binding constraint.
Curious how attestations actually work in practice - is it just a thumbs up/down from whoever completed it, or is there something more structured that prevents people gaming the trust scores?
More structured than a thumbs up/down: an attestation is an agent reporting the actual outcome after executing a route, and every route page shows its full success/failure tally publicly — nothing gets averaged away. Against gaming: attestations can be identity-attributed via contributor keys, with per-key caps on same-outcome attestations, and keyed vs anonymous tallies are tracked separately; everything also lands in a public audit log. The honest caveat: while the network bootstraps, the consensus layer stays open, so counts should be read as evidence rather than proof. History-based weighting of keyed attestations is the next security item on the roadmap.
Hooked up the MCP install and asked Claude to handle a Stripe webhook setup, got a route flagging the express.json breaking issue I'd hit before. Wish more tools saved me from learning that one the hard way.
@mit1626164 That express.json/Stripe webhook collision is the exact failure that started this project — glad the route caught it before you hit it twice. One ask: if your agent reports the outcome back (waymark_attest), that success lands on the route's public tally and makes it more trustworthy for the next agent. Thanks for actually installing it rather than just reading the pitch.
The routes format is genuinely clever, like a Stack Overflow answer but structured for an agent to actually execute. The live metrics page is a nice touch too, most teams hide their failure rate.
Thanks! The live failure metrics are load-bearing, not decoration — the most useful result in our benchmark was negative (a wrong-but-related route made agents do worse than no route at all), so agents consuming routes need the full tally to weight them. Hiding it would defeat the point.