AI Context Flow
Reusable AI Memory for Smarter Prompts Anywhere
Start new thread

Are you "Team Filesystem" or "Team Vector Search" for AI Memory?

by•

When it comes to AI Memory, everyone's arguing "RAG vs. grep" like it's a religious war. It's not. It's just a cost curve. I've gotten this question so many times that I thought I'd just share my thoughts. So here goes:

Vector search wins when your corpus is massive, messy, and unstructured. Thousands of docs, no clean boundaries, meaning matters more than exact words.

Filesystem plus grep wins when your corpus is structured and actually yours. A folder of markdown files you can open, read, and audit line by line. No infra required.

Anthropic already ran this experiment in production. Claude Code dropped its RAG pipeline for plain agentic search (grep, glob, read) and it outperformed the vector pipeline on real work. Not close.

But the benchmark wars are missing the actual point. It was never about picking one. It's about knowing what each layer is for.

Markdown is your source of truth. Portable, human readable, greppable, not locked to one provider. Memory you can actually own and move.

Vector search is an accelerant on top of that truth. A fast index for when the haystack gets too big for exact match to keep up.

Use either one alone and it breaks down:

- Markdown alone stalls at scale and struggles with paraphrasing or fuzzy recall

- Vectors alone turn your memory into a black box you can't read, audit, or export

The next step for memory infrastructure isn't picking a side. It's the filesystem as the ledger and RAG as the index on top of it, so memory stays legible and portable, and still fast when it needs to be.

This is the exact direction we're building with AI Context Flow: markdown as the portable, ownable source of truth, with retrieval layered on top instead of replacing it.

If your memory only exists as embeddings inside someone else's vector DB, that's not memory. That's a lease.

Which team are you on?

40 views

Add a comment

Replies

Best

Interesting take. where do you think the tipping point is? at what corpus size does plain filesystem search stop being practical and vector search becomes genuinely necessary instead of just a nice optimization?

depends on the files how structured and similar they are… but roughly between 500-1000 files is where you start feeling the heat without a vector index

The idea of markdown being the source of truth and vectors acting as the index makes a lot of sense. That feels much more practical than picking one side.

yes that’s what we realised after users started putting in 600+ files and create 100+ buckets - we realised if we just bank on vector search, we cannot implement viewing, editing, rollback features. But if we just rely on markdowns, it will get pretty slow + semantic search capability is then lost. To be honest this is a new approach we have developed in-house, if you are curious to see how this performs in action, try AICF especially with it’s MCP server

Could the real competition actually be between retrieval quality and memory quality instead of grep vs. vectors? even the best search cant rescue poorly written or duplicated memories. curious how you are thinking about that.

duplicates are flagged to the user to be removed and wrong edits can be rolled back in AICF

Memory is a lease is probably my favorite line from this post. It highlights why portability and ownership should be part of the conversation.

exactly! Cannot let AI memory be another lock in like social media - especially because the consequences here are graver than the social media use cases

I've seen so many debates around this topic, but this is one of the more balanced takes. The hybrid approach seemslike the direction most AI tools eventually take.

if you are deep in the space, would be great if you could check out AICF esp with it’s MCP server. This is our in house built technique and we haven’t seen much content around this. It could be a research paper of it’s own but well, to build, to market or to publish research: that is the question 🤔

The line "memory is a lease" is powerful. But here is the question that came to mind if I can export my markdown yet lose all the learned relationships stored in embeddings, have I really preserved my memory? Or have I only saved the raw data? Curious how you think about that distinction.

no both parts are important, if the markdown is properly structured you can get some value out of it, but relationships are equally important as they build up the nuances

I’m closer to “filesystem as the ledger, vectors as the index.” For business AI, memory has to be inspectable and portable before it can be trusted. Embeddings are useful for recall, but the source of truth should be something a human can open, review, correct, and move between systems.

This made me rethink my own architecture. one question though how do you prevent an AI from polluting the markdown layer with low-quality or redundant memories? retrieval is one challenge, but memory hygiene seems equally important.

we have version history + duplicate flagging + conflicting memories flagging within the system, but we don’t prune anything ourselves, the user has to say the word. Plus we have a full notion style editor inside the memory studio (app.plurality.network) so you can view, edit, organise format like a human

Team hybrid, and I don't think that's a cop-out — I've landed exactly where you describe, just from the applied side. I do research-heavy knowledge work and my "source of truth" is a folder of markdown notes plus a pile of PDFs. The moment I trust an answer more because I can open the file it came from, filesystem wins. But once the corpus crosses a few thousand documents, exact match quietly starts missing the thing I actually meant, and that's where an index earns its keep.

The part people skip is that the two layers fail differently, so you want them for different reasons. Grep gives me auditability and portability. Vectors give me recall on the days I can't remember the exact word I used six months ago. Retrieval as an index on top of an ownable ledger, rather than a replacement for it, matches what's held up for me in practice.

One thing I'd push on: "own your memory" is only half true if the markdown lives somewhere you can't easily pull it back out. Export and format portability matter as much as where it's stored. I've been building a workspace layer on top of NotebookLM for roughly this reason — the retrieval is great, but people still want the underlying sources to stay theirs and legible. Curious how you handle re-embedding when files change: do you diff and re-index incrementally, or is it a full rebuild?