When you sit down to build a retrieval system in 2026, the path is so worn you can follow it in your sleep. Chunk the documents. Embed the chunks. Push the vectors into a database. Embed the query, grab the nearest neighbors, stuff them in the prompt. There are a hundred tutorials and a dozen managed services for every step.
I built a tool called grove — a local knowledge engine that answers questions over your own docs and notes — and I didn't start there. No chunking, no vector database. Embeddings do show up eventually, but as one voice in the room, not the whole conversation. This is the log of why, and what I built instead.
What was bugging me about the default
I wrote a whole post on embeddings being a map: great for "what's near this," useless for "is this the place I need." That critique is fair but abstract. What actually pushed me off the vector path was sitting and looking at the corpora I wanted grove to handle — my Obsidian vault, a folder of work docs, a docs site.
They all had one thing in common: they were already organized. Folders grouped by topic. Headings that were a table of contents if you squinted. Notes that linked to each other on purpose. Someone — me, a teammate, a docs team — had already done the work of giving this knowledge a shape.
And the first thing the standard pipeline does is destroy that shape. You run a chunker over it, and now "the auth section of the architecture doc" is just chunk #4,812, sitting in a vector space next to a chunk from an unrelated postmortem because they both mention tokens. The structure that told you where things are is gone, replaced by geometric proximity in a space no human will ever look at. That felt backwards. I was paying — in infrastructure, in a lossy transform — to throw away information I wanted.
The alternative: let the model read the table of contents
Here's the idea grove is built on. If your documents already form a hierarchy, don't flatten it. Turn it into an index the model can navigate.
grove builds a tree per source. Each node — a folder, a topic cluster, eventually a document — gets a short LLM-generated title and summary describing what's underneath it. The primary move when answering a question isn't a similarity lookup — it's what you'd do walking into a library:
- Pick the relevant trees. With one source it skips this; with several, a quick model call narrows to the trees worth opening.
- Descend. Starting at the root, the model sees the current node's children — their titles and summaries — and picks which to walk into. Then it does it again, one level down. It's reading a table of contents and flipping to the right chapter.
- Assemble. When it reaches leaves, grove collects those documents, deduped and capped.
- Synthesize. A final call writes a cited answer from the assembled text, with a
retrieval_tracerecording every step it took to get there.
I didn't invent tree descent — PageIndex does it well for navigating a single long document, and it's good prior art. grove's bet is different: do it across many heterogeneous sources at once, as a forest, with the source's own structure as the tree. I read their public approach and wrote grove's navigation prompts clean-room rather than forking, partly for licensing hygiene and partly because the multi-source case asks different questions of the prompt.
Descent is the spine, not the whole skeleton
Navigating structure is grove's bet, but it has blind spots, and I'd be lying if I dressed it up as one clever trick that needs nothing else. A descent is only as smart as the summaries it reads — it can walk straight past the right document because the title undersold it. Exact-term matches, an error code or a function name, are something plain keyword search just nails. And meaning phrased differently from the question is exactly what embeddings are good at catching.
So grove doesn't pick one. It runs three retrievers and fuses them: the tree descent, a full-text keyword search, and a semantic pass over document embeddings — merged with reciprocal rank fusion, then filtered by a quick yes/no relevance check on each survivor. Each retriever covers the others' blind spots.
The embeddings are real, and they earn their seat. But notice what grove still doesn't do: it embeds whole documents, not a pile of chunks, and those vectors live in the same local SQLite file as everything else. There's no chunker dissolving your structure, and no separate vector database to stand up and keep in sync. Embeddings are a tool grove reaches for, not the foundation the whole thing is poured on. "No vector database" was never the same promise as "no vectors" — the distinction is the point.
The part I'm proudest of: rebuilds are free
The obvious worry with "an LLM writes a summary for every node" is cost. Build a forest over a few hundred documents and you've made a few hundred model calls. Do that every time a single note changes and it's unusable.
So every node is content-addressed. Its cache key is a hash of (the node's input content, the prompt version, the model that built it). Rebuild an unchanged forest and grove makes zero model calls — every node is a cache hit. Change one note and only the branches that touch it regenerate; the rest come straight off disk. grove sync --watch leans on this: edit a file, and a second later only that slice of the tree has rebuilt. The whole index is SQLite plus content-addressed JSON on your disk, so it's inspectable, diffable, and yours, not an opaque blob in someone's cloud.
That property fell out of taking structure seriously. Because the tree mirrors the source, "what changed" is a local question with a local answer.
Where this is honestly worse
I'm building this in the open, so here's the other side, plainly.
Descent costs model calls at query time. A vector lookup is microseconds of arithmetic; grove's descent is a few sequential LLM calls per question. It's slower per query and, if you're querying a cloud model, not free. For grove that's an acceptable trade — build cost is amortized to near zero by the cache, and the queries are the rare event — but if you're firing thousands of queries a second, vectors win on latency, full stop.
It leans on good summaries. The descent is only as smart as the titles and summaries the model wrote for each node. A lazy summary sends the navigator down the wrong branch. Vector search has no such dependency — it doesn't care what the chunk "means," only where it lands.
Structure-poor corpora benefit less. Ten thousand untitled notes dumped in one folder don't have much of a tree to navigate. grove clusters loose notes into topics to fight this, but if your knowledge genuinely has no shape, the thing descent exploits isn't there. This is part of why the keyword and embedding retrievers exist alongside it — they don't care whether your knowledge has a shape. And for when you want speed over navigation, a --fast path skips the model descent entirely and answers from just the keyword and semantic hits, no LLM in the retrieval loop.
What I'm not claiming yet
I want to be careful here, because the internet is full of retrieval systems with confident accuracy numbers and no methodology. grove does not have published benchmark numbers yet. Until it does, I'm making no quality claims — not "better than vector RAG," not anything. The next real milestone is a benchmark over a messy corpus with hand-written question/answer pairs, scored against a BM25-and-embeddings baseline, published honestly including the cases where grove loses.
What I can say is that the architecture matches the problem I actually have: knowledge that already has structure, that I'd rather navigate than dissolve. If that sounds like your knowledge too, grove is here, and the build continues.