Everyone is building “company brains” and people seem to want to ingest every email, deal note, Slack thread or even “deck out every room with microphones” (Tom Blomfield, YC batch talk…hello 1984 yikes 😬), then let AI retrieval sort it out. I think the approach is wrong and some early research suggests the same.
You don’t have to go far from how a company operates effectively to get to a well designed and orchestrated learning brain. In the early days at Uber, we had well maintained documentation on Confluence. Each page had an owner who kept it updated. This discipline allowed me to launch a city within 3 weeks of starting because key insights and learnings were kept updated in a centralised doc. As the company progressed, this discipline was lost and knowledge dissolved into 1000s of gdocs floating around.
I believe this discipline is key. The design I have been developing recognises that companies, agentic or not, have the potential to generate data and outcomes at volumes greater than any human can remember them. The tempting answer that Y Combinator seems to be championing is to hoover all of it into one searchable store and then say something like ‘AI will figure it out’.
But a store that accepts everything degrades as noise accumulates. An AI that learns from all of your raw operational chatter doesn’t necessarily fix your inefficiencies… worst case, it can codify them instead. There’s some early research on this from a small study by Zahn and Chana. A memory store that accepted everything scored 13% accuracy once near-misses (called distractors) built up. But a data store that filtered at the point of writing scored 100% on the same test.
The ingest-everything approach includes thousands of items, most of them noise. Just reflect on how much genuinely valuable information comes out of all the meetings you have…ingesting everything would include half-formed observations, one-off anecdotes and contradictory notes from different weeks.
Then, when an agent asks a question, a search layer runs over that store, pulls back the handful of entries that look the most relevant, and ranks them.
“Looks most relevant” equals a similarity judgement, not a truth judgement. Near-misses would often resemble the right answer closely - a stale finding from an A/B experiment can read almost identically to the current one. Even filtering at read time can’t save you from a noisy store, because a search layer can only rerank what’s already there.
The key is controlling what gets written in, how it’s updated, and keeping a dated log of what was rejected and why. If only verified, supported claims are allowed in, then everything the search returns is trustworthy by construction i.e. there’s nothing to filter out.
Which is really just those early Uber Confluence pages again: every page had an owner who kept it current, except now an agent does the tedious part and the human owner approves (edits) or rejects it.