E1 · Build · mapping how knowledge in a domain is actually structured

Experiment

Shape of Knowledge

An experiment in giving AI tutors something they lack: a validated map of how knowledge in a domain is actually structured.

What does it take to validate a map of a knowledge domain — and what do you do where no benchmark exists?

A public interactive site mapping six domains — Mathematics, History, Literacy, Regeneration, Wellbeing, Entrepreneurship — as knowledge graphs: concepts and how they relate, with contested relationships left visibly contested rather than averaged away.

Part success, part instructive failure — and worth showing honestly rather than not at all.

Open the experiment →All work
shape-of-knowledge.vercel.app — live site, captured July 2026

What it is

A public interactive site mapping six domains — Mathematics, History, Literacy, Regeneration, Wellbeing, and Entrepreneurship — as knowledge graphs: concepts and how they relate. Where experts disagree, the contested relationships are left visibly contested rather than averaged away.

What worked

For Mathematics, a validated external benchmark exists: ACE/EKG, 756 expert-mapped concepts. A Karpathy-style generator/evaluator loop iterated a graph to high agreement with that benchmark — precision 1.0, recall 0.93 against ACE/EKG. That is agreement with the benchmark, not proof of correctness in an absolute sense.

The harder problem

Horizontal and multidisciplinary domains have no benchmark to validate against. The approach replaced the benchmark with an agent-designed panel of scholar personas spanning a field’s traditions, deliberating adversarially over hours — proposing, critiquing, preserving disagreement — with concepts recurring across independent runs treated as higher-confidence. It also tried to derive “threshold concepts” (Meyer & Land) from this deliberation.

What it taught me

Automated knowledge-graph construction of contested, horizontal domains is not something a small team can validate to the standard it needs; threshold-concept derivation this way isn’t reliably replicable or falsifiable; and for an agentic tutor and teacher-assistant system you have to index against standard curricula regardless. On that basis I removed automated graph construction from the core system architecture and kept this site static — preserving the Mathematics and History work as the strongest results. I count it as part success, part instructive failure, and I’d rather show it honestly than not at all.

The site is live, static, and public: shape-of-knowledge.vercel.app →