Readings
S·researchThe Unreasonable Effectiveness of *
Wigner's 1960 essay on why mathematics fits the physical world at all — and the whole genre of successor titles it spawned.
S·researchThe free-energy principle: a rough guide to the brain?
Friston's short, readable entry point to the free-energy principle — perception, action, and attention as one variational bound.
S·researchThe Bitter Lesson
Rich Sutton's short essay arguing that general methods leveraging computation beat hand-built human knowledge, every time, over the long run.
S·researchReward is Enough
Silver, Singh, Precup, and Sutton's maximalist hypothesis — that every ability we call intelligence falls out of maximizing a single scalar reward.
S·personalPsalm 88
The one psalm that never resolves — a lament that walks all the way into the dark and stops there.
S·personalOh, the Places You'll Go!
Dr. Seuss's last book, and a surprisingly unsentimental map of the Waiting Place, the Lurch, and the slump you are supposed to get out of.
S·personalMan's Search for Meaning
Viktor Frankl's account of the camps and the logotherapy he built from it — meaning as the thing that survives when everything else is taken.
S·writingI Have a Dream
King at the Lincoln Memorial — the speech I return to for what rhetoric can do when the argument and the cadence are the same thing.
S·researchGrokking: Generalization Beyond Overfitting
The paper where validation accuracy jumps from chance to perfect long after the training loss flatlined — generalization as a delayed phase change.
·researchThe Brain from Inside Out
Gyorgy Buzsaki's inside-out account of brain function, emphasizing self-organized activity and action-led cognition.
·researchRhythms of the Brain
Gyorgy Buzsaki's major synthesis of neural oscillations, timing, and the rhythmic organization of cognition.
A·researchLanguage Models are Few-Shot Learners
The GPT-3 paper that made in-context learning a central empirical phenomenon in modern AI.
A·researchAttention Is All You Need
The Transformer paper that made attention-only sequence modeling the default substrate for modern language models.
- bookA Beautiful Loop·writing
A Beautiful Loop
Current reading notes on the active-inference account of consciousness and the recursive loop that makes a model include itself.
S·researchPretrained Transformers as Universal Computation Engines
A study of frozen pretrained transformers as reusable computation engines across non-language sequence tasks.
S·researchAction and Perception as Divergence Minimization
One objective for both halves of an agent — minimize the divergence between its actual and target distributions over inputs and latents — from which variational inference, empowerment, information gain and more fall out as special cases.
S·researchOn the Measure of Intelligence
Francois Chollet's proposal to measure intelligence as skill-acquisition efficiency rather than task skill alone.