The World is a Cloud
A Reader’s Guide to David Chapman
Note: this is my attempt at a brief guide to the work of one of my key intellectual influences, David Chapman. I’ve been sitting on it for many months and, rather than rewrite it for the third time, I figured some people would find this helpful now. I hope to find a slightly more stable home for this soon.
Introduction — Context on Chapman — Meaning — Adult Development — Culture and Civilization — Buddhism — Ethics — Science and Thinking — AI / Technical Work
Introduction
What is truth?
Our most powerful scientific models of the world are, to varying degrees, wrong. We know Newtonian mechanics isn't fully true because it can’t explain for Mercury's orbit, the timing of pings from GPS satellites, and light bending around massive objects. Even general relativity conflicts with quantum mechanics. Even quantum field theory, our most precise physical theory, struggles to incorporate gravity. Yet all these theories reliably describe patterns we can observe within their domains. What is the real truth then?
How about ethics? If we can’t build scientific theories that are absolutely true, then what hope do we have for finding true ethical theories? Major frameworks like utilitarianism and deontology make logically compelling arguments but contradict each other. Even within utilitarianism, we don’t know whether to maximize average or total utility. How are we supposed to figure out the right way to be if there's no perfect ethical framework?
Or what about meaning in life?
Are we free to choose what our lives mean? Or are our lives meaningless because we’re really a collection of atoms on a tiny planet in an indifferent universe?
Most of my adult life I have been worrying about some version of these questions. Studying computer science at MIT and being encultured in a deeply scientific-analytic worldview, I was actually upset that I had no good answer to these questions about truth, ethics, and meaning. Partly in response, I re-ignited the Buddhist Students Club at MIT and began to meditate.
Years later, I realized that these questions were to be dissolved, not resolved. I learned this largely though the writing of David Chapman, an AI-researcher-turned-entrepreneur-turned-Tantric-Buddhist-internet-writer.
The core message from Chapman is very simple: pattern (that which is reliable, clear, distinct, definable) and nebulosity (that which is insubstantial, amorphous, non-separable, ambiguous) cannot ultimately be separated. Recognizing this dissolves the earlier questions. Everything that Chapman writes is downstream of this realization.
Unfortunately, though it is simple, it is typically not easy to internalize. Chapman uses straightforward observations to make this realization more apparent. For example, when faced with the trolley problem, you may be used to arguing for or against flipping the switch, but what about considering the limitations of ethical thought experiments in general? Chapman is not in the business of offering frameworks, but rather pointers to a vantage point from which you can recognize the limitations of all frameworks.
The purpose of this essay is to make the case for reading David Chapman, and then to suggest a few ways in.
Context
Who is David Chapman?
Brief context:
After graduating from MIT with a pure math degree in 1982, Chapman begins his PhD at MIT's AI Lab focused on artificial intelligence planning. At the time, the dominant AI paradigm (based on formal symbolic reasoning) was showing serious limitations. Chapman's influential paper "Planning for Conjunctive Goals" demonstrates mathematically why certain common approaches to AI planning were fundamentally intractable. His work helps show why purely logical, symbol-manipulation approaches to artificial intelligence couldn't succeed - putting a nail in the coffin of GOFAI.
In the mid-90s, after some involvement with a Wiccan pagan spiritual practices, Chapman begins studying and practicing Vajrayana Buddhism (aka Buddhist Tantra, also commonly called Tibetan Buddhism1). He initially practices with Shambhala, one of the larger Western Vajrayana organizations, before joining a smaller, more traditional teaching lineage called the Aro gTer
In the late 90s, he starts a software company targeted to pharmaceutical companies in Silicon Valley. He sells this at the peak of the dot-com bubble.
He begins engaging more with the internet Buddhist scene around 2008. His writing with Vividness helps lay the seeds for his spouse Charlie Awbery’s creation of Evolving Ground, an online contemporary Vajrayana community.
Chapman’s influences are varied. Among the most fundamental are Heidegger, Nietzsche, various Vajrayana teachings, Kegan, and Dreyfus.
Who might benefit from reading David Chapman?
People who:
Enjoy thinking analytically but feel frustrated at how 'meaning' fits in with materialism/any of the questions raised at the start of the essay.
Like Buddhism but don't want to be celibate monks
Want to understand, in broad but useful terms, the changing shape of culture over the 20th century
Appreciate vampire fantasy
Where to start?
David Chapman is highly prolific. He has written, variously, a technical guide for engineering, a book on Vajrayana Buddhism, a Buddhist Vampire Romance serial, an extended reflection on meaning, along with dozens of essays, and technical papers on classical AI.
I'll organize my favorite essays of his by theme:
Meaning
Most of Chapman’s writing is in some way related to meaning. However, it’s the first few sections of Meaningness that most explicitly address meaning.
The short introductory pages under Why meaningness are a great place to begin. Meaningfulness and meaninglessness, Pattern, Nebulosity are also probably a good idea before jumping around.
After that, my favorites are:
Stances trump systems, and The Big Three stance combinations. Chapman’s identification of stances, rather than systems like utilitarianism, are very clarifying. (In contrast to a system, which at least attempts to be coherent or complete, a stance is just a compelling pattern of thinking and feeling: “good people follow rules”, “what goes around comes around”, etc). It has lifted me out of a repeated mistake in relating to meaning. I still think in terms of his presentation of monism-dualism and eternalism-nihilism.
No cosmic meaning. “We’re just a tiny collection of atoms in the huge meaningless universe”, right? This long essay is an excellent antidote to the nihilist sales pitch.
Sartre’s ghost and the corpse of God discusses and dismisses existentialism in less than 500 words. Having read more existentialism since, I think Chapman is spot-on and I could have taken him at face value.
If you are more partial to audio, I found Jake Orthwein’s exposition of Chapman’s thought to be very helpful over a few podcast episodes.
On Stages of Development
Before diving more into Chapman's work, it's helpful to understand the developmental framework he uses, based on Robert Kegan's model. He does this best in "Developing ethical, social, and cognitive competence", but I’ll briefly summarize it here too.
Kegan’s stages are much deeper than they initially appear. You can think of each stage as an increase in cognitive complexity that transcends and includes the previous level of complexity. For instance, a child may contribute to a group project because they fear a bad grade (Stage 2/ self-interested mode), or because they care about not letting other people down (Stage 3/ communal mode). Kegan empirically found that the child operating out of the communal mode can simulate the self-interested child, but not vice versa. It turns out that these stages apply across a lot of domains, and are pretty helpful for understanding civilizations as a whole.
The three stages most common in adults:
Stage 3 (Pre-rational/Communal): People at this stage primarily organize their thinking around social relationships and shared values. They tend to accept traditional ways of doing things without questioning systematic justification. When faced with contradictions, they focus more on maintaining social harmony than logical consistency.
Stage 4 (Rational/Systematic): This is where most STEM education aims to get people. At this stage, people can work with formal systems, appreciate logical consistency, and systematically analyze problems. They seek universal principles and tend to believe there must be one correct framework or answer. Most modern institutions require this kind of thinking to function.
Stage 5 (Meta-rational/Fluid): At this stage, people can both use systematic reasoning when appropriate while also recognizing its limitations. They can work with multiple frameworks simultaneously, understanding that different approaches may be useful in different contexts. Rather than seeking absolute truth in any one system, they focus on skillful deployment of various methods based on context.
There are many ways that stages are misunderstood. Chapman addresses some common misunderstandings here.
Culture and Civilization
“The only defense against invented traditions and timeworn futures is to study the history of ideas.”
How meaning fell apart is Chapman’s series about systems of meaning over the twentieth century. It is the most concise explanation of the rise and collapse of modernity I have come across2. It tracks our transition from ‘the choiceless mode’ (-1700) to ‘the systematic mode’ (1450-1914) and its collapse (1914-1980), which includes Countercultures (1964-1990), Subcultures (1975-2001), and Atomization (2001 - now). See A gigantic chart that explains absolutely everything for a bird’s eye view.
A bridge to meta-rationality vs civilizational collapse describes how postmodernism sabotages the bridge to rationality, and the barrier between rationality and metarationality. In so doing, it is the best direct explanation of the function of David Chapman’s Meaningness and Metarationality projects in the first place.
Other favorites:
In praise of choicelessness. Before the rise of systems was what Chapman calls ‘the choiceless mode’.
Invented traditions and timeworn futures. The Scottish kilt was invented by an Englishman in the 1720s. Cranberry sauce with Turkey on Thanksgiving was invented by the marketing arm of the cranberry industry. Much of British royal ritual was invented in the last two centuries. A primer on defense against ideological time-distortion.
The collapse of rational certainty. A longer essay laying the intellectual history behind the rise of modernity. Starting with Euclid’s elements and ending with Kuhn. This article has served as the structure I’ve continued to fill in with my reading of the twentieth century.
Buddhism
Chapman’s writing about Buddhism is split between Vividness, his collection of essays about Vajrayana Buddhism, and Buddhism for Vampires, his fantasy romance series.
What do you want out of Buddhism? I found Chapman’s chapter "Approaching systems of meaning" very helpful for suggesting the useful questions to ask. What is the principle and function behind this practice? Is it a good personal fit? Can I lighten my grip on certainty?
I will caution wading into his writing on Consensus Buddhism and Sutrayana, especially if you feel somewhat happy with a current mindfulness-style practice. If you are not familiar with the modern history of Buddhism, it will likely blow up your conception of Buddhism and leave you reeling for months. You may also happen to be someone who has a better personal fit with Consensus Buddhism3. It may be hard to pursue the practice in the same way for a while. Read onwards with caution.
The "How Buddhism became Consensus" series unveils contemporary western Buddhism as a 'Protestant Reformation of Buddhism', using anthropologist David L McMahon’s excellent The Making of Buddhist Modernism as inspiration. The titles are self-recommending: Modern Buddhism: Forged as anti-colonial weapon, Zen vs the U.S. Navy, The King of Siam invents Western Buddhism, Theravada reinvents meditation. What the Buddha REALLY said provides an excellent discussion of authenticity in religion.
"Buddhism for Vampires" is a delightful work of serialized fiction. The story consists of 24 short chapters and is currently unfinished. The existing chapters, though, are fantastic. The topics range from tantric warfare to tender love to cannibalistic witches. Most fantasy I have read is rooted in Christianity and medieval Europe. “Buddhism for Vampires” feels quite unique in telling a compelling story from the basis of tantric Buddhism and the myths of around the medieval himalayas. My enjoyment of Buddhism for Vampires is on a par with Brandon Sanderson’s The Emperor's Soul (Chapman is much more thoughtful but his plotting and pacing is much more sporadic) and even some of the stories from Ken Liu’s The Paper Menagerie.
His metablog provides historical context for the novel.
Ethics
"Buddhist ethics is a fraud" - Introduces a series on the historical context of moralities as they relate to Buddhism.
"Meaningness > Ethics" - A stub that lays out the deep assumptions inherent in all ethical systems.
"Developing ethical, social, and cognitive competence" - Introduces an influential and very helpful framework, Robert Kegan's stage theory, for understanding cognitive, social, and ethical development. Obviously, stage theory itself is a system for understanding development, but the categories of communal stage 3 to the systematic stage 4 to the fluid stage 5 are very helpful in making sense of a broad range of human development.
In a sense this essay provides the background for his project of metarationality, which is an attempt to move the reader from the systematic stage 4 to the fluid stage 5.
Science and Thinking
In the cells of the eggplant is Chapman’s book on metarationality, which aims to be directly helpful in STEM work. The most practical sections have yet to be written but what is present so far I have found very helpful4.
I think Chapman's writing here is helpful for developing a better relationship with the notion of 'truth'. One of his central points is that to develop theories of the world and to build things that work, it is not enough to 'seek truth'. Finding 'truths' and 'updating beliefs' are certainly involved in the quest, but one ought to orient towards better ways of knowing, rather than centrally prizing 'the quest for truth'.
These essays are particularly meant to be read in order, but I’ll still highlight my favorite if you want to get a flavor.
Part I: Taking Rationalism Seriously provides a quick and helpful overview of how various rationalist5 approaches to truth and how they fail. Some essays I like from the collection:
Positive and logical talks about the collapse of logical positivism and what a good practice of reasoning can learn from its failure.
The world is everything that is the case talks about the appeal and problems with Aristotelian logic and how it is still present in much informal thought today. ("Beliefs are very rarely absolutely true or false, but rather nebulous.")
*Objects, objectively discusses the appeal but ultimate inadequacy of discussions about 'objective truth' in the world of categories we live in. Chapman draws a distinction here between 'the eggplant sized world' — i.e. the world of objects we live in and interact with— and the atomic/subatomic sized world. ("An electron's mass may be measurable to the 18th sig fig, but a chair, person, or any category of things we care about is not simply a collection of subatomic particles.")
*Leaving the casino helpfully frames probability theory as another cognitive tool rather than an absolute truth about the world.
Reference: rationalism's reality problem discusses the correspondence theory of truth and its inadequacy as a theory. It is mostly helpful if you have been exposed to discussions of rationalist epistemology.
Part II: Taking Reasonableness Seriously explores everyday (non-rational) thought and action (“reasonableness”), which Chapman then uses to reconstruct an account of metarationality.
This is not cognitive science explains why cognitive science is not necessary for his account. He argues against the superficial similarity between Kahneman’s System 1 and System 2, and answers the question “what good is this if it isn’t science?”
The ethnomethodological flip is tremendously helpful perspective reversal of the sort Heidegger attempted to convey in Being and Time. Basically, we tend to think that the optimal way of making decisions is through rationality, and that we make mistakes because we are boundedly rational and have suboptimal biological hardware. Things make a lot more sense, however, if we can recognize rationality as a mode of cognition that is appropriate in a particular context.
Parts III and IV have not yet been written.
Favorite additional essays
"Nutrition: The Emperor Has No Clothes” is a devastating critique of nutrition science. Chapman at his most entertainingly acerbic.
"How to Think Real Good" - An informal response to LessWrong community's fixation with Bayesian probability theory (particularly relevant for those familiar with LessWrong).
"Making a kangling for chod" - "Putting a dead person's leg bone to your mouth—to blow the kangling while practicing—is an intimate reminder of your own mortality."
Diverse forms of agency as a counterweight to the common conception of a monolithic AI.
AI / Technical Work
Chapman’s technical papers are probably mostly interesting to those who want to understand the intellectual history of AI.
Chapman’s technical contributions are largely in conversation with a common approach to AI (and cognitive science) circa ~1980-1990: planning. He and his close collaborator Phil Agre developed their ideas within the ‘interactionist’ paradigm, which took seriously the Heideggerian/Dreyfusian critique of planning. It’s very close in spirit to enactivism, or what is also frequently called 4E cognitive science.
I have not read much of Chapman’s older research6, but I do have a sense of his role within the intellectual lineage of AI research.
Three decades after the giddy optimism of the fateful Dartmouth Summer Research Project on Artificial Intelligence, it was becoming apparent that all the dominant approaches to building AI were not going to work. But at the time, this was more of a feeling than a principled conclusion; many researchers knew something was amiss, but couldn’t precisely say what.
The classical approach to AI in the 1970s and early 1980s relied mainly on symbolic representation and formal logic for problem-solving. It assumed that intelligent behavior could be reduced to manipulation of symbols using explicit rules. The failure of this approach is captured by ‘the frame problem’: the impossibility of formally specifying what features in the environment matter as the world changes. The AIs back then would have a ‘world model database’ filled with reference facts (e.g. temperature(room) = 72, color(walls)=white) which it would update after taking actions. Moving a cup to the bedroom would look formally like location(cup) = kitchen → location(cup) = bedroom. The frame problem: how do you decide what else in your world model needs to be updated? There is no systematic protocol to choose what does and doesn’t change in the world model.
Chapman’s contribution was to demonstrate the intractability of the frame problem under a very common set of assumptions among AI researchers at the time. In his widely cited master’s thesis, “Planning for Conjunctive Goals”, he provides a formal proof to demonstrate the intractability of linear and nonlinear planners for general purpose problem solving.
His PhD thesis, “Vision, instruction, and action”, describes Sonja, a multimodal semi-autonomous AI that can play a video game. What’s striking about it is how much attention is given to a coherent explanation of what’s actually going on. He has sections on the psychophysics of attention and visual search, and extensive discussion of computational linguistics. This is very surprising to someone coming from AlexNet or the papers on Transformers, where it’s just about the parameters and architecture search.
For a more informal overview of Chapman’s line of thinking, I like this article that provides an overview of Phil Agre’s legacy (which shares much with Chapman).
It has been a great privilege to be in dialogue with one of my deep intellectual influences. For his latest writing, you can subscribe to his substack.
Acknowledgements
Thank you to Saffron Huang and Henry Williams for the encouragement, Tuomas Oikarinen and Nick Stares, Alexey Guzey, and Kasra Koushan for feedback and, of course, David Chapman for his conversations and writing.
Most people in the west conflate Vajrayana with Tibetan Buddhism, which, as Chapman argues, is not quite right. Much of Vajrayana is from outside of Tibet (e.g. Bhutan) and much of Buddhism in Tibet is not Vajrayana.
The closest rival is Charles Taylor’s much denser Sources of the Self.
Two friends of mine have seriously attempted Vajrayana and concluded that it was not a good personal fit, turning back instead to Consensus Buddhism or other forms of practice. One of these friends said that David’s writing made it hard to go back to old practices that might have been helpful.
For one stupid example, it finally gave me a principled reason to not use expected utility to decide whether I should stay with my girlfriend
Historical rationalism, very different from Bay-Area style LessWrong rationalism.
By the time I was an undergrad, no one was talking explicitly about the planning approach to AI, though I did sit in on one relevant lecture with Patrick Winston as a freshman.