We need hot existential risk research
The importance of creativity for studying the end of the world
Existential risk studies look at events that have the chance of either ending humanity or its potential (for example, by locking us in an eternal pre-industrial state). These kinds of events can vary wildly. Humanity could be wiped out by nuclear war, a pandemic, a supervolcanic eruption and a variety of other causes. Those events need knowledge from all kinds of fields, ranging from geology to medicine. However, they have one thing in common: We don’t know much about them. Civilization-ending events are unprecedented by definition. This implies that we might have to do something a bit different here than in a lot of other fields. To better be able to think about this, Adrian Currie introduces the idea of “hot science” and “cold science” in their paper “Existential risk, creativity & well-adapted science” (1). This is a metaphor based on the movement of molecules. Hot molecules move around quite a bit, while cold molecules mostly stay where they are. But how does this relate to science?
Cold Science
This refers to science on a well trodden path. Imagine a scientist who has mapped out a field of research they want to look at, for example in engineering. They want to create a new variation for an already existing piece of machinery and need to develop all the parts required to construct it. While this might not be an easy task, and there will probably be setbacks, the direction is clear. Just go through all the parts you need, tinker around until you finish something,and finally construct the complete machine. This is cold science. You have a small area of research, you methodologically go through it and try to exhaust it as completely as possible. The results will likely be incremental. This does not imply that cold science is bad in any way. For many fields and ideas, cold science is the exact right thing to do. Mostly this is the case if you want to find the very best solution there is. Making sure to find the best solution means that you have to sample your solution space as thoroughly as possible. But if you cannot map out your research directions so easily, you need something different.
Hot Science
This is science that is more in explore than in exploit mode. A science that puts less weight on its priors. Your priors become less helpful, the less well known the area you are moving in. As you put less weight on your priors, your search becomes more random. This makes it harder to do the best possible solution, as you might leave research areas before you have looked at everything within them. Finding the very best solution implies that you have a clear metric of quality for your research. This metric is missing when you explore something completely new. How should you know what the best solution is, if you barely know what kind of solutions you are even looking for? In this case it is better to look at lots of ideas and possible solutions and to let your creativity flow. Doing so will give you an overview of the research area. Over time, it might even allow you to make your research colder again, as you get a better feel for what you are searching for.
Is contemporary science hot or cold?
The examples above show that doing science hot or cold does not say anything about the quality of the research. The question is more about what you need when. In addition, scientists don’t work alone. A collection of cold scientists, for example, could still do hot science overall, as long as their starting places in the space of possible solutions are far enough apart. Still, Currie makes the argument that today's research is often done cold. This is less a choice made by individual scientists, but more the overall incentive structure they face. Currie (basing it on research by Stanford (2015) (2)) lists several reasons on why science is mostly done cold (the links to further research in this direction are by me):
Professionalization: If you want to get ahead in science, you have to produce science that your scientific peers believe is legitimate and significant, which likely leads to a science of the least common denominator.
Peer review: Peer review gives scientists the chance to influence the ideas of others, enforcing a trend towards consensus, by questioning the legitimacy of novel ideas.
Big science: Large scale science projects need standardization to work. However, the more you enforce a standard, the harder it is to do something novel.
Increasing influence of senior scientists (3): Today, more and more funding and thus influence is funneled to senior scientists. This makes it harder for new scientists with disruptive ideas to establish themselves.
More finely grained scientific disciplines: While interdisciplinary research is often held high, most research is sorted by disciplines. This makes inter- and transdisciplinary research much harder, because there is no natural place for it (e.g. when it comes to university departments). (see also the idea of epistemic diversity (4))
Cold science is easier to teach: Teaching people to exhaustively sample a small area is easier than to lead them to explore novel ideas and fields. This is also highlighted in “Constructing Research Questions” (5).
All this leads to more gap spotting and less risky research overall (6), meaning that most of today’s research is done cold.
The case for hot existential risk research
Now let's get back to existential risk research. We have established that cold science is the default today. Does this also work for existential risk research? The answer is no. Existential risk research has to be hot for several reasons:
Uniqueness: Most of the events considered in existential risk research haven’t happened yet or only a limited number of times (this is the case for natural existential risks like asteroid impacts). This sparse data makes it impossible to calibrate simple models.
Interference and noise: For many existential risks, it is difficult to determine cause and effect, because they are interconnected with other risks. Currie gives the example of solar flares. Their direct effects could be devastating, for example we could lose all satellites. However, this would disrupt global communication, which in turn might lead to a breakdown of global trade, possibly causing a global famine. Such complex cascades are hard to analyze, and novel ideas are needed to do so.
2nd-Order Uncertainty: We don’t know yet which research questions are hypotheses are the most valuable to explore. We, therefore, have to cast the net of research widely.
This means existential risk research needs to be hot, while most incentives are making it cold. We have to actively work against this by making sure that diverse voices are heard, researchers actively leave their scientific comfort zone, read widely, and many fields are considered as inspiration. These are just the first steps. What will work best will be established by our hot pursuit of the end of the world.
References
1. A. Currie, Existential risk, creativity & well-adapted science. Studies in History and Philosophy of Science Part A 76, 39–48 (2019).
2. P. K. Stanford, Unconceived Alternatives and Conservatism in Science: The Impact of Professionalization, Peer-Review, and Big Science. Synthese, 1–18 (2015).
3. M. Clancy, Age and the Impact of Innovations. What’s New Under the Sun (2022) (December 25, 2022).
4. N. Oreskes, Why trust science? (Princeton University Press, 2021).
5. M. Alvesson, J. Sandberg, Constructing Research Questions: Doing Interesting Research (SAGE Publications Ltd, 2013) https:/doi.org/10.4135/9781446270035 (December 2, 2022).
6. M. Clancy, Biases Against Risky Research. What’s New Under the Sun (2023) (April 6, 2023).