David Thorstad engages with the philosophy of longtermism and navigates its implications for existential risk mitigation. The grounding concept of longtermism revolves around the potential vastness of the future, considering the numerous lives and experiences that could exist. Within this framework, Thorstad probes the concept of astronomical waste, a central tenet for longtermists, which posits that the obliteration of potential future lives due to existential catastrophe would result in a colossal loss. Thorstad accepts this foundational proposition, which is also widely recognized within the longtermist community. However, his exploration does not halt here; it extends into an elaborate scrutiny of the complexities and uncertainties that may surface while operationalizing this perspective in the realm of existential risk mitigation.
Thorstad does not simply acquiesce to longtermist conventions; instead, he examines and dissects these principles to draw out latent uncertainties and nuances. With each section of his paper, he scrutinizes the longtermist premises and its implications for existential risk mitigation. By doing so, he reveals hidden layers within the longtermist argument and uncovers practical and ethical concerns that often get overshadowed by the basic premises of this philosophy. Thorstad’s work, therefore, stands not as a mere affirmation of longtermism, but as an essential critique that brings to light the intricate moral and practical conundrums that lurk within its core propositions. His thorough examination aims to enrich the understanding of longtermism, laying the groundwork for future debates and discussions on the philosophy of existential risk mitigation.
Intergenerational Coordination Problem, Background Risk, and Population Dynamics
A noteworthy aspect of Thorstad’s analysis is his framing of existential risk mitigation as an intergenerational coordination problem. Thorstad postulates that for humanity to accrue significant benefits from such mitigation, it must suppress cumulative risk over prolonged periods. This poses a challenge, as each generation must ensure that future generations continue to reduce risk. According to Thorstad, this coordination problem is difficult due to four reasons. Firstly, desired risk levels are quite low and might require considerable sacrifice. Secondly, each generation carries only a fraction of the cost of potential catastrophe, requiring an unusual concern for future generations. Thirdly, this level of concern is challenging to instill given human impatience and limited altruism. Finally, enforcement is complicated because monitoring and punishing future generations’ potential selfishness is difficult, increasing the temptation to defect from a collectively optimal solution. By placing the problem within this frame, Thorstad opens up pertinent questions around feasibility and ethical considerations regarding intergenerational coordination.
Thorstad also critically examines the relationship between background existential risk and the value of mitigation efforts, producing insights that challenge conventional views. The argument rests on the premise that if background risk levels are raised, the value of mitigating any specific risk, such as biosecurity threats, significantly diminishes. This counterintuitive relationship suggests that a world with reduced biosecurity risks would still be a risky world, thus more vulnerable to future catastrophes. Thorstad extends this point by demonstrating that pessimistic assumptions about the background level of existential risk can drastically lessen the value of a fixed relative or absolute risk reduction. Intriguingly, this argument suggests a “dialectical flip” in debates on existential risk. Paradoxically, higher levels of background risk tend to lower the importance of risk mitigation, and lower levels enhance it. This revelation has potential implications for the Time of Perils Hypothesis, a foundational principle stating that risk will soon decline substantially and stay low for the rest of human history. Thorstad underscores that this hypothesis is crucial for arguing the astronomical importance of existential risk mitigation when background risk is high. However, he and others question its validity, implying further doubts about the value of existential risk mitigation.
The exploration into population dynamics, demographic interventions, and the significance of digital minds imparts another dimension to the discourse on longtermism. Thorstad scrutinizes the interplay between the potential number of lives a region can sustain and the likely number of lives it will support, given the dynamics of human populations. This insight implies that efforts to increase future human population size could be as important as mitigating existential risk. However, Thorstad notes the intricacies of this assertion as it depends on the framework of population axiology. Further, Thorstad introduces the potential role of digital minds, arguing that digital populations programmed to value expansion might outperform humans in expanding to a meaningful proportion of their maximum possible size. This argument suggests that future efforts might need to prioritize the development and safety of digital populations, possibly at the expense of future human populations, accentuating the profound ethical implications surrounding longtermism and its practical execution.
The Cluelessness Problem and Model Uncertainty, Connections to the Broader Philosophical Discourse
The cluelessness problem, as Thorstad explains, lies in the immense difficulty of predicting the consequences of our actions on the distant future, an issue further exacerbated when considering the global stakes of existential risks. Some longtermists believe that existential risk mitigation could alleviate this problem, as current risks can be identified and strategies for mitigation can be crafted today. However, Thorstad offers an alternative perspective, suggesting that cluelessness may persist due to ‘model uncertainty.’ His argument posits that the complexity inherent in valuing existential risk mitigation could mean there are still unknown variables or considerations that have been overlooked or misrepresented in the current models. This presents a cautionary note, suggesting that the escape from cluelessness via existential risk mitigation may be an overoptimistic assumption. Thorstad leaves readers contemplating the level of model uncertainty and the potential for other unexplored variables in longtermist thinking.
Thorstad’s article contributes significantly to the broader philosophical discourse, especially in the context of moral philosophy and ethical futures studies. By articulating the intergenerational coordination problem, he engages with concepts central to intergenerational justice, a core topic within the ethics of long-term thinking. Further, his exploration of ‘background risk’ and ‘the time of perils’ hypothesis contributes to the discourse around existential risk philosophy, offering a novel viewpoint that challenges traditional assumptions about existential risk mitigation. Moreover, his argument concerning population dynamics and digital minds intersects with philosophy of mind and metaphysics, advancing the philosophical understanding of these complex notions. Thorstad’s discussion on the ‘cluelessness problem’ and ‘model uncertainty’ carries implications for epistemology and decision theory, underlining the complexities associated with making predictions about the distant future and creating models for such projections. His study not only scrutinizes the presuppositions within longtermist philosophy, but also invites further inquiry into the associated philosophical dimensions, thereby expanding the theoretical terrain of futures studies.
Abstract
Longtermists have recently argued that it is overwhelmingly important to do what we can to mitigate existential risks to humanity. I consider three mistakes that are often made in calculating the value of existential risk mitigation: focusing on cumulative risk rather than period risk; ignoring background risk; and neglecting population dynamics. I show how correcting these mistakes pushes the value of existential risk mitigation substantially below leading estimates, potentially low enough to threaten the normative case for existential risk mitigation. I use this discussion to draw four positive lessons for the study of existential risk: the importance of treating existential risk as an intergenerational coordination problem; a surprising dialectical flip in the relevance of background risk levels to the case for existential risk mitigation; renewed importance of population dynamics, including the dynamics of digital minds; and a novel form of the cluelessness challenge to longtermism.
Three mistakes in the moral mathematics of existential risk
