At TFI, we feel that urgent action is needed to address risks from existing AI systems and frontier AI systems, and we utilize our expertise in strategic foresight and other domains to help mitigate these risks and guide humanity toward vibrant and prosperous futures. TFI’s research agenda includes a variety of projects that all have one thing in common—they are intended to inform policymakers and decision makers about the societal-scale risks of advanced AI and how such risks might be best managed. The research we conduct is both practical and methodological, but all aimed at helping us to identify promising policy interventions that can increase the likelihood of vibrant and prosperous futures for humanity.
Our core research effort involves research at the intersection of forecasting and scenario planning. We are conducting two forecasting tournaments: 1) a first pilot evaluating the value of using a Bayesian network tool for improving conditional forecasts and 2) a second pilot evaluating the value of this tool for improving accuracy and rationales in forecasting tournaments. In separate work, we are using a scenario network mapping process to identify societal-scale risks on a two to six year horizon. We will use the results of these workshops to inform Delphi processes for question generation (see more here). The novel scenario mapping and Delphi process we are developing is intended to help us identify the most critical questions to include in our AI forecasting tournament as well as early warning indicators for risks or dangerous emergent capabilities that could come from frontier AI systems.
Because we focus on societal-scale AI risks, we also are conducting a survey of AI experts and U.S. voters to elicit opinion regarding perceptions of the likelihood and the potential scale of impacts of various societal-scale risks. We are also working to better define societal-scale AI risks, and we plan to release a white paper later this year reporting the results of all of this work.
Our secondary research focus is the neglected domain of structural AI risks. Foresight and forecasting are essential to evaluating structural risks from advanced AI systems, and we are actively involved in several research projects in this area.
Evaluations are critical—capabilities, misuse, and structural—and TFI led an initial effort to develop a proposal for an international consortium for evaluations of societal-scale risks from advanced AI. The initial proposal has been published, and is linked to in the research publications section below. We are continuing to work to establish an organization like the proposed consortium to coordinate evaluations, set standards for evaluations and evaluations reporting, and to serve as an accrediting body for AI risk evaluators.
TFI is also working directly on evaluations of structural risks through a few different approaches. For one, we are using foresight and forecasting techniques to try to identify structural risks from advanced AI. This is important because systemic, structural risks may be the most difficult to anticipate. This effort is just an extension of our foresight work using scenario mapping for forecasting question generation, only it will specifically target systemic, structural risks in particular domains.
Another effort addresses reasoning-under-uncertainty functionality (e.g., prediction & forecasting), and the temptation of human decision-makers to use AI tools to assist them in this regard, when both are in fact poorly suited. The work outlines multiple risk pathways, their potentially compounding nature, linking current AI risk to salient extreme risk research, and describes necessary solutions which may yield outsized advantages to global problems. After these exploratory efforts, we will begin developing evaluations for these risks.
Yet another project addresses the system dynamics invoked by both AI researchers and policy-makers in their attempts to understand (and potentially curb) rates of AI development and (mis)use. Specifically, we are using complex system modelling to quantify (via simulation) the AI race/coordination problem, wherein the non-linear and interdependent dynamics of the multi-actor, multi-incentive problem is captured. This will enable us to test current assumptions in the AI forecasting field, determine likely policy efficacy in the AI governance space, and assess test AI race scenarios.
October 2023—An International Consortium for Evaluating Societal-Scale Risks from Advanced AI