Computing Power and the Governance of Artificial Intelligence

Computing Power and the Governance of Artificial Intelligence Girish Sastry,∗†1 Lennart Heim,∗†2 Haydn Belfield,∗†3 Markus Anderljung,∗2 Miles Brundage,∗1 Julian Hazell,∗2,4 Cullen O’Keefe,∗1,5 Gillian K. Hadfield,∗6,7 Richard Ngo,1 Konstantin Pilz,8 George Gor,9 Emma Bluemke,2 Sarah Shoker,1 Janet Egan,10 Robert F. Trager,11 Shahar Avin,12 Adrian Weller,13 Yoshua Bengio,14 Diane Coyle15 1OpenAI, 2Centre for the Governance of AI (GovAI), 3Leverhulme Centre for the Future of Intelligence, Uni. of Cambridge, 4Oxford Internet Institute, 5Institute for Law & AI, 6University of Toronto 7Vector Institute for AI, 8Georgetown University, 9ILINA Program, 10Harvard Kennedy School, 11AI Governance Institute, Uni. of Oxford, 12Centre for the Study of Existential Risk, Uni. of Cambridge, 13Uni. of Cambridge, 14Uni. of Montreal / Mila, 15Bennett Institute, Uni. of Cambridge February 14, 2024

Abstract

Computing power, or "compute," is crucial for the development and deployment of artificial intelligence (AI) capabilities. As a result, governments and companies have started to leverage compute as a means to govern AI. For example, governments are investing in domestic compute capacity, controlling the flow of compute to competing countries, and subsidizing compute access to certain sectors. However, these efforts only scratch the surface of how compute can be used to govern AI development and deployment. Relative to other key inputs to AI (data and algorithms), AI-relevant compute is a particularly effective point of intervention: it is detectable, excludable, and quantifiable, and is produced via an extremely concentrated supply chain. These characteristics, alongside the singular importance of compute for cutting-edge AI models, suggest that governing compute can contribute to achieving common policy objectives, such as ensuring the safety and beneficial use of AI. More precisely, policymakers could use compute to facilitate regulatory visibility of AI, allocate resources to promote beneficial outcomes, and enforce restrictions against irresponsible or malicious AI development and usage. However, while compute-based policies and technologies have the potential to assist in these areas, there is significant variation in their readiness for implementation. Some ideas are currently being piloted, while others are hindered by the need for fundamental research. Furthermore, naïve or poorly scoped approaches to compute governance carry significant risks in areas like privacy, economic impacts, and centralization of power. We end by suggesting guardrails to minimize these risks from compute governance. Each author contributed ideas and/or writing to the paper. However, being an author does not imply agreement with every claim made in the paper, nor does it represent an endorsement from any author’s respective organization. ∗ Denotes primary authors, who contributed most significantly to the direction and content of the paper. Both primary authors and other authors are listed in approximately descending order of contribution. † Indicates the corresponding authors: Girish Sastry (girish@openai.com), Lennart Heim (lennart.heim@governance.ai), and Haydn Belfield (hb492@cam.ac.uk). Figures can be accessed at https://github.com/lheim/CPGAI-Figures.

Appendix B

These drawbacks underline the need for technically-enabled enforcement to be accompanied with traditional methods of enforcement. They cannot operate effectively in isolation and should be complemented by other governance regimes, including methods to verify the integrity of these mechanisms. 112They are therefore more robust to failures of allocation, such as allowing bad actors to possess large quantities of compute. 113Of course, technically-enabled enforcement may not always be the best way to enforce rules for AI. The regulatory application of this tool requires sensitivity to its context (see e.g. Mulligan (2008)). 114There is no mechanism that differentiates “good AI” from “bad AI.” Rather, these assurances, and their corresponding mechanisms, are wide-ranging: from influencing the cost of AI model training to delaying deployment, increasing compute costs, or even applying specific constraints like preventing chips from training models on biological data. The desirability of each assurance is eventually informed by the threat model. Enforcing “compute caps” by technically limiting chip-to-chip networking Our first example is a relatively blunt method of leveraging compute to prevent violations of a rule. Training highly capable AI systems currently requires accumulating and orchestrating thousands of AI chips; if these systems are potentially dangerous, then limiting this accumulated computing power could serve to limit the production of potentially dangerous AI systems. How might this be accomplished? Instead of broadly limiting access to AI chips to prevent the development of potentially dangerous AI systems, regulators can implement a more targeted approach. This strategy would involve restricting the networking capabilities of these highperformance chips to prevent them from linking together to form large, powerful clusters. A mechanism for restricting cluster scalability could involve limiting communication outside of a pre-authorized number of chips. While communication between pre-authorized chips could occur at unrestricted bandwidth, communication with external chips or systems could be drastically limited. This confined communication limits the scalability into the large clusters required for the efficient training of large AI models. Determining the optimal bandwidth limit for external communication is an area that merits further research. Implementing limits on chip-to-chip networking could relax some of the trade-offs involved with broadly denying access to chips. However, the challenge lies in making these mechanisms as targeted as possible. It is true that current frontier AI training runs are extremely communication-intensive and require record-breaking numbers of AI chips, and yet imposing new limitations could also inadvertently affect other workloads. This suggests that the chip-level interventions required to limit large accumulations of compute should be designed to leave consumer use cases unaffected.115 Hardware-based remote enforcement In situations where AI systems pose catastrophic risks, it could be beneficial for regulators to verify that a set of AI chips are operated legitimately or to disable their operation (or a subset of it) if they violate rules. Modified AI chips may be able to support such actions, making it possible to remotely attest to a regulator that they are operating legitimately, and to cease to operate if not. Remote enforcement at the chip level could leverage existing cryptographic technology (Sommerhalder 2023; Sabt, Achemlal, and Bouabdallah 2015). One potential application of this technology is in enabling (ex post) visibility of workloads, but it can also be used for automatically enforcing rules.116 115For example, the consumer gaming experience does not benefit from large numbers of accumulated GPUs. 116Wherein an AI developer uses chips that store privacy-preserving logs of their workloads, and a Consider export controls on AI chips. Using traditional methods of enforcement incurs high administrative costs and inflates the scope of the controls as they have to focus on who accesses the chips, rather than what they are being used for.117 If remote authorization mechanisms are used, these export controls could be “digitized” (Reinsch and Benson 2021; US BIS 2020). Specialized co-processors that sit on the chip could hold a cryptographically signed digital “certificate,” and updates to the use-case policy could be delivered remotely via firmware updates. The authorization for the on-chip license could be periodically renewed by the regulator, while the chip producer could administer it.118 An expired or illegitimate license would cause the chip to not work, or reduce its performance.119 Remote enforcement mechanisms come with significant downsides, and may only be warranted if the expected harm from AI is extremely high. Notably, such mechanisms could themselves pose significant security (R. Anderson and Fuloria 2010) and privacy risks, as well as potential for the abuse of power. The inclusion of a mechanism to disable the device remotely could be manipulated by malicious actors or even misaligned autonomous AI systems to disable or otherwise manipulate computing infrastructure. This could lead to substantial financial losses or even pose risks to human safety in certain scenarios. Thus, if this approach is desirable at all, these mechanisms should focus on a specific subset of AI development and scenarios—for example, where rapid enforcement is particularly valuable. Preventing risky training runs via multiparty control Another future-oriented, speculative proposal, which may be justified only in extreme scenarios, involves a strategy to prevent undesirable AI training runs. This would operate by distributing the control over the metaphorical “start switch” either among multiple parties or to a governing third party. The power to decide how large regulator verifies after the fact that the developer is adhering to any requirements for their workloads (we discuss this in Section 4.A). 117That is, they are targeted broadly at the level of countries and organizations (users) on the theory that those targets run an unacceptable risk of using compute for harmful purposes. This user-level targeting is by necessity, as it is not currently possible for governments to reliably monitor or control how these chips are being exported. These export controls can have the drawback of limiting beneficial or benign use cases (e.g., scientific research or innovation in societally beneficial domains), even those that might benefit the countries imposing export controls in the first place. Additional side effects include increasing incentives for domestic development of semiconductor development by targeted countries, curbing the revenue of semiconductor companies located in democracies, increasing geopolitical tensions, and conveying the impression that researchers from certain backgrounds are being targeted as people (rather than the harmful use cases themselves). 118In principle, remote enforcement need not be “baked in” at the hardware level; one can imagine higher-level software that enforces rules on a data center; indeed, many cloud computing providers operate similar software. 119It is not just regulators who would benefit from these mechanisms. For example, chip producers could automatically enforce violations of their own terms of service. amounts of compute are used could be allocated via digital “votes” and “vetoes,” with the aim of ensuring that the most risky training runs and inference jobs are subject to increased scrutiny. The implementation of this could parallel the previous example of remote enforcement; multilateral control could be implemented through the use of multisignature cryptographic protocols (Cramer, Damgård, and Nielsen 2015). The software and hardware for AI chips could be modified to initiate processing instructions only when the workload is cryptographically signed by all parties. Institutionally, a number of configurations seem worthy of exploration. In a domestic setting, the control rights can be distributed to government regulators, independent auditors, or an international body, who should be incentivized to accurately assess the risk of the training run. While this may appear drastic relative to the current state of largely unregulated AI research, there is precedent in the case of other high-risk technologies: nuclear weapons use similar mechanisms, called permissive action links (“PALs”). PALs are security systems that require multiple authorized individuals in order to unlock nuclear weapons for possible use. By requiring the involvement of multiple parties, the system reduces the risk of human error or malicious intent, and increases the level of accountability for decisions related to nuclear weapons use. From one perspective, this mechanism could diffuse power, by making it harder for lone actors to unilaterally take actions with massive externalities (Bostrom, Douglas, and Sandberg 2016). But from another perspective, it could concentrate enormous power in the hands of every party that has the right to veto potential technical advances. We have seen how well-intentioned efforts to give many stakeholders the ability to veto decisions that could affect them can block various desirable forms of progress (e.g., VerWey (2021)), including progress towards the very goals that vetocratic policies aimed to advance (Fukuyama 2022). As with all policy measures, the substantive and procedural elements of this policy will determine its desirability. A separate problem is information security. Vote- and veto-holders must be informed of the relevant features of the training run to make an informed decision. But some details of the training run could be sensitive—either to individuals or commercial actors.120 The information shared with vote- and veto-holders would therefore have to be very carefully scoped. It may also be possible to construct “zero-knowledge” proofs of certain claims about proposed training runs that do not disclose sensitive information. More research into this possibility seems valuable (e.g., Buterin (2023),

Appendix B)

120We discuss these issues further in Section 5.A. Digital norm enforcement In some cases, enforcement via compute can enable more flexible and fine-grained prevention and response. One example involves implementing digital controls over compute resources from infrastructure-as-a-service (IaaS) entities, like cloud computing providers. Instead of outright denying access to chips, regulators can set restrictions on the total amount of compute usage permitted. These restrictions are digitally enforced by the IaaS companies themselves. Access to large-scale compute resources could be made conditional upon users complying with risk-reducing policies. For example, an AI developer (building on the IaaS’s compute) planning a large-scale deployment could be required to submit audit results of their AI model as a precondition for access (Egan and Heim 2023). Access could be easily restricted at any time if potential violations were detected. Ideally, decision-making regarding these conditional accesses should not be left at the discretion of IaaS companies, since they face flawed incentives (such as a profit incentive to overgrant access). An alternative would be to have decision-making governed by regulatory mandates and rely on the technical capabilities of IaaS companies for enforcement. As discussed in Section 5.A, this approach is akin to how digital services are shut down for legal violations, such as hosting illegal online drug markets. This method allows for more flexible and context-sensitive regulation than broad brush policies (like denying chips). Regulation could adapt to the rapidly evolving landscape of AI development and deployment while ensuring compliance with established legal and ethical standards. 5 Risks of Compute Governance and Possible Mitigations While governing AI via compute has significant potential as discussed above, pushing compute governance to extremes—especially when used as a tool for visibility and enforcement—bears significant risks that policymakers should carefully evaluate. As we have tried to emphasize above, compute governance is a double-edged sword: it can be used to promote widely shared objectives like safety, but it can also be used to infringe on civil liberties, prop up the powerful, and entrench authoritarian regimes. We discuss examples of such unintended consequences of compute governance below, including: threats to privacy; additional opportunities for leakage of commercially sensitive information; other negative economic impacts; and risks from centralization and concentration of power. Further, compute governance is a promising tool for AI governance in large part due to empirical factors that could change. We discuss such limitations to the feasibility and efficacy of compute governance. These include: algorithmic and hardware progress; low-compute specialized models with dangerous capabilities; and evasion, circumvention, and decoupling. To close out this section, we provide several overarching recommendations for guarding against some of these concerns. These include focusing on AI chips that are designed for AI supercomputers (excluding consumer-grade hardware as far as possible), using privacy-preserving practices and technologies, favoring compute-based measures for risks where ex ante measures are justified, periodically revisiting controlled computing technologies, implementing all controls with substantive and procedural safeguards, and using governable compute to protect society against risks from ungovernable compute. 5.A Limitations Unintended Consequences Threats to personal privacy In modern society, computational activity is core to most aspects of virtually every person’s life. The economic, social, political, cultural, intellectual, recreational, and health spheres are all largely enabled and mediated by computation. Thus, it is possible that any revelation or monitoring of an actors’ computational activities could reveal private and sensitive information. A number of the compute governance possibilities we explore (e.g., required reporting of large-scale training compute usage from cloud providers and AI developers, international AI chip registry, and privacy-preserving workload monitoring.) involve giving some actor more visibility into specific computational activities. For example, required reporting from cloud providers on customer usage could reveal sensitive information about companies or individuals. This visibility may reveal information about computational activities in which individuals have a legitimate privacy interest,121 or in which companies have a trade secret interest. It is reasonable to worry, then, that increasing visibility into AI-relevant computation could carry significant risks to privacy and civil liberties (e.g., Thierer (2023); Howard (2023)). Even in the context of large computing clusters, trade-offs between monitoring and privacy or security arise and cannot be addressed solely through means previously discussed, such as structured access via APIs. For example, cloud computing raises “tenant” privacy considerations—where customers seek assurance that their cloud provider is not, for example, stealing their IP—that need to be protected strictly and that pose challenges for AI-related monitoring. Government (especially military) data centers may be particularly sensitive to disclosure, and the semiconductor supply chain is regularly targeted for espionage purposes, which could compromise some efforts discussed here absent significant effort. Opportunities for leakage of sensitive strategic and commercial information Many of the compute governance ideas discussed above—especially those in Section 4.A—involve sharing information about compute and compute usage with policymakers. As discussed, there can be large benefits to this sort of visibility. But where these approaches have poor information security or are overly broad, they could create opportunities for the disclosed information to leak, to the competitive detriment of the regulated companies. Such leaks could also undermine trust and exacerbate racing dynamics, making it more challenging to establish effective policy for the governance of AI. Frontier AI labs increasingly withhold information about the processes used to create their flagship models, including the amount of compute used to create them.122 Revealing this information could, for example, help commercial competitors and geopolitical rivals understand how great of an investment would be needed to replicate 121However, we note that most of the visibility mechanisms we discuss above are targeted at corporate model developers, not consumers. 122For example, compare GPT-2 (Radford et al. 2019) with GPT-4 (OpenAI et al. 2023). the capabilities of an existing model. In some instances, the details sought by regulators may be considered highly confidential within the frontier AI labs themselves, accessible to only a select group of employees. Thus, secrecy helps AI labs preserve their economic competitiveness, and also slows diffusion of capabilities advances to geopolitical rivals. However, as this information is made available to policymakers, additional opportunities for this information to leak arise. Similarly, cloud compute providers often do not release much information about the location, capacity, and operation of their large data centers. They invest a substantial amount in physical security and cybersecurity (Pilz and Heim 2023). Policymaker demands for access to or visibility into the supply chain or operation of these data centers could create additional vectors for attack or compromise of sensitive information. Poor information security could dramatically increase the costs of compliance for AI companies, leak trade secrets, and accelerate proliferation of potentially dangerous capabilities (Anderljung, Barnhart, et al. 2023). As discussed in Section 5.B, compute governance measures must therefore be carefully scoped and implemented with information security in mind. Negative economic impacts Research by the U.S. Bureau of Economic Analysis suggests that the digital economy accounts for 10% of U.S. GDP (Highfill and Surfield 2022).123 The “permissionless” nature of most computational activity is a large part of why digital technologies have been such a force for economic growth (Thierer 2014). It is therefore reasonable to worry that placing burdens on access to certain compute—the substrate of the digital economy—could impose meaningful economic costs (Thierer 2023). For example, we consider KYC requirements for access to large-scale computation above. A skeptic might worry that even a presently high threshold for KYC checks will ultimately cover a sizable portion of the AI industry as compute usage increases, causing significant frictions to economic activity. We also consider export controls, but the history of export control policy is replete with debates around the trade-offs between strategic benefits from controlling exports to rivals and increasing domestic production, including general skepticism toward the effectiveness of many controls (Mastanduno 1992). Some of the more dramatic governance approaches we explore above—such as the CERN for AI and multiparty control of large-scale compute usage—contemplate centralizing or concentrating the development of the most capable, compute-intensive, 123This number is expected to grow significantly; the revised definitions for GDP due to be adopted by the UN in 2025 will likely set out a consistent and more inclusive method for measuring the digital contribution across countries, and work is underway to define and measure the contribution of AI (Briggs and Kodnani 2023). general AI systems. However, if that is not accompanied by widespread ability to build on and deploy such systems, we may fail to harness the creativity of the market, with accompanying loss of economic growth. Risks from centralization and concentration of power Right now, control over computation is fairly widely distributed.124 Greater central regulatory or allocative authority over large concentrations of compute will increase centralized control over an increasingly crucial economic and political resource. This carries serious risks (Thierer 2023; Howard 2023). Some of the risks from centralized control are technical. Remote enforcement mechanisms like kill switches can introduce security risks and the potential for control or manipulation (R. Anderson and Fuloria 2010). Compute visibility mechanisms may create concentrated repositories of information that are attractive to bad actors. Other risks are political. With increased government control over AI-relevant compute, powerful actors—including corporations—may try to wield the power of the state for their own ends, e.g., attempting regulatory capture. More fundamentally, history shows that centralizing power can carry significant—and even catastrophic—downsides, such as entrenching existing inequalities (Golden and Londregan 2006), suppressing dissent (Wallach 1991), creating poor epistemic standards among governing powers (E. Anderson 2006), and promoting poor economic decision-making (Acemoglu and Robinson 2012; Scott 1999). Issues of Feasibility and Efficacy Algorithmic and hardware progress Compute governance is more effective when, all else equal, (1) it takes a large amount of compute to achieve a certain level of capabilities, (2) the cost per unit of compute is high, and (3) using a large amount of compute requires usage of a large data center.125 However, certain long-run trends are slowly weakening each of these. Due to algorithmic progress, it takes fewer and fewer computational operations each year to achieve a given level of AI performance (Hernandez and T. B. Brown 2020; Erdil and Besiroglu 124However, as discussed above, the supply chain for AI chips and large data centers is extremely concentrated. Existing compute providers do not seem to leverage this existing power for political or ideological purposes, though perhaps they will in the future. This dynamic resembles the leverage that social media and other communications platforms could (and often do) exercise over speech on their platform, which is the subject of ongoing controversy (e.g., Klonick (2017)). 125This is because larger data centers are (1) easier to detect, (2) more expensive to build, (3) less common, and (4) more likely to be used for larger training runs, given the efficiencies of hosting a training run in a single data center. 2022b).126 Due to Moore’s Law127 and more specialized architectures, a dollar can buy many more operations every year (Hobbhahn and Besiroglu 2022; Hobbhahn, Heim, and Aydos 2023). Also, major progress in communication-efficient training could allow more decentralized training—i.e., splitting a single training run across multiple data centers—allowing training runs of a constant size to be hosted on multiple smaller data centers (Yuan et al. 2022). This makes it harder to identify and distinguish data centers potentially useable for large training runs. It is unclear whether these trends will continue in the long run, and what their limits, if any, are. Thus, each year it becomes more feasible to train models to a given level of performance using less, cheaper, and more decentralized compute, and consequently somewhat less governable.128 The extent to which this effect undermines compute governance largely depends on the importance of relative versus absolute capabilities. Increases in compute efficiency make it easier and cheaper to access a certain level of capability, but as long as scaling continues to pay dividends, the highest-capability models are likely to be developed by a small number of actors, whose behavior can be regulated via compute (Pilz, Heim, and N. Brown 2023). This dynamic could change if the scaling paradigm diminishes in effectiveness (Lohn 2023) or if decentralized training becomes feasible.129 That is to say, over time the amount of compute needed to train a system with a particular level of capability (e.g. GPT-4 or Claude 2 level in 2023) will decrease, but the amount of compute needed to train a system with a frontier level capability (a hypothetical GPT-5 and GPT-6 or Claude 3 and Claude 4) will increase. Low-compute specialized models with dangerous capabilities Specialized models trained on high-quality data require significantly less training compute to reach a high level of performance on particular tasks, compared to today’s most well-known generally capable models. For example, AlphaFold 2 achieved superhuman 126Compute itself is arguably a significant driver of algorithmic progress (Barnett 2023), as it enables experimenting with more architectures, scaling up what works, and gaining insights that may be only visible at scale. 127Moore’s Law originally referred to the density of transistors on a chip (G. E. Moore 1998), but has since been used colloquially to refer to the general exponential improvements in the performance of chips (in large part due to increasing transistor density). 128Dramatically improved computing hardware would certainly change aspects of AI development, but might not necessarily alter the role or importance of compute governance. Semiconductors have powered computing for decades and will likely continue to do so. Alternative compute architectures seem to face significant challenges: quantum computing is likely still distant and poorly suited for training AI models (Sevilla and Riedel 2020). Neuromorphic chips are primarily useful for inference, and likely still require the silicon supply chain in the short term. Optical computing remains mostly speculative. While new hardware may improve efficiency, it would not eliminate the need for computational power to develop AI systems. 129While progress in decentralized training may allow more actors to train models of a certain capability, such efforts would likely still be enormously resource-intensive. performance on protein folding prediction using fewer than 1023 operations—two orders of magnitude less compute than models like GPT-4 (Epoch 2022). If such low-compute models could cause significant harm, compute governance could be ineffective or inadvertently impose on harmless activity. Compute governance seems most appropriate where risk originates from a small number of hugely compute-intensive general models. This fact is also recognized in the 2023 U.S. Executive Order on AI, where reporting requirements are imposed on models trained on biological sequence data using three orders of magnitude less compute than other models—1023 operations vs. 1026 operations (The White House 2023)—in light of such models’ potential for biological weapons design (J. B. Sandbrink 2023). Dangerous capabilities can also arise through changes made to AI systems post-training. For example, with just $200 and one GPU, researchers were able to untrain (via finetuning) the safety features of Meta’s Llama 2 Chat (the model’s weights were publicly available). This intervention caused the subverted model to respond to requests for harmful information in the vast majority of cases (Lermen, Rogers-Smith, and Ladish 2023). This was despite Meta’s investments in safety testing and red teaming (Touvron et al. 2023). A broader set of policy approaches will be needed to further investigate and mitigate these risks. Once trained, high-compute models can be run using less costly computational resources. Some important and (potentially dangerous) AI capabilities may be accessible without high-end compute. For instance, protein folding capabilities can be harnessed with only a handful of inferences (Jumper et al. 2021). One can imagine successor models trained on biological data that could potentially use small amounts of inference compute to identify novel pathogens. Moreover, there is growing interest in the downsizing of AI models to be compatible with consumer or edge devices like smartphones or laptops. For example, Stable Diffusion v1.5 (albeit operating slowly) can now run locally on a phone (Vincent 2023), potentially giving rise to the proliferation of visual “deepfakes.” In general, compute governance measures would be unable to reliably “reach” the computing hardware sufficient to create or run a small number of instances of such low-compute models. Regulation of such low-compute models will require other policy approaches. Incentives for diversion, evasion, circumvention, and decoupling Actors are likely to attempt to circumvent and evade compute governance interventions, especially where their access to AI chips or their privacy is severely affected. Cutting off access to compute, for example—either preemptively or reactively—is a blunt instrument and has many downsides. We are already seeing such dynamics play out as a result of U.S. export controls on AI chips to China (Fist, Heim, and Schneider 2023). In the short term, there are attempts to circumvent these AI chip export controls via chip smuggling, using non-controlled chips, or accessing cloud compute (Fist, Heim, and Schneider 2023; Grunewald and Aird 2023). Attempts by non-state groups to evade controls on other materials, such as explosives, chemicals, biological agents, and radioactive material, are common (Gregory C. Allen, Benson, and Reinsch 2022). In the medium and long term, however, denying compute could further incentivize other attempts to get around a limit. Squeezing one part of the supply chain puts pressure on other parts. Actors without access to high-end chips are incentivized to find ways to utilize larger quantities of lower-grade chips. Restricting Chinese access to AI chips creates even stronger economic incentives to build a supply chain free of U.S. involvement. Though this would be incredibly challenging, over time, this could potentially create a wholly separate supply chain, reducing strategic interdependence and the ability to govern AI using compute—often referred to as “decoupling.” Separately, additional scrutiny on training runs above a certain threshold could further incentivize research into algorithmic breakthroughs. However, those incentives are already very strong since they can increase one’s “effective compute” given a certain quantity of actual compute. 5.B Guardrails for Compute Governance Given these serious downside risks, compute governance efforts should be thoughtfully designed and executed, and include safeguards to protect against abuse. We explore some possible approaches to doing so here. A recurring theme of these heuristics is the need for compute governance measures to be carefully scoped to tackle the largest risks while reducing the impacts on consumers and individuals. Our five principles are:

Acknowledgments

Thanks to Alex Savard, Allan Dafoe, Andrew Lohn, Andrew Trask, Carrick Flynn, Chris Phenicie, David Robinson, Gretchen Krueger, Jaan Tallin, Jade Leung, Katarina Slama, Larissa Schiavo, Lewis Ho, Lucy Lim, Magnus Løiten, Matthijs Maas, Mauricio Baker, Michael Lampe, Paul Scharre, Rosie Campbell, Sam Manning, Sean O hEigeartaigh, Tim Fist, Tom Davidson, Tom Westgarth, and Yonadav Shavit for feedback on earlier versions of this paper, and Eden Beck for editorial revision. Thank you to Alex Savard for graphic design help. Miles dedicates this paper to the memory of his father, Jan Brundage. GPT-4 and Claude were used to suggest ideas and provide feedback during the writing process. A The Compute-Uranium Analogy There is a suggestive analogy between a key physical input to two powerful technologies: compute and data centers in the case of AI, and uranium and enrichment facilities in the case of nuclear energy or weapons.138 Uranium mining and enrichment and compute fabrication and training both lead to outputs that can be used for both safe and harmful purposes, require significant capital investments, and can be differentiated by quantitative measures of quality (e.g., operations per watt or the level of enrichment). One way of envisaging this analogy is shown in Figure 15.AI Training is Similar to Uranium Enrichment Uranium Enrichment AI Training Uranium Ore Yellowcake Low Enriched Uranium Low Capability AI model Highly Enriched Uranium High Capability AI model Chip Materials Advanced Chips Mining Centrifuge

Enrichment Training in

Data CenterFabrication Figure 15: The analogy between uranium enrichment and AI training. For both AI (chips) and nuclear energy (uranium), there is a key input that is difficult to produce and potentially regulable. Uranium ore goes through a process of mining to produce yellowcake, which then goes through a process of enrichment to produce either low or highly-enriched uranium. One can draw an analogy with compute: materials go through a process of fabrication to produce chips, which then are used in a process of training to produce a model (below or above some level of capability). Each process is lengthy, difficult, expensive, and potentially amenable to monitoring. This analogy, while imperfect,139 is encouraging. Risks associated with nuclear en138Nuclear weapons can also be made with plutonium, though similar considerations apply. 139Like other analogies, the comparison between training and uranium enrichment has its limitations. In particular, while both are dual-use, it is possible to infer a narrower set of potential uses for enriched uranium, while high-capability models can be applied to a wider variety of use cases. The analogy is inexact as the chips are the physical location where the process occurs, rather than the material that is processed—which is data using particular algorithms. So an individual AI chip processing data, say, can be compared to an individual centrifuge enriching uranium, while a data center can be richment have been (more or less) managed for decades. A wide variety of technical and political measures are used to control the production, flow, and use of nuclear materials. Many of these can be thought of as “accounting”: keeping and checking careful records of who is creating nuclear material, where it goes, how it’s used, and how it’s disposed of. There are national and international regimes to track and monitor mines, yellowcake, enrichment plants, and enriched uranium. These measures include export controls, inspections by the International Atomic Energy Administration (IAEA), remote monitoring, unique identifiers such as serial numbers, and regulations around the use and disposal of nuclear materials. Enrichment capacity has been a key focus of nuclear nonproliferation regimes, as it is the key determinant of “breakout time”: the minimum time for a state to produce enough weapons-grade enriched uranium fuel for a single nuclear weapon. Taken together, these measures have contributed to a low rate of proliferation and the prevention of a nuclear conflict since 1945. Analogously, we may also want to consider regimes to track and monitor chip fabrication, the resulting advanced AI chips, and their final destination in data centers. Much like how unique IDs and tamper-proofing are employed to track uranium, complemented by monitoring through inspections and intelligence sources like satellite footage, we could envision similar methodologies being theoretically applicable to the advanced AI chip supply chain (Baker 2023). Compute may even have some advantages over nuclear. For example, there are between 6 and 59 manufacturers for each of the dual-use goods under the Nuclear Suppliers Group’s purview (Doyle 2019). By contrast, some steps in the compute supply chain have only a single company. Our use of this analogy should not be interpreted to mean that we overlook its limitations. Difficult political battles were required to achieve today’s modern institutions, and we recognize that nonproliferation governance continues to be a contested space, as demonstrated in the last nuclear nonproliferation treaty review conference (UN 2022; Potter 2023). As Stewart (2023) writes, modern nuclear nonproliferation governance evolved over decades, through international crises rather than preemptively, resulting in a governance patchwork instead of a universally coherent standard. And we would not be able to apply the same technical methods used for monitoring uranium to compute. The nonradioactivity of compute makes it more difficult to detect at ports and other border crossings than nuclear material. Many efforts related to tracking nuclear material, including international inspections, depend on the ability to correlate radioactivity emissions with the precise chemical properties of uranium (as well as plutonium). There is another significant limitation associated with the nuclear analogy; namely, while the emphasis on hardware excludability advances nonproliferation aims, there are no obvious parallels in nuclear proliferation to the release of model weights. Recent compared to an enrichment plant. historical cases demonstrate that access to information about nuclear weapons design— non-rivalrous, easily replicable and transferable—is often an insufficient condition for nuclear proliferation (R. S. Kemp 2014; Ouagrham-Gormley 2014). We can reasonably conclude that a bad actor with access to scientific information would not seriously undermine the existence of the nuclear nonproliferation regime. The release of model weights, on the other hand, poses a significant threat to nonproliferation compute regimes, because their public availability would allow an individual with a moderate amount of machine learning expertise to bypass the large compute requirements needed for training a model.140 This is an argument not only for strong cybersecurity controls, but for avoiding a repeat of the historical setbacks suffered by nuclear nonproliferation regimes and adopting preemptive governance mechanisms before the wide availability of model weights potentially undermines safeguards. Despite these limitations with the analogy, it is striking that society has also safely produced fairly large quantities of nuclear power (Ritchie and Rosado 2023) and that there have been zero instances of nuclear terrorism in the nearly 80 years since the advent of nuclear weapons technology. The political and technical regimes that govern nuclear technology likely deserve some credit for this situation. While this system is certainly imperfect—rogue states like North Korea have still managed to build up their nuclear capacity in part via illegal proliferation networks (Chestnut 2007; Reiss and Galluci 2005)—it is nevertheless a proof of concept for an institutional design that governs a highly sought after, dual-use technology at global scale. B Research Directions Policy implications of increased compute efficiency Given ongoing algorithmic and hardware progress, an AI capability that is initially only available to a small number of well-resourced developers will slowly diffuse to increasingly compute-limited actors over time (Pilz, Heim, and N. Brown 2023). This makes it challenging to construct enduring compute-based policy. Instead, compute governance interventions may be about influencing not whether but when certain capabilities are made available, to whom, and for what purpose. This view suggests that society will need to use the time bought by certain compute governance interventions to prepare for the widespread diffusion of advanced AI capabilities. To what extent is this picture correct? If so, what can policymakers do to increase society’s 140Because the computing power necessary to run a model is much less than the computing power needed to train a model. resilience to the diffusion of increasingly capable AI systems? Relatedly, what does the offense-defense balance look like for different AI capabilities? One could potentially use the compute required to develop different capabilities as one input to the offense-defense balance. Will the proliferating systems favor the offense or defense? Will it be possible to use compute resources to counter harm, such as by using highly performing systems or deploying defensive applications on a large scale? Trustworthy verification of compute capabilities and usage Implementing compute governance requires the ability to verify claims about actors’ compute capabilities and usage (Brundage, Avin, J. Wang, et al. 2020). However, validating these claims often involves accessing sensitive information that actors wish to keep private. On-site inspections intended to verify the number of chips an actor has access to might inadvertently reveal other sensitive information, and monitoring computational workloads is likely unacceptably intrusive in the international context with current techniques. This presents a challenge for arms control, especially when trust is low and competition is high (Coe and Vaynman 2020). How severe is this trade-off? What sorts of hardware, software, and institutional techniques might help alleviate it? Regulatory flight as a result of compute governance measures Certain compute governance interventions can induce regulatory flight, where activities are moved to less heavily regulated jurisdictions. For example, recent U.S. chip export controls incentivize the creation of an advanced chip supply chain without U.S. inputs. Similarly, customers may prefer compute providers with the least ability to monitor their compute capabilities and usage. What compute governance interventions are most likely to see their effectiveness undermined by regulatory flight? How can the chance of such flight be reduced? For instance, mechanisms for detecting black market AI chips in smaller countries may be important, since this compute could potentially be targeted by malicious actors who exploit differences in regulation. Countries caught in geopolitical competition The compute supply chain is increasingly shaped by geopolitical competition between great powers. How will other countries respond? To what extent can they avoid getting involved in the conflict, or will they be forced to ally with one side? What effects would this have on semiconductor supply chains? Incentivizing responsible compute provision How should responsible compute provision practices be incentivized and enforced by policymakers? Certain practices may require enforcement from the government via regulation or export controls. Other practices could be incentivized via liability regimes, where compute providers are held partly liable for lax attempts at identifying and thwarting misuse. Others may be implemented voluntarily by the compute provider industry. Limits of scaling What are the fundamental and practical limits to compute scaling? The past decade’s growth in compute usage for notable AI systems has been driven by increases in spending as well as reductions in the cost of compute. If these trends continue, training a state-of-the-art AI system would cost approximately 2.2% of U.S. GDP in 2032, similar to the annual cost of the Apollo program (Heim 2023b; Lohn and Musser 2022). Such spending would only be possible if the returns to scaling compute are immense. Further, many have argued that reductions in compute efficiency are likely to slow down, as Moore’s Law starts to hit its limits (Shalf 2020). Piloting compute governance levers To have confidence in deploying some of the levers described above at large scale, pilot studies may be needed to test their viability (such as the compute measurement pilot suggested by Brundage (Brundage, Avin, J. Wang, et al. 2020)), much as the Joint Verification Experiment demonstrated the feasibility of seismographic detection of nuclear testing (US Government 1988; Sykes and Ekström 1989). Security-privacy trade-offs Researchers should also more carefully analyze the trade-offs related to security and privacy, and assess what sorts of hardware, software, and institutional techniques might help alleviate such trade-offs. Further piloting, analysis, and debate are needed in order to more fully understand how compute can and can’t enable effective AI

References

Abramoff, Michael D. et al. (Oct. 2023). “Autonomous Artificial Intelligence Increases RealWorld Specialist Clinic Productivity in a Cluster-Randomized Trial”. In: npj Digital Medicine 6.1, pp. 1–8. ISSN: 2398-6352. DOI: 10.1038/s41746- 023- 00931- 7. URL: ht tps://www.nature.com/articles/s41746- 023- 00931- 7 (visited on 01/12/2024). Acemoglu, Daron, Camilo García-Jimeno, and James A. Robinson (2015). “State Capacity and Economic Development: A Network Approach”. In: The American Economic Review 105.8, pp. 2364–2409. ISSN: 0002-8282. JSTOR: 43821344. URL: https://www.jstor. org/stable/43821344 (visited on 01/13/2024). Acemoglu, Daron and James A. Robinson (2012). Why Nations Fail: The Origins of Power, Prosperity, and Poverty. 1. edition. New York, NY: Crown Publishing. ISBN: 978-0-307-719232. Adan, Sumaya Nur (Oct. 2023). The Case for Including the Global South in AI Governance Discussions. URL: https://www.governance.ai/post/the- case- forincluding-the-global-south-in-ai-governance-conversations (visited on 01/13/2024). Ahmed, Nur and Muntasir Wahed (Oct. 2020). The De-Democratization of AI: Deep Learning and the Compute Divide in Artificial Intelligence Research. arXiv. DOI: 10.48550/arX iv.2010.15581. URL: http://arxiv.org/abs/2010.15581 (visited on 01/13/2024). Aleph Alpha (Nov. 2023). Aleph Alpha Raises a Total Investment of More than Half a Billion US Dollars from a Consortium of Industry Leaders and New Investors. URL: https:// aleph-alpha.com/aleph-alpha-raises-a-total-investment-ofmore-than-half-a-billion-us-dollars-from-a-consortium-ofindustry-leaders-and-new-investors/ (visited on 01/12/2024). Allen, Gregory C. (Oct. 2022). Choking off China’s Access to the Future of AI. Center for Strategic & International Studies. URL: https://www.csis.org/analysis/chokingchinas-access-future-ai (visited on 01/12/2024). – (May 2023). “China’s New Strategy for Waging the Microchip Tech War”. In: URL: https: //www.csis.org/analysis/chinas-new-strategy-waging-microc hip-tech-war (visited on 01/12/2024). Allen, Gregory C., Emily Benson, and William Alan Reinsch (Nov. 2022). Improved Export Controls Enforcement Technology Needed for U.S. National Security. URL: https://ww w.csis.org/analysis/improved-export-controls-enforcementtechnology-needed-us-national-security (visited on 01/13/2024). Amazon (Oct. 2023). Amazon EC2 P4d Instances. URL: https://web.archive.org/ web / 20231006023059 / https : / / aws . amazon . com / ec2 / instance types/p4/ (visited on 01/13/2024). – (2024). AWS GovCloud (US). URL: https://aws.amazon.com/govcloud-us/ ?whats-new-ess.sort-by=item.additionalFields.postDateTime &whats-new-ess.sort-order=desc (visited on 01/13/2024). Amodei, Dario and Danny Hernandez (May 2018). AI and Compute. URL: https://openai. com/research/ai-and-compute (visited on 01/12/2024). Anderljung, Markus, Joslyn Barnhart, et al. (Nov. 2023). Frontier AI Regulation: Managing Emerging Risks to Public Safety. arXiv. DOI: 10.48550/arXiv.2307.03718. URL: http://arxiv.org/abs/2307.03718 (visited on 01/12/2024). Anderljung, Markus and Julian Hazell (Mar. 2023). Protecting Society from AI Misuse: When Are Restrictions on Capabilities Warranted? arXiv. DOI: 10.48550/arXiv.2303.09377. URL: http://arxiv.org/abs/2303.09377 (visited on 01/12/2024). Anderson, Elizabeth (2006). “The Epistemology of Democracy”. In: Episteme: A Journal of Social Epistemology 3.1, pp. 8–22. ISSN: 1750-0117. DOI: 10.1353/epi.0.0000. URL: http : / / muse . jhu . edu / content / crossref / journals / episteme / v003/3.1anderson.html (visited on 01/13/2024). Anderson, Ross and Shailendra Fuloria (Oct. 2010). “Who Controls the off Switch?” In: 2010 First IEEE International Conference on Smart Grid Communications, pp. 96–101. DOI: 10.1 109/SMARTGRID.2010.5622026. URL: https://ieeexplore.ieee.org/ abstract/document/5622026 (visited on 01/13/2024). Anthropic (Sept. 2023). Anthropic’s Responsible Scaling Policy. URL: https://www.anthr opic.com/index/anthropics-responsible-scaling-policy (visited on 01/12/2024). – (2024). Anthropic Partners with Google Cloud. URL: https : / / www . anthropic . com / index / anthropic - partners - with - google - cloud (visited on 01/12/2024). Apple (June 2022). Deploying Transformers on the Apple Neural Engine. URL: https://mach inelearning.apple.com/research/neural-engine-transformers (visited on 01/12/2024). Armstrong, Stuart, Nick Bostrom, and Carl Shulman (May 2016). “Racing to the Precipice: A Model of Artificial Intelligence Development”. In: AI & SOCIETY 31.2, pp. 201–206. ISSN: 1435-5655. DOI: 10.1007/s00146-015-0590-y. URL: https://doi.org/ 10.1007/s00146-015-0590-y (visited on 01/13/2024). Arrow, Kenneth J. (June 1996). “The Economics of Information: An Exposition”. In: Empirica 23.2, pp. 119–128. ISSN: 1573-6911. DOI: 10 . 1007 / BF00925335. URL: https : //doi.org/10.1007/BF00925335 (visited on 01/12/2024). Askell, Amanda, Miles Brundage, and Gillian Hadfield (July 2019). The Role of Cooperation in Responsible AI Development. arXiv. DOI: 10.48550/arXiv.1907.04534. URL: http://arxiv.org/abs/1907.04534 (visited on 01/13/2024). Bach, Deborah (Sept. 2023). How a Small City in Iowa Became an Epicenter for Advancing AI. URL: https://news.microsoft.com/source/features/ai/westdes-moines-iowa-ai-supercomputer/ (visited on 01/12/2024). Bai, Yuntao et al. (Dec. 2022). Constitutional AI: Harmlessness from AI Feedback. URL: https: //arxiv.org/abs/2212.08073v1 (visited on 01/12/2024). Baily, Martin Neil, Erik Brynjolfsson, and Anton Korinek (May 2023). Machines of Mind: The Case for an AI-powered Productivity Boom. URL: https : / / www . brookings . edu/articles/machines-of-mind-the-case-for-an-ai-poweredproductivity-boom/ (visited on 01/12/2024). Baker, Mauricio (Apr. 2023). Nuclear Arms Control Verification and Lessons for AI Treaties. arXiv. URL: http://arxiv.org/abs/2304.04123 (visited on 01/13/2024). Barnett, Matthew (May 2023). A Compute-Based Framework for Thinking about the Future of AI. URL: https://epochai.org/blog/a-compute-based-frameworkfor-thinking-about-the-future-of-ai (visited on 01/13/2024). Barnett, Matthew and Tamay Besiroglu (Apr. 2023). The Direct Approach. URL: https : //epochai.org/blog/the-direct-approach (visited on 01/13/2024). Belfield, Haydn (Jan. 2020). Activism by the AI Community: Analysing Recent Achievements and Future Prospects. arXiv. DOI: 10.48550/arXiv.2001.06528. URL: http: //arxiv.org/abs/2001.06528 (visited on 01/12/2024). – (May 2023). Great British Cloud and BritGPT: The UK’s AI Industrial Strategy Must Play to Our Strengths. URL: https://www.labourlongterm.org/briefings/greatbritish-cloud-and-britgpt-the-uks-ai-industrial-strategymust-play-to-our-strengths (visited on 01/12/2024). Belfield, Haydn and Shin-Shin Hua (Aug. 2022). “Compute and Antitrust: Regulatory Implications of the AI Hardware Supply Chain, from Chip Design to Cloud APIs”. In: Verfassungsblog. DOI: 10.17176/20220819-181907-0. URL: https://verfassungsblog. de/compute-and-antitrust/ (visited on 01/12/2024). Belfield, Haydn and Christian Ruhl (July 2022). Why Policy Makers Should Beware Claims of New ’Arms Races’. URL: https://thebulletin.org/2022/07/why-poli cy-makers-should-beware-claims-of-new-arms-races/ (visited on 01/13/2024). Benaich, Nathan and Ian Hogarth (Oct. 2022). State of AI Report 2022. ONLINE. URL: https: //docs.google.com/presentation/d/1WrkeJ9-CjuotTXoa4ZZlB3U PBXpxe4B3FMs9R9tn34I (visited on 01/12/2024). Bengio, Yoshua (Sept. 2023). AI and Catastrophic Risk. URL: https://www.journalof democracy.org/ai-and-catastrophic-risk/ (visited on 01/13/2024). Berglund, Lukas et al. (Sept. 2023). Taken out of Context: On Measuring Situational Awareness in LLMs. arXiv. DOI: 10.48550/arXiv.2309.00667. URL: http://arxiv.org/ abs/2309.00667 (visited on 01/12/2024). Berke, Allison (Nov. 2023). Can’t Quite Develop That Dangerous Pathogen? AI May Soon Be Able to Help. URL: https://thebulletin.org/2023/11/cant-quite-developthat-dangerous-pathogen-ai-may-soon-be-able-to-help/ (visited on 01/12/2024). Besiroglu, Tamay, Sage Andrus Bergerson, et al. (Jan. 2024). The Compute Divide in Machine Learning: A Threat to Academic Contribution and Scrutiny? URL: https://arxiv.org/ abs/2401.02452v2 (visited on 01/12/2024). Besiroglu, Tamay, Lennart Heim, and Jaime Sevilla (Mar. 2022). Projecting Compute Trends in Machine Learning. URL: https://epochai.org/blog/projecting-comput e-trends (visited on 01/13/2024). Birhane, Abeba (Aug. 2020). “Algorithmic Colonization of Africa”. In: SCRIPT-ed 17.2, pp. 389– 409. ISSN: 17442567. DOI: 10.2966/scrip.170220.389. URL: https://scri pt-ed.org/?p=3888 (visited on 01/13/2024). Bluemke, Emma et al. (Mar. 2023). Exploring the Relevance of Data Privacy-Enhancing Technologies for AI Governance Use Cases. arXiv. DOI: 10.48550/arXiv.2303.08956. URL: http://arxiv.org/abs/2303.08956 (visited on 01/13/2024). Boakye, Bridget, Pete Furlong, and Kevin Zandermann (Oct. 2022). Reaping the Rewards of the next Technological Revolution: How Africa Can Accelerate AI Adoption Today. URL: https: //www.institute.global/insights/tech- and- digitalisation/ reaping-rewards-next-technological-revolution-how-africacan-accelerate-ai-adoption-today (visited on 01/13/2024). Boakye, Bridget, Melanie Garson, et al. (Dec. 2023). State of Compute Access: How to Bridge the New Digital Divide. URL: https://www.institute.global/insights/techand-digitalisation/state-of-compute-access-how-to-bridgethe-new-digital-divide (visited on 01/13/2024). Bommasani, Rishi (Nov. 2023). Drawing Lines: Tiers for Foundation Models. URL: https: //crfm.stanford.edu/2023/11/18/tiers.html (visited on 01/12/2024). Bommasani, Rishi et al. (July 2022). On the Opportunities and Risks of Foundation Models. arXiv. DOI: 10.48550/arXiv.2108.07258. URL: http://arxiv.org/abs/ 2108.07258 (visited on 01/12/2024). Bostrom, Nick, Thomas Douglas, and Anders Sandberg (July 2016). “The Unilateralist’s Curse and the Case for a Principle of Conformity”. In: Social Epistemology 30.4, pp. 350–371. ISSN: 0269-1728. DOI: 10.1080/02691728.2015.1108373. pmid: 27499570. URL: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4959137/ (visited on 01/13/2024). Briggs, Joseph and Devesh Kodnani (Mar. 2023). The Potentially Large Effects of Artificial Intelligence on Economic Growth. URL: https://www.gspublishing.com/co ntent/research/en/reports/2023/03/27/d64e052b- 0f6e- 45d7967b-d7be35fabd16.html (visited on 01/13/2024). Browne, Ryan (Apr. 2023). Europe Approves Its $47 Billion Answer to Biden’s CHIPS Act — Here’s Everything That’s in It. URL: https://www.cnbc.com/2023/04/19/europeapproves- its- 47- billion- answer- to- bidens- chips- act.html (visited on 01/12/2024). Brundage, Miles, Shahar Avin, Jack Clark, et al. (Feb. 2018). The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation. arXiv. DOI: 10.48550/arXiv.1802. 07228. URL: http://arxiv.org/abs/1802.07228 (visited on 01/13/2024). Brundage, Miles, Shahar Avin, Jasmine Wang, et al. (Apr. 2020). Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims. URL: https://arxiv.org/ abs/2004.07213v2 (visited on 01/12/2024). Brynjolfsson, Erik, Danielle Li, and Lindsey R. Raymond (Apr. 2023). Generative AI at Work. National Bureau of Economic Research, Cambridge. URL: https://www.nber.o rg/system/files/working_papers/w31161/w31161.pdf (visited on 01/12/2024). Buchanan, Ben (Aug. 2020). The AI Triad and What It Means for National Security Strategy. Center for Security and Emerging Technology. URL: https://cset.georgetown. edu/publication/the-ai-triad-and-what-it-means-for-nation al-security-strategy/ (visited on 01/12/2024). Buterin, Vitalik (Nov. 2023). My Techno-Optimism. URL: https://vitalik.eth.limo/ general/2023/11/27/techno_optimism.html. CAIS (2024). Statement on AI Risk. Center for AI Safety. URL: https://www.safe.ai/ statement-on-ai-risk (visited on 01/12/2024). Caplan, Bryan (July 2008). “The Totalitarian Threat”. In: Global Catastrophic Risks. Ed. by Martin J Rees, Nick Bostrom, and Milan M Cirkovic. Oxford University Press. ISBN: 978-0-19857050-9. DOI: 10.1093/oso/9780198570509.003.0029. URL: https://d oi.org/10.1093/oso/9780198570509.003.0029 (visited on 01/13/2024). Carlsmith, Joseph (June 2022). Is Power-Seeking AI an Existential Risk? arXiv. URL: http: //arxiv.org/abs/2206.13353 (visited on 01/12/2024). Cave, Stephen and Seán S. ÓhÉigeartaigh (Dec. 2018). “An AI Race for Strategic Advantage: Rhetoric and Risks”. In: Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society. AIES ’18. New York, NY, USA: Association for Computing Machinery, pp. 36–

- Bureau of Industry and Security. URL: https://www.bis.doc.gov/index. php/documents/update- 2017/2124- facilitating- end- users- 1kurland-eco-panel-opening-v2/file. Ladish, Jeffrey and Lennart Heim (May 2022). Information Security Considerations for AI and the Long Term Future. URL: https://blog.heim.xyz/information-securityconsiderations-for-ai/ (visited on 01/12/2024). Lawrance, Cate (Sept. 2023). Xavier Niel Invests =C200M to Bolster AI Sovereignty and Retain French Talent but the Country Is Lagging Far behind the US. URL: https://tech.eu/ 2023/09/29/xavier-niel-invests-eur200m-to-boost-ai-sovere ignty-in-retain-talent-in-france/ (visited on 01/12/2024). Leike, Jan (Dec. 2022). Distinguishing Three Alignment Taxes. URL: https://aligned. substack.com/p/three-alignment-taxes (visited on 01/13/2024). Lermen, Simon, Charlie Rogers-Smith, and Jeffrey Ladish (Oct. 2023). LoRA Fine-Tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B. arXiv. DOI: 10.48550/arXi v.2310.20624. URL: http://arxiv.org/abs/2310.20624 (visited on 01/13/2024). Licklider, Roy E. (1970). “The Missile Gap Controversy”. In: Political Science Quarterly 85.4, pp. 600–615. ISSN: 0032-3195. DOI: 10.2307/2147598. JSTOR: 2147598. URL: https://www.jstor.org/stable/2147598 (visited on 01/13/2024). Liu, Qianer (Nov. 30, 2023). “How Huawei Surprised the US with a Cutting-Edge Chip Made in China”. In: Financial Times. URL: https://www.ft.com/content/327414d2fe13-438e-9767-333cdb94c7e1 (visited on 01/22/2024). Lohn, Andrew (Dec. 2023). Scaling AI - Cost and Performance of AI at the Leading Edge. Center for Security and Emerging Technology. URL: https://cset.georgetown.edu/ publication/scaling-ai/ (visited on 01/13/2024). Lohn, Andrew and Micah Musser (Jan. 2022). AI and Compute. Center for Security and Emerging Technology. URL: https://cset.georgetown.edu/publication/ ai-and-compute/ (visited on 01/13/2024). Luccioni, Alexandra Sasha, Sylvain Viguier, and Anne-Laure Ligozat (Nov. 3, 2022). Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Model. DOI: 10.48550/ arXiv.2211.02001. arXiv: 2211.02001 [cs]. URL: http://arxiv.org/ abs/2211.02001 (visited on 02/06/2024). preprint. Luciana, Cingolani (2013). “The State of State Capacity : A Review of Concepts, Evidence and Measures”. In: MERIT Working Papers. URL: https://ideas.repec.org//p/ unm/unumer/2013053.html (visited on 01/12/2024). Maas, Matthijs (Aug. 2022). “Paths Untaken: The History, Epistemology and Strategy of Technological Restraint, and Lessons for AI”. In: Verfassungsblog. DOI: 10.17176/20220 810-061602-0. URL: https://verfassungsblog.de/paths-untaken/ (visited on 01/13/2024). Maas, Matthijs M. (Nov. 2023a). Advanced AI Governance: A Literature Review of Problems, Options, and Proposals. SSRN Scholarly Paper. Rochester, NY. DOI: 10.2139/ssrn. 4629460. URL: https://papers.ssrn.com/abstract=4629460 (visited on 01/12/2024). – (Oct. 2023b). Concepts in Advanced AI Governance: A Literature Review of Key Terms and Definitions. SSRN Scholarly Paper. Rochester, NY. DOI: 10.2139/ssrn.4612473. URL: https://papers.ssrn.com/abstract=4612473 (visited on 01/12/2024). Maes, Roel (2013). “Physically Unclonable Functions: Properties”. In: Physically Unclonable Functions: Constructions, Properties and Applications. Ed. by Roel Maes. Berlin, Heidelberg: Springer, pp. 49–80. ISBN: 978-3-642-41395-7. DOI: 10.1007/978-3-642-413957_3. URL: https://doi.org/10.1007/978-3-642-41395-7_3 (visited on 01/13/2024). Mastanduno, Michael (Jan. 1992). Economic Containment: Cocom and the Politics of East-West Trade. Ithaca, N.Y: Cornell University Press. ISBN: 978-0-8014-9996-8. Matheny, Jason (Aug. 2023). “Here’s a Simple Way to Regulate Powerful AI Models”. In: Washington Post. ISSN: 0190-8286. URL: https://www.washingtonpost.com/ opinions/2023/08/16/ai- danger- regulation- united- states/ (visited on 01/12/2024). Maug, Nicole, Aidan O’Gara, and Tamay Besiroglu (2024). Biological Sequence Models in the Context of the AI Directives. URL: https://epochai.org/blog/biologicalsequence-models-in-the-context-of-the-ai-directives. Mazzucato, Mariana (Jan. 2021). Mission Economy. URL: https://marianamazzucat o.com/books/mission-economy (visited on 01/13/2024). Menghani, Gaurav (Mar. 2023). “Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better”. In: ACM Computing Surveys 55.12, 259:1–259:37. ISSN: 0360-0300. DOI: 10.1145/3578938. URL: https://doi.org/10.1145/ 3578938 (visited on 01/12/2024). Microsoft (2024a). AI for Earth Data Sets. URL: https://microsoft.github.io/ AIforEarthDataSets/ (visited on 01/13/2024). – (2024b). Azure Products by Region. URL: https://azure.microsoft.com/enus/explore/global-infrastructure/products-by-region/ (visited on 01/13/2024). Miller, Chris (Oct. 2022). Chip War. Scribner. ISBN: 978-1-982172-00-8. URL: https:// www.simonandschuster.com/books/Chip-War/Chris-Miller/9781 982172008 (visited on 01/13/2024). Miller, Kyle and Andrew Lohn (Oct. 2023). Techniques to Make Large Language Models Smaller: An Explainer. URL: https://cset.georgetown.edu/publication/techn iques-to-make-large-language-models-smaller-an-explainer/ (visited on 01/12/2024). Moore, Gordon E (1998). “Cramming More Components onto Integrated Circuits”. In: PROCEEDINGS OF THE IEEE 86.1. URL: https://www.cs.utexas.edu/~fussell/ courses/cs352h/papers/moore.pdf. Moore, Samuel K. (July 2020). “A Better Way to Measure Progress in Semiconductors”. In: IEEE Spectrum. URL: https://spectrum.ieee.org/a- better- way- tomeasure-progress-in-semiconductors (visited on 01/12/2024). Morgan, Timothy Prickett (July 2023). H100 GPU Instance Pricing on AWS: Grin and Bear It. URL: https://www.nextplatform.com/2023/07/27/h100-gpu-instancepricing-on-aws-grin-and-bear-it/ (visited on 01/13/2024). – (Jan. 2024). The Datacenter GPU Gravy Train That No One Will Derail. URL: https:// www.nextplatform.com/2024/01/11/the-datacenter-gpu-gravytrain-that-no-one-will-derail/ (visited on 01/13/2024). Mouton, Christopher A., Caleb Lucas, and Ella Guest (Oct. 2023). The Operational Risks of AI in Large-Scale Biological Attacks: A Red-Team Approach. RAND Corporation. URL: https: //www.rand.org/pubs/research_reports/RRA2977-1.html (visited on 01/12/2024). Muggah, Robert and Ilona Szabo (May 29, 2023). Artificial Intelligence Will Entrench Global Inequality. URL: https://foreignpolicy.com/2023/05/29/ai-regulat ion-global-south-artificial-intelligence/ (visited on 01/13/2024). Mulligan, Christina (2008). “Perfect Enforcement Of Law: When To Limit And When To Use Technology”. In: Richmond Journal of Law & Technology 14.4. URL: https://sc holarship.richmond.edu/cgi/viewcontent.cgi?article=1295& context=jolt. Musser, Micah et al. (Apr. 2023). "The Main Resource Is the Human". Center for Security and Emerging Technology. URL: https://cset.georgetown.edu/publication/ the-main-resource-is-the-human/ (visited on 01/13/2024). NAIRR and The White House (Jan. 2023). National Artificial Intelligence Research Resource Task Force Releases Final Report. The White House. URL: https://www.whitehou se.gov/ostp/news- updates/2023/01/24/national- artificial- intelligence-research-resource-task-force-releases-finalreport/ (visited on 01/12/2024). Negus, Mitchell Gardiner (2021). “Privacy-Preserving Computation for Nuclear Safeguards”. PhD thesis. UC Berkeley. URL: https://escholarship.org/uc/item/1df8z 24f (visited on 01/13/2024). Nellis, Stephen and Max A. Cherney (Aug. 2023). “US Curbs AI Chip Exports from Nvidia and AMD to Some Middle East Countries”. In: Reuters. URL: https://www.reuters.com/ technology/us-restricts-exports-some-nvidia-chips-middleeast-countries-filing-2023-08-30/ (visited on 01/13/2024). Nellis, Stephen, Chavi Mehta, and Chavi Mehta (June 2023). “With No Big Customers Named, AMD’s AI Chip Challenge to Nvidia Remains Uphill Fight”. In: Reuters. URL: https:// www . reuters . com / technology / amd - likely - offer - details - ai chip-challenge-nvidia-2023-06-13/ (visited on 01/12/2024). Nevo, Sella et al. (Oct. 2023). Securing Artificial Intelligence Model Weights: Interim Report. RAND Corporation. URL: https://www.rand.org/pubs/working_papers/ WRA2849-1.html (visited on 01/12/2024). Ngo, Richard, Lawrence Chan, and Sören Mindermann (Sept. 2023). The Alignment Problem from a Deep Learning Perspective. arXiv. DOI: 10.48550/arXiv.2209.00626. URL: http://arxiv.org/abs/2209.00626 (visited on 01/12/2024). NTI (Sept. 2015). Information Protection & Information Barriers. URL: https://www.nti. org/analysis/articles/information- protection- informationbarriers/ (visited on 01/13/2024). NVIDIA (Mar. 2023). AWS and NVIDIA Collaborate on Next-Generation Infrastructure for Training Large Machine Learning Models and Building Generative AI Applications. URL: http:// nvidianews.nvidia.com/news/aws-and-nvidia-collaborate-onnext- generation- infrastructure- for- training- large- machin e-learning-models-and-building-generative-ai-applications (visited on 01/12/2024). – (2024a). NVIDIA DGX H100 Datasheet. URL: https://resources.nvidia.com/ en-us-dgx-systems/ai-enterprise-dgx (visited on 01/12/2024). – (2024b). NVIDIA H100 Tensor Core GPU. URL: https://www.nvidia.com/enus/data-center/h100/ (visited on 01/13/2024). O’Neill, Philip (Dec. 2009). Verification in an Age of Insecurity: The Future of Arms Control Compliance. Oxford University Press. ISBN: 978-0-19-977096-0. OECD (Nov. 2022). Measuring the Environmental Impacts of Artificial Intelligence Compute and Applications: The AI Footprint. Paris: OECD. DOI: 10.1787/7babf571-en. URL: https://www.oecd-ilibrary.org/science-and-technology/meas uring-the-environmental-impacts-of-artificial-intelligenc e-compute-and-applications_7babf571-en (visited on 01/12/2024). – (Feb. 2023). A Blueprint for Building National Compute Capacity for Artificial Intelligence. Paris: OECD. DOI: 10.1787/876367e3-en. URL: https://www.oecd-ilib rary.org/science-and-technology/a-blueprint-for-buildingnational-compute-capacity-for-artificial-intelligence_876 367e3-en (visited on 01/12/2024). – (2024). Expert Group on Compute & Climate. URL: https://oecd.ai/en//networ k-of-experts/working-group/1136 (visited on 01/12/2024). OES (2024). What We Believe. Organization for Ethical Source. URL: https://ethicals ource.dev/what-we-believe/ (visited on 01/13/2024). OFCOM (Apr. 2023). Cloud Services Market Study. OFCOM. URL: https://www.ofcom. org.uk/__data/assets/pdf_file/0029/256457/cloud-servicesmarket-study-interim-report.pdf (visited on 01/12/2024). ÓhÉigeartaigh, Seán S. et al. (Dec. 2020). “Overcoming Barriers to Cross-Cultural Cooperation in AI Ethics and Governance”. In: Philosophy & Technology 33.4, pp. 571–593. ISSN: 22105433, 2210-5441. DOI: 10.1007/s13347-020-00402-x. URL: https://link. springer.com/10.1007/s13347-020-00402-x (visited on 01/12/2024). Okolo, Chinasa, Kehinde Aruleba, and George Obaido (Jan. 2023). “Responsible AI in Africa – Challenges and Opportunities”. In: pp. 35–64. ISBN: 978-3-031-08214-6. DOI: 10.1007/ 978-3-031-08215-3_3. Okolo, Chinasa T. (Nov. 2023). AI in the Global South: Opportunities and Challenges towards More Inclusive Governance. URL: https://www.brookings.edu/articles/aiin-the-global-south-opportunities-and-challenges-towardsmore-inclusive-governance/ (visited on 01/13/2024). OPCW (2023). Chemical Weapons Convention - Article II. Organization for the Prohibition of Chemical Weapons. URL: https://www.opcw.org/chemical- weaponsconvention / articles / article - ii - definitions - and - criteria (visited on 01/12/2024). Open Knowledge (2024). Open Definition 2.1. Open Definition. URL: https://opendefi nition.org/od/2.1/en/ (visited on 01/13/2024). OpenAI (Dec. 2023). Preparedness. URL: https://openai.com/safety/prepare dness (visited on 01/12/2024). OpenAI et al. (Dec. 2023). GPT-4 Technical Report. arXiv. URL: http://arxiv.org/abs/ 2303.08774 (visited on 01/12/2024). Ortiz, Eleazar (Sept. 2021). Google Cloud Research Credits Expand to Nonprofit Researchers. URL: https://cloud.google.com/blog/topics/public-sector/googlecloud- research- credits- expand- nonprofit- researchers (visited on 01/13/2024). OSI (July 2006). The Open Source Definition. URL: https://opensource.org/osd/ (visited on 01/13/2024). Ostrom, Elinor (Sept. 2015). Governing the Commons: The Evolution of Institutions for Collective Action. Cambridge University Press. DOI: 10.1017/CBO9781316423936. URL: ht tps://www.cambridge.org/core/books/governing-the-commons/ A8BB63BC4A1433A50A3FB92EDBBB97D5 (visited on 01/12/2024). Ouagrham-Gormley, Sonia Ben (2014). Barriers to Bioweapons: The Challenges of Expertise and Organization for Weapons Development. Cornell University Press. ISBN: 978-0-8014-5288-8. JSTOR: 10.7591/j.ctt1287dk2. URL: https://www.jstor.org/stable/ 10.7591/j.ctt1287dk2 (visited on 01/13/2024). Owen, David (Jan. 2024). How Predictable Is Language Model Benchmark Performance? arXiv. DOI: 10.48550/arXiv.2401.04757. URL: http://arxiv.org/abs/2401. 04757 (visited on 01/12/2024). Pan, Che (Feb. 2023). Tech War: Chinese Chip Firms Stockpile Equipment Ahead of US-japannetherlands Agreement on Tightening Export Controls. URL: https://www.scmp.com/ tech/tech-war/article/3211416/tech-war-chinese-chip-firms -stockpile-equipment-ahead-us-japan-netherlands-agreementtightening (visited on 01/12/2024). Patel, Dylan (Jan. 2023). The Gaps in the New China Lithography Restrictions – ASML, SMEE, Nikon, Canon, EUV, DUV, ArFi, ArF Dry, KrF, and Photoresist. URL: https : / / www . semianalysis.com/p/the-gaps-in-the-new-china-lithography (visited on 01/12/2024). Patel, Dylan, Afzal Ahmad, and Myron Xie (Sept. 2023). China AI & Semiconductors Rise: US Sanctions Have Failed. URL: https://www.semianalysis.com/p/china-aiand-semiconductors-rise (visited on 01/12/2024). Patterson, David, Joseph Gonzalez, Urs Hölzle, et al. (Apr. 2022). The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink. arXiv. DOI: 10 . 48550 / arXi v.2204.05149. URL: http://arxiv.org/abs/2204.05149 (visited on 01/12/2024). Patterson, David, Joseph Gonzalez, Quoc Le, et al. (Apr. 2021). Carbon Emissions and Large Neural Network Training. arXiv. DOI: 10.48550/arXiv.2104.10350. URL: http: //arxiv.org/abs/2104.10350 (visited on 01/12/2024). Peng, Sida et al. (Feb. 2023). The Impact of AI on Developer Productivity: Evidence from GitHub Copilot. URL: https://arxiv.org/abs/2302.06590v1 (visited on 01/12/2024). Peterson, Dahlia and Samantha Hoffman (June 2022). “Geopolitical Implications of AI and Digital Surveillance Adoption”. In: URL: https://www.brookings.edu/articl es/geopolitical-implications-of-ai-and-digital-surveillan ce-adoption/. Philippe, Sébastien, Alexander Glaser, and Edward W. Felten (Jan. 2, 2019). “A Cryptographic Escrow for Treaty Declarations and Step-by-Step Verification”. In: Science & Global Security 27.1, pp. 3–14. ISSN: 0892-9882, 1547-7800. DOI: 10.1080/08929882.2019.1 573483. URL: https://www.tandfonline.com/doi/full/10.1080/ 08929882.2019.1573483 (visited on 01/19/2024). Pilz, Konstantin and Lennart Heim (Nov. 2023). Compute at Scale: A Broad Investigation into the Data Center Industry. arXiv. DOI: 10.48550/arXiv.2311.02651. URL: http://arxiv.org/abs/2311.02651 (visited on 01/12/2024). Pilz, Konstantin, Lennart Heim, and Nicholas Brown (Nov. 2023). Increased Compute Efficiency and the Diffusion of AI Capabilities. arXiv. DOI: 10.48550/arXiv.2311.15377. URL: http://arxiv.org/abs/2311.15377 (visited on 01/13/2024). Potter, William C. (Aug. 2023). “Behind the Scenes: How Not to Negotiate an Enhanced NPT Review Process”. In: Arms Control Today 53.8, pp. 18–23. URL: https://www. proquest.com/openview/870a5ed6cabd018ee6c0853374b3ccf4/1? pq-origsite=gscholar&cbl=37049 (visited on 01/13/2024). Radford, Alec et al. (2019). Language Models Are Unsupervised Multitask Learners. URL: https: //d4mucfpksywv.cloudfront.net/better-language-models/lang uage_models_are_unsupervised_multitask_learners.pdf. preprint. Reinsch, William Alan and Emily Benson (Dec. 2021). Digitizing Export Controls: A Trade Compliance Technology Stack? URL: https://www.csis.org/analysis/dig itizing-export-controls-trade-compliance-technology-stack (visited on 01/13/2024). Reiss, Mitchell B. and Robert Galluci (Mar. 2005). “Red-Handed”. In: Foreign Affairs 84.2. ISSN: 0015-7120. URL: https://www.foreignaffairs.com/articles/asia/ 2005-03-01/red-handed (visited on 01/13/2024). Richter, Felix (Aug. 2023). Infographic: Amazon Maintains Lead in the Cloud Market. URL: https://www.statista.com/chart/18819/worldwide-market-sha re-of-leading-cloud-infrastructure-service-providers (visited on 01/12/2024). Ritchie, Hannah and Pablo Rosado (Dec. 2023). “Nuclear Energy”. In: Our World in Data. URL: https://ourworldindata.org/nuclear-energy (visited on 01/13/2024). Russell, Stuart (Oct. 2019). Human Compatible. Penguin Random House. ISBN: 978-0-52555863-7. URL: https://www.penguinrandomhouse.com/books/566677/ human-compatible-by-stuart-russell/ (visited on 01/12/2024). Sabt, Mohamed, Mohammed Achemlal, and Abdelmadjid Bouabdallah (Aug. 2015). “Trusted Execution Environment: What It Is, and What It Is Not”. In: 2015 IEEE Trustcom / BigDataSE / ISPA. Helsinki, Finland: IEEE, pp. 57–64. ISBN: 978-1-4673-7952-6. DOI: 10.1109/ Trustcom.2015.357. URL: http://ieeexplore.ieee.org/document/ 7345265/ (visited on 01/13/2024). Samuelson, Paul A. (1954). “The Pure Theory of Public Expenditure”. In: The Review of Economics and Statistics 36.4, pp. 387–389. ISSN: 0034-6535. DOI: 10.2307/1925895. JSTOR: 1925895. URL: https://www.jstor.org/stable/1925895 (visited on 01/12/2024). Sandbrink, Jonas et al. (Sept. 2022). Differential Technology Development: An Innovation Governance Consideration for Navigating Technology Risks. SSRN Scholarly Paper. Rochester, NY. DOI: 10 . 2139 / ssrn . 4213670. URL: https : / / papers . ssrn . com / abstract=4213670 (visited on 01/12/2024). Sandbrink, Jonas B. (Dec. 2023). Artificial Intelligence and Biological Misuse: Differentiating Risks of Language Models and Biological Design Tools. arXiv. DOI: 10.48550/arXiv.2306. 13952. URL: http://arxiv.org/abs/2306.13952 (visited on 01/13/2024). Schaeffer, Rylan, Brando Miranda, and Sanmi Koyejo (2023). Are Emergent Abilities of Large Language Models a Mirage? arXiv: 2304.15004 [cs.AI]. Scharre, Paul (June 2021). Debunking the AI Arms Race Theory. URL: https://tnsr .org/2021/06/debunking- the- ai- arms- race- theory/ (visited on 01/13/2024). – (Jan. 2023). Decoupling Wastes U.S. Leverage on China. URL: https://foreignpol icy.com/2023/01/13/china-decoupling-chips-america/ (visited on 01/12/2024). Schleich, Matthew and William Alan Reinsch (Sept. 26, 2023). “Contextualizing the National Security Concerns over China’s Domestically Produced High-End Chip”. In: URL: https: //www.csis.org/analysis/contextualizing-national-securityconcerns-over-chinas-domestically-produced-high-end-chip (visited on 01/14/2024). Schuett, Jonas (Nov. 27, 2023). “Three Lines of Defense against Risks from AI”. In: AI & SOCIETY. ISSN: 1435-5655. DOI: 10.1007/s00146-023-01811-0. URL: https: //doi.org/10.1007/s00146-023-01811-0 (visited on 01/22/2024). Scott, James (Mar. 1999). Seeing like a State: How Certain Schemes to Improve the Human Condition Have Failed. Revised ed. edition. New Haven, CT London: Yale University Press. ISBN: 978-0-300-07815-2. Sefala, Raesetje et al. (Dec. 6, 2021). “Constructing a Visual Dataset to Study the Effects of Spatial Apartheid in South Africa”. In: Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1. URL: https://datasets-benchmark s-proceedings.neurips.cc/paper/2021/hash/07e1cd7dca89a167 8042477183b7ac3f-Abstract-round2.html (visited on 02/06/2024). Seger, Elizabeth, Noemi Dreksler, et al. (2023). “Open-Sourcing Highly Capable Foundation Models: An Evaluation of Risks, Benefits, and Alternative Methods for Pursuing OpenSource Objectives”. In: SSRN Electronic Journal. ISSN: 1556-5068. DOI: 10.2139/ssrn. 4596436. URL: https://www.ssrn.com/abstract=4596436 (visited on 01/12/2024). Seger, Elizabeth, Aviv Ovadya, et al. (Aug. 2023). Democratising AI: Multiple Meanings, Goals, and Methods. arXiv. DOI: 10.48550/arXiv.2303.12642. URL: http://arxiv. org/abs/2303.12642 (visited on 01/12/2024). Semiconductor Industry Association (2003). Overall Roadmap Technology Characteristics 2003. Semiconductor Industry Association. URL: https://www.semiconductors.org/ wp-content/uploads/2018/08/2003Overall-Roadmap-TechnologyCharacteristics.pdf (visited on 01/12/2024). Sevilla, Jaime, Lennart Heim, Anson Ho, et al. (July 2022). “Compute Trends across Three Eras of Machine Learning”. In: 2022 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. DOI: 10 . 1109 / IJCNN55064 . 2022 . 9891914. URL: http : //arxiv.org/abs/2202.05924 (visited on 01/14/2024). Sevilla, Jaime, Lennart Heim, Marius Hobbhahn, et al. (Jan. 2022). Estimating Training Compute of Deep Learning Models. Epoch AI. URL: https://epochai.org/blog/ estimating-training-compute (visited on 01/12/2024). Sevilla, Jaime and C. Jess Riedel (Dec. 2020). Forecasting Timelines of Quantum Computing. arXiv. DOI: 10.48550/arXiv.2009.05045. URL: http://arxiv.org/abs/ 2009.05045 (visited on 01/13/2024). Shalf, John (Jan. 2020). “The Future of Computing beyond Moore’s Law”. In: Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 378.2166, p. 20190061. DOI: 10.1098/rsta.2019.0061. URL: https://royalsocie typublishing.org/doi/full/10.1098/rsta.2019.0061 (visited on 01/13/2024). Shavit, Yonadav (May 2023). What Does It Take to Catch a Chinchilla? Verifying Rules on LargeScale Neural Network Training via Compute Monitoring. arXiv. URL: http://arxiv. org/abs/2303.11341 (visited on 01/13/2024). Shepardson, David (Mar. 2022). “U.S. Senate Approves $52 Bln Chips Bill in Bid to Reach Compromise”. In: Reuters. URL: https://www.reuters.com/world/us/ussenate- approves- 52- bln- chips- bill- bid- reach- compromise2022-03-28/ (visited on 01/12/2024). Shevlane, Toby (Apr. 2022). Structured Access: An Emerging Paradigm for Safe AI Deployment. arXiv. DOI: 10.48550/arXiv.2201.05159. URL: http://arxiv.org/abs/ 2201.05159 (visited on 01/13/2024). Shevlane, Toby and Allan Dafoe (Jan. 2020). The Offense-Defense Balance of Scientific Knowledge: Does Publishing AI Research Reduce Misuse? arXiv. DOI: 10 . 48550 / arXiv . 2001 . 00463. URL: http://arxiv.org/abs/2001.00463 (visited on 01/12/2024). Shevlane, Toby, Sebastian Farquhar, et al. (May 2023). Model Evaluation for Extreme Risks. URL: https://arxiv.org/abs/2305.15324v2 (visited on 01/12/2024). Shoker, Sarah et al. (Aug. 2023). Confidence-Building Measures for Artificial Intelligence: Workshop Proceedings. arXiv. DOI: 10.48550/arXiv.2308.00862. URL: http:// arxiv.org/abs/2308.00862 (visited on 01/12/2024). Smith, Brad (Sept. 2023). Developing and Deploying AI Responsibly: Elements of an Effective Legislative Framework to Regulate AI. URL: https://blogs.microsoft.com/onthe-issues/2023/09/12/developing-and-deploying-ai-respon sibly- elements- of- an- effective- legislative- framework- toregulate-ai/ (visited on 01/12/2024). Sommerhalder, Maria (2023). “Hardware Security Module”. In: Trends in Data Protection and Encryption Technologies. Ed. by Valentin Mulder et al. Cham: Springer Nature Switzerland, pp. 83–87. ISBN: 978-3-031-33386-6. DOI: 10.1007/978-3-031-33386-6_16. URL: https://doi.org/10.1007/978- 3- 031- 33386- 6_16 (visited on 01/13/2024). Steers, Sam (Mar. 2022). “Top 10 Underground Data Centres”. In: DataCentre. URL: https: //datacentremagazine.com/data-centres/top-10-undergrounddata-centres (visited on 01/12/2024). Stewart, Ian J. (June 2023). Why the IAEA Model May Not Be Best for Regulating Artificial Intelligence. URL: https://thebulletin.org/2023/06/why-the-iaeamodel-may-not-be-best-for-regulating-artificial-intellige nce/ (visited on 01/13/2024). Stix, Charlotte (Aug. 2022). “Foundations for the Future: Institution Building for the Purpose of Artificial Intelligence Governance”. In: AI and Ethics 2.3, pp. 463–476. ISSN: 2730-5961. DOI: 10.1007/s43681-021-00093-w. URL: https://doi.org/10.1007/ s43681-021-00093-w (visited on 01/13/2024). Sutton, Rich (Mar. 2019). The Bitter Lesson. URL: https://www.cs.utexas.edu/ ~eunsol/courses/data/bitter_lesson.pdf (visited on 01/12/2024). Sykes, Lynn R. and Göran Ekström (May 1989). “Comparison of Seismic and Hydrodynamic Yield Determinations for the Soviet Joint Verification Experiment of 1988”. In: Proceedings of the National Academy of Sciences 86.10, pp. 3456–3460. DOI: 10.1073/pnas.86. 10.3456. URL: https://www.pnas.org/doi/abs/10.1073/pnas.86. 10.3456 (visited on 01/13/2024). Tamkin, Alex et al. (Feb. 2021). Understanding the Capabilities, Limitations, and Societal Impact of Large Language Models. arXiv. DOI: 10.48550/arXiv.2102.02503. URL: http://arxiv.org/abs/2102.02503 (visited on 01/12/2024). Tarasov, Katie (Mar. 2022). ASML Is the Only Company Making the $200 Million Machines Needed to Print Every Advanced Microchip. Here’s an inside Look. URL: https://www.cnbc.co m/2022/03/23/inside-asml-the-company-advanced-chipmakersuse-for-euv-lithography.html (visited on 01/12/2024). Thadani, Akhil and Gregory C Allen (May 2023). “Mapping the Semiconductor Supply Chain: The Critical Role of the Indo-Pacific Region”. In: CSIS Briefs. URL: https://www.csis. org/analysis/mapping-semiconductor-supply-chain-criticalrole-indo-pacific-region. The White House (Oct. 2022). Remarks by National Security Advisor Jake Sullivan on the Biden-Harris Administration’s National Security Strategy. The White House. URL: https: //www.whitehouse.gov/briefing-room/speeches-remarks/2022/ 10/13/remarks-by-national-security-advisor-jake-sullivan- on-the-biden-harris-administrations-national-security-str ategy/ (visited on 01/12/2024). The White House (Oct. 2023). Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence. The White House. URL: https://www.whitehouse. gov/briefing-room/presidential-actions/2023/10/30/executi ve- order- on- the- safe- secure- and- trustworthy- developmentand-use-of-artificial-intelligence/ (visited on 01/12/2024). Thierer, Adam (2014). Permissionless Innovation: The Continuing Case for Comprehensive Technological Freedom. Mercatus Center, George Mason University. ISBN: 978-0-9892193-4-1. – (June 2023). “Existential Risks and Global Governance Issues around AI and Robotics”. In: R Street Policy Study 291. Thomson Reuters Foundation (Nov. 2023). AI Governance for Africa Toolkit - Parts 1 and 2. Thomson Reuters Foundation. URL: https://www.trust.org/dA/97390870 db/pdfReport/AI%20Governance%20for%20Africa%20Toolkit%20%20Part%201%20and%202.pdf (visited on 01/13/2024). Thrush, Glenn (Nov. 2021). “‘Ghost Guns’: Firearm Kits Bought Online Fuel Epidemic of Violence”. In: The New York Times. U.S. ISSN: 0362-4331. URL: https://www.nytimes. com/2021/11/14/us/ghost-guns-homemade-firearms.html (visited on 01/13/2024). Ting-Fang, Cheng (June 27, 2023). “ASML Says Decoupling Chip Supply Chain Is Practically Impossible”. In: Financial Times. URL: https://www.ft.com/content/317be8 b3-48d9-411e-b763-261a179c9d0d (visited on 01/22/2024). Touvron, Hugo et al. (July 2023). Llama 2: Open Foundation and Fine-Tuned Chat Models. arXiv. DOI: 10.48550/arXiv.2307.09288. URL: http://arxiv.org/abs/2307. 09288 (visited on 01/13/2024). Trask, Andrew et al. (Dec. 2020). Beyond Privacy Trade-Offs with Structured Transparency. arXiv. DOI: 10.48550/arXiv.2012.08347. URL: http://arxiv.org/abs/2012. 08347 (visited on 01/12/2024). U. S. Embassy in Ghana (Sept. 2023). U.S. Announces New Support for Ghana’s Civil Nuclear Energy Program under the FIRST Capacity Building Program. URL: https://gh.us embassy.gov/u- s- announces- new- support- for- ghanas- civilnuclear-energy-program-under-the-first-capacity-buildingprogram/ (visited on 01/13/2024). UK Cabinet Office (Mar. 2021). Global Britain in a Competitive Age. CP 403. HM Government. URL: https://assets.publishing.service.gov.uk/media/60644 e4bd3bf7f0c91eababd/Global_Britain_in_a_Competitive_Age_the_Integrated_Review_of_Security__Defence__Development_ and_Foreign_Policy.pdf. UK CMA (July 2021). NVIDIA - Arm: A Report to the Secretary of State for Digital, Culture, Media & Sport on the Anticipated Acquisiton by NVIDIA Corporation of Arm Limited. UK Competition and Markets Authority. URL: https://assets.publishing.service.gov .uk/media/6193a87bd3bf7f0559e1d976/GOV.UK_- _NVIDIA_Arm__CMA_Report_to_DCMS__Web_Accessible_.pdf (visited on 01/12/2024). – (Feb. 2022). NVIDIA / Arm Merger Inquiry. UK Competition and Markets Authority. URL: https://www.gov.uk/cma-cases/nvidia-slash-arm-merger-inqu iry (visited on 01/12/2024). UK CMA (Sept. 2023). AI Foundation Models: Initial Report. UK Competition and Markets Authority. URL: https://www.gov.uk/government/publications/aifoundation-models-initial-report (visited on 01/12/2024). UK DSIT (Oct. 2023). Frontier AI: Capabilities and Risks – Discussion Paper. UK Department for Science, Innovation & Technology. URL: https://www.gov.uk/government/pu blications/frontier-ai-capabilities-and-risks-discussionpaper/frontier-ai-capabilities-and-risks-discussion-paper (visited on 01/12/2024). UK Government (Sept. 2023). Bristol Set to Host UK’s Most Powerful Supercomputer to Turbocharge AI Innovation. URL: https : / / www . gov . uk / government / news / bristol- set- to- host- uks- most- powerful- supercomputer- toturbocharge-ai-innovation (visited on 01/13/2024). UN (Aug. 2022). Non-Proliferation Treaty Review Conference Ends without Adopting Substantive Outcome Document Due to Opposition by One Member State. United Nations. URL: https: //press.un.org/en/2022/dc3850.doc.htm (visited on 01/13/2024). – (2024). Goal 17: Revitalize the Global Partnership for Sustainable Development. United Nations. URL: https://www.un.org/sustainabledevelopment/global partnerships/ (visited on 01/13/2024). US BIS (Dec. 2020). Comment on FR Doc # 2020-22443. Bureau of Industry and Security. URL: https://www.regulations.gov/comment/BIS-2020-0029-0056 (visited on 01/13/2024). – (Oct. 2022a). Export Controls on Semiconductor Manufacturing Items. US Department of Commerce - Bureau of Industry and Security. URL: https://www.bis.doc.gov/i ndex.php/documents/federal-register-notices-1/3352-10-1623-semiconductor-equipment-controls/file (visited on 01/13/2024). – (Oct. 2022b). Implementation of Additional Export Controls: Certain Advanced Computing and Semiconductor Manufacturing Items; Supercomputer and Semiconductor End Use; Entity List Modification. Bureau of Industry and Security. URL: https://www.federalre gister.gov/documents/2022/10/13/2022-21658/implementationof-additional-export-controls-certain-advanced-computingand-semiconductor (visited on 01/12/2024). – (Nov. 17, 2023). Export Administration Regulations (ECCN 9A515, 4A001). Bureau of Industry and Security. URL: https://www.bis.doc.gov/index.php/documents/ regulations-docs/2335-ccl4-5/file (visited on 01/19/2024). US FTC (Dec. 2021). FTC Sues to Block $40 Billion Semiconductor Chip Merger. US Federal Trade Commission. URL: https://www.ftc.gov/news- events/news/pressreleases/2021/12/ftc-sues-block-40-billion-semiconductorchip-merger (visited on 01/12/2024). US Government (Aug. 1988). White House Statement on the Soviet-United States Joint Verification Experiment for Nuclear Testing. URL: https://www.reaganlibrary.gov/arc hives/speech/white- house- statement- soviet- united- statesjoint-verification-experiment-nuclear (visited on 01/13/2024). – (2024). Data.Gov. Data.gov. URL: https://data.gov/ (visited on 01/15/2024). US NAIRR (Jan. 2023). Strengthening and Democratizing the U.S. Artificial Intelligence Innovation Ecosystem: An Implementation Plan for a National Artificial Intelligence Research Resource. National Artificial Intelligence Research Resource Task Force. URL: https : //www.ai.gov/wp- content/uploads/2023/01/NAIRR- TF- FinalReport-2023.pdf. Vaswani, Ashish et al. (June 2017). Attention Is All You Need. URL: https://arxiv.org/ abs/1706.03762v7 (visited on 01/12/2024). Verdegem, Pieter (Apr. 2022). “Dismantling AI Capitalism: The Commons as an Alternative to the Power Concentration of Big Tech”. In: AI & SOCIETY. ISSN: 1435-5655. DOI: 10.1007/ s00146-022-01437-8. URL: https://doi.org/10.1007/s00146-02201437-8 (visited on 01/12/2024). VerWey, John (Oct. 2021). No Permits, No Fabs. Center for Security and Emerging Technology. URL: https://cset.georgetown.edu/wp- content/uploads/CSETNo-Permits-No-Fabs.pdf (visited on 01/13/2024). Villalobos, Pablo (Jan. 2023). Scaling Laws Literature Review. Epoch AI. URL: https:// epochai . org / blog / scaling - laws - literature - review (visited on 01/12/2024). Villalobos, Pablo and David Atkinson (July 2023). Trading off Compute in Training and Inference. Epoch AI. URL: https://epochai.org/blog/trading-off-compute-intraining-and-inference (visited on 01/12/2024). Villalobos, Pablo and Anson Ho (Sept. 2022). Trends in Training Dataset Sizes. URL: https: //epochai.org/blog/trends-in-training-dataset-sizes (visited on 01/12/2024). Vincent, James (Feb. 2023). Qualcomm Demos Fastest Local AI Image Generation with Stable Diffusion on Mobile. URL: https://www.theverge.com/2023/2/23/236 11668/ai- image- stable- diffusion- mobile- android- qualcommfastest (visited on 01/13/2024). Vinuesa, Ricardo et al. (Jan. 2020). “The Role of Artificial Intelligence in Achieving the Sustainable Development Goals”. In: Nature Communications 11.1, p. 233. ISSN: 2041-1723. DOI: 10.1038/s41467-019-14108-y. URL: https://www.nature.com/ articles/s41467-019-14108-y (visited on 01/13/2024). Wallach, Amei (1991). “Censorship in the Soviet Bloc”. In: Art Journal 50.3, pp. 75–83. ISSN: 0004-3249. DOI: 10.2307/777221. JSTOR: 777221. URL: https://www.jstor. org/stable/777221 (visited on 01/13/2024). Wanat, Zosia (Oct. 2023). “‘Eu’s AI Act Could Kill Our Company,’ Says Mistral’s Cédric O”. In: Sifted. URL: https://sifted.eu/articles/eu-ai-act-kill-mistralcedric-o/ (visited on 01/12/2024). Wei, Jason (May 3, 2023). Common Arguments Regarding Emergent Abilities — Jason Wei. URL: https://perma.cc/F48V-XZHC (visited on 01/26/2024). Wei, Jason et al. (June 2022). Emergent Abilities of Large Language Models. URL: https: //arxiv.org/abs/2206.07682v2 (visited on 01/12/2024). Weinstein, Emily S. and Kevin Wolf (July 2023). For Export Controls on AI, Don’t Forget the "Catch-all" Basics. URL: https://cset.georgetown.edu/article/dontforget - the - catch - all - basics - ai - export - controls/ (visited on 01/12/2024). Whittlestone, Jess et al. (Mar. 2023). Response to the UK’s Future of Compute Review. Center for the Governance of AI. URL: https://www.governance.ai/researchpaper/response-to-the-uks-future-of-compute-review (visited on 01/12/2024). Williams, Joe and Max A. Cherney (Nov. 2022). Biden Administration on Quantum Computing Export Controls - Protocol. URL: https://www.protocol.com/enterprise/ quantum-computing-export-controls (visited on 01/13/2024). Yu, Danni, Hannah Rosenfeld, and Abhishek Gupta (Jan. 2023). The ‘AI Divide’ between the Global North and Global South. URL: https : / / www . weforum . org / agenda / 2023/01/davos23-ai-divide-global-north-global-south/ (visited on 01/13/2024). Yuan, Binhang et al. (June 2, 2022). Decentralized Training of Foundation Models in Heterogeneous Environments. arXiv. DOI: 10.48550/arXiv.2206.01288. URL: http: //arxiv.org/abs/2206.01288 (visited on 01/13/2024). Zhang, Baobao et al. (Aug. 2021). “Ethics and Governance of Artificial Intelligence: Evidence from a Survey of Machine Learning Researchers”. In: Journal of Artificial Intelligence Research 71, pp. 591–666. ISSN: 1076-9757. DOI: 10.1613/jair.1.12895. URL: https://j air.org/index.php/jair/article/view/12895 (visited on 01/12/2024). Zhang, Daniel et al. (May 2022). Enhancing International Cooperation in AI Research: The Case for a Multilateral AI Research Institute. URL: https://hai.stanford.edu/whi te-paper-enhancing-international-cooperation-ai-researchcase-multilateral-ai-research-institute (visited on 01/13/2024). Zwetsloot, Remco and Jack Corrigan (July 2022). AI Faculty Shortages: Are U.S. Universities Meeting the Growing Demand for AI Skills? Issue Brief. Center for Security and Emerging Technology, p. 56. URL: https : / / cset . georgetown . edu / wp - content / uploads/CSET-AI-Faculty-Shortages.pdf (visited on 01/12/2024).