Open-Sourcing Highly Capable Foundation Models: An Evaluation of Risks, Benefits, and Alternative Methods for Pursuing Open-Source Objectives

This resource is the second of two on the benefits and risks of open-weights model release. In contrast, this paper expresses strong skepticism toward releasing highly capable foundation model weights, arguing that the risks may outweigh the benefits. While recognizing the advantages of openness, such as encouraging innovation and external oversight, it warns that making models publicly available increases the potential for misuse, including cyberattacks, biological weapon development, and disinformation. The article emphasizes that malicious actors could easily disable safeguards, fine-tune models for harmful purposes, and exploit vulnerabilities. Instead of fully open releases, it advocates for safer alternatives like democratic oversight, structured access, and staged model release, which can provide some benefits of openness while mitigating the extreme risks posed by advanced AI systems.

Original text: https://cdn.governance.ai/Open-Sourcing_Highly_Capable_Foundation_Models_2023_GovAI.pdf

Author(s): Elizabeth Seger, Noemi Dreksler, Richard Moulange, Emily Dardaman, Jonas Schuett, K. Wei, Christoph Winter, Mackenzie Arnold, Seán Ó hÉigeartaigh, Anton Korinek, Markus Anderljung, Ben Bucknall, Alan Chan, Eoghan Stafford, Leonie Koessler, Aviv Ovadya, Ben Garfinkel, Emma Bluemke, Michael Aird, Patrick Levermore, Julian Hazell, Abhishek Gupta

A podcast by BlueDot Impact.

Learn more on the AI Safety Fundamentals website.