GPT: To ban or not to ban, that is the question

On March 14th, OpenAI launched GPT-4 , which took the world by surprise and storm. Almost everybody, including people within the AI community, was stunned by its capabilities. A week later, the Future of Life Institute (FLI) published an open letter calling on the world’s AI labs to pause the development of larger versions of GPT (generative pre-trained transformer) models until their safety can be ensured.

Recent episodes of this podcast have presented arguments for and against this call for a moratorium. Jaan Tallin, one of the co-founders of FLI, made the case in favour. Pedro Domingos, an eminent AI researcher, and Kenn Cukier, a senior editor at The Economist, made variants of the case against. In this episode, co-hosts Calum Chace and David Wood highlight some key implications and give our own opinions. Expect some friendly disagreements along the way.

Follow-up reading:
https://futureoflife.org/open-letter/pause-giant-ai-experiments/
https://www.metaculus.com/questions/3479/date-weakly-general-ai-is-publicly-known/

Topics addressed in this episode include:

*) Definitions of Artificial General Intelligence (AGI)
*) Many analysts knowledgeable about AI have recently brought forward their estimates of when AGI will become a reality
*) The case that AGI poses an existential risk to humanity
*) The continued survival of the second smartest species on the planet depends entirely on the actions of the actual smartest species
*) One species can cause another to become extinct, without that outcome being intended or planned
*) Four different ways in which advanced AI could have terrible consequences for humanity: bugs in the implementation; the implementation being hacked (or jail broken); bugs in the design; and the design being hacked by emergent new motivations
*) Near future AIs that still fall short of being AGI could have effects which, whilst not themselves existential, would plunge society into such a state of dysfunction and distraction that we are unable to prevent subsequent AGI-induced disaster
*) Calum's "4 C's" categorisation of possible outcomes regarding AGI existential risks: Cease, Control, Catastrophe, and Consent
*) 'Consent' means a superintelligence decides that we humans are fun, enjoyable, interesting, worthwhile, or simply unobjectionable, and consents to let us carry on as we are, or to help us, or to allow us to merge with it
*) The 'Control' option arguably splits into "control while AI capabilities continue to proceed at full speed" and "control with the help of a temporary pause in the development of AI capabilities"
*) Growing public support for stopping AI development - driven by a sense of outrage that the future of humanity is seemingly being decided by a small number of AI lab executives
*) A comparison with how the 1983 film "The Day After" triggered a dramatic change in public opinion regarding the nuclear weapons arms race
*) How much practical value could there be in a six-month pause? Or will the six-months be extended into an indefinite ban?
*) Areas where there could be at least some progress: methods to validate the output of giant AI models, and choices of initial configurations that would make the 'Consent' scenario more likely
*) Designs that might avoid the emergence of agency (convergent instrumental goals) within AI models as they acquire more intelligence
*) Why 'Consent' might be the most likely outcome
*) The longer a ban remains in place, the larger the risks of bad actors building AGIs
*) Contemplating how to secure the best upsides - an "AI summer" - from advanced AIs

Music: Spike Protein, by Koi Discovery, available under CC0 1.0 Public Domain Declaration