Peter Lee: Computing Theory and Practice, and GPT-4's Impact

Episode 133

I spoke with Peter Lee about:

* His early work on compiler generation, metacircularity, and type theory

* Paradoxical problems

* GPT-4s impact, Microsoft’s “Sparks of AGI” paper, and responses and criticism

Enjoy—and let me know what you think!

Peter is President of Microsoft Research. He leads Microsoft Research and incubates new research-powered products and lines of business in areas such as artificial intelligence, computing foundations, health, and life sciences. Before joining Microsoft in 2010, he was at DARPA, where he established a new technology office that created operational capabilities in machine learning, data science, and computational social science. Prior to that, he was a professor and the head of the computer science department at Carnegie Mellon University. Peter is a member of the National Academy of Medicine and serves on the boards of the Allen Institute for Artificial Intelligence, the Brotman Baty Institute for Precision Medicine, and the Kaiser Permanente Bernard J. Tyson School of Medicine. He served on President Obama’s Commission on Enhancing National Cybersecurity. He has testified before both the US House Science and Technology Committee and the US Senate Commerce Committee. With Carey Goldberg and Dr. Isaac Kohane, he is the coauthor of the best-selling book, “The AI Revolution in Medicine: GPT-4 and Beyond.” In 2024, Peter was named by Time magazine as one of the 100 most influential people in health and life sciences.

Find me on Twitter for updates on new episodes, and reach me at [email protected] for feedback, ideas, guest suggestions.

I spend a lot of time on this podcast—if you like my work, you can support me on Patreon :) You can also support upkeep for the full Gradient team/project through a paid subscription on Substack!

Subscribe to The Gradient Podcast:  Apple Podcasts  | Spotify | Pocket Casts | RSSFollow The Gradient on Twitter

Outline:

* (00:00) Intro

* (00:50) Basic vs. applied research

* (05:20) Theory and practice in computing

* (10:28) Traditional denotational semantics and semantics engineering in modern-day systems

* (16:47) Beauty and practicality

* (20:40) Metacircularity in the polymorphic lambda calculus: research directions

* (24:31) Understanding the nature of difficulties with metacircularity

* (26:30) Difficulties with reflection, classic paradoxes

* (31:02) Sparks of AGI

* (31:41) Reproducibility

* (38:04) Confirming and disconfirming theories, foundational work

* (42:00) Back and forth between commitments and experimentation

* (51:01) Dealing with responsibility

* (56:30) Peter’s picture of AGI

* (1:01:38) Outro

Links:

* Peter’s Twitter, LinkedIn, and Microsoft Research pages

* Papers and references

* The automatic generation of realistic compilers from high-level semantic descriptions

* Metacircularity in the polymorphic lambda calculus

* A Fresh Look at Combinator Graph Reduction

* Sparks of AGI

* Re-envisioning DARPA

* Fundamental Research in Engineering