Sebastian Bubeck is a Senior Principal Research Manager at Microsoft Research, where he leads the Machine Learning Foundations group. He has been with Microsoft Research since 2014 and has studied the foundations of machine learning. He has published research papers and articles on the topic, which can be found on his Google Scholar page.
Sebastian Bubeck talks about GPT-4, a language model developed by OpenAI. He starts by explaining how GPT-4 works and how it’s different from its previous versions. He then shows some examples of GPT-4’s capabilities, including generating text and completing tasks. The speaker then discusses the implications of GPT-4’s development, including its potential impact on society and healthcare. He concludes by addressing the question of whether GPT-4 is intelligent and suggests that its usefulness is more important than its classification as “intelligent” or not. Finally, he encourages society to move beyond the technical aspects of GPT-4 and consider its broader implications for humanity.
Sebastian’s talk is about the scientific part of his study on GPT-4 and how it has led to sparks of AGI (Artificial General Intelligence). He acknowledges that GPT-4 is entirely OpenAI’s creation and that the experiments were done on an early version of the model that was text input and output only. He clarifies that due to further modifications, the answers to the problems shown in his presentation may differ from what others might get. The talk aims to convince the audience that there is some intelligence in the GPT-4 system that should be called an intelligent system.
He is discussing the capabilities and limitations of large language models, specifically GPT-4. They give an example of a puzzle involving stacking objects and explain how GPT-4 can successfully answer this type of question using common sense. The speaker then moves on to discussing the topic of “theory of mind” and how GPT-4 may or may not have the ability to understand human motives and emotions. They mention a paper that argues that GPT-4 has a theory of mind, and the potential implications for the field of machine learning and interpretability. Bubeck then presents an example from a paper that shows how GPT-4 fails at a simple theory of mind question involving a cat being moved from a basket to a box. He implies that GPT-4 may have some limitations but is still capable of understanding some human-like concepts. A new paper about this topic is mentioned that will be available soon.
He talks about GPT-4’s ability to solve problems, think abstractly, comprehend complex ideas, learn quickly and learn from experience. Although GPT-4 is a large language model, it cannot update itself, and each session with GPT-4 is a new session. He believes that GPT-4 can be considered intelligent if we evaluate its ability to solve problems, think abstractly, and comprehend complex ideas. Sebastian suggests that we assess GPT-4’s intelligence by asking it to perform creative tasks that are outside of what it has seen. The speaker then presents an example of a creative task – asking GPT-4 to write a proof of the infinitude of primes with every line that rhymes. He is amazed at the quality of the response, which is correct and rhymes, but the speaker emphasizes the importance of not stopping there and suggests a broader range of domains to test GPT-4’s intelligence, such as vision, theory of mind, coding, mathematics, and privacy.
Sebastian Bubeck showed the audience GPT-4’s ability to understand the concept of a unicorn and draw it, and compared it to ChatGPT’s unicorn. GPT-4 is also able to use tools to improve its drawings.
Sebastian talks about their experience with GPT-4, which can write fully functional 500 to 1,000 lines of code. They compare GPT-4 with Chat GPT, which is already a significant improvement, and Codex and GitHub Copilot, which can auto-complete code snippets. The speaker also shows two animations produced by GPT-4, demonstrating its expert level of coding. They highlight the power and creativity that GPT-4 unlocks and discuss its weaknesses, including its lack of memory and tendency to make arithmetic mistakes. However, GPT-4 is intelligent enough to use tools and external resources, which allows it to perform complex tasks. Bubeck also mentions GPT-4’s superhuman coding abilities, as it can pass interviews and beat 100% of human users in mock interviews.
Sebastian discusses how a language model, specifically GPT-4, performs when it comes to understanding abstract concepts and arithmetic. They describe a test where the model is asked to find a function that satisfies certain conditions and, while it struggles at first, eventually arrives at the correct conclusion. He also describes how the model sometimes produces incorrect answers to arithmetic problems but is able to explain its reasoning, indicating that it is using its internal representation to arrive at answers. The speaker notes that the model’s ability to overcome errors in its prompts is a sign of its training.
Sebastian Bubeck concludes their speech by stating that with more training, GPT-4 can learn a lot more than what it currently can do. While GPT-4 may lack certain abilities, it is already useful and will change the world. He suggests that it may be an opportunity to rethink what intelligence is with this new process. GPT-4 is just the beginning, and there is much more on the horizon. Society should move beyond discussions of whether GPT-4 is copy paste or statistics and focus on the important questions it raises. GPT-4 has many potential uses, such as data analysis, medical knowledge, and gaming, among others. He recommends a book about using GPT-4 for healthcare titled “The AI Revolution in Medicine“.