Andras Gyorgy: Statistical vs. Exact Learning and Artificial General Intelligence

András György is a Senior Staff Research Scientist at Google DeepMind, London, UK. He received his Ph.D. from the Budapest University of Technology and Economics, Hungary. He was a postdoctoral researcher at Queen’s University, Canada, and held research positions at the Institute for Computer Science and Control (SZTAKI), Hungary, leading the Machine Learning Research Group, and at the University of Alberta, Canada. He was also a faculty member at the Department of Electrical and Electronic Engineering, Imperial College London, UK. His research interests include machine learning, statistical learning theory, sequential decision making, optimization and, more recently, large language models. Among others, Dr. György received a best paper award at the 7th IEEE Global Conference on Signal and Information Processing (GLOBALSIP2019) in 2019, a best paper runner-up award at the 34th Annual Conference on Learning Theory (COLT 2021), the Gyula Farkas prize of the János Bolyai Mathematical Society in 2001, and the Academic Golden Ring of the President of the Republic of Hungary in 2003.Title: Statistical vs. Exact Learning and Artificial General Intelligence

Title: Statistical vs. Exact Learning and Artificial General Intelligence

Abstract: Sound deductive reasoning—the ability to derive new knowledge from existing facts and rules—is an indisputably desirable aspect of general intelligence. Despite the major advances of AI systems in areas such as math and science, especially since the introduction of transformer architectures, it is well-documented that even the most advanced frontier systems regularly and consistently falter on easily-solvable deductive reasoning tasks. Hence, these systems are unfit to fulfill the dream of achieving artificial general intelligence capable of sound deductive reasoning. We argue that their unsound behavior is a consequence of the statistical learning approach powering their development. To overcome this, we contend that to achieve reliable deductive reasoning in learning-based AI systems, researchers must fundamentally shift from optimizing for statistical performance against distributions on reasoning problems and algorithmic tasks to embracing the more ambitious exact learning paradigm, which demands correctness on all inputs. We argue that exact learning is both essential and possible, and that this ambitious objective should guide algorithm design. Finally, we study the limitations of achieving exact learning on a simple logic reasoning task.

Based on joint work with Csaba Szepesvari, Nevena Lazic, Tor Lattimore, and Liam Fowl

Please email informed-ai@bristol.ac.uk if you’d like to register and join the seminar either in person at Imperial College London or online.

Location: Room 909B EEE Building, Imperial College London