Stuart Russell is a distinguished artificial intelligence researcher, a Professor of Computer Science at the University of California, Berkeley, an Adjunct Professor of Neurological Surgery at the University of California, San Francisco, and leads the Center for Human-Compatible Artificial Intelligence at UC Berkeley. Along with Peter Norvig, Stuart is the author of Artificial Intelligence: A Modern Approach, the most widely used textbook on artificial intelligence. In his most recent book, Human Compatible: AI and the Problem of Control, Stuart proposes a fundamentally new approach to developing AI.
In this interview, Stuart warns that AI is reshaping society in unintended ways. For example, social media content selection algorithms that choose what individuals watch and read do not even know that human beings exist. As AI becomes more capable, he suggests that we are going to see bigger failures unless we change the way we think about AI altogether. Stuart argues that to ensure AI is provably beneficial for human beings, we must design machines to be inherently uncertain about human preferences. This way, we can ensure they are humble, altruistic, and committed to pursuing our objectives even as they set their own goals. We also discuss why AI needs regulation similar to civil engineering and medicine, the impact AI is going to make over the next decade, autonomous vehicles, and a variety of other topics.
(To listen to a podcast version of this interview, please click this link. To read future articles like this one in the future, please follow me on Twitter @PeterAHigh.)
Peter High: You are a Professor of Computer Science at the University of California, Berkeley and an Adjunct Professor of Neurological Surgery at the University of California, San Francisco. Further, you founded and lead the Center for Human-Compatible Artificial Intelligence at UC Berkeley. You have written a number of books, perhaps most notably along with Peter Norvig, Artificial Intelligence: A Modern Approach and most recently without a coauthor, Human Compatible: AI and the Problem of Control. What drew you to the field of artificial intelligence [AI]?
Stuart Russell: I first became aware of the field in the mid-70s when I got a Cambridge programmable calculator. I had always been curious about how intelligence and the human mind work, and I finally had a tool in hand to explore that question by writing intelligent programs. The calculator had a capacity for a 36-keystroke-program, which was not enough to do much AI. I got permission to go to Imperial College and use its giant CDC 6600 supercomputer, and I wrote a chess program on that. That is how I got hooked on AI. At the time, I was aspiring to be a physicist, which is what I studied as an undergraduate. However, the lure of AI drew me, so I did my Ph.D. at Stanford and have been at Berkeley ever since.
High: You got your Ph.D. in 1986, and the field of AI has gone back much further than that. The AI space has gone through various peaks and troughs in terms of development and optimism. There has been a confluence of factors that have come together in recent years that have powered much of the revolution that is happening of late. Where do you see us in the overall evolution of AI?
Russell: You mentioned there have been some ups and downs, and when I finished my Ph.D. in ’86, that was during one of the ups. At the time, the expert systems revolution was taking place where AI was going out into the real world. People were building expert systems, which were essentially collections of rules for reasoning about particular types of problems. This could have involved a medical diagnosis, how to configure a supercomputer, or something along those lines. Thousands of those systems were built, but many of them did not work well in practice. The more you added rules to them, the more they started to make mistakes. That led by the late ’80s to what we call the AI winter where interest waned. My AI course was down to 25 students at one point, but it is back up to 900 students right now. We are in the middle of another enormous explosion of interest, excitement, and investment just as we were in the mid ‘80s.
The technology itself is separate from the level of interest in Silicon Valley and in the media. The technology itself does not go in booms and busts. Instead, it gradually accumulates as we solve more problems, develop better theories, and improve the ones we have. Our capabilities increased monotonically. There are periods where they accelerate faster, and this is one of those because of the development of deep learning. Deep learning is a method of training large tunable circuits from data so that those circuits do a good job of tasks such as speech recognition, recognizing objects in images, machine translation, and so on. Machine learning is a technology that has been around since the 1950s, and we have gradually increased the size and scope of the types of data that we can handle. For example, it used to be that training with images was infeasible because the volume of data was typically too big to even handle on an ordinary computer. However, due to the advances in hardware, we were able to start training larger circuits with larger amounts of data. Beginning around 2012, we saw a rapid improvement in the ability of these systems to recognize objects, images, and other tasks. That is what is happening right now and that stimulated the enormous quantity of investment, many startups, and many interesting applications such as self-driving cars.
High: You mentioned some of the most prominent of the innovations that have arisen in recent years as a result of artificial intelligence’s progress. As you look to the near future, are those areas that you think are going to have the most profound impact either for consumers or for enterprises?
Russell: We are already seeing a significant impact. The technology that is coming down the pipe that is going to be most obvious to everybody is the self-driving car, which will revolutionize a trillion-dollar industry. It will hopefully make everyone much safer on the road, provide mobility for many people who do not have it right now, and reduce traffic jams. In doing so, it will likely reshape our cities, many of which devote large areas to parking. As a result, you will be able to walk down the streets without having to navigate parked cars climbing up on the sidewalk. These aspects will be quite obvious to people.
There are also many aspects that are invisible that we get used to quickly. It was not long ago that search engines did not exist, and search engines are AI algorithms that are constantly running to scan the web, find new material, and integrate it into a model of what you are going to be interested in. When your query comes along, the system tries to figure out the most likely response it should send that you are going to be interested in. Search engines represent an extremely complicated and large AI system that runs all the time and is used by billions of people a day.
In the next decade, we should see real progress on understanding language so that the system will not just be able to recognize what you are saying to your phone or your home assistant. Instead, it will be able to understand it, have a conversation with you, and learn about you and your life so that it will become a truly useful assistant. It will be comparable to having an expensive personal assistant, who these days, typically works for a politician or a CEO. Instead of it costing you one hundred thousand dollars annually, it will cost you 99 cents a year. This will make it obvious to people that AI is part of their lives. They will be interacting with it every day, and it will be essential to managing their daily life.
Education is another big area. While AI can spit out predigested material to help you learn something, you cannot ask AI a question. This is going to change, and we will have AI tutoring systems that can bring students up to a high level of understanding and competence much faster than a typical classroom could do. These are areas where there will be a significant impact on daily life.
If we look further down the road, when you start to put these capabilities together, you get into the idea of general-purpose AI. This includes systems that are not just capable of solving a particular task that they are programmed for, but they are able to cope with the whole range of actions that human beings can take.
High: Given the progress that we have made, how far do you feel like we are from that reality?
Russell: It is difficult to predict because it is not a quantitative problem. It is not as if we just need bigger machines or more data. A bigger machine only gets you the wrong answer more quickly, so that is not the solution. Instead, we need some real conceptual breakthroughs. A real understanding of language currently eludes computers, so they are sort of similar to parrots. They recognize variants of communication that they already know how to respond to, but in terms of being able to take a physics book and read it and then go away and design a radio telescope, they are nowhere close to that level. A conceptual breakthrough is needed for real understanding of language, for long-range planning in the real world, and for the type of cumulative discovery processes that we see in science. Machines are not able to do this right now and each of these requires one or more conceptual breakthroughs, and those are extremely hard to predict. That said, given the level of investment happening now, which is unprecedented, we will likely see some of those breakthroughs happening within the next decade. Most AI researchers believe that sometime around the middle of the century, we will have something that looks similar to general-purpose AI.
High: In your latest book, you note that there is a problem of control that we need to grapple with. As the machines become more intelligent than we are, they may draw their own conclusions that counter what we believe to be beneficial. You go on to note that machines are intelligent to the extent that their actions can be expected just to achieve their objectives. Further, machines are beneficial to the extent that their actions can be expected to achieve our objectives. You go on to note that AI needs to be provably beneficial for humans. Could you elaborate on this?
Russell: Let me unpack this a bit. The standard model for AI, which is where a machine is intelligent to the extent that its actions can be expected to achieve its objectives, is the first version that you mentioned. We copied this notion back in the 1950s from our conception of human intelligence. This idea of action that achieves objectives goes back thousands of years, and the idea was gradually made more precise and mathematical in philosophy and economics over the 18th, 19th, and 20th centuries. That was the notion we copied when we were looking for a practical definition of machine intelligence. Despite what you might read in the media that AI is about passing the Turing test, which is about intelligence as being a direct copy of human intellectual activity, that is not what we have pursued in AI. No one within the field makes serious efforts to pass the Turing test. Instead, we have tried to instantiate this more mathematical definition of intelligence as rational behavior. That is fine as long as the machine is relatively stupid and is confined to the lab or a small virtual world, such as a virtual chessboard. This is because outcomes go wrong when you specify the objective incorrectly, which is not a new notion. The legend of King Midas shows you what happens when you get what you asked for. He asked for everything that he touched to turn to gold, and he got exactly what he wished for. Of course, his food, his drink, and his family all turned to gold, so he died in misery and starvation. The same idea goes for the Genie in the lamp in which the third wish is “Please undo the first two wishes because I just ruined everything.”
That notion is in every human culture we understand. The problem with the standard model of AI is that we cannot specify the objective completely and correctly. When the machine is stupid in the lab and doing something within a limited scope of impact on the real world, you can set the objectives, see how it behaves, and then realize that you forgot something important. From there, you can go back, revise the objective, press the reset button, and try again.
One of my favorite examples of that is in simulated evolution, which is where people make little simulated environments where creatures can evolve. They have a simulated genetic code, and you supply a fitness function that decides which creatures get to reproduce. The designers wanted to see if they could evolve creatures that could run extremely fast. They set the function to how fast the creature can move its center of gravity. They said this simulated environment would let the creatures grow up, it would calculate how fast they moved, and then it would make more of the ones that were moving faster. What ended up evolving was not creatures that could run fast, but creatures that could grow enormously tall and just fall over. Being 100 miles tall in this simulated environment, they moved extraordinarily fast when they fell over, so they won the evolutionary race. It was not what the designers intended at all, but it was a solution to the problem they set. That is amusing, and it is a simulated environment, so no one got killed by these falling trees. However, when you get out into the real world, there is enormous collateral damage. For example, the social media content selection algorithms choose what you read and what you watch. Those algorithms are learning algorithms that optimize a fixed objective which turns out to be click-through, eyeball time, or some other monetization metric. The collateral damage this causes is an early warning that creating intelligent systems that optimize a fixed objective can be extremely dangerous when the algorithms operate in the real world on a global scale. Right now, the social media algorithms are simple, and they do not even know that human beings exist. However, as AI becomes more capable, we are going to see bigger failures of this kind unless we change the way we think about AI altogether.
High: You noted that we need to ensure that it is provably beneficial to humans. What is the method of doing so?
Russell: If we accept the premise that we cannot specify the objective completely and correctly, then you do not want machines pursuing the objectives that we put into them. At best, they should understand that objective as an incomplete and possibly erroneous attempt to communicate what human beings want. The machine has to operate in the knowledge that it does not know what the true objective is, which is the radical break from the standard way of using AI.
We have always assumed that the machine has the correct objective, but that is a mistake. Interestingly, for the last 40 years or so, we have understood perfectly well that the machine is not going to have correct knowledge of the physics of the world. It may have some partial knowledge, so it may be able to make weak probabilistic predictions about how something is going to turn out. For example, it might be able to predict if it is going to rain tomorrow, but we know that machines cannot exactly predict the weather. They cannot precisely predict how humans are going to behave, so they have to deal with partial uncertain knowledge of the world. However, for some reason, we forgot that the same applies to the objective. AI is not going to know exactly what the objective is.
When you change the way the whole notion of intelligence is defined so that the machine knows that it does not know what our true preferences and objectives are, then you get a much different type of behavior. The goal is still to satisfy our preferences about how the future should unfold. However, if the machine knows that it does not know what those preferences are and it comes up with a plan that messes with some part of the world and is unsure whether we would be unhappy or happy with it pursuing that, then it has an incentive to ask us. It wants to make the right choice, but it needs to gather more information to do that. You get machines that ask permission and machines that allow themselves to be switched off because they do not want to do whatever it is that would cause us to switch them off. They are happy if we decide to switch them off because that way, they avoided some catastrophic outcome. None of this is possible for a machine that believes it has the correct objective. In the language of economics, we call this a game-theoretic formulation.
Game theory studies how to make correct decisions when there is more than one decision-making entity in the world. They use that to study warfare, bidding in auction, and all types of real-world problems. Here, we are saying that there is a decision problem, not just for the machine alone, but it is in a coupled system with the human being. It is trying to help the human, but the human is the one who has more information about what the real objective is. The machine is going to solve this two-entity or mini-entity game, and the solutions to the game produce these behaviors that we think of as deferential; asking permission, allowing yourself to be switched off, and so on. They fall out not because we program them in, but because they are the solutions to this mathematical problem that we set up for the machine to solve. That is the sense in which they are provably beneficial. You can show that when the machine solves these games, the results are beneficial to a human. To get that mathematical theorem, you have to make a few assumptions about the human, such as that the human is a rational participant in this game. Of course, that is an approximation because humans are not perfectly rational. Humans clearly deviate from rationality in many interesting ways. One of the big parts of our research agenda is to figure out how to accommodate a human imperfection in the way that the machine interprets our behavior.
High: As you note, humans have imperfections, and we are not perfectly rational, and not all humans are good actors. What do you see as the role of the government and broader regulatory forces to ensure that what you are describing comes to pass?
Russell: We need to develop all of the technology on this new foundation because we have had 70 years of developing technology on the old foundation, and some of it is pretty useful. People are not going to give it up and then wait another decade for the new version to arrive. Because of this, we have to develop the algorithms for the various types of decision-making scenarios that people need. This includes different types of machine learning algorithms, planning algorithms, and reinforcement learning for algorithms that robots use to learn how to behave in the physical world. All of these have to be rebuilt from scratch in some way. When we have done that, we will have templates for each category of AI application saying, “If you conform to this template, you have some guarantee of safety. You have some guarantee that you will be able to switch off the machine, and if it is misbehaving, you have some guarantee that the machine is not interfering with parts of the world that you do not want to be interfering with.” At that point, it would make sense to start setting up standards through the standard organizations and regulations. This is because when you start to have systems that are potentially more intelligent than human beings, there is a serious downside.
This is not similar to someone getting in a crash with their self-driving car. Instead, similar to what is happening with social media, it would have huge negative effects on a global scale. We obviously do not currently allow scientists to experiment with making disease organisms more contagious and then release them into the environment to see what happens. We do not allow people to develop nuclear weapons in their physics lab, except in the national labs under extremely strict conditions. It is not unreasonable that there should be legal regulations on the types of AI systems that can be deployed.
If you think we currently have a problem with malware, cybercrime, and computer viruses that get released that have far worse global effects than even their evil creators intended, this would be far worse than that. People will have to accept that when people create powerful technology, it is reasonable that there be constraints to protect society from it. We will have regulations saying, “You have to have provable conformity with these templates.” There is a technology called proof-carrying code, which I believe will turn out to be exceedingly useful. Proof-carrying code means a program that comes along with a proof that the program does what it is supposed to do. Even if you do not understand the program and even if you do not understand the proof, you can have another program that checks the proof against the program that is supposed to be behaving properly and verify that the program is going to do what it says because the proof shows that it does and the proof agrees with what the program is saying. That type of technology can be used to say, “Okay, here is a piece of AI software coming from some Russian software company. I can check if it comes as a proof-carrying code and that it is going to conform to the standards and behave itself.” It may not be useful, but at least it is going to behave itself. It will be similar to App Store rules that require that apps conform to all these software checks so that it does not cause your phone to seize up or steal data from other apps.
High: Are you optimistic that we will be able to accomplish provable conformity as you have described?
Russell: Yes, I believe we can do that. It is part of a gradual maturing of the entire IT industry. Other areas such as civil engineering in which people build bridges and skyscrapers that people’s lives depend on have developed codes of conduct and legal building codes over the centuries. That is perfectly normal. We do not think about that as onerous government regulation, but we are glad that the skyscraper conforms to the building codes when we are on the 72nd floor of a building. The IT industry is going to have to start becoming more similar to civil engineering and medicine where people have a mature acceptance that the wellbeing of humans are in their hands, and they take that responsibility seriously. I do not believe the IT industry has quite gotten around to this idea that they have a serious effect on the world and not necessarily a good one.
Peter High is President of Metis Strategy , a business and IT advisory firm. His latest book is Implementing World Class IT Strategy . He is also the author of World Class IT: Why Businesses Succeed When IT Triumphs . Peter moderates the Technovation podcast series. He speaks at conferences around the world. Follow him on Twitter @PeterAHigh.
I am the president of Metis Strategy, a business and IT strategy firm that I founded in 2001. I have advised many of the best chief information officers at multi-billion dollar corporations in the United States and abroad. I’ve written for the Wall Street Journal, CIO Magazine, CIO Insight, Information Week and several other periodicals. I am also the author of Getting to Nimble (Kogan Page, March 2021), Implementing World Class IT Strategy (Wiley Press, September 2014) and World Class IT (Wiley Press, December 2009). Since 2008, I have moderated a widely listened to podcast entitled “Technovation with Peter High,” which features a wide array of IT thought-leaders, and is available at https://www.metisstrategy.com/technovation-podcast/ on a twice per week basis. I have been the keynote speaker at a host of corporate conferences and universities in the US, Canada, Mexico, the United Kingdom, the Republic of Ireland, the Czech Republic, Spain, China, India, Australia, and Saudi Arabia. You can reach me at peter.high [at] metisstrategy.com or on Twitter @PeterAHigh
Original post: https://www.forbes.com/sites/peterhigh/2020/01/13/leading-ai-luminary-has-an-idea-to-ensure-humans-remain-in-control/amp/?__twitter_impression=true