AlphaGo wasn’t based on rules programmed by human experts; instead, it programmed its own rules using machine learning. It was trained on a database of more than 30 million past moves by expert Go players. Additionally, it played millions of games against itself— more than “the number of games played by the entire human race since the creation of Go,” according to the research team that created it. This preparation came to fruition most vividly in the second game of the Sedol- AlphaGo match of March 2016. After a resounding defeat in game 1, Sedol was playing more carefully, and even though AlphaGo had gained a slight early edge, the contest was still anyone’s to win. Then move 37 happened.
AlphaGo, playing black, made an unusual move on the center-right side of the board. “I thought it was a mistake,” said one commentator within seconds of the play. Sedol stepped out of the room briefly and took nearly fifteen minutes to respond. Fan Hui, a European Go champion who had previously lost to AlphaGo, remarked “It’s not a human move. I’ve never seen a human play this move.” He later described it as “so beautiful. So beautiful.” AlphaGo went on to win the game (and I hope was a good sport about it).
Move 37 was, in fact, in some ways an utterly human move— the kind of mysterious, creative, and unpredictable action of which humans are routinely capable but that we don’t expect from machines. What made it unusual was that it couldn’t be understood by AlphaGo’s human developers, let alone programmed by them. They had simply provided the input— millions of Go moves from past games— and stood back to observe the output: that stunning 37th move.
Given that such deep learning systems independently combine simple concepts to create abstract patterns from the data, computer scientists don’t actually know what’s going on under the hood of their systems. How or why AlphaGo and its peers behave in certain ways is often not clear even to their designers.
Why might this be disturbing? On the one hand, there is the sheer eeriness of computers displaying the sort of creativity that we believed belonged to humans alone. But there is also a more practical concern. Think about the way we teach young children to operate in the world. We might simply give them sets of rules and watch, satisfied, as they follow them: wash your hands before eating; wait your turn; wrestle that winter coat on by laying it on the floor, sticking your hands in the arm openings, and flipping it over your head. But children also routinely surprise us, pretending to have washed their hands when they’ve patently not done so; finding another activity rather than waiting in line to do the original one; deciding that they’re coordinated enough to put on a jacket the way they’ve seen adults do it, one arm at a time. Such abilities are not the result of following sets of rules they have been taught but rather stem from observing people around them and picking up new skills on their own. A capacity to surprise is core to a child’s normal development. It also makes them admittedly difficult to handle at times.
The same is true of algorithms. Today, some of the most accurate machine learning models that computer scientists can build are also the most opaque. As machines become more intelligent and dynamic they also become more unpredictable. This suggests a fundamental conundrum in algorithm design. You can either create intelligent algorithms in highly curated environments— for example, programming explicit rules they might follow, expert systems style— to ensure they are highly predictable in behavior, while accepting that they will run up against problems they weren’t prepared for and therefore can’t solve; or, you can expose them to messy real- world data to create resilient but also unpredictable algorithms. I call this a predictability- resilience paradox.
The unpredictability of a game- playing algorithm may not be particularly troublesome to anyone outside the Go universe. However, when life- altering decisions are placed in the cybernetic hands of algorithms, the ability to understand and predict their decisions becomes more urgent. We can’t always know why an airline pilot pulled rather than pushed the control wheel amid turbulence; why a manager decided to hire someone with nontraditional qualifications; why a doctor decided not to order a blood test for her patient. And yet most of us find unfathomable the idea that we would let computers make these decisions and still not know the reasoning. But as much as we may desire fully explainable and interpretable algorithms, the balance between predictability and resilience inevitably seems to be tilting in the latter direction.
[This excerpt from ‘A Human’s Guide to Machine Intelligence’ by Kartik Hosanagar has been reproduced with permission from Penguin Random House]
