Opinion: Artificial intelligence has come a long way since World War Two, but general purpose machine intelligence remains a distant prospect

Multiplying large numbers is difficult. It takes time, you have to concentrate, and if you can do it in your head people will think you are a genius. Electronic computers were invented in World War Two to automate this sort of heavy-duty thinking and they were spectacularly successful: the US Army’s ENIAC could calculate artillery trajectories thousands of times faster than a human mathematician, while Britain’s Colossus played a crucial role in breaking German codes. 

Not surprisingly, early computer scientists were optimistic about the broader potential of their machines. "The brain will carry out mathematical research", Maurice Wilkes told a Daily Mail journalist in 1947. "It may make sensational discoveries in engineering, astronomy, and atomic physics. It may even solve economic and philosophic problems too complicated for the human mind. There are millions of vital questions we wish to put to it." 

Maurice Wilkes at work at the University Mathematical Laboratory at Cambridge in 1949. Photo: Harry Todd/Fox Photos/Getty Images

Wilkes was referring to Cambridge University’s EDSAC, a device vastly outclassed in raw computing power by modern dishwashers. By 1952, it was able to play Tic-Tac-Toe, but the sensational scientific discoveries were not forthcoming. And the "philosophic" problems? Well, suffice to say that computers still cannot converse at the level of a four-year-old.

"The main lesson of 35 years of AI research", wrote psychologist Steven Pinker in 1994, "is that the hard problems are easy and the easy problems are hard." Arithmetic is hard for humans but easy for computers - so easy that we are no longer impressed by mere calculators. Freewheeling conversation lies at the other extreme: trivial for us, but exceptionally difficult for computers. 

Building a machine that can master this trick is often seen as the holy grail of AI research. It is a fair bet that if it were accomplished, artificial scientists and philosophers – and Wilkes’s "sensational discoveries" – would not be far behind. Justification: if human-level general intelligence can be simulated at all, it can likely be simulated at scale – millions of interacting agents – and at speed. 

How computers were transported back in the day: a Moscow-bound National-Elliot 803 computer being loaded onto a customised Austin van in 1960 at the Elliott-Automation factory. Photo: Fox Photos/Getty Images

Few believe this goal is near. Thus, the traditional subfields of AI focus on more specific "easy for us, hard for computers" tasks. Some involve natural language – for example, detecting the grammatical structure of a sentence, or conversing about a narrowly defined topic. Others have to do with learning from experience – for example, learning to recognise cats by looking at pictures of cats.

One reason why it is difficult to program computers to do these things is that it is difficult to describe how we do them. When we do a multiplication problem, we consciously follow rules. It is a simple matter to articulate those rules and pass them to a computer. When we understand an English sentence, or learn to spot cats, we do not consciously follow rules. 

So how should the AI researcher who wishes to automate these abilities proceed? She might guess that we unconsciously follow rules. The idea here is not that the relevant brain processes obey the laws of chemistry and physics - that goes without saying. Rather, the idea is that at some high (but nevertheless unconscious) functional level, far removed from the messy details of neuron physiology, the brain manipulates symbols that correspond (somehow) to ordinary concepts, and that it does so according to rules that are not drastically unlike the kind we follow consciously when we do arithmetic or solve a logic problem.

It became clear that tasks such as translation, speech recognition and normal conversation require an enormous amount of general knowledge

If this guess is right, the AI researcher’s best bet is probably to attempt to reverse-engineer the brain’s high-level symbolic computations. She might turn to cognitive scientists and psychologists for help. She won’t bother neuroscientists, because she doesn’t need to know how the brain’s symbol-juggling routines are implemented at the physiological level.

The approach just described – "symbolic" or "Good Old Fashioned" AI – was dominant until the mid 1980s. It had some significant successes. Chess programs began to defeat club-level human players in the late 1960s. Around the same time, Terry Winograd’s SHRDLU demonstrated a limited ability to converse about a simple virtual world of blocks and pyramids.

Chess programs continued to improve (they now outclass even the strongest human players), but progress on the linguistic front soon ran out of steam. It became clear that tasks such as translation, speech recognition and normal conversation require an enormous amount of general knowledge. In practice and perhaps also in principle, human programmers cannot codify this knowledge as lists of high-level rules. Symbolic AI had even less success with visual processing and object-identification tasks; the basic obstacle seemed to be the same.

A facial recognition system for law enforcement during the NVIDIA GPU Technology Conference in Washington in 2017. Photo: Saul Loeb AFP/Getty Images

Ideally, an AI system that needed a lot of general knowledge would acquire it the way humans do – by learning from experience in an open-ended and flexible way. No one knew how to make a symbolic AI program do that. However from the earliest days of AI, some researchers had favoured a radically different approach. Rather than trying to emulate human conceptual abilities directly, they took their cue from the physiology of the brain. They built systems in which many simple units ("neurons") were interconnected to form complex – and highly adaptable – networks. Early neural networks couldn’t do very much, but they acquired their limited abilities through experience. 

For example, if a researcher wished to build a neural network to calculate the AND function, she could begin with a generic do-nothing network, and present it with a few dozen examples of correctly evaluated ANDs. By using a general-purpose training algorithm to update the connection weights in response to each example, she would quickly teach the network to calculate the desired function. The hope was that larger neural networks would eventually be able to perform much more impressive tricks: translate a Russian sentence into English; reliably take dictation; judge whether a photograph depicts a cat.

In the internet age, no research project will stall for lack of cat pictures

But training large neural networks is not easy. It is computationally expensive, and you need a lot of data - tens or hundreds of thousands of training examples. These were prohibitive drawbacks in the 1960s and 1970s. 

Since then, computers have become vastly more powerful, and data more plentiful. In the internet age, no research project will stall for lack of cat pictures. Multi-layer neural networks now drive state-of-the-art systems in image classification, speech recognition, machine translation and many other fields. Deep learning is the hottest thing in tech. 

Still, it is important not to overstate what has been achieved. Artificial general intelligence is not just a few years away. Chatbots are still very unconvincing. And if we are ever to build a system that can reason about cats as well as recognise them, we will almost certainly have to integrate some of the methods of Good Old Fashioned, symbolic AI.


The views expressed here are those of the author and do not represent or reflect the views of RTÉ