Until
next week,More AI History 3: Neural Networks
Dateline: August 17, 1997
SYMBOLIC AI in the form of expert systems began to run into size and complexity problems in the 1980s, as we saw in last week's article. But help was at hand.
In the late 1950s/early 1960s, while Marvin Minsky was pursuing mainstream symbolic AI, his old high school classmate Frank Rosenblatt of Cornell University was taking a different tack. With a background in psychology, Rosenblatt naturally approached AI as an exercise in modeling the human brain – which is, of course, a neural net. His first effort was called the Perceptron.
The Perceptron got a lot of press on the basis of its touted potential, but in reality it couldn't do very much. The press of 1958 reported the Perceptron could distinguish between a cat and a dog, but that was not true. It could do so in principle, but not (yet) in fact.
The Perceptron was essentially a blending of the Pandemonium concept (we met earlier) of demons (agents) with the neural net computing function described in 1943 by mathematicians Warren McCulloch and Walter Pitts and the network learning function demonstrated in 1949 by Donald Hebb. The demons were photocells which informed the neurons how much light was being sensed. The neurons weighed the relative strengths of the light input reported by the demons. If a weight was higher than a threshold predetermined by the programmers than a neuron would "fire." That is, it would tell the next level of neurons, which could light up pixels on a screen, about the light intensity. Eventually a pattern -- such as a letter of the alphabet if words were the target object -- would emerge on the screen.
Similar technology is a now applied widely; in scanners, for example. AI, like the space program, does not have to succeed in the equivalent of landing a human on Mars for it to produce many valuable spinoffs.
Rosenblatt's principle, however, that neural nets could be trained to respond correctly to stimuli, was the foundation on which neural network technology was built, and many researchers took to it during the 1960s, until Minsky and Seymour Papert (of LOGO fame) published an influential book criticizing the Perceptron. There followed what Daniel Crevier, in his book AI: The Tumultuous History of the Search for Artificial Intelligence, calls a "research vacuum" in neural network development that lasted until the 1980s, and Minsky and Papert have been held responsible by the neural network community for the delay in progress of what was to turn out to be a very fruitful approach for AI.
In 1974, Paul Werbos developed a neural network training procedure called "back-propagation" which not only overcame the Minsky/Papert objections but which, it has recently been discovered, is a fundamental feature of biological neural nets. It is simply a form of iterative feedback -- cybernetics -- in which the results of the neural process are fed back to the network so it can learn what is a "good" response to a given input.
Back-propagation was independently rediscovered by David Rumelhart and David Parker in the early 1980s. By 1986, Rumelhart, James McClelland, Nobel laureate Francis Crick (co-discoverer of DNA) and others had developed this and other neural network ideas into a two-volume work which is today regarded as the bible of the connectionist school.
In the 1990s, connectionist technology grew rapidly and socked its symbolic sibling in the eye with successes in automatic speech recognition and autonomous land vehicle systems.
But life is a compromise, and AI is part of life. The symbolic and connectionist approaches are now commonly used together. Each has strengths that make up for the other's weaknesses.
Incidentally, AI critic Hubert Dreyfus, whom we met briefly in a previous article, was not so much antagonistic to AI in general as he was to the symbolic approach. He tended to side with the connectionists.
Until
next week,
NEXT WEEK: Dragon NaturallySpeaking. A follow-up to my earlier review of this award-winning, neural net-based automatic speech recognition software from Dragon Systems, now that I have the benefit of a better sound card.