Machine Learning: Body and Soul of AI - Artificial Intelligence

A NEWBORN baby is not what we would call intelligent. Not in the sense of being able to seek and acquire knowledge of, make sense of, and make decisions from, high-level information such as that contained in a plot of quark scatterings from a particle collision, or in the sound of raindrops spattering against the window pane.

The baby is conscious and sentient, for sure, and it also has a level of intelligence already built in; "hard wired" into its chromosomes and to some extent pre-patterned into its neural structure. But what is most important is that the baby arrives in the world already coded to learn—to acquire knowledge through experience.

Some machines are now being built that way, too, using techniques drawn from Mother Nature. Currently the major techniques being researched and applied in machine learning are genetic programming and neural networks, but that is not all. Avrim Blum of Carnegie Mellon University's Machine Learning Group calls machine learning "a melting pot of ideas from a wide variety of disciplines."

At the basic science level, machine learning is concerned with computer programs that automatically improve their performance through experience, so to practice in the field you’ll need a background in computer science, programming, and math.

Typical areas of inquiry include understanding how concepts are formed and how we learn search heuristics (rules of thumb—for example, as the baby acquires knowledge, it will eventually learn to check to see if it is raining when deciding what to wear before going out.) How do we learn to break a too-big problem up into smaller, more manageable sub-problems? How does the baby learn to improve its motor skills, so it can raise spoon from bowl to lips? How do we learn to explain new concepts in terms of existing concepts (much as I am trying to do in these articles)? How do we learn to revise and extend existing theories in light of new evidence, and discover scientific laws and theories?

To answer these questions, and to be able to apply the answers in a machine context, requires even deeper exploration into issues of knowledge representation and reasoning performance using a variety of alternative approaches including rule-based (expert) systems, frame-like structures, probabilistic concepts (such as Bayesian networks), and neural networks.

Getting right down to the theoretical nitty-gritty—the basic science—of machine learning requires addressing such questions as "What kinds of guarantees can one prove about learning algorithms? What could one hope to prove? What are good models that are both amenable to mathematical analysis and make sense empirically? Can we use these models to come up with improved algorithms? What can we say about the inherent ease or difficulty of learning problems?" To address such questions, the researcher must be able to combine "notions and ideas from statistics, complexity theory, cryptography, and on-line algorithms, as well as empirical machine learning and neural network research."

The above quotes are from Blum’s description of his course in machine learning at CMU. It’s not hard to see why he calls machine learning "a melting pot," nor why a computer science background helps.

Machine learning methods have been successfully applied to problems such as getting a vehicle to drive itself, getting a machine to learn to recognize human speech (see Previous Features for my essay on automatic speech recognition), detect credit card fraud, improve the yield of integrated circuits produced from silicon wafers, and strategies for game playing and simulations. The following table, from CMU’s Web site, illustrates some of the general application areas of specific machine learning approaches.

Basic Science	Applications
Learning from structured text, images, sound, ...	Medicine
Active experimentation, exploration	Manufacturing
Learning across multiple sources (e.g., databases, newsfeeds, the Web)	Financial
Learning embedded in decision aids	Intelligence analysis
Automatic refinement of decision policies	Public policy
Cumulative Learning	Marketing

Among the reasons machine learning is currently limited to solving sub-problems in very specific and finite domains, as opposed to solving major problems in general human and social domains, are these:

Several efforts are underway to take care of the context issue. We came across Doug Lenat’s CYC project and the Canadian MISTIC project in an earlier article (see Previous Features). Another example is Belgium’s Principia Cybernetica project, which aims to turn the Web into the machine’s learning library.

The system complexity issue is being tackled using AI techniques themselves to create self-designing systems. Hugo de Garis’ "Brain Builder" project in Japan is about a cellular automata machine (CAM) that grows its own complex circuits using genetic algorithms, just as the human being is "grown" through genetic programming.

Probably the biggest challenge facing would-be creators of Machina sapiens is getting it to reason using not just logic but also the illogic of emotion.

No-one is quite sure what a "mind" is, but we do know it’s something more than the sum of a brain, a body, a nervous system, sense organs, and some limbs. With just those attributes, a machine could make a pretty good showing as an intelligent, reasoning being; but if it could not experience and manifest joy and despair, passion and tranquility, anger and resignation—in short, emotion—then we’d think of it as a soulless being, would we not?

IBM’s chess-playing computer, Deep Blue, has beaten world champion Gary Kasparov once and will undoubtedly soon be made powerful enough to defeat him every time. But oh! What an arid win! Deep Blue could not anticipate and enjoy the game, savor it afterwards, feel elated at a win or deflated at a loss. More importantly, it is likely that those emotions in Kasparov are what help to make him such a brilliant chess player even without Deep Blue’s massive computational prowess.

(On a side note, I would propose that not one but two identical Deep Blues be created, and that Kasparov be teamed with one of the machines against the other, with the power to override the moves proposed by his ally.)

Neurophysiologists (in particular, Dr. Antonio Damasio—see his book Descartes Error: Emotion, Reason, and the Human Brain) are uncovering evidence not only that the human mind is made up of reasoning PLUS emotion, but also that emotion has a neurophysiological basis. Given this knowledge, it will remain only to apply it to a reasoning machine and we may well have then created a truly sentient being. Our understanding of the mechanism for emotion is far behind our understanding of the mechanisms for genetic development and adaptations, and for neural structures and functioning, but as we have seen these latter two areas of understanding are already being applied with great success in machine learning.

It seems only a matter of time before the mechanism of emotional intelligence is understood sufficiently to become programmable at some level.

The scariest aspect of HAL, the intelligent machine in the movie 2001: A Space Odyssey (another topic covered in a Previous Issue) is not his cold, logical, reasoning. The film merely hints that it might be HAL’s emotion that underlies his murderous behavior toward the crew, but the hint is enough to send shivers up the spine. We surely could live with robots that kill (we already do—auto factory workers have been killed by robots), but how will we fare with robots that may learn how to lie, cheat, and hate?

As every major religion and philosophy tells us, you cannot appreciate happiness without experiencing sadness. If we are going to create, or help create, a truly intelligent being, then we have to accept that it will learn to hate, as well as to love. It will know Bad, as well as Good, Wrong, as well as Right. At this point in our discussion, we are moving away from the scientific and technical aspects of machine learning and into the (to me, much more interesting!) philosophical aspects.

We’ll pursue some of those aspects next week, in examining the extraordinary learning of Pierre Teilhard de Chardin.

NEXT WEEK: French geologist and paleontologist Father Pierre Teilhard de Chardin, S.J., 1881–1955, coined the word Noosphere to name a sphere of consciousness above the geological and biological spheres that preceded it in evolution. Was he just a bit before his time?

POSTSCRIPT: "Ever since computers were invented, it has been natural to wonder whether they might be made to learn. If we could understand how to program them to learn the impact would be dramatic. The practice of computer programming would be revolutionized as many tedious hand-coding tasks would be replaced by automatic learning methods. And a successful understanding of how to make computers learn would most likely yield a better understanding of human learning abilities and disabilities as well.'' (from Tom Mitchell's book Machine Learning.)