When Watson, the artificial intelligence supercomputer, competed against former “Jeopardy” winners Ken Jennings and Brad Rutter in Feb. 2011, there was a 74 percent chance of Watson winning and a 26 percent chance of David Ferrucci losing his job.
David Ferrucci was the principal investigator for the IBM research team that brought the world Watson, which would eventually win “Jeopardy” with $77,147, which was $53,147 more than the runner-up. Ferrucci, who received both his master’s and doctorate from Rensselaer Polytechnic Institute, was posed in 2006 with the challenge of creating a computer capable of open-domain question answering, or responding correctly to questions posed in natural language.
“This has been a dream of AI and it’s been a longstanding expectation over the course of science fiction,” Ferrucci said.
Open-domain question answering, Ferrucci said, is a challenge because it requires a good deal of background information and context to process both the question being asked and the possible answers.
“The real difficulty comes from understanding the expression of the question and the expression of the content, and literally mapping the meaning,” Ferrucci said. “We have no idea what the question is going to be ahead of time, never mind the answer.”
In his presentation at TEDx Sunday, Ferrucci used the sentence, “The bat came flying toward the window” to show how homophones, or words with multiple meanings, can confuse the meaning of a sentence. Taken out of context, this sentence could have multiple meanings, given the fact that “bat” could refer to either the flying mammal or a piece of baseball equipment.
“You need more context to map those symbols to that experience,” Ferrucci said.
Deciphering the meaning of a sentence, Ferrucci said, requires a lot of interpretation and depends on being able to relate to shared experiences.
“The more context you have, the more you can sort of map those symbols’ actual meaning, and that meaning is founded in that common experience,” Ferrucci said. “The richer the context, the more we can narrow the possibilities and zoom in on what was really intended by the symbol.”
Ferrucci said that deciphering meaning is crucial as the volume and complexity of information is outpacing humans’ ability to digest and understand natural language.
“The amount of information is growing incredibly rapidly and we have more and more complexity in life,” Ferrucci said. “We have to get at that knowledge and we have to get at it in a more precise and more concise way so we can make better decisions.”
Watson was introduced to the world through the “Jeopardy” challenge, but the computer’s capabilities also have potential to be useful in the medical field. Ferrucci is turning his attention toward another area of studying human behavior — economics.
Ferrucci is working with a company called Bridgewater to study macroeconomics from a behavioral, rather than formulaic, perspective.
“Probably the two most important things in civilization are health care and … economics, because the stability of our governments, the stability of our societies, depend so much on managing the economic system,” Ferrucci said. “Thinking about the economic system as a system that’s driven by human behavior is fascinating to me.”
Q&A
Pipe Dream: What kinds of research did you participate in while you were attending school?
David Ferrucci: As a junior in college I started switching over to the computer science major, I started targeting computer science graduate schools and I did a thesis on medical expert systems where I researched different programs that were being developed at the time to kind of mimic diagnostic reasoning. And then in graduate school I did my own actual full-fledged medical expert system as well as what’s called a semantic network to model how humans represent primitive concepts and build more abstract concepts on those primitive concepts.
PD: How did the Watson project start?
DF: It was actually the semantic network stuff that got me an interview at IBM research and I worked there for a couple years on an AI project and then went back to get my PhD at RPI. My thesis was in automatic configuration, getting the computer to automatically configure cars or computers or different kinds of objects based on the different requirements that a person might have. So all these projects really focused on AI, and then when I got to IBM the second time, which was in 1995, I started to build a team focused on lots of different projects but all of them geared toward a combination of AI and software engineering and software architecture. So by the time it came around in 2006, I had a really strong team of Articficial Intelligence experts who were good at machine learning and knowledge representation and reasoning, I had software architects and software engineers, so we were in a perfect position to take on the Jeopardy project.
PD: What was it like working on the project?
DF: It was a dream come true for me, because it was a project that was going to test everything I hag been learning and researching for years and I was going to be able to learn, “can these technologies all come together to do something to compete with humans at this very knowledge-intensive, language-intensive task?” And whether I failed or succeeded I knew I was going to learn a lot about AI, so it was an incredibly exciting project in that regard. And ultimately we succeeded, and the next really exciting thing I was going to be able to do was convince IBM to take this technology and pursue medicine. So it sort of rounded out a 30-year journey for me, where using Watson to help doctors do diagnostic reasoning and treatment evaluation was something I dreamed about doing going all the way back to high school and college.
PD: What’s next for Watson?
DF: IBM is taking Watson into the health care industry in a really big way and they have a number of really interesting projects, one with Memorial-Sloan Kettering, another one they have with Cleveland Clinic, so it’s really exciting what they’re doing with it. But if you think about trying to understand what’s going on, and take lung cancer for example, there’s just so much information about so many different patients … there’s an incredible variability in the experiences they go through, the symptoms that they have, the treatments that work and don’t work, and how well they work. And you want to kind of relate that to information about the patients themselves so that you can make more informed decisions, you can do a better job at evaluating treatments, and quickly figuring out what treatment is more likely or less likely to work.
This is one of the areas in which Watson can really help to dive in and do analysis and try to derive insights into that data and help them do a better job at advancing treatments and evaluating the right treatments for the right patient, because it’s really so personalized.
Making those connections from what you’re observing with the patient and what the potential diagnosis is and what the treatment is—making those connections by navigating existing documents, existing knowledge in the language humans use to communicate with one another, is much more efficient and much more powerful than imagining humans sitting down and carefully encoding all the knowledge and “IF THEN” rules. By the time they’re done, the knowledge is old. So you want to take the journals and the papers and the books and everything that’s written and kind of feed it into Watson and do that.
PD: What project(s) are you currently working on?
DF: One of the other things that really interest me is macroeconomics at a very very fundamental level. Probably the two most important things in civilization are health care and … economics, because the stability of our governments, the stability of our societies, depend so much on managing the economic system. And thinking about the economic system as a system that’s driven by human behavior is fascinating to me.