The secret headquarters of The Legion of Doom, where the three-day Jeopardy! event was shown while Lex Luthor and Gorilla Grodd plotted their latest scheme to defeat The Justice League. Image via Wikipedia

I watched the IBM Jeopardy! Challenge at RPI’s EMPAC facility last evening with an unhealthy combination of skepticism and ignorance.

The three-day event coincided with the airing of three episodes of the long-running game show “Jeopardy!” hosted by Alex Trebek that pitted two former champions (Ken Jennings and Brad Rutter) against Watson, an artificial intelligence software and supercomputer developed by IBM.

Watson won, which didn’t surprise me. What did surprise me was the complexity behind the mechanizations. Despite its appearance on television, Watson is actually a room filled with 90 servers processing a complex network of hardware and software and combined information retrieval with language processing, reasoning, and other factors. If you’re interested in the specifics, IBM has an entire website devoted to it.

I was far more interested in the crowd reaction to the machine. Students and community members from in and around Rensselaer Polytechnic Institute – some of whom participated in the development of Watson – packed the auditorium to what I estimate was at least ninety percent capacity. When the facade of the computer appeared, they roared in approval. Correct answers garnered enthusiastic applause while the machine’s wildly inaccurate guesses were met with laughter that ranged from slight titters to agonizingly forced guffaws. Members of the crowd buzzed with anticipation while some showed off t-shirts that read “I <3 Watson”.

The endearment shown towards the IBM project was understandable given the school’s technical focus, though no less distressing to someone like myself who has far more appreciation for things he can grasp and appreciate; things such as endurance, creativity, and feeling. To put it kindly a more arts-minded fellow, though it may be more accurate to say “a dunce.”

Students made t-shirts and cheered for Watson, rather than the underdog humans. (Image courtesy The Approach).

Still, I couldn’t help but not be as impressed by the machine’s performance given my limited understanding of its intricacies and infrastructure. I had to ask the person I accompanied to the event – former Times Union intern and local blogger Erin Morelli – if I was correct in my analysis that the machine was essentially the equivalent of a very effective Google search. I suspected, of course, that wasn’t the case, and it isn’t. Erin was kind enough to tell me as such without flooding me with details I couldn’t possibly wrap my head around.

Discussion occurred before the program, during commercial breaks, and after the event with a panel that included IBM representatives and RPI alumni Adam Lally and Dr. Chris Welty, as well as members of RPI’s faculty, including Cognitive Science Department Head Dr. Selmer Bringsjord. They fielded questions from the audience and spoke on the progress of artificial intelligence and its application in this instance.

Dr. Bringsjord in particular introduced some exciting and insightful talking points, which made me curious about his view of the project. Having already decided to devote this space to the event, I was going to find him after the event and pick his brain, but by 8:30pm I hadn’t yet eaten dinner and was reminded that the Times Union does not actually pay me for my efforts.

Another time, Dr. Bringsjord. You’ll be hearing from me soon.

The real story to me, though, was the reaction I mentioned earlier to the machine. Aspiring scientists and engineers in the crowd treated the contraption with an eagerness and endearment that, though sometimes masked through attempts at ironic humor, was clearly affection for the machine. It was all in fun and a good sign for those of us who recognize our country’s quiet crisis in the maths and sciences, but I still felt slightly disheartened that people were rooting for the inorganic to triumph over the human underdogs.

When I cheered for a Jennings response early in the evening, Erin leaned over and said “you’re going to be in the minority here tonight.”

“I know,” I sighed.

As the Double Jeopardy round reached its apex, Watson hit the second Daily Double. Former champion Ken Jennings, who at one point in the program had given the machine a run for its money and put a scare into the pro-Watson crowd, leaned against his podium with a smirk of bemused resignation. He knew that with the given category and two-day totals, he had been caught in a checkmate. There would be more questions in the round and Final Jeopardy, but nothing barring a wagering error in Final Jeopardy that the computer was literally incapable of making was going to change the outcome.

"A man ain’t nothin’ but a man, but if you bring that steam drill round, I’ll beat it fair and honest. I’ll die with my hammer in my hand, but I’ll be laughing, because you can't replace a steel drivin’ man."

As I saw the sporting Jennings accept his fate, I was reminded of the American folk tale of John Henry. I looked at Jennings and imagined Henry, hammer in hand, competing against the steam-powered hammer that the railroad company designed to replace him.

In the case of artificial intelligence, we can take solace in the fact that there is no Watson to replace human ingenuity, creativity, and determination. As Dr. Bringsjord pointed out to the crowd that remained after the program, Watson is wholly incapable of actually creating thought. It is a series of processes that searches for an answer and can “learn” from its input and mistakes, but it cannot create a solution from original thought.

Based on my limited understanding of the subjects of artificial intelligence and human thought, that’s what makes ventures such as this an interesting conundrum. Human beings are trying to bridge a divide between a simulation of human problem solving and actual human thought, but with a limited understanding of exactly how, where, and why things like human thought and ingenuity are generated in the three pounds of hardware and software contained in our skulls. We are pushing machines like Watson to approach an endpoint, but don’t know where and what that endpoint is. Thankfully, this means the science fiction doomsday scenarios regarding artificial intelligence remain firmly rooted in the “fiction” and not so much in the “science.”

Still, there was a part of me that was disappointed when Watson’s two-day total was shown. It had completely trounced its human competition, with Jennings a distant second and Rutter an inconsequential third.

And as John Henry lay dead on the ground, this time defeated by the steam-powered hammer, the crowd roared with approval and heralded the coming of an exciting new age.

29 Responses to At RPI, crowd cheers for machine as IBM’s Watson beats human competition on Jeopardy

  1. Rob Madeo says:

    Open the pod bay doors, HAL.

  2. Angelos says:

    Of course I was cheering for Watson on Monday and Tuesday, when I was there. By which I mean I was cheering for the people who built it.

    This was a huge technical achievement; the future applications in medicine, language, and who knows what else are quite exciting.

  3. Hopeful says:

    So glad someone finally wrote about it. I found it pretty interesting to watch and was glad Jennings came around and gave Watson a run for his (it’s?) money. It was obvious Watson was going to win early in the first match. But Watson didn’t have that human element of knowing how to properly wager with Daily Doubles or knowing to work a category from low to high – he’d jump all over the board, which, from my understanding, isn’t a good game plan for Jeopardy. But it didn’t seem to faze him… I’m not a computer person, but I think IBM should be proud of it’s accomplishment.

  4. Tony Barbaro says:

    we’re all doomed…..Terminator..didn’t know it was non-fiction

  5. Alec Perkins says:

    @hopeful Funny enough, its least human skill, being wicked good at math, was its biggest strength with the Daily Double and Final Jeopardy wagers. According to Adam Lally, Watson’s bets were based on how sure it was about the given category, how much money it had, and how much money the opponents had. In the end, it bet precisely the most that it could, and still ensure win by 1$ if it got the question wrong.

    Also, the jumping around was Watson hunting for Daily Doubles, a strategy also employed by Jennings.

    I’m very excited by the outcome, but it’s clear there is a long way to go before a computer can handle things like implied meaning the way humans do naturally. And then there’s reasoning. Jennings didn’t *know* the first Final Jeopardy answer was Chicago, but he reasoned it from a very limited base of evidence, something Watson cannot do. All of Watson’s strengths are things we’re bad at, and its weaknesses things we’re good at. Sounds like good qualities for something made to assist people.

  6. Hopeful says:

    Alec – didn’t realize that. Thanks. I also really liked seeing the answers on the bottom when Watson didn’t ring in first to see how far off he was.

  7. Iceman says:

    While it was thoroughly entertaining, it seemed like Watson had an unbelievable advantage at ringing in.. seemed to be frustrating Ken and Brad (and myself a little bit)..

  8. Alec Perkins says:

    The most interesting part was definitely the process. I’m glad they showed Watson’s confidence in its answers, and the other answers it was considering. I’d really like to know the confidence of the humans in some of their answers.

    The buzzing was interesting, because unlike humans, Watson would only buzz if it had an answer that it was confident about (though not necessarily correct). Humans can and do buzz in before they have the answer, since they have an additional five seconds to respond. The short questions of the Actors Who Direct category gave the advantage to the humans, since Watson was still working on the answers by the time buzzing was enabled. This makes the speed of the system all the more impressive.

    • I wonder, though, if it was an unfair test in the sense that it didn’t have to account for those delays in physical reactions. My understanding is that it was test of thought process, yet there were some moments where I felt Jennings had arrived at the answer as quickly (if not quicker) than Watson, but was held back by his reflexes.

  9. John says:

    I thought the best part was Mr. Jennings’ final answer as the unrelated comment about welcoming our computer overlords was hiliarious.

  10. Alec Perkins says:

    Dr Welty insisted that humans could have the advantage, because of the way the buzzing system works. I wouldn’t be surprised if Watson is one of the fastest buzzers on Jeopardy, but there were a few times where Rutter and Jennings were able to beat Watson, even on longer questions.

    It’s funny how easily we anthropomorphize Watson. The name and voice certainly don’t hurt. Nearly everyone called Watson ‘he’, except that one girl during the Q&A (props to her for asking if Watson is a girl). It was also easy to forget how unaware of context Watson is. Not a surprise, I guess. So much harder to relate to, and root for, an it.

  11. MBAMom says:

    Thanks for the spoiler in the title. We had to DVR last night’s episode and were planning on watching it later. After watching the first 2, I had a feeling what the outcome was going to be but still wanted to be surprised.

    • MBAMom – You said it yourself – DVR’ed. Sorry, but all bets are off, particularly since it was an event and has become a news story that is widely being reported on today. You’d have a point if this were written yesterday, but not today.

      It’s like a live sporting event. If you miss its airing, it’s not the responsibility of everybody else to shield you from results.

  12. beavis518 says:

    Now once we have Waston, or another AI computer work on developing a better AI Computer then that computer developing it’s successor – then things will be interesting..

  13. Mark Ramirez says:

    One point I think you’re missing is the fact that what Watson “did” is a human achievement, not a computer one. That an “inorganic” can accomplish these things is a credit to the engineers at IBM, and so there’s nothing strange about cheering for their creation to succeed.

  14. Johnnyboy says:

    So I see everyone is missing a big point here: Kevin won’t get paid for this excellent commentary. Maybe Mike and Rex can find a free lunch in the budget. Time to pony up boys. This piece is good writing.

  15. ds says:

    during K. Jennings mini “run” last night, I actually thought something was wrong with WATSON!!! HAHA

  16. MBAMom says:

    Kevin – You’re right, and I agree with you. Here’s the thing though – this is a blog, not a news story so the primary intent (emphasis on primary) is not to report but to provide commentary so it’s a little different. So yes, if I didn’t want to know the outcome right away I would not look at a news story. Blogs are a little different and others (take for example WSJ’s Speakeasy blog which regularly writes about TV series), put spoiler alerts on their posts.

    Anyway, I digress. The entire Watson project is fascinating. It was entertaining to see Watson go up against the best players to ever have played Jeopardy. The ability to see Watson’s “thought process” by displaying the 3 top answers with probabilities was pretty amazing too. You certainly don’t get that with 3 players. To me, what the competition was about was, can a machine act like a human? On an “academic” basis, yes the machine is going win each time, but the decision to respond, and when, not to mention the little cultural “interpretations” one has to make with the Jeopardy clues, those are the true achievements here.

  17. EZ says:

    SPOILER ALERT !!
    Verbal Kint is Kaiser Soze

  18. Will Gilchryst says:

    SPOILER ALERT(S):
    Mubarak has resigned; and
    Darth Vader is Luke Skywalker’s dad.

  19. A.C. says:

    Love it, great post. I agree, it was interesting (and frustrating) to have the answer and see it in Ken Jennings’ face that he had the answer, but Watson buzzed in first. I was starting to get excited when Jennings and Rutter had their run of buzzing in before Watson, but – alas!

    Fascinating to watch how a computer parses human language. I’d be interested to see if they can program recognition of proper names and so forth: so Watson could say “Who is” instead of “What is.” Nuances like that must be hard to program in among all the other stuff. Or to put it in words I can understand: algorithms is hard.

    Pet peeve: Watson was credited for a correct answer when it said, “What is Maxwell’s silver hammer,” when the correct answer was actually, “Who is Maxwell.”

  20. Kingstonian says:

    IBM Research employee here – love the post, Kevin – and love the intelligent and civil discourse in the comments. So nice to see something other than references to SkyNet, HAL and IBM as an evil, soul crushing corporate entity.

    As you well know – we love the RPI. Dr. David Ferrucci – Watson’s ‘father’ – and our director of Research, John Kelly, are RPI alums.

    It’s only fitting that RPI got to watch and celebrate too. Great post.

  21. xxhaimbondxx says:

    They can definitely go away with the host now and save some dough.

  22. twisted says:

    Maybe the spinoff will be searching the web for hints of terrorism or our public records for hints of tax evasion! RPI does much research and development under the guise of goodness, such as the retinal scanning rapid identification algorithym program.

  23. Cihan says:

    Er, twisted?

    Watson was not developed at RPI, RPI has nothing to do with it. Some of its developers are RPI alum, and that’s why they decided to hold the event at RPI.

    Retinal scanning rapid identification algorithms sounds flashy, but what project are you actually referencing? Research is no secret at RPI, can you show me a reference? I don’t defend everything that the school does, and I have distaste for plenty, but you’re sounding quite Beckian here.

  24. cp retiree says:

    And Toronto is ours!!

    Impressive achievement, but it’s a little scary to think about how it may be applied in the future.

    I also think the humans had a very slight buzzing-in advantage over Watson, in that they could anticipate the end of the question from the inflection in Alex’s voice.

  25. EZ says:

    CP, the probalem with human anticipation is if you miss it, you’re blocked out for like a second before you can buzz back in. In football terms a great DE who anticipates a snap will likely be called for offsides 50% of the time. Watson will never go offsides, so if he figures the answer out before the buzz-in time is allowed, his pure reaction time is much greater than a human’s. Alex has often commented on buzzer mechanics and states that he thinks he would probably lose a Jeopardy game if he was playing it with younger people who have an edge with reaction times.

Leave a Reply to Hopeful Cancel reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>