We in the chess world saw computers trump humans in our favorite game more than 10 years ago. Soon, will fans of Jeopardy! witness the same?
The long-awaited match up between the world's best Jeopardy! contestants and IBM's Watson is coming in February. Judging from this news story and the following video, the 'test' rounds are not looking good for the flesh and blood set.
The Watson project is IBM's long-awaited follow-up to their Deep Blue project, which defeated Kasparov in a chess match in 1996. Despite that famous victory, Deep Blue was essentially a failure for IBM -- it turned out that all the specialized chess-playing hardware and software was not readily adaptable to "real-world" problems.
They turned to tackling one of artificial intellegence's most challenging problems-- the ability to flexibly work with the complexities of real language. Jeopardy!, which combines word play, cryptic clues, and trivia, embodies this problem perfectly. And this time, the real-world applications are nearly endless. Imagine a voice-responsive Google that answers complex human questions, or a call-in directory that no longer requires pushing 0 to speak to an operator.
I imagine that they are very close to satisfying the Turing Test, as well.
Stay tuned as we continue to cover Watson's attack on Jeopardy! As chessplayers, we understand how this goes.
The Chess Blog of the Boylston Chess Club... The Boylston Chess Club (BCC) is a part of the Boylston Chess Foundation (BCF).
4 comments:
Watson sounds impressive, but I think passing the Turing Test is a long ways off yet. Whatever the timeline, Watson's capabilities, even if it aces Jeopardy, are child-like compared to what a successful candidate for the Turing Test needs.
Then again, how many actual people in a chatroom could pass the Turing Test?
Do they still have chatrooms? Remember how mindless those conversations could be??
For a computer to pass the Turing Test, the computer has to be able to win the "imitation game", in which a human interrogator (A) has a conversation with the computer and a second human (B), A and B both know that only one of A's conversation partners is human, A wants to identify the human and B is trying to prove to A that he or she is the human. (And they can only communicate via text messages.) Failure to discriminate between computer conversation and mindless chatroom conversation isn't sufficient. For example, the interrogator can ask each conversation partner to write a poem and can then ask specific questions about their poems and choice of words, their meanings, etc. The computer also fails if the interrogator asks mathematical questions (or perhaps questions about chess) and identifies the computer because the computer answers correctly and too quickly to be human. (Both examples are taken from Turing's article "Computing Machinery and Intelligence", where he introduced the test.)
Turing proposed the question "Can a machine pass the imitation game?" as a "replacement" for the question "Can machines think?", which he thought was too fraught with ambiguities to be capable of a meaningful answer. Turing, writing in 1950, was confident that by 2000, a computer could be built that would pass the test at least 30% of the time with an "average interrogator" afforded five minutes for questioning. It's obviously a tough test, but certainly a computer like Watson should give pause to anyone who doubts that it will ever be passed. But we might need another 50 years.
That is the strict original version of the Turing Test. But it is too easy to program a computer to do math slower or to make typing mistakes that mimic a human. That isn't really what I have in mind when I think of the Turing Test anymore. (I was influenced by a recent Turing-like test, in which teams of computers and teams of humans play Halo and outside observers determine the computer teams. One team did really well by programming in a lot of random bad moves, jerky action, goof-off jumping, etc. To me, that is like making Rybka play chess worse just to make it more human like.)
Instead, imagine calling up a company and Watson answers, "Hello. How can I help you?" and then you say something like, "Well, I've had my phone for only two weeks, and it was working fine, but now the battery is dead already." and Watson says, "I see. Well, have you tried recharging it?"
"Of course. I'm not an idiot."
Watson laughs, "I'm sorry, but you would be surprised at how many times that is the right answer. Can you plug it into the wall for me and read out the display on the front?"
"OK, I'm doing that, but I don't see anything. It is totally blank."
"I see. I'm looking up your customer number now. Can you confirm that your phone serial number is 123456789?"
"Yes."
"Good news! You fall under the 30 day warranty with that particular phone. Would you like me to send you a replacement phone?"
etc.
I think if a computer can do THAT, I'm going to be blown away. And I don't need it to act like a jerk, accidentially hang up on me, or not be able to answer my question to be satisfied that the computer can do what Turing had in mind in his test.
Now, this doesn't mean the computer is "thinking" the way Turing's original test was intended to determine. But I think the "Chinese Room" (and the modern day "Chat Room") already makes it clear that Turing's test doesn't answer that question.
Post a Comment