Join Early Retirement Today
Reply
 
Thread Tools Display Modes
Watson on Jeopardy
Old 02-14-2011, 10:37 PM   #1
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
 
Join Date: Jul 2006
Posts: 11,401
Watson on Jeopardy

Did anyone watch Jeopardy this evening? Normally I would never spend time watching a game show, but I was intrigued by the scientific experiment. "Watson" is the name of a supercomputer that IBM has been working on for several years. IBM approached the Jeopardy team with the proposal in 2007; they wanted to build a computer that could search knowledge, judge the probability of a correct answer, and make a decision to volunteer an answer based on reaching a critical confidence level of being right. That's mostly data mining and reporting, but the neat thing is the interpretation of language cues, which are sometimes pretty complex in Jeopardy clues.

So Watson is ready, and is competing with the two top human Jeopardy champions of all time: the guy who kept the crown for 74 shows, and the guy who won the most money. Part 1 was tonight. Parts 2 and 3 are being broadcast tomorrow and the day after.

Watson (represented on the set by an avatar) gets the questions as text files. It started off in the lead. But some of the questions really threw it off and at the end of tonight's round it was in a dead heat with a human. IT was "sweating" too! Viewers could see Watson's top ranked answers and what I saw suggests to me that the algorithms on popular culture are the strongest - perhaps reflecting the age group of the engineers designing Watson! I've also heard a radio piece that suggested that Watson has demonstrated the ability to learn from the answers of the other Jeopardy players.

I got annoyed that Alex Trebek was referring to Watson as "he"; Watson's "voice" is male but that was an arbitrary human decision.

Anyhow, I find artificial intelligence fascinating.
Meadbh is offline   Reply With Quote
Join the #1 Early Retirement and Financial Independence Forum Today - It's Totally Free!

Are you planning to be financially independent as early as possible so you can live life on your own terms? Discuss successful investing strategies, asset allocation models, tax strategies and other related topics in our online forum community. Our members range from young folks just starting their journey to financial independence, military retirees and even multimillionaires. No matter where you fit in you'll find that Early-Retirement.org is a great community to join. Best of all it's totally FREE!

You are currently viewing our boards as a guest so you have limited access to our community. Please take the time to register and you will gain a lot of great new features including; the ability to participate in discussions, network with our members, see fewer ads, upload photographs, create a retirement blog, send private messages and so much, much more!

Old 02-15-2011, 04:28 AM   #2
Thinks s/he gets paid by the post
 
Join Date: Mar 2009
Location: Pittsburgh, PA suburbs
Posts: 1,796
I watch Jeopardy most evenings, but I made a point of viewing last night. I think that language and AI are interesting, not that I know much about these fields. I saw a CMU professor interviewed last week on a Pittsburgh TV station program "Night Talk" as he and some of his grad students also worked on the Watson project. I am rooting for one of the humans to win. It seems that Watson might lose ground on nuance but is very fast with the answers. I am presuming that since Watson would most often be a male given name, the host is saying "he". I believe Thomas Watson was the founder of IBM if I am not mistaken.
I plan to tune in again tonight.
WhoDaresWins is offline   Reply With Quote
Old 02-15-2011, 04:56 AM   #3
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
clifp's Avatar
 
Join Date: Oct 2006
Posts: 7,733
Darn I've been hearing about this match for months and seen several programs about it, NOVA, Newshour etc. and I missed it...

I couldn't find the Day 1 anywhere online. Anybody found a link for it.
clifp is offline   Reply With Quote
Old 02-15-2011, 05:24 AM   #4
Thinks s/he gets paid by the post
Rustward's Avatar
 
Join Date: Apr 2006
Posts: 1,684
Quote:
Originally Posted by Meadbh View Post
I got annoyed that Alex Trebek was referring to Watson as "he"; Watson's "voice" is male but that was an arbitrary human decision.

Anyhow, I find artificial intelligence fascinating.
Could it be that this project is named after the real Watson who was a "he"?
Rustward is offline   Reply With Quote
Old 02-15-2011, 06:59 AM   #5
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
 
Join Date: Sep 2005
Location: Northern IL
Posts: 26,819
Quote:
Originally Posted by clifp View Post
Anybody found a link for it.
I missed also, should get to see it today. Would love a link to the 1st episode.

I agree with OP, the understanding of the clue is the hard thing. Words can have such twisted meanings. Trying to figure out what they are getting at isn't a very direct kind of thing. I think the 'response' is the easy part (for a computer' - figuring out the clue is the hard part.

I was watching when Ken Jennings was on his streak. One thing I noticed is he seemed to 'size up' the clue, and sometimes buzz in as soon as he could, even if he hadn't figured out the response yet. Buzz in, then figure it out. And I could sorta, kinda do it on occasion if I tried (and got lucky!) Sometimes, you can get a pretty good feeling, just skimming the terms quickly that "I know this!", but it takes another second or two to actually formulate and phrase the 'response' correctly.

The computer can't do that, it needs fully understand the clue, evaluate responses and weigh them, but the faster the computer is the less that human advantage is. The language recognition part is just amazing though.

-ERD50
ERD50 is offline   Reply With Quote
Old 02-15-2011, 07:08 AM   #6
Thinks s/he gets paid by the post
grumpy's Avatar
 
Join Date: Jul 2004
Posts: 1,321
I often watch Jeopardy and was watching last night when Watson was competing. Very interesting. One thing I noticed was that Watson seemed to beat the humans on buzzing in, not that the humans didn't know the "questions", just that they didn't get the chance to answer. It seemed that the human reaction time couldn't compete with the machine's.

I wonder what the outcome would be if the contestants were allowed to buzz in BEFORE the question reading ended. I suspect that the humans would sometimes be able to make an intuitive leap to the correct question before Watson could complete "his" algorithm process. As it is, Watson may already be well into his "thought" process before the reading of the question ends.

I wonder if the programmers were embarrassed when one of the humans gave the wrong answer and then Watson buzzed in and gave the same wrong answer. An obvious oversight in their logic.
__________________
...you can check out any time you like, but you can never leave...
grumpy is offline   Reply With Quote
Old 02-15-2011, 07:47 AM   #7
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
 
Join Date: Sep 2005
Location: Northern IL
Posts: 26,819
Quote:
Originally Posted by grumpy View Post
I wonder if the programmers were embarrassed when one of the humans gave the wrong answer and then Watson buzzed in and gave the same wrong answer. An obvious oversight in their logic.

I laughed when I read about that. That's a mistake that only a bad player would make. Does it mean the programmers never considered their machine wouldn't buzz in first, or that it couldn't learn from others? Says something about the programmers.

I also wonder if the timing of the buzzer is 'fair'. A machine has some advantage here. I never really understood exactly how that works on Jeopardy. Do they have to wait for Alex to stop talking to buzz in? Is there a light or something? I know they get locked out for a few seconds or something if they buzz in too early. A human might make that reaction mistake, but a machine wouldn't.

-ERD50
ERD50 is offline   Reply With Quote
Old 02-15-2011, 08:29 AM   #8
Thinks s/he gets paid by the post
Nodak's Avatar
 
Join Date: Feb 2010
Location: Cavalier
Posts: 2,317
Last week there was an extremely interesting NOVA program on PBS about the design and programming of Watson. If you can find a re-run of it it's well worth watching.
__________________
"Don't take life so serious, son. It ain't nohow permanent." Pogo Possum (Walt Kelly)
Nodak is offline   Reply With Quote
Old 02-15-2011, 09:25 AM   #9
Thinks s/he gets paid by the post
walkinwood's Avatar
 
Join Date: Jul 2006
Location: Denver
Posts: 3,504
I thought that the computer was going to "listen" or "read" to the clue. It is still impressive, but would be so much better if it got the clue in the same way the other players did.

Thanks for the heads up. Will record it tonight.
walkinwood is offline   Reply With Quote
Old 02-15-2011, 09:41 AM   #10
Recycles dryer sheets
glippy's Avatar
 
Join Date: Mar 2010
Location: New England
Posts: 197
Me and my fellow nerd friends are having a party on Wednesday for the final episode. I've followed advancements in Artificial Intelligence and Machine Learning for years and what IBM has achieved with Watson is nothing short of a breakthrough.

We've got a pool going, I'm the only one betting on the machine.

I lot of good can come out of the advancements IBM is making here. Imagine a program with a google like, text-input interface where you can type in a natural language question and get the answer you're looking for 9 out of 10 times. - Or the help it could provide to medical doctors in making a diagnosis.

The change from simply a human putting in a thousands of custom rules to the computer being able to figure out, on its own, what rules to follow in order to succeed makes the "intelligence" and "knowledge" of the computer grow exponentially faster than if people had to sit there and punch in every single rule.

@ERD50
Does it mean the programmers never considered their machine wouldn't buzz in first, or that it couldn't learn from others? Says something about the programmers.

They considered the scenario, and it happened in practice sessions. But Watson doesn't have a microphone or voice recognition software. If he did, they could avoid this problem. Not sure why they didn't bother. Probably a matter of too much to do and not enough time to do it in.
glippy is offline   Reply With Quote
Old 02-15-2011, 09:57 AM   #11
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
ziggy29's Avatar
 
Join Date: Oct 2005
Location: North Oregon Coast
Posts: 16,483
Quote:
Originally Posted by glippy View Post
I lot of good can come out of the advancements IBM is making here. Imagine a program with a google like, text-input interface where you can type in a natural language question and get the answer you're looking for 9 out of 10 times. - Or the help it could provide to medical doctors in making a diagnosis.
Or a machine that can "think" more cheaply and efficiently than human labor, and thus throwing even more people out of work.

I'm not advocating against this; I'm certainly no Luddite. I'm just saying that technology has a high social price tag to it when it comes to making labor unnecessary. And too many of the new jobs that develop this technology that's throwing our people out of work are being created in places like China and India.
__________________
"Hey, for every ten dollars, that's another hour that I have to be in the work place. That's an hour of my life. And my life is a very finite thing. I have only 'x' number of hours left before I'm dead. So how do I want to use these hours of my life? Do I want to use them just spending it on more crap and more stuff, or do I want to start getting a handle on it and using my life more intelligently?" -- Joe Dominguez (1938 - 1997)
ziggy29 is offline   Reply With Quote
Old 02-15-2011, 10:00 AM   #12
Recycles dryer sheets
glippy's Avatar
 
Join Date: Mar 2010
Location: New England
Posts: 197
@ziggy29

I agree. The more adept robots and artificial intelligence gets, the more we'll have to think of an alternative to an economy where we sell our labor.
glippy is offline   Reply With Quote
Old 02-15-2011, 10:34 AM   #13
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
 
Join Date: May 2005
Location: Lawn chair in Texas
Posts: 14,183
Quote:
Originally Posted by glippy View Post
@ziggy29

I agree. The more adept robots and artificial intelligence gets, the more we'll have to think of an alternative to an economy where we sell our labor.
I'm working on that alternative as we speak...
__________________
Have Funds, Will Retire

...not doing anything of true substance...
HFWR is offline   Reply With Quote
Old 02-15-2011, 11:17 AM   #14
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
Midpack's Avatar
 
Join Date: Jan 2008
Location: NC
Posts: 21,202
Quote:
Originally Posted by Nodak View Post
Last week there was an extremely interesting NOVA program on PBS about the design and programming of Watson. If you can find a re-run of it it's well worth watching.
I don't watch Jeopardy, but I caught the NOVA episode and was so intrigued that I DVRd the Mon, Tue & Wed episodes. Watched the first last night and it was fascinating IMO. Watson got off to an incredible start and it looked "he" was going to embarass the two champs, but the first episode ended with all three in the money. I am looking forward to the next two episodes and I agree you will better appreciate the AI challenge if you see the NOVA episode. Arming Watson with lots of answers turned out to be the easy part, and some of the earlier versions were amusing .
__________________
No one agrees with other people's opinions; they merely agree with their own opinions -- expressed by somebody else. Sydney Tremayne
Retired Jun 2011 at age 57

Target AA: 50% equity funds / 45% bonds / 5% cash
Target WR: Approx 1.5% Approx 20% SI (secure income, SS only)
Midpack is online now   Reply With Quote
Old 02-15-2011, 11:18 AM   #15
Moderator
braumeister's Avatar
 
Join Date: Feb 2010
Location: Flyover country
Posts: 25,198
Quote:
Originally Posted by ERD50 View Post
I also wonder if the timing of the buzzer is 'fair'. A machine has some advantage here. I never really understood exactly how that works on Jeopardy. Do they have to wait for Alex to stop talking to buzz in? Is there a light or something? I know they get locked out for a few seconds or something if they buzz in too early. A human might make that reaction mistake, but a machine wouldn't.
I've taken the in-person audition twice, and believe me there is a real art to using the buzzer. It becomes activated as soon as Alex stops talking, but there is a person backstage listening to him, and the backstage person activates all the contestants' buzzers by pressing another buzzer.

If a contestant buzzes too soon, he is locked out for a brief time (they were a little unclear on exactly how long, but I'm pretty sure it's not over a second).

You definitely want to be the first, because you'll still have a split second to keep trying to come up with the answer, and that was Ken Jennings' real strength. He's a virtuoso on the buzzer.
__________________
I thought growing old would take longer.
braumeister is offline   Reply With Quote
Old 02-15-2011, 02:33 PM   #16
Recycles dryer sheets
Arnie's Avatar
 
Join Date: Dec 2010
Posts: 230
I am rooting for the humans. Can you spell skynet?
__________________

Arnie is offline   Reply With Quote
Old 02-15-2011, 03:06 PM   #17
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
clifp's Avatar
 
Join Date: Oct 2006
Posts: 7,733
Quote:
Originally Posted by ERD50 View Post
I laughed when I read about that. That's a mistake that only a bad player would make. Does it mean the programmers never considered their machine wouldn't buzz in first, or that it couldn't learn from others? Says something about the programmers.

I also wonder if the timing of the buzzer is 'fair'. A machine has some advantage here. I never really understood exactly how that works on Jeopardy. Do they have to wait for Alex to stop talking to buzz in? Is there a light or something? I know they get locked out for a few seconds or something if they buzz in too early. A human might make that reaction mistake, but a machine wouldn't.

-ERD50
There is a red light behind Alex that turns of when he has finished reading the question. At the point you can buzz in, if you do so before the light goes off you are penalized.

Evidently Watson is feed the text of the question (which is some what cheating since voice recognition is to me part of AI) but doesn't hear the other contestants answers.
clifp is offline   Reply With Quote
Old 02-15-2011, 03:08 PM   #18
Give me a museum and I'll fill it. (Picasso)
Give me a forum ...
clifp's Avatar
 
Join Date: Oct 2006
Posts: 7,733
Quote:
Originally Posted by Arnie View Post
I am rooting for the humans. Can you spell skynet?

I for one welcome our new silicon-based overlords.
You will be assimilated resistance is futile.
clifp is offline   Reply With Quote
Old 02-15-2011, 04:46 PM   #19
Thinks s/he gets paid by the post
 
Join Date: Jun 2010
Location: Palma de Mallorca
Posts: 1,419
Quote:
Originally Posted by grumpy View Post
I wonder if the programmers were embarrassed when one of the humans gave the wrong answer and then Watson buzzed in and gave the same wrong answer. An obvious oversight in their logic.
Quote:
Originally Posted by ERD50 View Post
I laughed when I read about that. That's a mistake that only a bad player would make. Does it mean the programmers never considered their machine wouldn't buzz in first, or that it couldn't learn from others? Says something about the programmers.
I find it slightly strange to see a comment "Says something about the programmers". Are you seriously imagining that they overlooked this? We're dealing here with some of the finest minds in artificial intelligence, with a very large budget. This is not just a bunch of college freshmen on a 100-line Pascal assignment.

There's a much simpler explanation for this error. Watson receives the questions by text input, not by listening to what others say. (I presume that the question is pre-typed, and someone presses "Enter" when Alex stops or starts speaking.) Listening to the voices of the contestants or host is not in the system's specification. And AFAIK the operators don't (or don't have time to) input what the other contestants are saying.
BigNick is offline   Reply With Quote
Old 02-15-2011, 06:43 PM   #20
Thinks s/he gets paid by the post
GregLee's Avatar
 
Join Date: Oct 2010
Location: Waimanalo, HI
Posts: 1,881
Quote:
Originally Posted by BigNick View Post
There's a much simpler explanation for this error. Watson receives the questions by text input, not by listening to what others say. (I presume that the question is pre-typed, and someone presses "Enter" when Alex stops or starts speaking.) Listening to the voices of the contestants or host is not in the system's specification. And AFAIK the operators don't (or don't have time to) input what the other contestants are saying.
Yes, and it's not exactly an "error". AI is often construed as an enterprise to understand human intelligence, so when the machine makes exactly the same error as the human, that's a triumph. It shows that the machine reasons like the human.
__________________
Greg (retired in 2010 at age 68, state pension)
GregLee is offline   Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Baby Boomers Retirements in Jeopardy Retire Soon FIRE and Money 7 12-30-2010 02:43 PM

» Quick Links

 
All times are GMT -6. The time now is 09:45 AM.
 
Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2024, vBulletin Solutions, Inc.