h a l f b a k e r yWe are investigating the problem and will update you shortly.
add, search, annotate, link, view, overview, recent, by name, random
news, help, about, links, report a problem
browse anonymously,
or get an account
and write.
register,
|
|
|
Please log in.
Before you can vote, you need to register.
Please log in or create an account.
|
Imagine having a ten-minute typed conversation with an invisible stranger, using only the letters 0 and 1.
you> 0
???> 0
you> 1
???> 1
you> 01
???> 10
you> 100101010001010
???> 0
etc.
After ten minutes of this, can you tell whether you've been talking to
a human or a computer?
One problem with the traditional Turing test's use of full language is the breadth and detail of human experience. Even if you're an AI genius, collecting all this experience on carpets, bodies, windows, breakfast, weather, --- will damn near kill you. There have been turing tests with a limited subject area, but just doing grammar is hard. This restriction levels the playing field as much as possible, leaving both parties to develop their own grammar on the spot.
Q: So, how would one go about detecting another human through such a narrow channel?
A: Well, if I knew that, I wouldn't have proposed this test.
In general terms, I think you'd make an intuitive judgment about the complexity of the person at the other end, and if it's complicated, stateful, but fits some emotional storyline, you'll call it human.
For example, two humans could have this brief conversation (first one player speaks, than the other):
0, 0; 10000, 0; 01000, 0; 00100, 00010; 1, 1; 0, 0; 0, 10000; 0, 11000; 11100, 0; 0, 01100; 00110, 1; 11111111111, 0.
What's going on? Well, first both players exchange zeroes just to see if the channel works. Then the first player initiates a game where they start a pattern - 10000, 01000, 00100, ... - and the second player tries to guess what comes next. (While they don't have a guess, they just send a 0.) They guess correctly and are rewarded with a score of 1. Then the winner starts a sequence themselves; the other player jumps the gun and guesses wrong, once, and then guesses again, this time correctly. The 11111111111 is a mock-exaggerated expression of triumph (or perhaps protest).
Small puzzle pieces in this that are human, or intelligent, or whatever you want to call it: Sending 0 when you have nothing to say; getting bored and initiating a game; understanding that the other player is "doing something" and that watching more of it may reveal its meaning; guessing what the game could be; guessing a pattern; understanding that 0 isn't an attempt at a solution, but a non-response; understanding a "reward"; "taking turns" in initiating another game with the same rules; "retrying" when a guess fails.
[Side notes: Yes, I agree this has some similarity to the problem of "talking to aliens".
Expanding the alphabet into anything other than natural language would probably do just as well; the 0/1 thing is kind of cute, but has also turned out to be a bit of a red herring.
I had no idea just how controversial the classic Turing test is in the AI crowd - seems a lot of people really hate it as a distraction.]
Wikipedia: Turing test
http://en.wikipedia.org/wiki/Turing_test
Background. [jutta, Nov 20 2008]
Loebner Prize
http://www.loebner..../loebner-prize.html
Real-life turing test settings. No winners so far, just best losers - and the attempts are still pretty sad. [jutta, Nov 20 2008]
Pioneer plaque
http://en.wikipedia...wiki/Pioneer_plaque
[hippo, Nov 21 2008]
[link]
|
| |
It seems that your solution to the test being weighted towards humans is to weight it towards the computer. From reading the description on the link a machine will only pass the test once it has an understanding of, and ability to use, the breadth of experience you mention. No-one said it was easy. |
|
| |
Interesting question, because it raises the question of
what exactly the Turing test is for. The Turing test is,
surely, just a pragmatic test for "intelligence", and passing
it isn't an end in itself. I suspect that reducing it to a
binary conversation would lead to one of two results.
Either (a) the topic of conversation becomes simple
arithmetic, patterns etc, which might not give enough
scope to make it worthwhile (all you prove is that
computers can recognise patterns and do arithmetic) or
(b) the two participants evolve a coding system to
represent more complex ideas (like windows and weather),
in which case the computer is back to square one. |
|
| |
I see the point of a simplified vocabulary, but I don't think
using binary achieves the desired result. Perhaps better
to use a restricted vocabulary and grammar which are
closer to human language? |
|
| |
Mony, my objective is not to "weigh" the test in one direction or the other, but to make visible - perhaps testable - elements of intelligence and communication that are usually obscured by others. |
|
| |
Max, you're probably right about the pattern detection - but I think building a program to detect patterns at a human level is a difficult and interesting task. (Alas, merely managing the conversational structure of this already requires social knowledge - but less.) |
|
| |
So, there's no code to the binary? Just flat out grunts and grumbling? What does it prove? I like the restriced voc. and grammar approach. |
|
| |
I am not certain that judging a computer's "ability to think" should necessarily imply "ability to think like a human". |
|
| |
For example, suppose an alien vessel suddenly appears in earth orbit. It appears willing to communicate, although there are no shared bases for communication. You'd like to know whether there's something alive in there, or if it's just a piece of equipment. There'd be no reason for an alien computer to think like a human; for that matter, a putative alien crewmember wouldn't, either. |
|
| |
As [MaxB] says, Turing used a pragmatic approach, but his assumptions (language & background) limit (eliminate?) the test's usefulness. Is it even possible to create a test that isn't based on comparison-to-human, but doesn't boil down to a teraflops benchmark? |
|
| |
<edit> looks like [boysparks] was thinking the same as me, but typing faster. (So whatever I am, [boysparks] is, too. Right?) |
|
| |
With such a binary test, what would be the hypotetical result if the respondent were (a) a trained chimpanzee, (b) a dolphin or killer whale ? (given that the mechanical problems of providing suitable immersible displays/sounders and switches could be solved). |
|
| |
//building a program to detect patterns at a human level
is a difficult and interesting task.// Indeed it is, and a
human might well be better than (or at least different
from) a computer. |
|
| |
However, the problem then bites you in the arse: the
purpose of the Turing test is to let a *human* decide
whether the subject is a human or a computer. This
means that not only does the computer being tested have
to emulate "human" responses, but the human tester has
to be able to distinguish "human" from "computerish"
responses. I think that even if the computer was
generating "computerish" (non-humanish) binary
responses, the human observer might not be able to
distinguish them. In other words, for this to work, the
language really has to be equally natural to the computer,
the human subject (the computer's rival), and the human
observer. |
|
| |
hmm. Wouldn't this be fundamentally the same challenge as an ET conversation, i.e. starting without common symbols? |
|
| |
Let's assume tomorrow we get a signal from Fomalhaut, saying 0101. We bounce it back. They send 0000. We bounce it back. They go 1111, we bounce it back. |
|
| |
We've now convinced ourselves there is intelligent life at Fomalhaut, but that intelligent life could be the entity that created the program, and not the program itself, and not a way to distinguish the two. |
|
| |
And yet clearly, what I described is too simple to match a true Turing test. |
|
| |
How about this idea: The humans on the other side of the Turing test should be nonenglish speakers. Probably some familiarity with a keyboard would be good. This eliminates a lot of the language / background difficulty. |
|
| |
Err, what happened to our annotation about the chimps and the dolphins ? |
|
| |
We refuse to answer on the grounds that we may incriminate ourselves. |
|
| |
"halfbakery is brought to you today by the letters 0 and 1" |
|
| |
//you> 100101010001010//
If I can cheat, I can tell. The idea is to converse with the suspect, and decide if it's human or not. But I might try typing an extremely long string of numbers. If it hasn't been programmed to handle a particular overflow, replies may be formatted in a weird way. And if I crash it, it's likely a computer. |
|
| |
But limited to binary, we may be back to the dolphins and chimps. |
|
| |
Thinking more about Turing tests, another novel Turing test would be interaction with an avatar in a videogame wike Warcraft but with no speech allowed. |
|
| |
This test would be mostly pointless. The human could not tell the difference between a human, computer, or random set of 0's ans 1's. Furthermore, people have a tendancy to see patterns where none exist. If you make it an hour this might function as a kind of intelligence test for both sides, (can they create a new language?) but I don't think thats good enough. |
|
| |
How about a compromise? How about a set of images or symbols? Something that conveys more meaning than binary, but might be less subtle than English. |
|
| |
Two people who did not speak the same language or use the same alphabet would be confined to the symbols they could generate with the keyboard. |
|
| |
//How about a set of images or symbols//
Or a drawing program, to cooperatively create a moderately complex image. |
|
| |
But the idea is all about simplification, and what could be simpler than working with just two symbols (logic high and low) -- particularly since that's what a computer "brain" uses! The question remains whether this is too simple to be useful, and the poster doesn't explain how we might arrive at a decision after our base-2 conversation. |
|
| |
// the poster doesn't explain how we might arrive at a decision after our base-2 conversation. |
|
| |
[lengthy explanation moved into main post, as per daseva's recommendation below.] |
|
| |
The Turing test is a bit artificial in the sense that intelligence doesn't require an ability to talk about the weather any more than an ability to talk about the weather denotes intelligence, but I like the idea of formalising the communication a bit to reduce the dependence of the test on being able to parse English, with all its reliance on idiom and irregular verbs. Possibly the most interesting attempt to communicate intelligence without the use of language has been the plaque attached to the Pioneer space probe (link) which is chiefly symbolic and diagrammatic - perhaps your revised test should allow simple geometric shapes too? |
|
| |
Aaah, but, an obsession with the weather is characteristic of the English. |
|
| |
You want to simplify the 'bakery further? |
|
| |
The game example should be in the post. It clarifies and qualifies. + |
|
| |
The zeros and ones mean things to a processor. They're numbers, characters, and machine code. By trying to elimitate the language barrier, the idea wades deeply into computer language. And (as [Voice] mentions) when the computer responds, whether it's simply adding numbers, or following sequences, there will be many human-like responses. |
|
| |
But in the binary example, a computer might be better than a human at detecting humans. |
|
| |
0001 0010 0011 0100 0101 0110 0111 1000 1001 1010 1011 1100 1101 1110 1111 |
|
| |
This is more of a reverse Turing test, where a computer decides if it's talking to a human or another computer. Though it's not much of a test in any case, as it will be obvious in milliseconds which it is. One bit per second? An organic, obviously. |
|
| |
//One bit per second? An organic//
...or Windows Vista. |
|
| |
What about refrigerator magnets. Limited common
vocabulary with forced content. I think it's really worth a
try. You could set up a website with a "refrigerator"
playground and let the AI loose to play. Anyone have an
opinion about how that would tip. I could see it going
either way, but i suspect that an adaptive or evolutionary
AI would eventaully master the situation. |
|
| |