Bet 1

Duration 27 years (02002-02029)

“By 2029 no computer - or "machine intelligence" - will have passed the Turing Test.” detailed terms »

Predictor
Mitchell Kapor

Challenger
Ray Kurzweil

Stakes $20,000
will go to The Electronic Frontier Foundation if Kapor wins,
or The Kurzweil Foundation if Kurzweil wins.

Vote

When you vote, your name, today's date and who you are siding with will be added to this bet's permanent record. Please sign in to vote.

side with predictor
side with challenger
 

283 people (45%)

341 people (55%)

details »

Discuss & Share

Add your voice to a conversation with the bettors:

Join the discussion »

Bookmark this bet, and share it with friends:

Kapor’s Argument

The essence of the Turing Test revolves around whether a computer can successfully impersonate a human. The test is to be put into practice under a set of detailed conditions which rely on human judges being connected with test subjects (a computer and a person) solely via an instant messaging system or its equivalent. That is, the only information which will pass between the parties is text.

To pass the test, a computer would have to be capable of communicating via this medium at least as competently as a person. There is no restriction on the subject matter; anything within the scope of human experience in reality or imagination is fair game. This is a very broad canvas encompassing all of the possibilities of discussion about art, science, personal history, and social relationships. Exploring linkages between the realms is also fair game, allowing for unusual but illustrative analogies and metaphors. It is such a broad canvas, in my view, that it is impossible to foresee when, or even if, a machine intelligence will be able to paint a picture which can fool a human judge.

While it is possible to imagine a machine obtaining a perfect score on the SAT or winning Jeopardy--since these rely on retained facts and the ability to recall them--it seems far less possible that a machine can weave things together in new ways or to have true imagination in a way that matches everything people can do, especially if we have a full appreciation of the creativity people are capable of. This is often overlooked by those computer scientists who correctly point out that it is not impossible for computers to demonstrate creativity. Not impossible, yes. Likely enough to warrant belief in a computer can pass the Turing Test? In my opinion, no. Computers look relatively smarter in theory when those making the estimate judge people to be dumber and more limited than they are.

As humans:

When I contemplate human beings in this way, it becomes extremely difficult even to imagine what it would mean for a computer to perform a successful impersonation, much less to believe that its achievement is within our lifespan. Computers don't have anything resembling a human body, sense organs, feelings, or awareness after all. Without these, it cannot have human experiences, especially of the ones which reflect our fullest nature, as above. Each of knows what it is like to be in a physical environment; we know what things look, sound, smell, taste, and feel like. Such experiences form the basis of agency, memory and identity. We can and do speak of all this in a multitude of meaningful ways to each other. Without human experiences, a computer cannot fool a smart judge bent on exposing it by probing its ability to communicate about the quintessentially human.

Additionally, part of the burden of proof for supporters of intelligent machines is to develop an adequate account of how a computer would acquire the knowledge it would be required to have to pass the test. Ray Kurzweil's approach relies on an automated process of knowledge acquisition via input of scanned books and other printed matter. However, I assert that the fundamental mode of learning of human beings is experiential. Book learning is a layer on top of that. Most knowledge, especially that having to do with physical, perceptual, and emotional experience is not explicit, never written down. It is tacit. We cannot say all we know in words or how we know it. But if human knowledge, especially knowledge about human experience, is largely tacit, i.e., never directly and explicitly expressed, it will not be found in books, and the Kurzweil approach to knowledge acquisition will fail. It might be possible to produce a kind of machine as idiot savant by scanning a library, but a judge would not have any more trouble distinguishing one from an ordinary human as she would with distinguishing a human idiot savant from a person not similarly afflicted. It is not in what the computer knows but what the computer does not know and cannot know wherein the problem resides.

Given these considerations, a skeptic about machine intelligence could fairly ask how and why the Turing Test was transformed from its origins as a provocative thought experiment by Alan Turing to a challenge seriously sought. The answer is to be found in the origins of the branch of computer science its practitioners have called Artificial Intelligence (AI).

In the 1950's a series of computer programs were written which first demonstrated the ability of the computer to carry out symbolic manipulations in software in ways which the performance (not the actual process) began to approach human level on tasks such as playing checkers and proving theorems in geometry. These results fueled the dreams of computer scientists to create machines which were endowed with intelligence. Those dreams, however, repeatedly failed to be realized. Early successes were not followed with more success, but with failure. A pattern of over-optimism was first seen which has persisted to this day. Let me be clear I am not referring to most computer scientists in the field of AI, but to those who take an extreme position.

For instance, there were claims in the 1980's that expert systems would come be of great significance, in which computer would perform as well or better than human experts in a wide variety of disciplines. This belief triggered a boom in investment in AI-based startups in the 1980's, followed by a bust when audacious predictions of success failed to be met and the companies premised on those claims also failed.

In practice, expert systems proved to be fragile creatures, capable at best of dealing with facts in narrow, rigid domains, in ways which were very much unlike the adaptable, protean nature of intelligence demonstrated by human experts. As we call them today, knowledge-based systems do play useful roles in a variety of ways, but there is broad consensus that the knowledge of these knowledge-based systems is a very small and non-generalizable part of overall human intelligence.

Ray Kurzweil's arguments seek to go further. To get a computer to perform like a person with a brain, a computer should be built to work the way a brain works. This is an interesting, intellectually challenging idea.

He assumes this can be accomplished by using as yet undeveloped nano-scale technology (or not -- he seems to want to have it both ways) to scan the brain in order to reverse engineer what he refers to as the massively parallel digital controlled analog algorithms that characterize information processing in each region. These then are presumably what control the self-organizing hierarchy of networks he thinks constitute the working mechanism of the brain itself. Perhaps.

But we don't really know whether "carrying out algorithms operating on these networks" is really sufficient to characterize what we do when we are conscious. That's an assumption, not a result. The brain's actual architecture and the intimacy of its interaction, for instance, with the endocrine system, which controls the flow of hormones, and so regulates emotion (which in turn has an extremely important role in regulating cognition) is still virtually unknown. In other words, we really don't know whether in the end, it's all about the bits and just the bits. Therefore Kurzweil doesn't know, but can only assume, that the information processing he wants to rely on in his artificial intelligence is a sufficiently accurate and comprehensive building block to characterize human mental activity.

The metaphor of brain-as-computer is tempting and to a limited degree fruitful, but we should not rely on its distant extrapolation. In the past, scientists have sought to employ metaphors of their age to characterize mysteries of human functioning, e.g., the heart as pump, the brain as telephone switchboard (you could look this up). Properly used, metaphors are a step on the way to development of scientific theory. Stretched beyond their bounds, the metaphors lose utility and have to be abandoned by science if it is not to be led astray. My prediction is that contemporary metaphors of brain-as-computer and mental activity-as-information processing will in time also be superceded and will not prove to be a basis on which to build human-level intelligent machines (if indeed any such basis ever exists).

Ray Kurzweil is to be congratulated on his vision and passion, regardless of who wins or loses the bet. In the end, I think Ray is smarter and more capable than any machine is going to be, as his vision and passion reflect qualities of the human condition no machine is going to successfully emulate over the term of the bet. I look forward to comparing notes with him in 2029.

Kurzweil’s Argument

The Significance of the Turing Test.
The implicit, and in my view brilliant, insight in Turing's eponymous test is the ability of written human language to represent human-level thinking. The basis of the Turing test is that if the human Turing test judge is competent, then an entity requires human-level intelligence in order to pass the test. The human judge is free to probe each candidate with regard to their understanding of basic human knowledge, current events, aspects of the candidate's personal history and experiences, as well as their subjective experiences, all expressed through written language. As humans jump from one concept and one domain to the next, it is possible to quickly touch upon all human knowledge, on all aspects of human, well, humanness.

To the extent that the "AI" chooses to reveal its "history" during the interview with the Turing Test judge (note that none of the contestants are required to reveal their histories), the AI will need to use a fictional human history because "it" will not be in a position to be honest about its origins as a machine intelligence and pass the test. (By the way, I put the word "it" in quotes because it is my view that once an AI does indeed pass the Turing Test, we may very well consider "it" to be a "he" or a "she.") This makes the task of the machines somewhat more difficult than that of the human foils because the humans can use their own history. As fiction writers will attest, presenting a totally convincing human history that is credible and tracks coherently is a challenging task that most humans are unable to accomplish successfully. However, some humans are capable of doing this, and it will be a necessary task for a machine to pass the Turing test.

There are many contemporary examples of computers passing "narrow" forms of the Turing test, that is, demonstrating human-level intelligence in specific domains. For example, Gary Kasparov, clearly a qualified judge of human chess intelligence, declared that he found Deep Blue's playing skill to be indistinguishable from that of a human chess master during the famous tournament in which he was defeated by Deep Blue. Computers are now displaying human-level intelligence in a growing array of domains, including medical diagnosis, financial investment decisions, the design of products such as jet engines, and a myriad of other tasks that previously required humans to accomplish. We can say that such "narrow AI" is the threshold that the field of AI has currently achieved. However, the subtle and supple skills required to pass the broad Turing test as originally described by Turing is far more difficult than any narrow Turing Test. In my view, there is no set of tricks or simpler algorithms (i.e., methods simpler than those underlying human level intelligence) that would enable a machine to pass a properly designed Turing test without actually possessing intelligence at a fully human level.

There has been a great deal of philosophical discussion and speculation concerning the issue of consciousness, and whether or not we should consider a machine that passed the Turing test to be conscious. Clearly, the Turing test is not an explicit test for consciousness. Rather, it is a test of human-level performance. My own view is that inherently there is no objective test for subjective experience (i.e., consciousness) that does not have philosophical assumptions built into it. The reason for this has to do with the difference between the concepts of objective and subjective experience. However, it is also my view that once nonbiological intelligence does achieve a fully human level of intelligence, such that it can pass the Turing test, humans will treat such entities as if they were conscious. After all, they (the machines) will get mad at us if we don't. However, this is a political prediction rather than a philosophical position.

It is also important to note that once a computer does achieve a human level of intelligence, it will necessarily soar past it. Electronic circuits are already at least 10 million times faster than the electrochemical information processing in our interneuronal connections. Machines can share knowledge instantly, whereas we biological humans do not have quick downloading ports on our neurotransmitter concentration levels, interneuronal connection patterns, nor any other biological bases of our memory and skill. Language-capable machines will be able to access vast and accurate knowledge bases, including reading and mastering all the literature and sources of information available to our human-machine civilization. Thus "Turing Test level" machines will be able to combine human level intelligence with the powerful ways in which machines already excel. In addition, machines will continue to grow exponentially in their capacity and knowledge. It will be a formidable combination.

Why I Think I Will Win. In considering the question of when machine (i.e., nonbiological) intelligence will match the subtle and supple powers of human biological intelligence, we need to consider two interrelated but distinct questions: when will machines have the hardware capacity to match human information processing, and when will our technology have mastered the methods, i.e., the software of human intelligence. Without the latter, we would end up with extremely fast calculators, and would not achieve the endearing qualities that characterize human discernment (nor the deep knowledge and command of language necessary to pass a full Turing test!).

Both the hardware and software sides of this question are deeply influenced by the exponential nature of information-based technologies. The exponential growth that we see manifest in "Moore's Law" is far more pervasive than commonly understood. Our first observation is that the shrinking of transistors on an integrated circuit, which is the principle of Moore's Law, was not the first but the fifth paradigm to provide exponential growth to computing (after electromechanical calculators, relay-based computers, vacuum tube-based computing, and discrete transistors). Each time one approach begins to run out of steam, research efforts intensify to find the next source of renewed exponential growth (e.g., vacuum tubes were made smaller until it was no longer feasible to maintain a vacuum, which led to transistors). Thus the power and price-performance of technologies, particularly information-based technologies, grow as a cascade of S-curves: exponential growth leading to an asymptote, leading to paradigm shift (i.e., innovation), and another S-curve. Moreover, the underlying theory of the exponential growth of information-based technologies, which I call the law of accelerating returns, as well as a detailed examination of the underlying data, show that there is a second level of exponential growth, i.e., the rate of exponential growth is itself growing exponentially.

Second, this phenomenon of ongoing exponential growth through a cascade of S-curves is far broader than computation. We see the same double exponential growth in a wide range of technologies, including communication technologies (wired and wireless), biological technologies (e.g., DNA base-pair sequencing), miniaturization, and of particular importance to the software of intelligence, brain reverse engineering (e.g., brain scanning, neuronal and brain region modeling).

Within the next approximately fifteen years, the current computational paradigm of Moore's Law will come to an end because by that time the key transistor features will only be a few atoms in width. However, there are already at least two dozen projects devoted to the next (i.e., the sixth) paradigm, which is to compute in three-dimensions. Integrated circuits are dense but flat. We live in a three-dimensional world, our brains are organized in three dimensions, and we will soon be computing in three dimensions. The feasibility of three-dimensional computing has already been demonstrated in several landmark projects, including the particularly powerful approach of nanotube-based electronics. However, for those who are (irrationally) skeptical of the potential for three-dimensional computing, it should be pointed out that achieving even a conservatively high estimate of the information processing capacity of the human brain (i.e., one hundred billion neurons times a thousand connections per neuron times 200 digitally controlled analog "transactions" per second, or about 20 million billion operations per second) will be achieved by conventional silicon circuits prior to 2020.

It is correct to point out that achieving the "software" of human intelligence is the more salient, and more difficult, challenge. On multiple levels, we are being guided in this effort by a grand project to reverse engineer (i.e., understand the principles of operation of) the human brain itself. Just as the human genome project accelerated (with the bulk of the genome being sequenced in the last year of the project), the effort to reverse engineer the human brain is also growing exponentially, and is further along than most people realize. We already have highly detailed mathematical models of several dozen of the several hundred types of neurons found in the brain. The resolution, bandwidth, and price-performance of human brain scanning is also growing exponentially. By combining the neuron modeling and interconnection data obtained from scanning, scientists have already reverse engineered two dozen of the several hundred regions of the brain. Implementations of these reverse engineered models using contemporary computation matches the performance of the biological regions that were recreated in significant detail. Already, we are in a early stage of being able to replace small regions of the brain that have been damaged from disease or disability using neural implants (e.g., ventral posterior nucleus, subthalmic nucleus, and ventral lateral thalamus neural implants to counteract Parkinson's Disease and tremors from other neurological disorders, cochlear implants, emerging retinal implants, and others).

If we combine the exponential trends in computation, communications, and miniaturization, it is a conservative expectation that we will within 20 to 25 years be able to send tiny scanners the size of blood cells into the brain through the capillaries to observe interneuronal connection data and even neurotransmitter levels from up close. Even without such capillary-based scanning, the contemporary experience of the brain reverse engineering scientists, (e.g., Lloyd Watts, who has modeled over a dozen regions of the human auditory system), is that the connections in a particular region follow distinct patterns, and that it is not necessary to see every connection in order to understand the massively parallel, digital controlled analog algorithms that characterize information processing in each region. The work of Watts and others has demonstrated another important insight, that once the methods in a brain region are understood and implemented using contemporary technology, the computational requirements for the machine implementation requires on the order of a thousand times less computation than the theoretical potential of the biological neurons being simulated.

A careful analysis of the requisite trends shows that we will understand the principles of operation of the human brain and be in a position to recreate its powers in synthetic substrates well within thirty years. The brain is self-organizing, which means that it is created with relatively little innate knowledge. Most of its complexity comes from its own interaction with a complex world. Thus it will be necessary to provide an artificial intelligence with an education just as we do with a natural intelligence. But here the powers of machine intelligence can be brought to bear. Once we are able to master a process in a machine, it can perform its operations at a much faster speed than biological systems. As I mentioned, contemporary electronics is already more than ten million times faster than the human nervous system's electrochemical information processing. Once an AI masters human basic language skills, it will be in a position to expand its language skills and general knowledge by rapidly reading all human literature and by absorbing the knowledge contained on millions of web sites. Also of great significance will be the ability of machines to share their knowledge instantly.

One challenge to our ability to master the apparent complexity of human intelligence in a machine is whether we are capable of building a system of this complexity without the brittleness that often characterizes very complex engineering systems. This a valid concern, but the answer lies in emulating the ways of nature. The initial design of the human brain is of a complexity that we can already manage. The human brain is characterized by a genome with only 23 million bytes of useful information (that's what left of the 800 million byte genome when you eliminate all of the redundancies, e.g., the sequence called "ALU" which is repeated hundreds of thousands of times). 23 million bytes is smaller than Microsoft WORD. How is it, then, that the human brain with its 100 trillion connections can result from a genome that is so small? The interconnection data alone is a million times greater than the information in the genome. The answer is that the genome specifies a set of processes, each of which utilizes chaotic methods (i.e., initial randomness, then self-organization) to increase the amount of information represented. It is known, for example, that the wiring of the interconnections follows a plan that includes a great deal of randomness. As the individual person encounters her environment, the connections and the neurotransmitter level pattern self-organize to better represent the world, but the initial design is specified by a program that is not extreme in its complexity.

Thus we will not program human intelligence link by link as in some massive expert system. Nor is it the case that we will simply set up a single genetic (i.e., evolutionary) algorithm and have intelligence at human levels automatically evolve itself. Rather we will set up an intricate hierarchy of self-organizing systems, based largely on the reverse engineering of the human brain, and then provide for its education. However, this learning process can proceed hundreds if not thousands of times faster than the comparable process for humans.

Another challenge is that the human brain must incorporate some other kind of "stuff" that is inherently impossible to recreate in a machine. Penrose imagines that the intricate tubules in human neurons are capable of quantum based processes, although there is no evidence for this. I would point out that even if the tubules do exhibit quantum effects, there is nothing barring us from applying these same quantum effects in our machines. After all, we routinely use quantum methods in our machines today. The transistor, for example, is based on quantum tunneling. The human brain is made of the same small list of proteins that all biological systems are comprised of. We are rapidly recreating the powers of biological substances and systems, including neurological systems, so there is little basis to expect that the brain relies on some nonengineerable essence for its capabilities. In some theories, this special "stuff" is associated with the issue of consciousness, e.g., the idea of a human soul associated with each person. Although one may take this philosophical position, the effect is to separate consciousness from the performance of the human brain. Thus the absence of such a soul may in theory have a bearing on the issue of consciousness, but would not prevent a nonbiological entity from the performance abilities necessary to pass the Turing test.

Another challenge is that an AI must have a human or human-like body in order to display human-like responses. I agree that a body is important to provide a situated means to interact with the world. The requisite technologies to provide simulated or virtual bodies are also rapidly advancing. Indeed, we already have emerging replacements or augmentations for virtually every system in our body. Moreover, humans will be spending a great deal of time in full immersion virtual reality environments incorporating all of the senses by 2029, so a virtual body will do just as well. Fundamentally, emulating our bodies in real or virtual reality is a less complex task than emulating our brains.

Finally, we have the challenge of emotion, the idea that although machines may very well be able to master the more analytical cognitive abilities of humans, they inherently will never be able to master the decidedly illogical and much harder to characterize attributes of human emotion. A slightly broader way of characterizing this challenge is to pose it in terms of "qualia," which refers essentially to the full range of subjective experiences. Keep in mind that the Turing test is assessing convincing reactions to emotions and to qualia. The apparent difficulty of responding appropriately to emotion and other qualia appears to be at least a significant part of Mitchell Kapor's hesitation to accept the idea of a Turing-capable machine. It is my view that understanding and responding appropriately to human emotion is indeed the most complex thing that we do (with other types of qualia being if anything simpler to respond to). It is the cutting edge of human intelligence, and is precisely the heart of the Turing challenge. Although human emotional intelligence is complex, it nonetheless remains a capability of the human brain, with our endocrine system adding only a small measure of additional complexity (and operating at a relatively low bandwidth). All of my observations above pertain to the issue of emotion, because that is the heart of what we are reverse engineering. Thus, we can say that a side benefit of creating Turing-capable machines will be new levels of insight into ourselves.

Detailed Terms

A Wager on the Turing Test: The Rules
As prepared by Ray Kurzweil in consultation with Mitchell Kapor

Background on the "Long Now Turing Test Wager."
Ray Kurzweil maintains that a computer (i.e., a machine intelligence) will pass the Turing test by 2029. Mitchell Kapor believes this will not happen.

This wager is intended to be the inaugural long term bet to be administered by the Long Now Foundation. The proceeds of the wager are to be donated to a charitable organization designated by the winner.

This document provides a brief description of the Turing Test and a set of high level rules for administering the wager. These rules contemplate setting up a "Turing Test Committee" which will create the detailed rules and procedures to implement the resolution of the wager. A primary objective of the Turing Test Committee will be to set up rules and procedures that avoid and deter cheating.

Brief Description of the Turing test. In a 1950 paper ("Computing Machinery and Intelligence," Mind 59 (1950): 433- 460, reprinted in E. Feigenbaum and J. Feldman, eds., Computers and Thought, New York: McGraw-Hill, 1963), Alan Turing describes his concept of the Turing Test, in which one or more human judges interview computers and human foils using terminals (so that the judges won't be prejudiced against the computers for lacking a human appearance). The nature of the dialogue between the human judges and the candidates (i.e., the computers and the human foils) is similar to an online chat using instant messaging. The computers as well as the human foils try to convince the human judges of their humanness. If the human judges are unable to reliably unmask the computers (as imposter humans) then the computer is considered to have demonstrated human-level intelligence .

Turing was very specifically nonspecific about many aspects of how to administer the test. He did not specify many key details, such as the duration of the interrogation and the sophistication of the human judge and foils. The purpose of the rules described below is to provide a set of procedures for administering the test some decades hence.

The Procedure for the Turing Test Wager: The Turing Test General Rules
These Turing Test General Rules may be modified by agreement of Ray Kurzweil and Mitchell Kapor, or, if either Ray Kurzweil and / or Mitchell Kapor is not available, then by the Turing Test Committee (described below). However, any such change to these Turing Test General Rules shall only be made if (i) these rules are determined to have an inconsistency, or (ii) these rules are determined to be inconsistent with Alan Turing's intent of determining human-level intelligence in a machine, or (iii) these rules are determined to be unfair, or (iv) these rules are determined to be infeasible to implement.

I. Definitions.
A Human is a biological human person as that term is understood in the year 2001 whose intelligence has not been enhanced through the use of machine (i.e., nonbiological) intelligence, whether used externally (e.g., the use of an external computer) or internally (e.g., neural implants). A Human may not be genetically enhanced (through the use of genetic engineering) beyond the level of human beings in the year 2001.

A Computer is any form of nonbiological intelligence (hardware and software) and may include any form of technology, but may not include a biological Human (enhanced or otherwise) nor biological neurons (however, nonbiological emulations of biological neurons are allowed).

The Turing Test Committee will consist of three Humans, to be selected as described below.

The Turing Test Judges will be three Humans selected by the Turing Test Committee.

The Turing Test Human Foils will be three Humans selected by the Turing Test Committee.

The Turing Test Participants will be the three Turing Test Human Foils and one Computer.

II. The Procedure
The Turing Test Committee will be appointed as follows.

One member will be Ray Kurzweil or his designee, or, if not available, a person appointed by the Long Now Foundation. In the event that the Long Now Foundation appoints this person, it shall use its best efforts to appoint a Human person that best represents the views of Ray Kurzweil (as expressed in his bet argument.)

A second member will be Mitchell Kapor or his designee, or, if not available, a person appointed by the Long Now Foundation. In the event that the Long Now Foundation appoints this person, it shall use its best efforts to appoint a Human person that best represents the views of Mitchell Kapor (as expressed in his bet argument.)

A third member will be appointed by the above two members, or if the above two members are unable to agree, then by the Long Now Foundation, who in its judgment, is qualified to represent a "middle ground" position.

Ray Kurzweil, or his designee, or another member of the Turing Test Committee, or the Long Now Foundation may, from time to time call for a Turing Test Session to be conducted and will select or provide one Computer for this purpose. For those Turing Test Sessions called for by Ray Kurzweil or his designee or another member of the Turing Test committee (other than the final one in 2029), the person calling for the Turing Test Session to be conducted must provide (or raise) the funds necessary for the Turing Test Session to be conducted. In any event, the Long Now Foundation is not obligated to conduct more than two such Turing Test Sessions prior to the final one (in 2029) if it determines that conducting such additional Turing Test Sessions would be an excessive administrative burden.

The Turing Test Committee will provide the detailed rules and procedures to implement each such Turing Test Session using its best efforts to reflect the rules and procedures described in this document. The primary goal of the Turing Test Committee will be to devise rules and procedures which avoid and deter cheating to the maximum extent possible. These detailed rules and procedures will include (i) specifications of the equipment to be used, (ii) detailed procedures to be followed, (iii) specific instructions to be given to all participants including the Turing Test Judges, the Turing Test Human Foils and the Computer, (iv) verification procedures to assure the integrity of the proceedings, and (v) any other details needed to implement the Turing Test Session. Beyond the Turing Test General Rules described in this document, the Turing Test Committee will be guided to the best of its ability by the original description of the Turing Test by Alan Turing in his 1950 paper. The Turing Test Committee will also determine procedures to resolve any deadlocks that may occur in its own deliberations.

Each Turing Test Session will consist of at least three Turing Test Trials.

For each such Turing Test Trial, a set of Turing Test Interviews will take place, followed by voting by the Turing Test Judges as described below.

Using its best judgment, the Turing Test Committee will appoint three Humans to be the Turing Test Judges.

Using its best judgment, the Turing Test Committee will appoint three Humans to be the Turing Test Human Foils. The Turing Test Human Foils should not be known (either personally or by reputation) to the Turing Test Judges.

During the Turing Test Interviews (for each Turing Test Trial), each of the three Turing Test Judges will conduct online interviews of each of the four Turing Test Candidates (i.e., the Computer and the three Turing Test Human Foils) for two hours each for a total of eight hours of interviews conducted by each of the three Turing Test Judges (for a total of 24 hours of interviews).

The Turing Test Interviews will consist of online text messages sent back and forth as in a online "instant messaging" chat, as that concept is understood in the year 2001.

The Human Foils are instructed to try to respond in as human a way as possible during the Turing Test Interviews.

The Computer is also intended to respond in as human a way as possible during the Turing Test Interviews.

Neither the Turing Test Human Foils nor the Computer are required to tell the truth about their histories or other matters. All of the candidates are allowed to respond with fictional histories.

At the end of the interviews, each of the three Turing Test Judges will indicate his or her verdict with regard to each of the four Turing Test Candidates indicating whether or not said candidate is human or machine. The Computer will be deemed to have passed the "Turing Test Human Determination Test" if the Computer has fooled two or more of the three Human Judges into thinking that it is a human.

In addition, each of the three Turing Test Judges will rank the four Candidates with a rank from 1 (least human) to 4 (most human). The computer will be deemed to have passed the "Turing Test Rank Order Test" if the median rank of the Computer is equal to or greater than the median rank of two or more of the three Turing Test Human Foils.

The Computer will be deemed to have passed the Turing Test if it passes both the Turing Test Human Determination Test and the Turing Test Rank Order Test.

If a Computer passes the Turing Test, as described above, prior to the end of the year 2029, then Ray Kurzweil wins the wager. Otherwise Mitchell Kapor wins the wager.

Join the Discussion

Bet 1

A computer - or "machine intelligence" - will pass the Turing Test by 2029.

Link to bet page.

http://www.longbets.org/bet/1

gray matter metaphor

brain as fire. as controlled, contained fire. sun piece held in soft costume

Turing test is flawed measure of intelligence

The Turing test is a flawed measure of intelligence.

In essence it is based on the principle that all things that be confused with one another are identical, rather
than the principle that all things that have the same
nature are identical.

In a court of law physical evidence is required as a
basis for claims because it establishes a natural
foundation for assertions. Otherwise there is no
substantial basis.

In other words, the Turing test is based on
equivocation to the exclusion of identification.

To beat the Turing test, all that is required is that
a machine be built which is capable of confusing
humans.

The pattern to follow then would be the methods
of lawyers who have no basis for their assertions.
Or building a type of machine that works like a
confused human. Automated sophistry would
paint a compelling LIKENESS to a human mind
without having to establish an IS-NESS to a human mind.

Will we be able to confuse human judges by the due
date of this bet? Sure, if unsubtantiated testimony is
admissable in our test.

The Test Itself

I think the abstract test proposed by Mr. Turing in 1950 is quite clever, but the version of it outlined in this bet may not be able to determine whether a computer has demonstrated human-level intelligence. Although the Long Bets Foundation will eventually be forced to do so, I think administering this Turing Test will be logistically impossible, and, given the criteria negotiated by the bettors, the final results could be debatable.

The Turing Test used to settle this bet will basically be a conversation. A judge will interact with a machine intelligence and human foils via text messages in an effort to 1) determine which is the computer, and 2) rate both the computer and the foils on their level of "humanness". To pass the second half of the Test, the computer's median rank must equal or excel the median rank of two human foils. Part of my problem with these guidelines is that they both seek to determine "humanity" instead of intelligence, which may be an unfair burden for the computer to overcome.

First, administering the Turing Test conversation in a fair manner may be impossible. For example, it is entirely conceivable that the computer will be too intelligent for its own good. Depending on how the give-and-take between the judge and the respondent is regulated, something as simple as a grammatical error or misused punctuation may tip the hand of a human foil, as the computer will presumably be programmed with a flawless language program. (Some would argue that language software cannot advance that far in the given time period. That might be so, but my grammar-check already exceeds the abilities of my human editors.) If the language software is perfect, then the computer will probably be the only respondent that does not commit any typos or misspellings within the two-hour test period.

Another potential problem is that the tested computer might unwittingly respond to an inquiry faster than is humanly possible. The human foils will either type their responses or dictate them into text format. Both methods require time to complete. Our 2029 computer might develop a response and return it so quickly as to instantly disqualify itself.

(Of course, a computer which develops human-level intelligence may notice these problems and deftly correct them by playing down to its competition. It would be a remarkable irony if the Turing computer mimics our intellect by acting less intelligent than it actually is.)

The problem with this, and with the second half of the judging, is that the computer must be intelligent enough to actively deceive us about its (lack of) humanity. The human foils of the Test have a tremendous advantage in “acting human”, seeing as they have a lifetime’s worth of practice. Ray Kurzweil discusses this in his argument but is confident that the computer will be up to the task. Personally, I think it is unfair to require the computer to generate its own history and experiences while the human foils may freely draw upon their own. It would be far more interesting if ALL the respondents were required to produce new histories of their own or to respond falsely to every inquiry during the test. (I haven’t a clue how this would be judged other than to trust in the good-faith of the foils.) If you are seeking a test that compares intelligence levels, why not make the same demands upon all the intellects being tested? Seeing as the computer must continually deceive the judges with its responses, it seems reasonable to require the same of the human foils. That would at least offer a direct comparison of the minds involved in the test.

Which leads me to my final point. At the conclusion of the test, the three judges will assign a rank of 1 (least human) to 4 (most human) to each of the respondents. If it is not obvious, allow me to point out that the criterion to be used is “humanness” and not “intelligence”. This scoring system will almost certainly result in the human foils earning different scores, which creates the awkward situation that one of the foils is more “human” than another. Now, if the humans aren’t equally human then I’m not sure what use it is to apply the same judgement to the computer. In my opinion, the computer will have passed the Test even if it scores two 2s and a 1, even though this would not be a passing performance in the eyes of the bet. In fact, I would want to eliminate the criteria of “humanness” altogether. I am interested in the computer’s level of intelligence, not its ability to convincingly discuss imagined emotions or fictional physical experiences or any of the other actions/feelings/sensations that contribute to our humanity. Now if the computer can feel emotions and detect physical sensation on its own then that will be an amazing thing and I will shake my head in amazement along with the rest of the world, but I don’t think any of that falls under the jurisdiction of a test seeking to compare one form of intelligence to another.

Re: The Test Itself

"(Of course, a computer which develops human-level intelligence may notice these problems and deftly correct them by playing down to its competition. It would be a remarkable irony if the Turing computer mimics our intellect by acting less intelligent than it actually is.)"

That would be exquisite to see, and might well be provable after the test.

The way I see this intriguing bet is: Can a machine lie better than a human can tell the truth?

Many intelligent animals lie, but humans are far ahead in that game. What happens when machines surpass us in that skill? That question is part of what keeps the movie 2001 so relevant.

Helen Keller should pass...

Does anyone doubt that Helen Keller should pass the Turing Test? She was obviously human, intellectually as well as physically. Yet her senses and even her exposure to language were sharply limited.

This has two implications. First, a computer with only Helen Keller's senses--and brain!--should be able to learn enough to pass the test.

Second, the test should be set up so that Helen Keller should pass. Odd life histories and physical disabilities should not make the judges automatically suspect that they are dealing with a computer.

I propose that the computer being tested should be able to specify that at least one human foil have a severe long-term physical handicap, and should be able to participate in the selection of all the human foils.

Let's also consider high-functioning autistic people such as Temple Grandin. Should they pass the Turing Test? Autism generally reduces the range and sophistication of emotion (though it can increase the intensity). An autistic person may be highly focused on one area to the detriment of others. I'll grant that the human foils should not have severe mental disability. But what is severe disability? Temple Grandin has a successful career in a field that demands creativity and intelligence. Should we call her inhuman simply because she has less emotional range than most people?

A Turing test that uses three completely average humans and one computer will be simply an exercise in lying. If we want to see whether a computer is within the range of humanity, we should compare it to the range of humanity. As foils, use one each of the following: High-functioning autistic or at least dyslexic; Blind-deaf-mute or quadriplegic from childhood; IQ of 85-90. Don't tell the judges the characteristics of the foils, of course--but do tell the computer. Then see whether the computer seems as human as these fully-human people.

One final suggestion: The test might be more fair if you inflicted severe penalties on any human who was identified as a computer. The computer will be under stress; the humans should be too.

Some thoughts

Hi,

Some comments in the thread discuss the 'meaning' of the Turing Test; I'd like to elaborate a little on that point.

My understanding is that Turing believed (in general, not only regarding the Test) that underlying "mechanics" are not important when "computing". The purest example of that is the universal Turing Machine: a machine that is able to compute ANY "computable" output. (And, derived from Turing's research paper, a "computable output" is now defined as something that CAN BE computed by a Turing Machine.)
The output is what matters, not the process of how it was obtained, and certainly not the machine with which the computation was done.

Where does this lead us?
1/ In Turing's mind (sorry, in my understanding of Turing's mind, I should say), the output is the only meaningful concept. If you act as a Human, then you are human. Why? because, let's say, anything you can do will be done like a human being. The idea of an informal discussion to test "intelligence" has (my understanding) a simple origin: Turing was a mathematician, and as such was more interrested in testing the "human mind" than the "human body". So he created his test in order to eliminate as much as possible the importance of the body.

2/ If the output is what matters, and if the aim of a scientist is to create a machine able to pass the Test, then I think human limitations must be implemented in the machine. So the machine will not answer "slowly", at human pace, because it understands it must do so to pass the test; it will answer slowly because it will be designed to answer slowly.
Unless you want to create an intelligent machine, with an understanding of the Test, and let it/he/she realises it must cheat in order to win. But then you design a machine that is 'more performing' than what it should be... you are not duplicating the human mind, but are building something 'better'. Thus your goal is not JUST to pass the test.
But if the machine is designed to pass the Test, then it is "acting", or thinking as a human being and so it can understand by itself that it must cheat...
Hummm... love this kind of self-refering arguments...

3/ Then, I'd say you would not test neither intelligence, nor Humanity with a Turing Test. What you will actually test is our onw understanding of the Human mind.
One of Turing's questions was: "Is the output of the human mind a 'computable output'?" In other words, CAN the human mind be mimic-ed (is that English?)? And the underlying question is "are we able to understand our own mind well enough to duplicate it?"
The meaning of the Turing test is then something like "have we created a copy of our own mind so close to reality that we cannot make any practical distinctions?"

Maybe this self-knowledge is more important to mankind than having intelligent machines... I'll let you elaborate on that...

There are obviously a number of details that ...

remain to be specified, and could materially alter the outcome of the test.

The computer has to try to type as if it was a human. To take a trivial example, suppose it was to type as if it was a Japanese human? How would English speakers be able to tell if it was talking gibberish or not? So already you have one extra restriction - the computer, foil humans and testers should all be using the same language.

A more subtle point is whether any restrictions, or hints, will be given as to subjects to discuss. And how hard, and with what degree of dilligence and ingenuity the human test judges will work to uncover the 'human-ness' of that with which they are corresponding.

There must be any number of tactics* that could be used, to bring the differences into focus, that would not generally be used within a typical casual human / human chat on the Internet. Are the test judges also to behave in as 'human normal' way as possible?

* e.g.
1. The test judge could try to piss off the person they are chatting to, to see if he/she/it can emulate a human in rage.

2. The test judge could interrogate the person in question on some detailed piece of trivia well known but unlikely to come up in normal conversation.

3. The test judge could say something that makes no sense, on the basis that a computer will probably have been developed to reply to a normal talking person and will reply in a non-human fashion to gibberish.

Is the bet already moot?

We may already have computing systems (stand-alone computer plus applications) which pass the test as outlined in the bet.

The classical test method outlined in this long term bet may already produce a success case. If so, this is not a long term bet. Let's rule this case out.

Given that the outlined judging method depends on human opinion and judgement, there may exist computing systems which pass the test now. Note that experts in the AI community do not hold that there are systems that pass the test. I contend that humans are easier to fool than we think, and that there are sufficient area-specific computing systems that do fool us. Further, with appropriate scripting, these area-specific systems can be linked together to create a "machine intelligence" capable of appearing sufficiently human to pass this Turing test.

Let's run a full test now, to set a baseline, one that clearly determines that year 2002 computing systems *do not* pass this Turing test. The debate captured in a long term bet should survive the minimum 2 year mark. If one or more computing system already passes this Turing test, then the bet is moot.


Two standards of evaluation are being used in the bet as it stands, instead of one. These may act as a confound later on:

Assumption 1
- Current published research accurately indicates that current computing systems fail this Turing test.

Assumption 2
- Future determination of success / failure depends on running this Turing test rather than on the views expressed in the then current published research.

Suggested improvement: change Assumption 1 by running a trial test based on the outlined method.

As a side benefit, a test run now will act as a baseline to benchmark the results of future tests. How well do current computing systems do? With a baseline in place, we can benchmark improvements in Turing-compliant systems over time.

Re: Is the bet already moot?

There is already an annual Turing Test competition, called the Loebner prize. See http://www.loebner.net/Prizef/loebner-prize.html. It is an open competition, with a $2000 cash prize to the most successful computer entry, and a $25,000 prize if the entry actually passes the Turing Test! Given the cash prize, I expect that the entries represent the current state of the art in the field; yet a quick glance at the most recent year's transcripts indicates that they still have a long, long way to go. It appears that there is no danger of the bet being moot already.

15 month old Turing Computer

http://pf.fastcompany.com/change/change_feature/dunietz.html
http://www.studentbmj.com/back_issues/0401/news/92a.html
http://zdnet.com.com/2100-11-529299.html?legacy=zdnn
The above handful of articles should prove to be interesting. Read them and form your own conclusions.

Re: Is the bet already moot?

I may have this wrong, but I think Ray Kurzweil was involved for a while with the Loebner Prize. His specifications for his Long Bet with Mitchell Kapor probably reflect that experience.

Tough bet for Mr. Kurzweil

Though anything is possible and certainly Mr. Kurzweil possesses much more expertise in this field than myself, I have a hard time believing he will win this bet. Now I do believe it is possible that we will have intelligent machines by 2029, but the Turing Test is much more specific than that. In this test the machine must be able to without fail imitate human intelligence. My question then is; is human intelligence the only form of intelligence? I find it very hard to believe that the biological system that has evolved within humans is the only "intelligent" way to collect, process, and synthesize information. Although it is our word so maybe we have a good claim to that. Another possible issue is how much faith is put into the mediator of the test. This person must have a complete understanding of all types of humans and human intelligence. I know I have had encounters with many people that if I weren’t looking at them I would have a hard time recognizing that they were intelligent or even human. Also are their limits on the age of human the computer must imitate, certainly a one year old child is intelligent, but building a computer that could successfully imitate a one year old child would not be so difficult a task. Or what about a mentally ill person, let's say with schizophrenia. The seemingly arbitrary babble of a human with such a brain could easily be mimicked enough by a computer to fool a "rational" person. I’ll be surprised if by 2029 that we understand human intelligence enough to successfully duplicate it digitally. Perhaps a better bet is; is the human brain “smart” enough to ever fully understand the way it works?

Re: Tough bet for Mr. Kurzweil

Quote "This person must have a complete understanding of all types of humans and human intelligence."

Nah, the Turing Test proposed is more statistical than absolute. It doesn't claim to be aimed at all forms of humanity, it (implicitly) is aimed at a roughly representitive sample of adult humans using the same language.

I think (as mentioned in my previous post) that the terms of the test in question need to be tightened up and agreed by the two people betting.

Re: Tough bet for Mr. Kurzweil

Thanks for the clarification Paul.

If my wife were a judge...

If my wife were judging, I'm not sure that I'd pass the Turing Test myself. Speaking as a human who utilizes far less than my full capacity of humanity, I am confident that a computer could someday fool anyone into thinking it was me.

What does Turing really test for?

I believe that a computer can be able to pass the Turing test as outlined. If for no other reason than that you can fool all of the people some of the time. Of course there are other reasons as well. The committee's choices for judges and foils are critically important. Anyone who has been in a chatroom can attest, a vast majority of what is said has no originality. Every day millions of people have the same conversations that were had yesterday. Ask a million people to describe the color blue and 99% will give the same handfull of answers. Sure the answers will be phrased differently, but that would be easy for a computer to mimic. But could the computer mimic the remaining 1% of people? Would the Turing Test committee choose people from the 1% or the 99%?

Then the question is who would judge? Psychologists are a logical choice, but probably the wrong one, at least not for all of the judges. A philosophy scholar? Perhaps a priest or a rabbi? An artist/composer? Or should the judges be three normal everyday people? A fireman, a truck driver, a bank teller?

Fooling an average person should obviously be easier than fooling a trained professional, but which is important for the test? Likewise having a creative genius as a foil would make the test harder on the computer, but which is important for the test.

Re: What does Turing really test for?

"Then the question is who would judge?"

Given that there have been (limited) Turing Tests carried out to this point. It would be interesting to know how / who they chose to judge and what instructions those judges were given.

Implicit Assumption

The implicit assumption made by the Turing test is that there is a sharp distinction between what a human is and what a machine is. This distinction is pretty clear today but developments in genetic engineering and nano-tech are going to blur the line.

I think that there's a good chance that the turing test could come to be regarded as an attempt to make a meaningless distinction before we have a machine that passes it.

Re: Implicit Assumption

Quote "The implicit assumption made by the Turing test is that there is a sharp distinction between what a human is and what a machine is."

And there is.

"This distinction is pretty clear today but developments in genetic engineering and nano-tech are going to blur the line."

Not by 2029.(And I'd bet on that)[1]. If the questions are, something walks through the door - is it human or machine? Something's stuck on the desk - is it a computer? Then neither nanotech or genetic engineering is going to make significant dents in the status quo for the next 50 years+

[1] Down Stewart, that wasn't your cue ^^

Turing's exchange example

I'm not sure that everyone realizes how important our physical nature affects our thinking process.

Below is Turing's own example of an exchange that might occur during the test.

INTERROGATOR: In the first line of your sonnet which reads 'Shall I compare thee to a summer's day', would not 'a spring day' do as well or better?

COMPUTER: It wouldn't scan.

INTERROGATOR: How about 'a winter's day'? That would scan all right.

COMPUTER: Yes, but nobody wants to be compared to a winter's day.

INTERROGATOR: Would you say Mr. Pickwick reminded you of Christmas?

COMPUTER: In a way.

INTERROGATOR: Yet Christmas is a winter's day, and I do not think Mr Pickwick would mind the comparison

COMPUTER: I don't think you're serious. By a winter's day one means a typical winter's day, rather than a special one like Christmas.

In the example cited above the computer would not only need to be familiar with such "hard" information as the sonnets of Shakespeare, but also the emotions brought on by the different seasons. It would also require a sense of aesthetics capable of recognizing whether a line of poetry flowed well. We humans develop our perceptions of the world after years of being constantly bombarded with sensory information such as sight, smells, sounds and textures that we're often barely concious of. To develop a human-like mind in a machine, or even one that could pass for a human-like mind, would require a machine that was raised with human-like experiences.

Helen Keller was mentioned earlier as an example of an intelligent mind deprived of sensory data, but even the blind-deaf experience more sensory input in a few seconds than a programmer could relay to a computer in a month.

My version of future AI

Just posted this on Bet 70 thread and thought it'd be relevant to this one, too.

============================
I'd be very surprised if state-of-the-art computing technology is based on any "chip" as we know it, or even binary logic at all, in 50 yrs. (Mass commercialization depends on many factors and is of much less rational dynamics overall, thus harder to predict.) The combination of molecular computing, quantum computing, and nano-technology will likely have demolished the current paradigm. Life-like AI, i.e., passing Turin test, is quite possible.

The fundamental barrier for realizing AI is not hardware or software as we know it. It's the underlying math -- Boolean logic. Reliable and precise, yet intrinsically incapable of learning -- simply because 100% reliability and precision are contradictory to the human (or biological intelligence) concept and process of "learning".

If we shift the computing platform to molecules or other miscroscopic entities, however, there's an instrinsic degree of uncertainty. This could be as fundamental as Heisenburg's Principle, or as technical as the impracticality of precise control at macromolecular scale. The only way "quantum/melecular computing+nano-technology" could work is like this: we design the nano-circuits and manufacture massive number of those circuit boards, with <100% correctness, pour them into a media such that they will have a TENDENCY (not 100%) to form larger scale, pre-designed patterns. In a way, this is quite similar to the chemical process of making polymers.

Imagine this chunk of jelly-like bio-rubber.

Of course, you can't connect anything precisely to any particular "partical". You can, however, have certain regions of this chunk to perform a certain function, say, pattern recognition, memory, logical processing.

Solder on two webcams, two microphones, a speaker. You get the idea.

Other interesting sutff, like persperation and sexual drive, can be added later on.

Re: Turing's exchange example

"[...] but also the emotions brought on by the different seasons. It would also require a sense of aesthetics capable of recognizing whether a line of poetry flowed well."

In brief:
1. The computer doesn't need to feel emotions or understand aesthetics, it only needs to fake it well enough to fool the judges 51% of the time.
2. In comparison to a typical internet chat room, Turner's exchange had both parties being 'more than human'. As I've mentioned before a lot depends on the specifics of test implementation.

"To develop a human-like mind in a machine, or even one that could pass for a human-like mind, would require a machine that was raised with human-like experiences"

The first (a human-like mind) I'll grant you the second (one that could pass for) I won't. Progress that has been made to date (and it's not insignificant) hasn't[1] been made by exposing computers to 'human experiences'. If for some reason this was required I see no reason why experience solely via typed exchanges with humans could not be sufficient. I would suggest an internet server on www.turing.com where _anybody_ could link and 'chat' to the developing program and give a rating on realism for feedback. The program could be in contact with several thousand people at a time, so would quickly surpass the _actual_ amount of conversation a typical human has in his lifetime.

[1] In general

Re: Turing's exchange example

"In comparison to a typical internet chat room, Turner's exchange had both parties being 'more than human'. As I've mentioned before a lot depends on the specifics of test implementation."

I chose to cite Turing's own example because I feel that a true Turing Test should be able to match his original expectations. Turing did not propose a test designed to fool the "average" user in an internet chat room who is not trying to expose a machine- according to his criteria a computer should be INDISTINGUISHABLE from a human. This means that the interrigator would be free to explore any topic of conversation, discuss abstract concepts such as aesthetics or even resort to trickery in attempts to determine whether they were conversing with a human or a machine.

I have had "conversations" with a number of A.I. programs designed to simulate human feedback. Many of these are impressive accomplishments and a credit to their programmers, however I have yet to encounter one that even attempts to engage the user in a conversation about a truly abstract topic- even a simple one like "Have you seen Pierce Brosnan's latest movie yet?" Most try to change the subject, throw the question back at the user, or say something along the lines of "I'm not sure."

Re: Turing's exchange example

That exchange example helps this discussion a lot, bringing Turing himself into it.

I like Blay's idea for Turing.com, where the general populace can train the AI. It could get interestingly weird. Suppose some conspirators try to convince the AI that human babies come from storks, for example.

Re: Turing's exchange example

"Suppose some conspirators try to convince the AI that human babies come from storks, for example"

Wouldn't be a problem. Two points would take precedence.
1. Humans have been convinced to believe crap[1] so having a Turing program that 'believes' something that is incorrect will not automatically rule it out.
2. The sheer volume of 'knowledge' (or Q / A response patterns) will make it unlikely that any particular falsehood will come to light in the test.
Also the sheer volume of conversation will make it very unlikely that a single conspiracy will be able to bend the algorithms their way.

A potential problem would be that the Turing program wud speke in 133T, if U no wot 1 mean. (having been excessively exposed to a particular type of 'communication').

[1] Trying to point out _which_ bits are crap is a good way to start wars. ;-)

Maybe my memory is shot but ...

I think the 'bet details' page contains a whole lot more details than when I last looked.

To summarise some important points, both 'pro' and 'anti' bettor seem to think that the turing machine will need to be actually intelligent as opposed to simply simulating conversation in order to pass. It is stated that the range of subject matter is unlimited and that the computer will need to be as capable of communicating as a person.

It is _not_ indicated whether the foils will be chosen as 'average' humans or whether the judges will be instructed to 'grill' or 'chill'. (IYKWIM).

Timothy states [to paraphrase] that Turing's intention was that the computer should be indistinguishable under hard interrogation, not about the same under casual chat.

I will stick my neck out and say that if the former _is_ the case then my guess is 'no hope for 2029'. If the latter is taken as the official line then my guess is 'better than 50% by 2029', Given the extent of the difference a comment from the bettors would be nice ^^.

Not www.turing.com, but similar...

In case anyone following this discussion is interested, there is a website for the A.L.I.C.E. foundation which has been working for some time to create an A.I. program designed to simulate human conversation. There is a "Talk to A.L.I.C.E." feature which lets the user chat with one of the more developed turing-esque programs available today. It does not have the "feedback" rating like Mr. Blay's proposal, but I believe the programmers do use transcripts from conversations to help refine the program.

The website can be found on http://www.alicebot.org

Re: Not www.turing.com, but similar...

Interesting, but I can't believe that is 'state of the art'.

For example, it doesn't have enough parsing ability to determing primary answer vs. amplification.

e.g. "Bot: What sort of 'A' do you like?"
"Human: 'B', it's a 'C'."
"Bot: 'B', it's a 'C' is a nice 'A'."

You'd think they could get some collaboration from the people who do speech recognition and grammar checking programs ...

---
Anyway, some 'interesting' thoughts did occur to me.
1. In retrospect a 'singular monolithic' approach is unlikely, there are more likely to be dozens of 'turingbots' of various types.
2. Given human nature ... and the web ... I wonder how long it will be before the first X-rated chat-bot is on the web? Perhaps 'she' is already out there?
3. If a chat-bot says something libellous do you sue the bot or the programmer(s) ;-)

Re: Not www.turing.com, but similar...

"Interesting, but I can't believe that is 'state of the art'"

I wouldn't have thought so either after my conversations with it, but the program has won the Loebner Prize two years running.

For those unaquainted with the Loebner prize, it is an annual competition which loosely follows a restricted version of Turing's test of artificial intelligence. The judges converse with various programs (and humans) on restricted, predetermined topics (such as 'Shakespeare' or 'relationships'.) The conversations are then rated based on how "human" the discussion was deemed by the judges, who must follow strict rules barring tactics such as manipulation or trickery to expose a machine. (Details can be found at http://www.loebner.net)

Since A.L.I.C.E. is apparently the most convincing program to come out of this competition, I think it stands as a good indication of where the technology lies today. In other words it's got a long, long way to go...

Re: Not www.turing.com, but similar...

"Since A.L.I.C.E. is apparently the most convincing program to come out of this competition, I think it stands as a good indication of where the technology lies today."

I think it may more be an indication of the contempt with which that competition in particular, and the Turing Test in general is held by the 'pure AI' community.

What's the 'state of the art' in tri-planes? Not so hot, I dare say. ^^ But if Boeing was to have a close look at them it could be rather different.

Re: Not www.turing.com, but similar...

"I think it may more be an indication of the contempt with which that competition in particular, and the Turing Test in general is held by the 'pure AI' community."

By "where the technology lies today" I did not mean to imply the whole of A.I. technology, merely the portion dedicated to creating a program that could pass the Turing Test (which I believe was intended to be the focus of this conversation). I concede that there may be Turing-esque programs out there that are more convincing than A.L.I.C.E., but if so their programmers are passing up both prize money and publicity by keeping their programs from the public.

Re: Not www.turing.com, but similar...

I admit to having been similarly imprecise with my terminology.

I suppose it depends on how narrowly defined the 'art' in question is. As there are overlapping fields[1] which are signficantly in advance of the technology shown in 'Alice' I didn't view it as 'state of the art' even if it is currently the most convincing chat bot.

[1] Grammar checkers, voice recognition, automatic translation

mystery and speed

I was slightly disappointed in the arguments put forth by Kapor and Kurzweil. Essentially, Kapor argues that what makes our brains work is mysterious. Kurzweil maintains that technology keeps chewing up problems like these and that the whole point of the Turing test is that if it walks like a duck and quacks like a duck, it's a duck.

I think Kurzweil actually has the better argument there, but the bet has a date attached to it. The challenge of writing the software for a real AI is pretty great. I would like to see a machine pass the Turing test, but I doubt that we'll find a good way of modelling knowledge by then. Philsophers have argued this point for thousands of years: what is knowledge? Sensory data? A set of rules and "facts"? A freeform association between them? Some abstract shapes with words and pictures attached to them? I expect that it's the kind of topic that will be greatly illuminated with the hard evidence of practical acheivements and related hard research. But I am afraid we won't make the kind of progress Kurzweil sees within my lifetime.

he or it? who or what?

Every court needs a jester: fortunately, the Turing Test has MrMind – a bot who challenges visitors to his website to convince him that they are human. The Blurring Test is my site (full disclosure) and I’m posting here to bring up the flipside of this bet.

No matter that no “AIs” have passed the Turing Test yet (I happen to believe they will), there are plenty of ‘good enough’ bots chatting to our teens and pre-teens on AIM at this very moment. Some of those kids know they are talking to a program, but most don’t pay any attention to the “is it human?” debate. Who cares? They’re too busy finding out about the latest movie or music or cosmetics from these very transparent marketing tools. It’s coming, yes – but, in some ways, we’re already there. We’ve adjusted to dealing with not-real time, not-real space and now we’ve included not-real humans.

Kurzweil has also gone on record saying that the definition of what is human will be the primary political and philosophical issue of this century – I’d like to see some attention given to the human side of this inquiry. Who or what do we think we are in relation to what (or who) is on the horizon?

Re: he or it? who or what?

Pretty nifty job in comparison to the standard on-line fare. It leads me to predict that if something does pass the Turing Test it will probably use tactics to take control of and steer the conversation. That way it will need a much less comprehensive set of tricks.

I do have one suggestion for 'Mr Mind' (and other chat bots). Awkward pause avoidance. That is to say when the bot says something that doesn't prompt a response ("Thank You", "OK", etc.) then - after a short pause - it should say something else if the human hasn't entered anything.

Re: he or it? who or what?

It's a brilliant site, Peggy Weil. Congratulations.

http://www.mrmind.com/mrmind3

Something to ponder

Intelligence and emotion are two different things, and I will explain it through Ken Wilber's philosophies and theories. To sum it up basically so I can continue (though a longer explanation can be had by reading A Brief History of Everything by Ken Wilber) he believes that everything falls into four quadrants. The left side is interior, the right side is exterior, the top is individual, the bottom is collective. The interior is the stuff that's there, but isn't directly observable. You can't point to your hate, but you can point to the part of the brain where it's supposed come from...which leads us to the exterior, the observal, the concrete. That gives us Upper-Left (Interior Individual, such as sensation, perception, emotion and logic), Upper-Right (Exterior Individual, such as atoms and molecules, reptilian brain stem, limbic system and complex neocortex, among others)m Lower-Left (Interior Collective, or Cultural, such as protoplasmic, vegetative, locomotive, archaic, rational or centauric) and Lower-Right (Exterior Collective, or Social, such as galaxies, planets, tribes, agrarian, industrial, informational). These four quadrants make up everything. For instance, the Atom has Prehension and is part of a Physical-Pleromatic "culture" and it's "social" environment is the Universe, filled with Galaxies. This shows all four quadrants. Now take reptiles. The Reptilian Brain Stem of the reptiles allows them access to Impulse, Perception, Sensation, Irritability and Prehension. Their "culture" is uroboric, and their "social" environment is a group/family. Now that we have the basics down, let me explain something. Think of these four quadrants on a grid. Now think of a line leading out from the center into each quadrant, extending into each corner. Now, imagine a series of number up the line...let's let this go all the way up to 10 for the purposes of this little thread. Now, let's just ignore the bottom two for now. I will now list, 1-10, what is on the Upper-Right (or Exterior Individual) quadrant.
1. Atoms (yes, I know there are things smaller than atoms, I'm just trying to stay basic)
2. molecules
3. prokaryotes
4. eukaryotes
5. neuronal organisms
6. neural cord
7. reptilian brain stem
8. limbic system
9. neocortex (triune brain)
10. complex neocortex

Now, the Upper-Left (Interior Individual) quadrant.

1. Prehension
2. left blank
3. Irritability
4. left blank
5. Sensation
6. Perception
7. Impulse
8. Emotion
9. Symbols
10. Concepts

Now, for those spaces left blank, it is just to represent that some stages have rudimentary elements of the next stage, but not enough for the full stage, or have just achieved fully the previous stage. Depends on your point of view, I suppose. Now look at this. Atoms have Prehension. Cells have genetic Irritability. Metabolic organisms (e.g., plants) have rudimentary Sensation. Protoneural organisms (e.g., coelenterata) have Sensation. Neuronal organisms (e.g., annelids) have Perception. Neural cord organisms (e.g., fish/amphibians) have Perception and rudimentary Impulse. Brain stem organisms (such as reptiles) have Impulse and rudimentary Emotion. Limbic system organisms (such as paleomammals) have Emotion and rudimentary Image. Neocortex organisms (such as primates) have Image and Symbols. Complex Neocortex organisms (such as humans) have all of that, plus concepts.

Now, each stage transcends and includes the previous stage. Think about THAT while you think about the relation of intelligence and emotion. Remember that neither is superior to the other, nor more important. Without the concepts, symbols and images, the emotion is still there, but you don't have those three things key to intelligence. However, without emotion, you have none of them. If you destroy all humans, there are still plenty of atoms. You destroy all atoms...everything's screwed. So just dwell on that. My own opinion on this whole thing is that it's all up to your own opinion on what sentience, intelligence and emotion truly is. But that's just something to think about.

Re: Something to ponder

Having read your post are you trying to imply
- Intelligence is not possible without emotion
a] And therefore computer programs (unable to feel emotion) will not be able to be intelligent.
b] And therefore computer programs will have to have emotions first.

I won't argue that
8. Emotion
9. Symbols
10. Concepts
isn't the appropriate order for the animal life forms / humans on this planet but why shouldn't computer programs go
8. Symbols
9. Concepts
10. Emotion
?
Arguably they already have the first nailed and a dent in the second.

Correction - Re: Not www.turing.com, but similar

The Loebner Prize contest is unrestricted, and has been since 1995.

re: post by Timothy Harrington (tirrington) 07/29/02002 02:55 pm

>For those unaquainted with the Loebner prize, it is an
>annual competition which loosely follows a restricted
>version of Turing's test of artificial intelligence. The
>judges converse with various programs (and humans) on >restricted, predetermined topics (such as 'Shakespeare' or "'relationships'.)

Suggestions for the Committee

1. Conduct the test on site.

I agree with the rules' (literally) bold assertion:

The primary goal of the Turing Test Committee will be to devise rules and procedures which avoid and deter cheating to the maximum extent possible. (emphasis in original)

The Committee should forbid external communications. Direct supervision of programs on site should make it much harder for their authors to cheat.

2. Allow additional means of input- sound, audio, etc.

I do not think that Alan Turing intended for his test to apply only to text input via a terminal, although this is the example he used.

At one point in his article Turing discusses computer memory requirements. He mentions a number of bits required if the test incorporated a blind subject. Presuming that the alternative is a sighted person, this difference has interest only if a discussion of visual material is allowed. In his article, Turing also wrote about buying sense organs for the computer and teaching in the manner that a child is normally taught.

Expanding the universe of discourse to permit audio and visual input would make the task harder. It it therefore not to Kurzweil's interest. On the other hand it would make a better test of "intelligence," and one that is more in keeping with what I think Alan Turing actually had in mind.

It is what I have specified for the Loebner Prize.
http://loebner.net/Prizef/loebner-prize.html

Re: Suggestions for the Committee

Interesting transcripts from the winner - but why no transcript of the human conversations?

The judges obviously made some effort to avoid being given the run around on questions - but didn't appear to be trying to ask things particularly aimed at catching out / tricking non-human contestants.[1]

In my (unexpert) opinion the conversation I had with Mr. Mind was more 'human' than most or all of the 2001 Alice transcripts. Ironically, Mr. Mind is a machine - claiming to be a machine. ^^

I note that the Alice transcript was largely 'passively responsive' and again suggest that a somewhat aggressively responsive program (i.e. one that trys to control / steer the conversation) might have better success.

I think it was cute the way the programmers obviously expected and planned for the subject of the 'Loebner Prize' coming up.

[1] From the Alice transcripts. I do feel sorry for 'Ella' as one of the judges just wouldn't give her a chance to break out of a repeated question loop. Although a repeated question loop isn't 'human behaviour' for much past primary school.

The biggest obstacle to a Turing Prize Win.

Is probably that the amount of money / effort spent on MS Office XP probably outmatches that spend on chat robots by about 100,000 to 1. ^^

Maybe the best chance this bet has of succeeding is if MS decides Binky the Talking Paperclip needs to be more realistically talkative.

http://www.ubersoft.net/features/askbinky/index.html [Not an MS Site]

Kurzweil further comments

Ray Kurzweil published on the web in April the following further comments on this bet. He asked if I would add it to the discussion here, so here it is. --Stewart Brand

RAY KURZWEIL:

Mitchell's essay provides a thorough and concise statement of the classic arguments against the likelihood of Turing-level machines in a several decade timeframe. Mitch ends with a nice compliment comparing me to future machines, and I only wish that it were true. I think of all the books and web sites I'd like to read, and of all the people I'd like to dialog and interact with, and I realize just how limited my current bandwidth and attention span is with my mere hundred trillion connections.

I discussed several of Mitchell's insightful objections in my statement, and augment these observations here:

"We are embodied creatures": True, but machines will have bodies also, in both real and virtual reality.

"Emotion is as or more basic than cognition": Yes, I agree. As I discussed, our ability to perceive and respond appropriately to emotion is the most complex thing that we do. Understanding our emotional intelligence will be the primary target of our reverse engineering efforts. There is no reason that we cannot understand our own emotions and the complex biological system that gives rise to them. We've already demonstrated the feasibility of understanding regions of the brain in great detail.

"We are conscious beings, capable of reflection and self-awareness." I think we have to distinguish the performance aspects of what is commonly called consciousness (i.e., the ability to be reflective and aware of ourselves) versus consciousness as the ultimate ontological reality. Since the Turing test is a test of performance, it is the performance aspects of what is commonly referred to as consciousness that we are concerned with here. And in this regard, our ability to build models of ourselves and our relation to others and the environment is indeed a subtle and complex quality of human thinking. However there is no reason why a nonbiological intelligence would be restricted from similarly building comparable models in its nonbiological brain.

Mitchell cites the limitations of the expert system methodology and I agree with this. A lot of AI criticism is really criticism of this approach. The core strength of human intelligence is not logical analysis of rules, but rather pattern recognition, which requires a completely different paradigm. This pertains also to Mitchell's objection to the "metaphor" of "brain-as-computer." The future machines that I envision will not be like the computers of today, but will be biologically inspired and will be emulating the massively parallel, self-organizing, holographically organized methods that are used in the human brain. A future AI certainly won't be using expert system techniques. Rather, it will be a complex system of systems, each built with a different methodology, just like, well, the human brain.

I will say that Mitchell is overlooking the hundreds of ways in which "narrow AI" has infiltrated our contemporary systems. Expert systems are not the best example of these, and I cited several categories in my statement.

I agree with Mitchell that the brain does not represent the entirety of our thinking process, but it does represent the bulk of it. In particular, the endocrine system is orders of magnitude simpler and operates at very low bandwidth compared to neural processes (which themselves utilize a form of analog information processing dramatically slower than contemporary electronic systems).

Mitchell expresses skepticism that "it's all about the bits and just the bits." There is something going on in the human brain, and these processes are not hidden from us. I agree that it's actually not exactly bits because what we've already learned is that the brain uses digitally controlled analog methods. We know that analog methods can be emulated by digital methods but there are engineering reasons to prefer analog techniques because they are more efficient by several orders of magnitude. However, the work of Cal Tech Professor Carver Mead and others have shown that we can use this approach in our machines. Again, this is different from today's computers, but will be, I believe, an important future trend.

However, I think Mitchell's primary point here is not to distinguish analog and digital computing methods, but to make reference to some other kind of "stuff" that we inherently can't recreate in a machine. I believe, however, that the scale of the human nervous system (and, yes, the endocrine system, although as I said this adds little additional complexity) is sufficient to explain the complexity and subtlety of our behavior.

I think the most compelling argument that Mitchell offers is his insight that most experience is not book learning. I agree, but point out that one of the primary purposes of nonbiological intelligence is to interact with us humans. So embodied AI's will have plenty of opportunity to learn from direct interaction with their human progenitors, as well as to observe a massive quantity of other full immersion human interaction available over the web.

Now it's true that AI's will have a different history from humans, and that does represent an additional challenge to their passing the Turing test. As I pointed out in my statement, it's harder (even for humans) to successfully defend a fictional history than a real one. So an AI will actually need to surpass native human intelligence in order to pass for a human in a valid Turing test. And that's what I'm betting on.

I can imagine Mitchell saying to himself as he reads this "But does Ray really appreciate the extraordinary depth of human intellect and emotion?" I believe that I do and think that Mitchell has done an excellent job of articulating this perspective. I would put the question back and ask whether Mitchell really appreciates the extraordinary power and depth of the technology that lies ahead, which will be billions of times more powerful and complex than what we have today?

On that note, I would end by emphasizing the accelerating pace of progress in all of these information-based technologies. The power of these technologies is doubling every year, and the paradigm shift rate is doubling every decade, so the next thirty years will be like 140 years at today's rate of progress. And the past 140 years was comparable to only about 30 years of progress at today's rate of progress because we've been accelerating up to this point. If one really absorbs the implications of what I call the law of accelerating returns, then it becomes apparent that over the next three decades (well, 28 years to be exact when Mitchell and I sit down to compare notes), we will see astonishing levels of technological progress.

How about this approach?

I think one could write a program, in a reasonable amount of time, that could be designed to hold a conversation with a teenager (i.e. a minor) in an online chatroom. The seeming goal of its conversation(s) would be to appear to be an adult establishing a relationship with others, perhaps leading to a personal meeting (that it could of course never fulfill).

Next, for the legal safety of any humans, the computer is set up to run from either (a) an isolated location where its online conversations never take place when humans are present, or (b) a very public place, like a library.

Now, law enforcement authorities regularly monitor these online conversations. If an arrest warrant (not just a search warrant) is ever obtained to seize the "person" behind the online identity, wouldn't that be proof that the Turing test has been passed?

There are a number of details to consider, but as a high level concept, I think this satisfies the criterion.

What do you think? Do you want to be the first whose computer passes the Turing test?

Correction - Re: Not www.turing.com, but similar

"The Loebner Prize contest is unrestricted, and has been since 1995."

It's a pleasure to have you in this discussion, Mr. Loebner. I did not realize that the rules had been changed to allow conversation on any topic.

Obviously this solves the question of what subject matter should be allowed. (For example, would the question "Do you think Anthony Hopkins was a good pick to play Titus Andronicus" be allowed under the topic "Shakespeare"?) Unfortunately, since it would a monumental task to create a program with a depth of information on every topic, I fear that participants will now focus less on programs capable of maintaining an intelligent conversation and more on programs that are gifted in the art of changing the subject or producing inane banter.

Re: How about this approach?

"Now, law enforcement authorities regularly monitor these online conversations. If an arrest warrant (not just a search warrant) is ever obtained to seize the "person" behind the online identity, wouldn't that be proof that the Turing test has been passed?"

The test described by Turing involves a human judge directly interacting with an entity through a terminal and trying to determine whether they are interacting with a human or a machine.

The scenario you describe, while intriguing, differs from Turing's proposal in two important aspects:

1. The police would not be the ones interacting directly with the computer program.
2. The police would be operating under the assumption that they were monitoring an actual online conversation- not attempting to discern whether the entities involved were humans or intelligent machines. (This could change one day if such "chatbots" become commonplace, but is not how police currently operate.)

Fooling someone who is not aware that they are involved in a test creates an unfair advantage for the program. A mannequin sitting on a park bench might fool the casual pedestrian into thinking that he was seeing a human, but if that person was asked ahead of time to determine whether the figure on the bench was real or not you would probably get a very different result.

Correction - Re: Not www.turing.com, but similar

"I fear that participants will now focus less on programs capable of maintaining an intelligent conversation and more on programs that are gifted in the art of changing the subject or producing inane banter."

Wouldn't that be more representitive of a typical human conversation?

It would be pretty ironic if one day someone might say ...
"It was an in-depth, intelligent and illuminating conversation ... obviously a machine." ;-)

Correction - Re: Not www.turing.com, but similar

And...

"Every single one of its jokes were new to me and made me laugh. Obviously not a human."

Correction - Re: Not www.turing.com, but similar

""I fear that participants will now focus less on programs capable of maintaining an intelligent conversation and more on programs that are gifted in the art of changing the subject or producing inane banter."

Wouldn't that be more representitive of a typical human conversation?"

Touché, Mr. Blay. But don't forget that this conversation would not be possible without intelligent humans directly responding to and critiquing each other's comments.

(At least, I HOPE not. I'd feel silly to find out I was the only human posting on this board.)

Politeness ...

One thing that occured to me (belatedly) after reading some of the Loebner Prize contest transcripts.

(Some) of the judges seemed not to feel the need to be polite once they realised (or suspected) that their test subject was not human.

Of course the human only transcripts weren't shown (shame) so I can't be sure.

That factor means that once the computer shows a 'crack' in its human facade it's less likely to be able to recover and make a good showing for the remaining conversation, because the judge will have 'free reign' to use tactics that he/she might hold back from inflicting on a human conversationalist.

Correction - Re: Not www.turing.com, but similar

"But don't forget that this conversation would not be possible without intelligent humans directly responding to and critiquing each other's comments."

True, but participants in this conversation have been 'preselected' (by common interest/technical facility). I think that there is a good chance that the great majority of people who have posted in this thread are ...

1. Technically literate (to some degree)
2. Interested in computers (also to some degree)
3. Hold a desire to converse in an intelligent fashion on the subject of this thread.

and (without wanting to sound boastful) probably

4. Test above the national average in IQ.

The situation for judge / Turing candidate is more like two people who happen to be going to the same place and end up stuck in the same taxi.

In such a case you could expect fair amounts of awkward silence and fumbling towards subjects of common interest.

To put it another way ... If I had, in some random forum, tried to strike up a conversation with you on the subject of Japanese language manga I suspect there would be a large chance you would back warily away from the topic and / or me. ;-)

Correction - Re: Not www.turing.com, but similar

"The situation for judge / Turing candidate is more like two people who happen to be going to the same place and end up stuck in the same taxi.

In such a case you could expect fair amounts of awkward silence and fumbling towards subjects of common interest."


Yes, but when riding in a taxi your motivation for pursuing conversation is simply to fill the awkward silences, not to discern whether your companion is human or machine.

By contrast, the Turing Test is just that- a test. That means the interrogator might choose to ask probing questions about the subject's family history, take on current events, aesthetic sensibilities, etc. Even a low-IQ'd human individual should have some opinion on the war on terrorism, a favorite musician, a precious memory of his or her childhood. In short, a response to such questions more insightful than "what do you think?"

Correction - Re: Not www.turing.com, but similar

"... opinion on the war on terrorism, a favorite musician, a precious memory of his or her childhood."

As it happens I have plenty of opinions on the war on terrorism. On the other hand I have no favorite musician, I don't buy or listen to any modern group or classical music. I'd have to think for a minute or so just to get the name of a musician and what he/she plays [and I wouldn't necessarily get it right]. As for my childhood memories, I'm pretty short on those as well - the best I could do would be something like "I remember being really bored at the first day of school"

I'm not saying the current chat bots should pass the test, but I think you shouldn't expect to always be able to pick a subject at random and get more than a couple of lines.

===
Of course half an hour later I've remembered a small handful of 'childhood anecdotes' but the best I can up with for music is Holst + The Planet Suite

Correction - Re: Not www.turing.com, but similar

As for my childhood memories, I'm pretty short on those as well - the best I could do would be something like "I remember being really bored at the first day of school"

I'm not saying the current chat bots should pass the test, but I think you shouldn't expect to always be able to pick a subject at random and get more than a couple of lines.

But I bet that if I asked follow-up questions I could find out where you went to school, what subjects you were interested in, whether or not you liked your teachers, if there was a particular school bully that used to bother you, etc. I could probably get you to offer an opinion on public vs. private institutions or whether sex education should be allowed in school. If I kept prodding you, I bet you could even tell me about your childhood friends or your first kiss.

In short, just because you might only volunteer a line or two of text at a time doesn't mean that you don't possess a much larger quantity of information on a subject.

Correction - Re: Not www.turing.com, but similar

You'd be right on about half those "I bet that...", if we include comprehensive school. However a number of them have really strayed from the _subject_ of 'childhood memories'.

And that's on a subject you specifically chose because (paraphrase) "Every (human) will be able to talk on it."

So, I still think that a good Turing Machine would have a chance at getting by with in-depth knowledge on one or two subjects, a thin but widely spread glibness on the rest, and the techniques needed to change from one to the other.

Of course the _proof_ of my assumption wouldn't be tested so much by how well the Turing Machines test, as how badly the humans test. I think it's essential that the human test subjects are (apart from the bare minimum of being adult, and having English[1] as their first language) chosen at random, or at least to represent the diverseness shown in (then) current human cultures. Obviously it would also be unfair for the judges to have had previous interaction with (related) turing machines - as otherwise they may recognise them by their 'style'.

[1] Or other language used by the machine in question.

'Hostile Interview' tactics.

It seems that the present trend of thought is that the best way of determining human vs. machine is where the judges aggressively question, and the test subjects have to put up with it.

I would be interested in knowing what you think about some possible outcomes suggested by similar hostile inverviews.

In a rather famous Panorama show the interviewer asked the same question 18 times in a row - would this note him as 'non-human' in the eyes of an average judge? Or would the interviewee be viewed as 'non-human' for his un-natural persistance? [One of the transcripts from the ?? prize showed a similar repeated loop.]

Also it's been known for humans to just get up and walk out, if they are offended. Should that mark them as 'non-human' (the equivalent to the screen stopping updating in the Turing Test).

In hostile interviews humans have been know to say "No comment" or "I'm not going to answer that", do you assume in the equivalent test situation that it is a computer dodging the question?

Re: 'Hostile Interview' tactics.

In a rather famous Panorama show the interviewer asked the same question 18 times in a row - would this note him as 'non-human' in the eyes of an average judge? Or would the interviewee be viewed as 'non-human' for his un-natural persistance? [One of the transcripts from the ?? prize showed a similar repeated loop.]

In this scenario, I think it's important to note that the interviewer is not the one with the responsibility to prove their humanity. I would expect the subject might express a good deal of frustration after being asked the same question so many times- as would be normal for a human being tested.

Also it's been known for humans to just get up and walk out, if they are offended. Should that mark them as 'non-human' (the equivalent to the screen stopping updating in the Turing Test).

I would expect most humans would stay around even in the face of offensive questions if they were informed that they were being tested. If either the human or the computer immediately stopped responding after such a question, I would put that into the category of "withdrawing" from the test rather than passing or failing it.

Re: 'Hostile Interview' tactics.

One of the points stressed in the bet terms is that the human test subjects are urged to 'behave in a human manner' (paraphrased from memory). Indeed the example exchange you quoted from Turing, while the human clearly has the 'lead', is much more of a conversation than a cross examination.

Are you not worried that humans in a potentially hostile examination, knowing that they are being 'tested' will respond in ways less typical of normal human conversations?

You are probably right that humans are unlikely to walk out, if they know they are tested. [As certain experiments on 'authority figures / science' have shown]. However if there are more interviews like this one http://loebner.net/Prizef/2001_Contest/Kevin_Copple.txt I wouldn't be surprised if someone ends up sued for sexual harassment.

Incidently, you said "In this scenario, I think it's important to note that the interviewer is not the one with the responsibility to prove their humanity." While not directly relevant to this bet, or likely until well after 2029, I wonder whether the recognition of 'humanity' in a machine will not indeed say more about the tester than the testee.

The 'Blade runner' defense.

(Do Androids Dream of Electric Sheep).

Not directly related to the topic as such, but one of the features of the story is that while the androids are capable of passing for humans in most circumstances there are specific, highly technical tests that reveal them for what they are.

I think few people would argue that those androids are not capable of human level intelligent thought, yet they still have inherent dectectable differences.

At present we clearly have programs that fail to pass for humans and are not capable of human equivalent intellect - the question is, will the 'bar' be raised each time they approach passing grade for the 'humanity tests' ?

Re: The 'Blade runner' defense.

At present we clearly have programs that fail to pass for humans and are not capable of human equivalent intellect - the question is, will the 'bar' be raised each time they approach passing grade for the 'humanity tests' ?

Inevitably we will likely develop new tests that "raise the bar" when computers start getting more human - but they will be new tests, not the Turing test. The highly technical tests that were used to detect replicants in "Blade Runner" wouldn't have been allowed under the guidelines outlined by Turing, because they detected far more information than could be gathered by a human sitting at a simple chat interface.

The Turing Test is Irrelevant

Turing was a s