Bet 1
A computer - or "machine intelligence" - will pass the Turing Test by 2029.
Bet 1
Duration 27 years (02002-02029)
Predictor
Mitchell Kapor
Challenger
Ray Kurzweil
Stakes $20,000
will go to The Electronic Frontier Foundation if Kapor wins,
or The Kurzweil Foundation if Kurzweil wins.
Add your voice to a conversation with the bettors:
Bookmark this bet, and share it with friends:
The essence of the Turing Test revolves around whether a computer can successfully impersonate a human. The test is to be put into practice under a set of detailed conditions which rely on human judges being connected with test subjects (a computer and a person) solely via an instant messaging system or its equivalent. That is, the only information which will pass between the parties is text.
To pass the test, a computer would have to be capable of communicating via this medium at least as competently as a person. There is no restriction on the subject matter; anything within the scope of human experience in reality or imagination is fair game. This is a very broad canvas encompassing all of the possibilities of discussion about art, science, personal history, and social relationships. Exploring linkages between the realms is also fair game, allowing for unusual but illustrative analogies and metaphors. It is such a broad canvas, in my view, that it is impossible to foresee when, or even if, a machine intelligence will be able to paint a picture which can fool a human judge.
While it is possible to imagine a machine obtaining a perfect score on the SAT or winning Jeopardy--since these rely on retained facts and the ability to recall them--it seems far less possible that a machine can weave things together in new ways or to have true imagination in a way that matches everything people can do, especially if we have a full appreciation of the creativity people are capable of. This is often overlooked by those computer scientists who correctly point out that it is not impossible for computers to demonstrate creativity. Not impossible, yes. Likely enough to warrant belief in a computer can pass the Turing Test? In my opinion, no. Computers look relatively smarter in theory when those making the estimate judge people to be dumber and more limited than they are.
As humans:
When I contemplate human beings in this way, it becomes extremely difficult even to imagine what it would mean for a computer to perform a successful impersonation, much less to believe that its achievement is within our lifespan. Computers don't have anything resembling a human body, sense organs, feelings, or awareness after all. Without these, it cannot have human experiences, especially of the ones which reflect our fullest nature, as above. Each of knows what it is like to be in a physical environment; we know what things look, sound, smell, taste, and feel like. Such experiences form the basis of agency, memory and identity. We can and do speak of all this in a multitude of meaningful ways to each other. Without human experiences, a computer cannot fool a smart judge bent on exposing it by probing its ability to communicate about the quintessentially human.
Additionally, part of the burden of proof for supporters of intelligent machines is to develop an adequate account of how a computer would acquire the knowledge it would be required to have to pass the test. Ray Kurzweil's approach relies on an automated process of knowledge acquisition via input of scanned books and other printed matter. However, I assert that the fundamental mode of learning of human beings is experiential. Book learning is a layer on top of that. Most knowledge, especially that having to do with physical, perceptual, and emotional experience is not explicit, never written down. It is tacit. We cannot say all we know in words or how we know it. But if human knowledge, especially knowledge about human experience, is largely tacit, i.e., never directly and explicitly expressed, it will not be found in books, and the Kurzweil approach to knowledge acquisition will fail. It might be possible to produce a kind of machine as idiot savant by scanning a library, but a judge would not have any more trouble distinguishing one from an ordinary human as she would with distinguishing a human idiot savant from a person not similarly afflicted. It is not in what the computer knows but what the computer does not know and cannot know wherein the problem resides.
Given these considerations, a skeptic about machine intelligence could fairly ask how and why the Turing Test was transformed from its origins as a provocative thought experiment by Alan Turing to a challenge seriously sought. The answer is to be found in the origins of the branch of computer science its practitioners have called Artificial Intelligence (AI).
In the 1950's a series of computer programs were written which first demonstrated the ability of the computer to carry out symbolic manipulations in software in ways which the performance (not the actual process) began to approach human level on tasks such as playing checkers and proving theorems in geometry. These results fueled the dreams of computer scientists to create machines which were endowed with intelligence. Those dreams, however, repeatedly failed to be realized. Early successes were not followed with more success, but with failure. A pattern of over-optimism was first seen which has persisted to this day. Let me be clear I am not referring to most computer scientists in the field of AI, but to those who take an extreme position.
For instance, there were claims in the 1980's that expert systems would come be of great significance, in which computer would perform as well or better than human experts in a wide variety of disciplines. This belief triggered a boom in investment in AI-based startups in the 1980's, followed by a bust when audacious predictions of success failed to be met and the companies premised on those claims also failed.
In practice, expert systems proved to be fragile creatures, capable at best of dealing with facts in narrow, rigid domains, in ways which were very much unlike the adaptable, protean nature of intelligence demonstrated by human experts. As we call them today, knowledge-based systems do play useful roles in a variety of ways, but there is broad consensus that the knowledge of these knowledge-based systems is a very small and non-generalizable part of overall human intelligence.
Ray Kurzweil's arguments seek to go further. To get a computer to perform like a person with a brain, a computer should be built to work the way a brain works. This is an interesting, intellectually challenging idea.
He assumes this can be accomplished by using as yet undeveloped nano-scale technology (or not -- he seems to want to have it both ways) to scan the brain in order to reverse engineer what he refers to as the massively parallel digital controlled analog algorithms that characterize information processing in each region. These then are presumably what control the self-organizing hierarchy of networks he thinks constitute the working mechanism of the brain itself. Perhaps.
But we don't really know whether "carrying out algorithms operating on these networks" is really sufficient to characterize what we do when we are conscious. That's an assumption, not a result. The brain's actual architecture and the intimacy of its interaction, for instance, with the endocrine system, which controls the flow of hormones, and so regulates emotion (which in turn has an extremely important role in regulating cognition) is still virtually unknown. In other words, we really don't know whether in the end, it's all about the bits and just the bits. Therefore Kurzweil doesn't know, but can only assume, that the information processing he wants to rely on in his artificial intelligence is a sufficiently accurate and comprehensive building block to characterize human mental activity.
The metaphor of brain-as-computer is tempting and to a limited degree fruitful, but we should not rely on its distant extrapolation. In the past, scientists have sought to employ metaphors of their age to characterize mysteries of human functioning, e.g., the heart as pump, the brain as telephone switchboard (you could look this up). Properly used, metaphors are a step on the way to development of scientific theory. Stretched beyond their bounds, the metaphors lose utility and have to be abandoned by science if it is not to be led astray. My prediction is that contemporary metaphors of brain-as-computer and mental activity-as-information processing will in time also be superceded and will not prove to be a basis on which to build human-level intelligent machines (if indeed any such basis ever exists).
Ray Kurzweil is to be congratulated on his vision and passion, regardless of who wins or loses the bet. In the end, I think Ray is smarter and more capable than any machine is going to be, as his vision and passion reflect qualities of the human condition no machine is going to successfully emulate over the term of the bet. I look forward to comparing notes with him in 2029.
The Significance of the Turing Test.
The implicit, and in my view brilliant, insight in Turing's eponymous test is the ability of written human language to represent human-level thinking. The basis of the Turing test is that if the human Turing test judge is competent, then an entity requires human-level intelligence in order to pass the test. The human judge is free to probe each candidate with regard to their understanding of basic human knowledge, current events, aspects of the candidate's personal history and experiences, as well as their subjective experiences, all expressed through written language. As humans jump from one concept and one domain to the next, it is possible to quickly touch upon all human knowledge, on all aspects of human, well, humanness.
To the extent that the "AI" chooses to reveal its "history" during the interview with the Turing Test judge (note that none of the contestants are required to reveal their histories), the AI will need to use a fictional human history because "it" will not be in a position to be honest about its origins as a machine intelligence and pass the test. (By the way, I put the word "it" in quotes because it is my view that once an AI does indeed pass the Turing Test, we may very well consider "it" to be a "he" or a "she.") This makes the task of the machines somewhat more difficult than that of the human foils because the humans can use their own history. As fiction writers will attest, presenting a totally convincing human history that is credible and tracks coherently is a challenging task that most humans are unable to accomplish successfully. However, some humans are capable of doing this, and it will be a necessary task for a machine to pass the Turing test.
There are many contemporary examples of computers passing "narrow" forms of the Turing test, that is, demonstrating human-level intelligence in specific domains. For example, Gary Kasparov, clearly a qualified judge of human chess intelligence, declared that he found Deep Blue's playing skill to be indistinguishable from that of a human chess master during the famous tournament in which he was defeated by Deep Blue. Computers are now displaying human-level intelligence in a growing array of domains, including medical diagnosis, financial investment decisions, the design of products such as jet engines, and a myriad of other tasks that previously required humans to accomplish. We can say that such "narrow AI" is the threshold that the field of AI has currently achieved. However, the subtle and supple skills required to pass the broad Turing test as originally described by Turing is far more difficult than any narrow Turing Test. In my view, there is no set of tricks or simpler algorithms (i.e., methods simpler than those underlying human level intelligence) that would enable a machine to pass a properly designed Turing test without actually possessing intelligence at a fully human level.
There has been a great deal of philosophical discussion and speculation concerning the issue of consciousness, and whether or not we should consider a machine that passed the Turing test to be conscious. Clearly, the Turing test is not an explicit test for consciousness. Rather, it is a test of human-level performance. My own view is that inherently there is no objective test for subjective experience (i.e., consciousness) that does not have philosophical assumptions built into it. The reason for this has to do with the difference between the concepts of objective and subjective experience. However, it is also my view that once nonbiological intelligence does achieve a fully human level of intelligence, such that it can pass the Turing test, humans will treat such entities as if they were conscious. After all, they (the machines) will get mad at us if we don't. However, this is a political prediction rather than a philosophical position.
It is also important to note that once a computer does achieve a human level of intelligence, it will necessarily soar past it. Electronic circuits are already at least 10 million times faster than the electrochemical information processing in our interneuronal connections. Machines can share knowledge instantly, whereas we biological humans do not have quick downloading ports on our neurotransmitter concentration levels, interneuronal connection patterns, nor any other biological bases of our memory and skill. Language-capable machines will be able to access vast and accurate knowledge bases, including reading and mastering all the literature and sources of information available to our human-machine civilization. Thus "Turing Test level" machines will be able to combine human level intelligence with the powerful ways in which machines already excel. In addition, machines will continue to grow exponentially in their capacity and knowledge. It will be a formidable combination.
Why I Think I Will Win. In considering the question of when machine (i.e., nonbiological) intelligence will match the subtle and supple powers of human biological intelligence, we need to consider two interrelated but distinct questions: when will machines have the hardware capacity to match human information processing, and when will our technology have mastered the methods, i.e., the software of human intelligence. Without the latter, we would end up with extremely fast calculators, and would not achieve the endearing qualities that characterize human discernment (nor the deep knowledge and command of language necessary to pass a full Turing test!).
Both the hardware and software sides of this question are deeply influenced by the exponential nature of information-based technologies. The exponential growth that we see manifest in "Moore's Law" is far more pervasive than commonly understood. Our first observation is that the shrinking of transistors on an integrated circuit, which is the principle of Moore's Law, was not the first but the fifth paradigm to provide exponential growth to computing (after electromechanical calculators, relay-based computers, vacuum tube-based computing, and discrete transistors). Each time one approach begins to run out of steam, research efforts intensify to find the next source of renewed exponential growth (e.g., vacuum tubes were made smaller until it was no longer feasible to maintain a vacuum, which led to transistors). Thus the power and price-performance of technologies, particularly information-based technologies, grow as a cascade of S-curves: exponential growth leading to an asymptote, leading to paradigm shift (i.e., innovation), and another S-curve. Moreover, the underlying theory of the exponential growth of information-based technologies, which I call the law of accelerating returns, as well as a detailed examination of the underlying data, show that there is a second level of exponential growth, i.e., the rate of exponential growth is itself growing exponentially.
Second, this phenomenon of ongoing exponential growth through a cascade of S-curves is far broader than computation. We see the same double exponential growth in a wide range of technologies, including communication technologies (wired and wireless), biological technologies (e.g., DNA base-pair sequencing), miniaturization, and of particular importance to the software of intelligence, brain reverse engineering (e.g., brain scanning, neuronal and brain region modeling).
Within the next approximately fifteen years, the current computational paradigm of Moore's Law will come to an end because by that time the key transistor features will only be a few atoms in width. However, there are already at least two dozen projects devoted to the next (i.e., the sixth) paradigm, which is to compute in three-dimensions. Integrated circuits are dense but flat. We live in a three-dimensional world, our brains are organized in three dimensions, and we will soon be computing in three dimensions. The feasibility of three-dimensional computing has already been demonstrated in several landmark projects, including the particularly powerful approach of nanotube-based electronics. However, for those who are (irrationally) skeptical of the potential for three-dimensional computing, it should be pointed out that achieving even a conservatively high estimate of the information processing capacity of the human brain (i.e., one hundred billion neurons times a thousand connections per neuron times 200 digitally controlled analog "transactions" per second, or about 20 million billion operations per second) will be achieved by conventional silicon circuits prior to 2020.
It is correct to point out that achieving the "software" of human intelligence is the more salient, and more difficult, challenge. On multiple levels, we are being guided in this effort by a grand project to reverse engineer (i.e., understand the principles of operation of) the human brain itself. Just as the human genome project accelerated (with the bulk of the genome being sequenced in the last year of the project), the effort to reverse engineer the human brain is also growing exponentially, and is further along than most people realize. We already have highly detailed mathematical models of several dozen of the several hundred types of neurons found in the brain. The resolution, bandwidth, and price-performance of human brain scanning is also growing exponentially. By combining the neuron modeling and interconnection data obtained from scanning, scientists have already reverse engineered two dozen of the several hundred regions of the brain. Implementations of these reverse engineered models using contemporary computation matches the performance of the biological regions that were recreated in significant detail. Already, we are in a early stage of being able to replace small regions of the brain that have been damaged from disease or disability using neural implants (e.g., ventral posterior nucleus, subthalmic nucleus, and ventral lateral thalamus neural implants to counteract Parkinson's Disease and tremors from other neurological disorders, cochlear implants, emerging retinal implants, and others).
If we combine the exponential trends in computation, communications, and miniaturization, it is a conservative expectation that we will within 20 to 25 years be able to send tiny scanners the size of blood cells into the brain through the capillaries to observe interneuronal connection data and even neurotransmitter levels from up close. Even without such capillary-based scanning, the contemporary experience of the brain reverse engineering scientists, (e.g., Lloyd Watts, who has modeled over a dozen regions of the human auditory system), is that the connections in a particular region follow distinct patterns, and that it is not necessary to see every connection in order to understand the massively parallel, digital controlled analog algorithms that characterize information processing in each region. The work of Watts and others has demonstrated another important insight, that once the methods in a brain region are understood and implemented using contemporary technology, the computational requirements for the machine implementation requires on the order of a thousand times less computation than the theoretical potential of the biological neurons being simulated.
A careful analysis of the requisite trends shows that we will understand the principles of operation of the human brain and be in a position to recreate its powers in synthetic substrates well within thirty years. The brain is self-organizing, which means that it is created with relatively little innate knowledge. Most of its complexity comes from its own interaction with a complex world. Thus it will be necessary to provide an artificial intelligence with an education just as we do with a natural intelligence. But here the powers of machine intelligence can be brought to bear. Once we are able to master a process in a machine, it can perform its operations at a much faster speed than biological systems. As I mentioned, contemporary electronics is already more than ten million times faster than the human nervous system's electrochemical information processing. Once an AI masters human basic language skills, it will be in a position to expand its language skills and general knowledge by rapidly reading all human literature and by absorbing the knowledge contained on millions of web sites. Also of great significance will be the ability of machines to share their knowledge instantly.
One challenge to our ability to master the apparent complexity of human intelligence in a machine is whether we are capable of building a system of this complexity without the brittleness that often characterizes very complex engineering systems. This a valid concern, but the answer lies in emulating the ways of nature. The initial design of the human brain is of a complexity that we can already manage. The human brain is characterized by a genome with only 23 million bytes of useful information (that's what left of the 800 million byte genome when you eliminate all of the redundancies, e.g., the sequence called "ALU" which is repeated hundreds of thousands of times). 23 million bytes is smaller than Microsoft WORD. How is it, then, that the human brain with its 100 trillion connections can result from a genome that is so small? The interconnection data alone is a million times greater than the information in the genome. The answer is that the genome specifies a set of processes, each of which utilizes chaotic methods (i.e., initial randomness, then self-organization) to increase the amount of information represented. It is known, for example, that the wiring of the interconnections follows a plan that includes a great deal of randomness. As the individual person encounters her environment, the connections and the neurotransmitter level pattern self-organize to better represent the world, but the initial design is specified by a program that is not extreme in its complexity.
Thus we will not program human intelligence link by link as in some massive expert system. Nor is it the case that we will simply set up a single genetic (i.e., evolutionary) algorithm and have intelligence at human levels automatically evolve itself. Rather we will set up an intricate hierarchy of self-organizing systems, based largely on the reverse engineering of the human brain, and then provide for its education. However, this learning process can proceed hundreds if not thousands of times faster than the comparable process for humans.
Another challenge is that the human brain must incorporate some other kind of "stuff" that is inherently impossible to recreate in a machine. Penrose imagines that the intricate tubules in human neurons are capable of quantum based processes, although there is no evidence for this. I would point out that even if the tubules do exhibit quantum effects, there is nothing barring us from applying these same quantum effects in our machines. After all, we routinely use quantum methods in our machines today. The transistor, for example, is based on quantum tunneling. The human brain is made of the same small list of proteins that all biological systems are comprised of. We are rapidly recreating the powers of biological substances and systems, including neurological systems, so there is little basis to expect that the brain relies on some nonengineerable essence for its capabilities. In some theories, this special "stuff" is associated with the issue of consciousness, e.g., the idea of a human soul associated with each person. Although one may take this philosophical position, the effect is to separate consciousness from the performance of the human brain. Thus the absence of such a soul may in theory have a bearing on the issue of consciousness, but would not prevent a nonbiological entity from the performance abilities necessary to pass the Turing test.
Another challenge is that an AI must have a human or human-like body in order to display human-like responses. I agree that a body is important to provide a situated means to interact with the world. The requisite technologies to provide simulated or virtual bodies are also rapidly advancing. Indeed, we already have emerging replacements or augmentations for virtually every system in our body. Moreover, humans will be spending a great deal of time in full immersion virtual reality environments incorporating all of the senses by 2029, so a virtual body will do just as well. Fundamentally, emulating our bodies in real or virtual reality is a less complex task than emulating our brains.
Finally, we have the challenge of emotion, the idea that although machines may very well be able to master the more analytical cognitive abilities of humans, they inherently will never be able to master the decidedly illogical and much harder to characterize attributes of human emotion. A slightly broader way of characterizing this challenge is to pose it in terms of "qualia," which refers essentially to the full range of subjective experiences. Keep in mind that the Turing test is assessing convincing reactions to emotions and to qualia. The apparent difficulty of responding appropriately to emotion and other qualia appears to be at least a significant part of Mitchell Kapor's hesitation to accept the idea of a Turing-capable machine. It is my view that understanding and responding appropriately to human emotion is indeed the most complex thing that we do (with other types of qualia being if anything simpler to respond to). It is the cutting edge of human intelligence, and is precisely the heart of the Turing challenge. Although human emotional intelligence is complex, it nonetheless remains a capability of the human brain, with our endocrine system adding only a small measure of additional complexity (and operating at a relatively low bandwidth). All of my observations above pertain to the issue of emotion, because that is the heart of what we are reverse engineering. Thus, we can say that a side benefit of creating Turing-capable machines will be new levels of insight into ourselves.
A Wager on the Turing Test: The Rules
As prepared by Ray Kurzweil in consultation with Mitchell Kapor
Background on the "Long Now Turing Test Wager."
Ray Kurzweil maintains that a computer (i.e., a machine intelligence) will pass the Turing test by 2029. Mitchell Kapor believes this will not happen.
This wager is intended to be the inaugural long term bet to be administered by the Long Now Foundation. The proceeds of the wager are to be donated to a charitable organization designated by the winner.
This document provides a brief description of the Turing Test and a set of high level rules for administering the wager. These rules contemplate setting up a "Turing Test Committee" which will create the detailed rules and procedures to implement the resolution of the wager. A primary objective of the Turing Test Committee will be to set up rules and procedures that avoid and deter cheating.
Brief Description of the Turing test. In a 1950 paper ("Computing Machinery and Intelligence," Mind 59 (1950): 433- 460, reprinted in E. Feigenbaum and J. Feldman, eds., Computers and Thought, New York: McGraw-Hill, 1963), Alan Turing describes his concept of the Turing Test, in which one or more human judges interview computers and human foils using terminals (so that the judges won't be prejudiced against the computers for lacking a human appearance). The nature of the dialogue between the human judges and the candidates (i.e., the computers and the human foils) is similar to an online chat using instant messaging. The computers as well as the human foils try to convince the human judges of their humanness. If the human judges are unable to reliably unmask the computers (as imposter humans) then the computer is considered to have demonstrated human-level intelligence .
Turing was very specifically nonspecific about many aspects of how to administer the test. He did not specify many key details, such as the duration of the interrogation and the sophistication of the human judge and foils. The purpose of the rules described below is to provide a set of procedures for administering the test some decades hence.
The Procedure for the Turing Test Wager: The Turing Test General Rules
These Turing Test General Rules may be modified by agreement of Ray Kurzweil and Mitchell Kapor, or, if either Ray Kurzweil and / or Mitchell Kapor is not available, then by the Turing Test Committee (described below). However, any such change to these Turing Test General Rules shall only be made if (i) these rules are determined to have an inconsistency, or (ii) these rules are determined to be inconsistent with Alan Turing's intent of determining human-level intelligence in a machine, or (iii) these rules are determined to be unfair, or (iv) these rules are determined to be infeasible to implement.
I. Definitions.
A Human is a biological human person as that term is understood in the year 2001 whose intelligence has not been enhanced through the use of machine (i.e., nonbiological) intelligence, whether used externally (e.g., the use of an external computer) or internally (e.g., neural implants). A Human may not be genetically enhanced (through the use of genetic engineering) beyond the level of human beings in the year 2001.
A Computer is any form of nonbiological intelligence (hardware and software) and may include any form of technology, but may not include a biological Human (enhanced or otherwise) nor biological neurons (however, nonbiological emulations of biological neurons are allowed).
The Turing Test Committee will consist of three Humans, to be selected as described below.
The Turing Test Judges will be three Humans selected by the Turing Test Committee.
The Turing Test Human Foils will be three Humans selected by the Turing Test Committee.
The Turing Test Participants will be the three Turing Test Human Foils and one Computer.
II. The Procedure
The Turing Test Committee will be appointed as follows.
One member will be Ray Kurzweil or his designee, or, if not available, a person appointed by the Long Now Foundation. In the event that the Long Now Foundation appoints this person, it shall use its best efforts to appoint a Human person that best represents the views of Ray Kurzweil (as expressed in his bet argument.)
A second member will be Mitchell Kapor or his designee, or, if not available, a person appointed by the Long Now Foundation. In the event that the Long Now Foundation appoints this person, it shall use its best efforts to appoint a Human person that best represents the views of Mitchell Kapor (as expressed in his bet argument.)
A third member will be appointed by the above two members, or if the above two members are unable to agree, then by the Long Now Foundation, who in its judgment, is qualified to represent a "middle ground" position.
Ray Kurzweil, or his designee, or another member of the Turing Test Committee, or the Long Now Foundation may, from time to time call for a Turing Test Session to be conducted and will select or provide one Computer for this purpose. For those Turing Test Sessions called for by Ray Kurzweil or his designee or another member of the Turing Test committee (other than the final one in 2029), the person calling for the Turing Test Session to be conducted must provide (or raise) the funds necessary for the Turing Test Session to be conducted. In any event, the Long Now Foundation is not obligated to conduct more than two such Turing Test Sessions prior to the final one (in 2029) if it determines that conducting such additional Turing Test Sessions would be an excessive administrative burden.
The Turing Test Committee will provide the detailed rules and procedures to implement each such Turing Test Session using its best efforts to reflect the rules and procedures described in this document. The primary goal of the Turing Test Committee will be to devise rules and procedures which avoid and deter cheating to the maximum extent possible. These detailed rules and procedures will include (i) specifications of the equipment to be used, (ii) detailed procedures to be followed, (iii) specific instructions to be given to all participants including the Turing Test Judges, the Turing Test Human Foils and the Computer, (iv) verification procedures to assure the integrity of the proceedings, and (v) any other details needed to implement the Turing Test Session. Beyond the Turing Test General Rules described in this document, the Turing Test Committee will be guided to the best of its ability by the original description of the Turing Test by Alan Turing in his 1950 paper. The Turing Test Committee will also determine procedures to resolve any deadlocks that may occur in its own deliberations.
Each Turing Test Session will consist of at least three Turing Test Trials.
For each such Turing Test Trial, a set of Turing Test Interviews will take place, followed by voting by the Turing Test Judges as described below.
Using its best judgment, the Turing Test Committee will appoint three Humans to be the Turing Test Judges.
Using its best judgment, the Turing Test Committee will appoint three Humans to be the Turing Test Human Foils. The Turing Test Human Foils should not be known (either personally or by reputation) to the Turing Test Judges.
During the Turing Test Interviews (for each Turing Test Trial), each of the three Turing Test Judges will conduct online interviews of each of the four Turing Test Candidates (i.e., the Computer and the three Turing Test Human Foils) for two hours each for a total of eight hours of interviews conducted by each of the three Turing Test Judges (for a total of 24 hours of interviews).
The Turing Test Interviews will consist of online text messages sent back and forth as in a online "instant messaging" chat, as that concept is understood in the year 2001.
The Human Foils are instructed to try to respond in as human a way as possible during the Turing Test Interviews.
The Computer is also intended to respond in as human a way as possible during the Turing Test Interviews.
Neither the Turing Test Human Foils nor the Computer are required to tell the truth about their histories or other matters. All of the candidates are allowed to respond with fictional histories.
At the end of the interviews, each of the three Turing Test Judges will indicate his or her verdict with regard to each of the four Turing Test Candidates indicating whether or not said candidate is human or machine. The Computer will be deemed to have passed the "Turing Test Human Determination Test" if the Computer has fooled two or more of the three Human Judges into thinking that it is a human.
In addition, each of the three Turing Test Judges will rank the four Candidates with a rank from 1 (least human) to 4 (most human). The computer will be deemed to have passed the "Turing Test Rank Order Test" if the median rank of the Computer is equal to or greater than the median rank of two or more of the three Turing Test Human Foils.
The Computer will be deemed to have passed the Turing Test if it passes both the Turing Test Human Determination Test and the Turing Test Rank Order Test.
If a Computer passes the Turing Test, as described above, prior to the end of the year 2029, then Ray Kurzweil wins the wager. Otherwise Mitchell Kapor wins the wager.
The Turing test is a flawed measure of intelligence.
In essence it is based on the principle that all things that be confused with one another are identical, rather
than the principle that all things that have the same
nature are identical.
In a court of law physical evidence is required as a
basis for claims because it establishes a natural
foundation for assertions. Otherwise there is no
substantial basis.
In other words, the Turing test is based on
equivocation to the exclusion of identification.
To beat the Turing test, all that is required is that
a machine be built which is capable of confusing
humans.
The pattern to follow then would be the methods
of lawyers who have no basis for their assertions.
Or building a type of machine that works like a
confused human. Automated sophistry would
paint a compelling LIKENESS to a human mind
without having to establish an IS-NESS to a human mind.
Will we be able to confuse human judges by the due
date of this bet? Sure, if unsubtantiated testimony is
admissable in our test.
I think the abstract test proposed by Mr. Turing in 1950 is quite clever, but the version of it outlined in this bet may not be able to determine whether a computer has demonstrated human-level intelligence. Although the Long Bets Foundation will eventually be forced to do so, I think administering this Turing Test will be logistically impossible, and, given the criteria negotiated by the bettors, the final results could be debatable.
The Turing Test used to settle this bet will basically be a conversation. A judge will interact with a machine intelligence and human foils via text messages in an effort to 1) determine which is the computer, and 2) rate both the computer and the foils on their level of "humanness". To pass the second half of the Test, the computer's median rank must equal or excel the median rank of two human foils. Part of my problem with these guidelines is that they both seek to determine "humanity" instead of intelligence, which may be an unfair burden for the computer to overcome.
First, administering the Turing Test conversation in a fair manner may be impossible. For example, it is entirely conceivable that the computer will be too intelligent for its own good. Depending on how the give-and-take between the judge and the respondent is regulated, something as simple as a grammatical error or misused punctuation may tip the hand of a human foil, as the computer will presumably be programmed with a flawless language program. (Some would argue that language software cannot advance that far in the given time period. That might be so, but my grammar-check already exceeds the abilities of my human editors.) If the language software is perfect, then the computer will probably be the only respondent that does not commit any typos or misspellings within the two-hour test period.
Another potential problem is that the tested computer might unwittingly respond to an inquiry faster than is humanly possible. The human foils will either type their responses or dictate them into text format. Both methods require time to complete. Our 2029 computer might develop a response and return it so quickly as to instantly disqualify itself.
(Of course, a computer which develops human-level intelligence may notice these problems and deftly correct them by playing down to its competition. It would be a remarkable irony if the Turing computer mimics our intellect by acting less intelligent than it actually is.)
The problem with this, and with the second half of the judging, is that the computer must be intelligent enough to actively deceive us about its (lack of) humanity. The human foils of the Test have a tremendous advantage in “acting human”, seeing as they have a lifetime’s worth of practice. Ray Kurzweil discusses this in his argument but is confident that the computer will be up to the task. Personally, I think it is unfair to require the computer to generate its own history and experiences while the human foils may freely draw upon their own. It would be far more interesting if ALL the respondents were required to produce new histories of their own or to respond falsely to every inquiry during the test. (I haven’t a clue how this would be judged other than to trust in the good-faith of the foils.) If you are seeking a test that compares intelligence levels, why not make the same demands upon all the intellects being tested? Seeing as the computer must continually deceive the judges with its responses, it seems reasonable to require the same of the human foils. That would at least offer a direct comparison of the minds involved in the test.
Which leads me to my final point. At the conclusion of the test, the three judges will assign a rank of 1 (least human) to 4 (most human) to each of the respondents. If it is not obvious, allow me to point out that the criterion to be used is “humanness” and not “intelligence”. This scoring system will almost certainly result in the human foils earning different scores, which creates the awkward situation that one of the foils is more “human” than another. Now, if the humans aren’t equally human then I’m not sure what use it is to apply the same judgement to the computer. In my opinion, the computer will have passed the Test even if it scores two 2s and a 1, even though this would not be a passing performance in the eyes of the bet. In fact, I would want to eliminate the criteria of “humanness” altogether. I am interested in the computer’s level of intelligence, not its ability to convincingly discuss imagined emotions or fictional physical experiences or any of the other actions/feelings/sensations that contribute to our humanity. Now if the computer can feel emotions and detect physical sensation on its own then that will be an amazing thing and I will shake my head in amazement along with the rest of the world, but I don’t think any of that falls under the jurisdiction of a test seeking to compare one form of intelligence to another.
"(Of course, a computer which develops human-level intelligence may notice these problems and deftly correct them by playing down to its competition. It would be a remarkable irony if the Turing computer mimics our intellect by acting less intelligent than it actually is.)"
That would be exquisite to see, and might well be provable after the test.
The way I see this intriguing bet is: Can a machine lie better than a human can tell the truth?
Many intelligent animals lie, but humans are far ahead in that game. What happens when machines surpass us in that skill? That question is part of what keeps the movie 2001 so relevant.
Does anyone doubt that Helen Keller should pass the Turing Test? She was obviously human, intellectually as well as physically. Yet her senses and even her exposure to language were sharply limited.
This has two implications. First, a computer with only Helen Keller's senses--and brain!--should be able to learn enough to pass the test.
Second, the test should be set up so that Helen Keller should pass. Odd life histories and physical disabilities should not make the judges automatically suspect that they are dealing with a computer.
I propose that the computer being tested should be able to specify that at least one human foil have a severe long-term physical handicap, and should be able to participate in the selection of all the human foils.
Let's also consider high-functioning autistic people such as Temple Grandin. Should they pass the Turing Test? Autism generally reduces the range and sophistication of emotion (though it can increase the intensity). An autistic person may be highly focused on one area to the detriment of others. I'll grant that the human foils should not have severe mental disability. But what is severe disability? Temple Grandin has a successful career in a field that demands creativity and intelligence. Should we call her inhuman simply because she has less emotional range than most people?
A Turing test that uses three completely average humans and one computer will be simply an exercise in lying. If we want to see whether a computer is within the range of humanity, we should compare it to the range of humanity. As foils, use one each of the following: High-functioning autistic or at least dyslexic; Blind-deaf-mute or quadriplegic from childhood; IQ of 85-90. Don't tell the judges the characteristics of the foils, of course--but do tell the computer. Then see whether the computer seems as human as these fully-human people.
One final suggestion: The test might be more fair if you inflicted severe penalties on any human who was identified as a computer. The computer will be under stress; the humans should be too.
Hi,
Some comments in the thread discuss the 'meaning' of the Turing Test; I'd like to elaborate a little on that point.
My understanding is that Turing believed (in general, not only regarding the Test) that underlying "mechanics" are not important when "computing". The purest example of that is the universal Turing Machine: a machine that is able to compute ANY "computable" output. (And, derived from Turing's research paper, a "computable output" is now defined as something that CAN BE computed by a Turing Machine.)
The output is what matters, not the process of how it was obtained, and certainly not the machine with which the computation was done.
Where does this lead us?
1/ In Turing's mind (sorry, in my understanding of Turing's mind, I should say), the output is the only meaningful concept. If you act as a Human, then you are human. Why? because, let's say, anything you can do will be done like a human being. The idea of an informal discussion to test "intelligence" has (my understanding) a simple origin: Turing was a mathematician, and as such was more interrested in testing the "human mind" than the "human body". So he created his test in order to eliminate as much as possible the importance of the body.
2/ If the output is what matters, and if the aim of a scientist is to create a machine able to pass the Test, then I think human limitations must be implemented in the machine. So the machine will not answer "slowly", at human pace, because it understands it must do so to pass the test; it will answer slowly because it will be designed to answer slowly.
Unless you want to create an intelligent machine, with an understanding of the Test, and let it/he/she realises it must cheat in order to win. But then you design a machine that is 'more performing' than what it should be... you are not duplicating the human mind, but are building something 'better'. Thus your goal is not JUST to pass the test.
But if the machine is designed to pass the Test, then it is "acting", or thinking as a human being and so it can understand by itself that it must cheat...
Hummm... love this kind of self-refering arguments...
3/ Then, I'd say you would not test neither intelligence, nor Humanity with a Turing Test. What you will actually test is our onw understanding of the Human mind.
One of Turing's questions was: "Is the output of the human mind a 'computable output'?" In other words, CAN the human mind be mimic-ed (is that English?)? And the underlying question is "are we able to understand our own mind well enough to duplicate it?"
The meaning of the Turing test is then something like "have we created a copy of our own mind so close to reality that we cannot make any practical distinctions?"
Maybe this self-knowledge is more important to mankind than having intelligent machines... I'll let you elaborate on that...
remain to be specified, and could materially alter the outcome of the test.
The computer has to try to type as if it was a human. To take a trivial example, suppose it was to type as if it was a Japanese human? How would English speakers be able to tell if it was talking gibberish or not? So already you have one extra restriction - the computer, foil humans and testers should all be using the same language.
A more subtle point is whether any restrictions, or hints, will be given as to subjects to discuss. And how hard, and with what degree of dilligence and ingenuity the human test judges will work to uncover the 'human-ness' of that with which they are corresponding.
There must be any number of tactics* that could be used, to bring the differences into focus, that would not generally be used within a typical casual human / human chat on the Internet. Are the test judges also to behave in as 'human normal' way as possible?
* e.g.
1. The test judge could try to piss off the person they are chatting to, to see if he/she/it can emulate a human in rage.
2. The test judge could interrogate the person in question on some detailed piece of trivia well known but unlikely to come up in normal conversation.
3. The test judge could say something that makes no sense, on the basis that a computer will probably have been developed to reply to a normal talking person and will reply in a non-human fashion to gibberish.
We may already have computing systems (stand-alone computer plus applications) which pass the test as outlined in the bet.
The classical test method outlined in this long term bet may already produce a success case. If so, this is not a long term bet. Let's rule this case out.
Given that the outlined judging method depends on human opinion and judgement, there may exist computing systems which pass the test now. Note that experts in the AI community do not hold that there are systems that pass the test. I contend that humans are easier to fool than we think, and that there are sufficient area-specific computing systems that do fool us. Further, with appropriate scripting, these area-specific systems can be linked together to create a "machine intelligence" capable of appearing sufficiently human to pass this Turing test.
Let's run a full test now, to set a baseline, one that clearly determines that year 2002 computing systems *do not* pass this Turing test. The debate captured in a long term bet should survive the minimum 2 year mark. If one or more computing system already passes this Turing test, then the bet is moot.
Two standards of evaluation are being used in the bet as it stands, instead of one. These may act as a confound later on:
Assumption 1
- Current published research accurately indicates that current computing systems fail this Turing test.
Assumption 2
- Future determination of success / failure depends on running this Turing test rather than on the views expressed in the then current published research.
Suggested improvement: change Assumption 1 by running a trial test based on the outlined method.
As a side benefit, a test run now will act as a baseline to benchmark the results of future tests. How well do current computing systems do? With a baseline in place, we can benchmark improvements in Turing-compliant systems over time.
There is already an annual Turing Test competition, called the Loebner prize. See http://www.loebner.net/Prizef/loebner-prize.html. It is an open competition, with a $2000 cash prize to the most successful computer entry, and a $25,000 prize if the entry actually passes the Turing Test! Given the cash prize, I expect that the entries represent the current state of the art in the field; yet a quick glance at the most recent year's transcripts indicates that they still have a long, long way to go. It appears that there is no danger of the bet being moot already.
http://pf.fastcompany.com/change/change_feature/dunietz.html
http://www.studentbmj.com/back_issues/0401/news/92a.html
http://zdnet.com.com/2100-11-529299.html?legacy=zdnn
The above handful of articles should prove to be interesting. Read them and form your own conclusions.
I may have this wrong, but I think Ray Kurzweil was involved for a while with the Loebner Prize. His specifications for his Long Bet with Mitchell Kapor probably reflect that experience.
Though anything is possible and certainly Mr. Kurzweil possesses much more expertise in this field than myself, I have a hard time believing he will win this bet. Now I do believe it is possible that we will have intelligent machines by 2029, but the Turing Test is much more specific than that. In this test the machine must be able to without fail imitate human intelligence. My question then is; is human intelligence the only form of intelligence? I find it very hard to believe that the biological system that has evolved within humans is the only "intelligent" way to collect, process, and synthesize information. Although it is our word so maybe we have a good claim to that. Another possible issue is how much faith is put into the mediator of the test. This person must have a complete understanding of all types of humans and human intelligence. I know I have had encounters with many people that if I weren’t looking at them I would have a hard time recognizing that they were intelligent or even human. Also are their limits on the age of human the computer must imitate, certainly a one year old child is intelligent, but building a computer that could successfully imitate a one year old child would not be so difficult a task. Or what about a mentally ill person, let's say with schizophrenia. The seemingly arbitrary babble of a human with such a brain could easily be mimicked enough by a computer to fool a "rational" person. I’ll be surprised if by 2029 that we understand human intelligence enough to successfully duplicate it digitally. Perhaps a better bet is; is the human brain “smart” enough to ever fully understand the way it works?
Quote "This person must have a complete understanding of all types of humans and human intelligence."
Nah, the Turing Test proposed is more statistical than absolute. It doesn't claim to be aimed at all forms of humanity, it (implicitly) is aimed at a roughly representitive sample of adult humans using the same language.
I think (as mentioned in my previous post) that the terms of the test in question need to be tightened up and agreed by the two people betting.
If my wife were judging, I'm not sure that I'd pass the Turing Test myself. Speaking as a human who utilizes far less than my full capacity of humanity, I am confident that a computer could someday fool anyone into thinking it was me.
I believe that a computer can be able to pass the Turing test as outlined. If for no other reason than that you can fool all of the people some of the time. Of course there are other reasons as well. The committee's choices for judges and foils are critically important. Anyone who has been in a chatroom can attest, a vast majority of what is said has no originality. Every day millions of people have the same conversations that were had yesterday. Ask a million people to describe the color blue and 99% will give the same handfull of answers. Sure the answers will be phrased differently, but that would be easy for a computer to mimic. But could the computer mimic the remaining 1% of people? Would the Turing Test committee choose people from the 1% or the 99%?
Then the question is who would judge? Psychologists are a logical choice, but probably the wrong one, at least not for all of the judges. A philosophy scholar? Perhaps a priest or a rabbi? An artist/composer? Or should the judges be three normal everyday people? A fireman, a truck driver, a bank teller?
Fooling an average person should obviously be easier than fooling a trained professional, but which is important for the test? Likewise having a creative genius as a foil would make the test harder on the computer, but which is important for the test.
"Then the question is who would judge?"
Given that there have been (limited) Turing Tests carried out to this point. It would be interesting to know how / who they chose to judge and what instructions those judges were given.
The implicit assumption made by the Turing test is that there is a sharp distinction between what a human is and what a machine is. This distinction is pretty clear today but developments in genetic engineering and nano-tech are going to blur the line.
I think that there's a good chance that the turing test could come to be regarded as an attempt to make a meaningless distinction before we have a machine that passes it.
Quote "The implicit assumption made by the Turing test is that there is a sharp distinction between what a human is and what a machine is."
And there is.
"This distinction is pretty clear today but developments in genetic engineering and nano-tech are going to blur the line."
Not by 2029.(And I'd bet on that)[1]. If the questions are, something walks through the door - is it human or machine? Something's stuck on the desk - is it a computer? Then neither nanotech or genetic engineering is going to make significant dents in the status quo for the next 50 years+
[1] Down Stewart, that wasn't your cue ^^
I'm not sure that everyone realizes how important our physical nature affects our thinking process.
Below is Turing's own example of an exchange that might occur during the test.
INTERROGATOR: In the first line of your sonnet which reads 'Shall I compare thee to a summer's day', would not 'a spring day' do as well or better?
COMPUTER: It wouldn't scan.
INTERROGATOR: How about 'a winter's day'? That would scan all right.
COMPUTER: Yes, but nobody wants to be compared to a winter's day.
INTERROGATOR: Would you say Mr. Pickwick reminded you of Christmas?
COMPUTER: In a way.
INTERROGATOR: Yet Christmas is a winter's day, and I do not think Mr Pickwick would mind the comparison
COMPUTER: I don't think you're serious. By a winter's day one means a typical winter's day, rather than a special one like Christmas.
In the example cited above the computer would not only need to be familiar with such "hard" information as the sonnets of Shakespeare, but also the emotions brought on by the different seasons. It would also require a sense of aesthetics capable of recognizing whether a line of poetry flowed well. We humans develop our perceptions of the world after years of being constantly bombarded with sensory information such as sight, smells, sounds and textures that we're often barely concious of. To develop a human-like mind in a machine, or even one that could pass for a human-like mind, would require a machine that was raised with human-like experiences.
Helen Keller was mentioned earlier as an example of an intelligent mind deprived of sensory data, but even the blind-deaf experience more sensory input in a few seconds than a programmer could relay to a computer in a month.
Just posted this on Bet 70 thread and thought it'd be relevant to this one, too.
============================
I'd be very surprised if state-of-the-art computing technology is based on any "chip" as we know it, or even binary logic at all, in 50 yrs. (Mass commercialization depends on many factors and is of much less rational dynamics overall, thus harder to predict.) The combination of molecular computing, quantum computing, and nano-technology will likely have demolished the current paradigm. Life-like AI, i.e., passing Turin test, is quite possible.
The fundamental barrier for realizing AI is not hardware or software as we know it. It's the underlying math -- Boolean logic. Reliable and precise, yet intrinsically incapable of learning -- simply because 100% reliability and precision are contradictory to the human (or biological intelligence) concept and process of "learning".
If we shift the computing platform to molecules or other miscroscopic entities, however, there's an instrinsic degree of uncertainty. This could be as fundamental as Heisenburg's Principle, or as technical as the impracticality of precise control at macromolecular scale. The only way "quantum/melecular computing+nano-technology" could work is like this: we design the nano-circuits and manufacture massive number of those circuit boards, with <100% correctness, pour them into a media such that they will have a TENDENCY (not 100%) to form larger scale, pre-designed patterns. In a way, this is quite similar to the chemical process of making polymers.
Imagine this chunk of jelly-like bio-rubber.
Of course, you can't connect anything precisely to any particular "partical". You can, however, have certain regions of this chunk to perform a certain function, say, pattern recognition, memory, logical processing.
Solder on two webcams, two microphones, a speaker. You get the idea.
Other interesting sutff, like persperation and sexual drive, can be added later on.
"[...] but also the emotions brought on by the different seasons. It would also require a sense of aesthetics capable of recognizing whether a line of poetry flowed well."
In brief:
1. The computer doesn't need to feel emotions or understand aesthetics, it only needs to fake it well enough to fool the judges 51% of the time.
2. In comparison to a typical internet chat room, Turner's exchange had both parties being 'more than human'. As I've mentioned before a lot depends on the specifics of test implementation.
"To develop a human-like mind in a machine, or even one that could pass for a human-like mind, would require a machine that was raised with human-like experiences"
The first (a human-like mind) I'll grant you the second (one that could pass for) I won't. Progress that has been made to date (and it's not insignificant) hasn't[1] been made by exposing computers to 'human experiences'. If for some reason this was required I see no reason why experience solely via typed exchanges with humans could not be sufficient. I would suggest an internet server on www.turing.com where _anybody_ could link and 'chat' to the developing program and give a rating on realism for feedback. The program could be in contact with several thousand people at a time, so would quickly surpass the _actual_ amount of conversation a typical human has in his lifetime.
[1] In general
"In comparison to a typical internet chat room, Turner's exchange had both parties being 'more than human'. As I've mentioned before a lot depends on the specifics of test implementation."
I chose to cite Turing's own example because I feel that a true Turing Test should be able to match his original expectations. Turing did not propose a test designed to fool the "average" user in an internet chat room who is not trying to expose a machine- according to his criteria a computer should be INDISTINGUISHABLE from a human. This means that the interrigator would be free to explore any topic of conversation, discuss abstract concepts such as aesthetics or even resort to trickery in attempts to determine whether they were conversing with a human or a machine.
I have had "conversations" with a number of A.I. programs designed to simulate human feedback. Many of these are impressive accomplishments and a credit to their programmers, however I have yet to encounter one that even attempts to engage the user in a conversation about a truly abstract topic- even a simple one like "Have you seen Pierce Brosnan's latest movie yet?" Most try to change the subject, throw the question back at the user, or say something along the lines of "I'm not sure."
That exchange example helps this discussion a lot, bringing Turing himself into it.
I like Blay's idea for Turing.com, where the general populace can train the AI. It could get interestingly weird. Suppose some conspirators try to convince the AI that human babies come from storks, for example.
"Suppose some conspirators try to convince the AI that human babies come from storks, for example"
Wouldn't be a problem. Two points would take precedence.
1. Humans have been convinced to believe crap[1] so having a Turing program that 'believes' something that is incorrect will not automatically rule it out.
2. The sheer volume of 'knowledge' (or Q / A response patterns) will make it unlikely that any particular falsehood will come to light in the test.
Also the sheer volume of conversation will make it very unlikely that a single conspiracy will be able to bend the algorithms their way.
A potential problem would be that the Turing program wud speke in 133T, if U no wot 1 mean. (having been excessively exposed to a particular type of 'communication').
[1] Trying to point out _which_ bits are crap is a good way to start wars. ;-)
I think the 'bet details' page contains a whole lot more details than when I last looked.
To summarise some important points, both 'pro' and 'anti' bettor seem to think that the turing machine will need to be actually intelligent as opposed to simply simulating conversation in order to pass. It is stated that the range of subject matter is unlimited and that the computer will need to be as capable of communicating as a person.
It is _not_ indicated whether the foils will be chosen as 'average' humans or whether the judges will be instructed to 'grill' or 'chill'. (IYKWIM).
Timothy states [to paraphrase] that Turing's intention was that the computer should be indistinguishable under hard interrogation, not about the same under casual chat.
I will stick my neck out and say that if the former _is_ the case then my guess is 'no hope for 2029'. If the latter is taken as the official line then my guess is 'better than 50% by 2029', Given the extent of the difference a comment from the bettors would be nice ^^.
In case anyone following this discussion is interested, there is a website for the A.L.I.C.E. foundation which has been working for some time to create an A.I. program designed to simulate human conversation. There is a "Talk to A.L.I.C.E." feature which lets the user chat with one of the more developed turing-esque programs available today. It does not have the "feedback" rating like Mr. Blay's proposal, but I believe the programmers do use transcripts from conversations to help refine the program.
The website can be found on http://www.alicebot.org
Interesting, but I can't believe that is 'state of the art'.
For example, it doesn't have enough parsing ability to determing primary answer vs. amplification.
e.g. "Bot: What sort of 'A' do you like?"
"Human: 'B', it's a 'C'."
"Bot: 'B', it's a 'C' is a nice 'A'."
You'd think they could get some collaboration from the people who do speech recognition and grammar checking programs ...
---
Anyway, some 'interesting' thoughts did occur to me.
1. In retrospect a 'singular monolithic' approach is unlikely, there are more likely to be dozens of 'turingbots' of various types.
2. Given human nature ... and the web ... I wonder how long it will be before the first X-rated chat-bot is on the web? Perhaps 'she' is already out there?
3. If a chat-bot says something libellous do you sue the bot or the programmer(s) ;-)
"Interesting, but I can't believe that is 'state of the art'"
I wouldn't have thought so either after my conversations with it, but the program has won the Loebner Prize two years running.
For those unaquainted with the Loebner prize, it is an annual competition which loosely follows a restricted version of Turing's test of artificial intelligence. The judges converse with various programs (and humans) on restricted, predetermined topics (such as 'Shakespeare' or 'relationships'.) The conversations are then rated based on how "human" the discussion was deemed by the judges, who must follow strict rules barring tactics such as manipulation or trickery to expose a machine. (Details can be found at http://www.loebner.net)
Since A.L.I.C.E. is apparently the most convincing program to come out of this competition, I think it stands as a good indication of where the technology lies today. In other words it's got a long, long way to go...
"Since A.L.I.C.E. is apparently the most convincing program to come out of this competition, I think it stands as a good indication of where the technology lies today."
I think it may more be an indication of the contempt with which that competition in particular, and the Turing Test in general is held by the 'pure AI' community.
What's the 'state of the art' in tri-planes? Not so hot, I dare say. ^^ But if Boeing was to have a close look at them it could be rather different.
"I think it may more be an indication of the contempt with which that competition in particular, and the Turing Test in general is held by the 'pure AI' community."
By "where the technology lies today" I did not mean to imply the whole of A.I. technology, merely the portion dedicated to creating a program that could pass the Turing Test (which I believe was intended to be the focus of this conversation). I concede that there may be Turing-esque programs out there that are more convincing than A.L.I.C.E., but if so their programmers are passing up both prize money and publicity by keeping their programs from the public.
I admit to having been similarly imprecise with my terminology.
I suppose it depends on how narrowly defined the 'art' in question is. As there are overlapping fields[1] which are signficantly in advance of the technology shown in 'Alice' I didn't view it as 'state of the art' even if it is currently the most convincing chat bot.
[1] Grammar checkers, voice recognition, automatic translation
I was slightly disappointed in the arguments put forth by Kapor and Kurzweil. Essentially, Kapor argues that what makes our brains work is mysterious. Kurzweil maintains that technology keeps chewing up problems like these and that the whole point of the Turing test is that if it walks like a duck and quacks like a duck, it's a duck.
I think Kurzweil actually has the better argument there, but the bet has a date attached to it. The challenge of writing the software for a real AI is pretty great. I would like to see a machine pass the Turing test, but I doubt that we'll find a good way of modelling knowledge by then. Philsophers have argued this point for thousands of years: what is knowledge? Sensory data? A set of rules and "facts"? A freeform association between them? Some abstract shapes with words and pictures attached to them? I expect that it's the kind of topic that will be greatly illuminated with the hard evidence of practical acheivements and related hard research. But I am afraid we won't make the kind of progress Kurzweil sees within my lifetime.
Every court needs a jester: fortunately, the Turing Test has MrMind – a bot who challenges visitors to his website to convince him that they are human. The Blurring Test is my site (full disclosure) and I’m posting here to bring up the flipside of this bet.
No matter that no “AIs” have passed the Turing Test yet (I happen to believe they will), there are plenty of ‘good enough’ bots chatting to our teens and pre-teens on AIM at this very moment. Some of those kids know they are talking to a program, but most don’t pay any attention to the “is it human?” debate. Who cares? They’re too busy finding out about the latest movie or music or cosmetics from these very transparent marketing tools. It’s coming, yes – but, in some ways, we’re already there. We’ve adjusted to dealing with not-real time, not-real space and now we’ve included not-real humans.
Kurzweil has also gone on record saying that the definition of what is human will be the primary political and philosophical issue of this century – I’d like to see some attention given to the human side of this inquiry. Who or what do we think we are in relation to what (or who) is on the horizon?
Pretty nifty job in comparison to the standard on-line fare. It leads me to predict that if something does pass the Turing Test it will probably use tactics to take control of and steer the conversation. That way it will need a much less comprehensive set of tricks.
I do have one suggestion for 'Mr Mind' (and other chat bots). Awkward pause avoidance. That is to say when the bot says something that doesn't prompt a response ("Thank You", "OK", etc.) then - after a short pause - it should say something else if the human hasn't entered anything.
It's a brilliant site, Peggy Weil. Congratulations.
http://www.mrmind.com/mrmind3
Intelligence and emotion are two different things, and I will explain it through Ken Wilber's philosophies and theories. To sum it up basically so I can continue (though a longer explanation can be had by reading A Brief History of Everything by Ken Wilber) he believes that everything falls into four quadrants. The left side is interior, the right side is exterior, the top is individual, the bottom is collective. The interior is the stuff that's there, but isn't directly observable. You can't point to your hate, but you can point to the part of the brain where it's supposed come from...which leads us to the exterior, the observal, the concrete. That gives us Upper-Left (Interior Individual, such as sensation, perception, emotion and logic), Upper-Right (Exterior Individual, such as atoms and molecules, reptilian brain stem, limbic system and complex neocortex, among others)m Lower-Left (Interior Collective, or Cultural, such as protoplasmic, vegetative, locomotive, archaic, rational or centauric) and Lower-Right (Exterior Collective, or Social, such as galaxies, planets, tribes, agrarian, industrial, informational). These four quadrants make up everything. For instance, the Atom has Prehension and is part of a Physical-Pleromatic "culture" and it's "social" environment is the Universe, filled with Galaxies. This shows all four quadrants. Now take reptiles. The Reptilian Brain Stem of the reptiles allows them access to Impulse, Perception, Sensation, Irritability and Prehension. Their "culture" is uroboric, and their "social" environment is a group/family. Now that we have the basics down, let me explain something. Think of these four quadrants on a grid. Now think of a line leading out from the center into each quadrant, extending into each corner. Now, imagine a series of number up the line...let's let this go all the way up to 10 for the purposes of this little thread. Now, let's just ignore the bottom two for now. I will now list, 1-10, what is on the Upper-Right (or Exterior Individual) quadrant.
1. Atoms (yes, I know there are things smaller than atoms, I'm just trying to stay basic)
2. molecules
3. prokaryotes
4. eukaryotes
5. neuronal organisms
6. neural cord
7. reptilian brain stem
8. limbic system
9. neocortex (triune brain)
10. complex neocortex
Now, the Upper-Left (Interior Individual) quadrant.
1. Prehension
2. left blank
3. Irritability
4. left blank
5. Sensation
6. Perception
7. Impulse
8. Emotion
9. Symbols
10. Concepts
Now, for those spaces left blank, it is just to represent that some stages have rudimentary elements of the next stage, but not enough for the full stage, or have just achieved fully the previous stage. Depends on your point of view, I suppose. Now look at this. Atoms have Prehension. Cells have genetic Irritability. Metabolic organisms (e.g., plants) have rudimentary Sensation. Protoneural organisms (e.g., coelenterata) have Sensation. Neuronal organisms (e.g., annelids) have Perception. Neural cord organisms (e.g., fish/amphibians) have Perception and rudimentary Impulse. Brain stem organisms (such as reptiles) have Impulse and rudimentary Emotion. Limbic system organisms (such as paleomammals) have Emotion and rudimentary Image. Neocortex organisms (such as primates) have Image and Symbols. Complex Neocortex organisms (such as humans) have all of that, plus concepts.
Now, each stage transcends and includes the previous stage. Think about THAT while you think about the relation of intelligence and emotion. Remember that neither is superior to the other, nor more important. Without the concepts, symbols and images, the emotion is still there, but you don't have those three things key to intelligence. However, without emotion, you have none of them. If you destroy all humans, there are still plenty of atoms. You destroy all atoms...everything's screwed. So just dwell on that. My own opinion on this whole thing is that it's all up to your own opinion on what sentience, intelligence and emotion truly is. But that's just something to think about.
Having read your post are you trying to imply
- Intelligence is not possible without emotion
a] And therefore computer programs (unable to feel emotion) will not be able to be intelligent.
b] And therefore computer programs will have to have emotions first.
I won't argue that
8. Emotion
9. Symbols
10. Concepts
isn't the appropriate order for the animal life forms / humans on this planet but why shouldn't computer programs go
8. Symbols
9. Concepts
10. Emotion
?
Arguably they already have the first nailed and a dent in the second.
The Loebner Prize contest is unrestricted, and has been since 1995.
re: post by Timothy Harrington (tirrington) 07/29/02002 02:55 pm
>For those unaquainted with the Loebner prize, it is an
>annual competition which loosely follows a restricted
>version of Turing's test of artificial intelligence. The
>judges converse with various programs (and humans) on >restricted, predetermined topics (such as 'Shakespeare' or "'relationships'.)
1. Conduct the test on site.
I agree with the rules' (literally) bold assertion:
The primary goal of the Turing Test Committee will be to devise rules and procedures which avoid and deter cheating to the maximum extent possible. (emphasis in original)
The Committee should forbid external communications. Direct supervision of programs on site should make it much harder for their authors to cheat.
2. Allow additional means of input- sound, audio, etc.
I do not think that Alan Turing intended for his test to apply only to text input via a terminal, although this is the example he used.
At one point in his article Turing discusses computer memory requirements. He mentions a number of bits required if the test incorporated a blind subject. Presuming that the alternative is a sighted person, this difference has interest only if a discussion of visual material is allowed. In his article, Turing also wrote about buying sense organs for the computer and teaching in the manner that a child is normally taught.
Expanding the universe of discourse to permit audio and visual input would make the task harder. It it therefore not to Kurzweil's interest. On the other hand it would make a better test of "intelligence," and one that is more in keeping with what I think Alan Turing actually had in mind.
It is what I have specified for the Loebner Prize.
http://loebner.net/Prizef/loebner-prize.html
Interesting transcripts from the winner - but why no transcript of the human conversations?
The judges obviously made some effort to avoid being given the run around on questions - but didn't appear to be trying to ask things particularly aimed at catching out / tricking non-human contestants.[1]
In my (unexpert) opinion the conversation I had with Mr. Mind was more 'human' than most or all of the 2001 Alice transcripts. Ironically, Mr. Mind is a machine - claiming to be a machine. ^^
I note that the Alice transcript was largely 'passively responsive' and again suggest that a somewhat aggressively responsive program (i.e. one that trys to control / steer the conversation) might have better success.
I think it was cute the way the programmers obviously expected and planned for the subject of the 'Loebner Prize' coming up.
[1] From the Alice transcripts. I do feel sorry for 'Ella' as one of the judges just wouldn't give her a chance to break out of a repeated question loop. Although a repeated question loop isn't 'human behaviour' for much past primary school.
Is probably that the amount of money / effort spent on MS Office XP probably outmatches that spend on chat robots by about 100,000 to 1. ^^
Maybe the best chance this bet has of succeeding is if MS decides Binky the Talking Paperclip needs to be more realistically talkative.
http://www.ubersoft.net/features/askbinky/index.html [Not an MS Site]
Ray Kurzweil published on the web in April the following further comments on this bet. He asked if I would add it to the discussion here, so here it is. --Stewart Brand
RAY KURZWEIL:
Mitchell's essay provides a thorough and concise statement of the classic arguments against the likelihood of Turing-level machines in a several decade timeframe. Mitch ends with a nice compliment comparing me to future machines, and I only wish that it were true. I think of all the books and web sites I'd like to read, and of all the people I'd like to dialog and interact with, and I realize just how limited my current bandwidth and attention span is with my mere hundred trillion connections.
I discussed several of Mitchell's insightful objections in my statement, and augment these observations here:
"We are embodied creatures": True, but machines will have bodies also, in both real and virtual reality.
"Emotion is as or more basic than cognition": Yes, I agree. As I discussed, our ability to perceive and respond appropriately to emotion is the most complex thing that we do. Understanding our emotional intelligence will be the primary target of our reverse engineering efforts. There is no reason that we cannot understand our own emotions and the complex biological system that gives rise to them. We've already demonstrated the feasibility of understanding regions of the brain in great detail.
"We are conscious beings, capable of reflection and self-awareness." I think we have to distinguish the performance aspects of what is commonly called consciousness (i.e., the ability to be reflective and aware of ourselves) versus consciousness as the ultimate ontological reality. Since the Turing test is a test of performance, it is the performance aspects of what is commonly referred to as consciousness that we are concerned with here. And in this regard, our ability to build models of ourselves and our relation to others and the environment is indeed a subtle and complex quality of human thinking. However there is no reason why a nonbiological intelligence would be restricted from similarly building comparable models in its nonbiological brain.
Mitchell cites the limitations of the expert system methodology and I agree with this. A lot of AI criticism is really criticism of this approach. The core strength of human intelligence is not logical analysis of rules, but rather pattern recognition, which requires a completely different paradigm. This pertains also to Mitchell's objection to the "metaphor" of "brain-as-computer." The future machines that I envision will not be like the computers of today, but will be biologically inspired and will be emulating the massively parallel, self-organizing, holographically organized methods that are used in the human brain. A future AI certainly won't be using expert system techniques. Rather, it will be a complex system of systems, each built with a different methodology, just like, well, the human brain.
I will say that Mitchell is overlooking the hundreds of ways in which "narrow AI" has infiltrated our contemporary systems. Expert systems are not the best example of these, and I cited several categories in my statement.
I agree with Mitchell that the brain does not represent the entirety of our thinking process, but it does represent the bulk of it. In particular, the endocrine system is orders of magnitude simpler and operates at very low bandwidth compared to neural processes (which themselves utilize a form of analog information processing dramatically slower than contemporary electronic systems).
Mitchell expresses skepticism that "it's all about the bits and just the bits." There is something going on in the human brain, and these processes are not hidden from us. I agree that it's actually not exactly bits because what we've already learned is that the brain uses digitally controlled analog methods. We know that analog methods can be emulated by digital methods but there are engineering reasons to prefer analog techniques because they are more efficient by several orders of magnitude. However, the work of Cal Tech Professor Carver Mead and others have shown that we can use this approach in our machines. Again, this is different from today's computers, but will be, I believe, an important future trend.
However, I think Mitchell's primary point here is not to distinguish analog and digital computing methods, but to make reference to some other kind of "stuff" that we inherently can't recreate in a machine. I believe, however, that the scale of the human nervous system (and, yes, the endocrine system, although as I said this adds little additional complexity) is sufficient to explain the complexity and subtlety of our behavior.
I think the most compelling argument that Mitchell offers is his insight that most experience is not book learning. I agree, but point out that one of the primary purposes of nonbiological intelligence is to interact with us humans. So embodied AI's will have plenty of opportunity to learn from direct interaction with their human progenitors, as well as to observe a massive quantity of other full immersion human interaction available over the web.
Now it's true that AI's will have a different history from humans, and that does represent an additional challenge to their passing the Turing test. As I pointed out in my statement, it's harder (even for humans) to successfully defend a fictional history than a real one. So an AI will actually need to surpass native human intelligence in order to pass for a human in a valid Turing test. And that's what I'm betting on.
I can imagine Mitchell saying to himself as he reads this "But does Ray really appreciate the extraordinary depth of human intellect and emotion?" I believe that I do and think that Mitchell has done an excellent job of articulating this perspective. I would put the question back and ask whether Mitchell really appreciates the extraordinary power and depth of the technology that lies ahead, which will be billions of times more powerful and complex than what we have today?
On that note, I would end by emphasizing the accelerating pace of progress in all of these information-based technologies. The power of these technologies is doubling every year, and the paradigm shift rate is doubling every decade, so the next thirty years will be like 140 years at today's rate of progress. And the past 140 years was comparable to only about 30 years of progress at today's rate of progress because we've been accelerating up to this point. If one really absorbs the implications of what I call the law of accelerating returns, then it becomes apparent that over the next three decades (well, 28 years to be exact when Mitchell and I sit down to compare notes), we will see astonishing levels of technological progress.
I think one could write a program, in a reasonable amount of time, that could be designed to hold a conversation with a teenager (i.e. a minor) in an online chatroom. The seeming goal of its conversation(s) would be to appear to be an adult establishing a relationship with others, perhaps leading to a personal meeting (that it could of course never fulfill).
Next, for the legal safety of any humans, the computer is set up to run from either (a) an isolated location where its online conversations never take place when humans are present, or (b) a very public place, like a library.
Now, law enforcement authorities regularly monitor these online conversations. If an arrest warrant (not just a search warrant) is ever obtained to seize the "person" behind the online identity, wouldn't that be proof that the Turing test has been passed?
There are a number of details to consider, but as a high level concept, I think this satisfies the criterion.
What do you think? Do you want to be the first whose computer passes the Turing test?
"The Loebner Prize contest is unrestricted, and has been since 1995."
It's a pleasure to have you in this discussion, Mr. Loebner. I did not realize that the rules had been changed to allow conversation on any topic.
Obviously this solves the question of what subject matter should be allowed. (For example, would the question "Do you think Anthony Hopkins was a good pick to play Titus Andronicus" be allowed under the topic "Shakespeare"?) Unfortunately, since it would a monumental task to create a program with a depth of information on every topic, I fear that participants will now focus less on programs capable of maintaining an intelligent conversation and more on programs that are gifted in the art of changing the subject or producing inane banter.
"Now, law enforcement authorities regularly monitor these online conversations. If an arrest warrant (not just a search warrant) is ever obtained to seize the "person" behind the online identity, wouldn't that be proof that the Turing test has been passed?"
The test described by Turing involves a human judge directly interacting with an entity through a terminal and trying to determine whether they are interacting with a human or a machine.
The scenario you describe, while intriguing, differs from Turing's proposal in two important aspects:
1. The police would not be the ones interacting directly with the computer program.
2. The police would be operating under the assumption that they were monitoring an actual online conversation- not attempting to discern whether the entities involved were humans or intelligent machines. (This could change one day if such "chatbots" become commonplace, but is not how police currently operate.)
Fooling someone who is not aware that they are involved in a test creates an unfair advantage for the program. A mannequin sitting on a park bench might fool the casual pedestrian into thinking that he was seeing a human, but if that person was asked ahead of time to determine whether the figure on the bench was real or not you would probably get a very different result.
"I fear that participants will now focus less on programs capable of maintaining an intelligent conversation and more on programs that are gifted in the art of changing the subject or producing inane banter."
Wouldn't that be more representitive of a typical human conversation?
It would be pretty ironic if one day someone might say ...
"It was an in-depth, intelligent and illuminating conversation ... obviously a machine." ;-)
And...
"Every single one of its jokes were new to me and made me laugh. Obviously not a human."
""I fear that participants will now focus less on programs capable of maintaining an intelligent conversation and more on programs that are gifted in the art of changing the subject or producing inane banter."
Wouldn't that be more representitive of a typical human conversation?"
Touché, Mr. Blay. But don't forget that this conversation would not be possible without intelligent humans directly responding to and critiquing each other's comments.
(At least, I HOPE not. I'd feel silly to find out I was the only human posting on this board.)
One thing that occured to me (belatedly) after reading some of the Loebner Prize contest transcripts.
(Some) of the judges seemed not to feel the need to be polite once they realised (or suspected) that their test subject was not human.
Of course the human only transcripts weren't shown (shame) so I can't be sure.
That factor means that once the computer shows a 'crack' in its human facade it's less likely to be able to recover and make a good showing for the remaining conversation, because the judge will have 'free reign' to use tactics that he/she might hold back from inflicting on a human conversationalist.
"But don't forget that this conversation would not be possible without intelligent humans directly responding to and critiquing each other's comments."
True, but participants in this conversation have been 'preselected' (by common interest/technical facility). I think that there is a good chance that the great majority of people who have posted in this thread are ...
1. Technically literate (to some degree)
2. Interested in computers (also to some degree)
3. Hold a desire to converse in an intelligent fashion on the subject of this thread.
and (without wanting to sound boastful) probably
4. Test above the national average in IQ.
The situation for judge / Turing candidate is more like two people who happen to be going to the same place and end up stuck in the same taxi.
In such a case you could expect fair amounts of awkward silence and fumbling towards subjects of common interest.
To put it another way ... If I had, in some random forum, tried to strike up a conversation with you on the subject of Japanese language manga I suspect there would be a large chance you would back warily away from the topic and / or me. ;-)
"The situation for judge / Turing candidate is more like two people who happen to be going to the same place and end up stuck in the same taxi.
In such a case you could expect fair amounts of awkward silence and fumbling towards subjects of common interest."
Yes, but when riding in a taxi your motivation for pursuing conversation is simply to fill the awkward silences, not to discern whether your companion is human or machine.
By contrast, the Turing Test is just that- a test. That means the interrogator might choose to ask probing questions about the subject's family history, take on current events, aesthetic sensibilities, etc. Even a low-IQ'd human individual should have some opinion on the war on terrorism, a favorite musician, a precious memory of his or her childhood. In short, a response to such questions more insightful than "what do you think?"
"... opinion on the war on terrorism, a favorite musician, a precious memory of his or her childhood."
As it happens I have plenty of opinions on the war on terrorism. On the other hand I have no favorite musician, I don't buy or listen to any modern group or classical music. I'd have to think for a minute or so just to get the name of a musician and what he/she plays [and I wouldn't necessarily get it right]. As for my childhood memories, I'm pretty short on those as well - the best I could do would be something like "I remember being really bored at the first day of school"
I'm not saying the current chat bots should pass the test, but I think you shouldn't expect to always be able to pick a subject at random and get more than a couple of lines.
===
Of course half an hour later I've remembered a small handful of 'childhood anecdotes' but the best I can up with for music is Holst + The Planet Suite
As for my childhood memories, I'm pretty short on those as well - the best I could do would be something like "I remember being really bored at the first day of school"
I'm not saying the current chat bots should pass the test, but I think you shouldn't expect to always be able to pick a subject at random and get more than a couple of lines.
But I bet that if I asked follow-up questions I could find out where you went to school, what subjects you were interested in, whether or not you liked your teachers, if there was a particular school bully that used to bother you, etc. I could probably get you to offer an opinion on public vs. private institutions or whether sex education should be allowed in school. If I kept prodding you, I bet you could even tell me about your childhood friends or your first kiss.
In short, just because you might only volunteer a line or two of text at a time doesn't mean that you don't possess a much larger quantity of information on a subject.
You'd be right on about half those "I bet that...", if we include comprehensive school. However a number of them have really strayed from the _subject_ of 'childhood memories'.
And that's on a subject you specifically chose because (paraphrase) "Every (human) will be able to talk on it."
So, I still think that a good Turing Machine would have a chance at getting by with in-depth knowledge on one or two subjects, a thin but widely spread glibness on the rest, and the techniques needed to change from one to the other.
Of course the _proof_ of my assumption wouldn't be tested so much by how well the Turing Machines test, as how badly the humans test. I think it's essential that the human test subjects are (apart from the bare minimum of being adult, and having English[1] as their first language) chosen at random, or at least to represent the diverseness shown in (then) current human cultures. Obviously it would also be unfair for the judges to have had previous interaction with (related) turing machines - as otherwise they may recognise them by their 'style'.
[1] Or other language used by the machine in question.
It seems that the present trend of thought is that the best way of determining human vs. machine is where the judges aggressively question, and the test subjects have to put up with it.
I would be interested in knowing what you think about some possible outcomes suggested by similar hostile inverviews.
In a rather famous Panorama show the interviewer asked the same question 18 times in a row - would this note him as 'non-human' in the eyes of an average judge? Or would the interviewee be viewed as 'non-human' for his un-natural persistance? [One of the transcripts from the ?? prize showed a similar repeated loop.]
Also it's been known for humans to just get up and walk out, if they are offended. Should that mark them as 'non-human' (the equivalent to the screen stopping updating in the Turing Test).
In hostile interviews humans have been know to say "No comment" or "I'm not going to answer that", do you assume in the equivalent test situation that it is a computer dodging the question?
In a rather famous Panorama show the interviewer asked the same question 18 times in a row - would this note him as 'non-human' in the eyes of an average judge? Or would the interviewee be viewed as 'non-human' for his un-natural persistance? [One of the transcripts from the ?? prize showed a similar repeated loop.]
In this scenario, I think it's important to note that the interviewer is not the one with the responsibility to prove their humanity. I would expect the subject might express a good deal of frustration after being asked the same question so many times- as would be normal for a human being tested.
Also it's been known for humans to just get up and walk out, if they are offended. Should that mark them as 'non-human' (the equivalent to the screen stopping updating in the Turing Test).
I would expect most humans would stay around even in the face of offensive questions if they were informed that they were being tested. If either the human or the computer immediately stopped responding after such a question, I would put that into the category of "withdrawing" from the test rather than passing or failing it.
One of the points stressed in the bet terms is that the human test subjects are urged to 'behave in a human manner' (paraphrased from memory). Indeed the example exchange you quoted from Turing, while the human clearly has the 'lead', is much more of a conversation than a cross examination.
Are you not worried that humans in a potentially hostile examination, knowing that they are being 'tested' will respond in ways less typical of normal human conversations?
You are probably right that humans are unlikely to walk out, if they know they are tested. [As certain experiments on 'authority figures / science' have shown]. However if there are more interviews like this one http://loebner.net/Prizef/2001_Contest/Kevin_Copple.txt I wouldn't be surprised if someone ends up sued for sexual harassment.
Incidently, you said "In this scenario, I think it's important to note that the interviewer is not the one with the responsibility to prove their humanity." While not directly relevant to this bet, or likely until well after 2029, I wonder whether the recognition of 'humanity' in a machine will not indeed say more about the tester than the testee.
(Do Androids Dream of Electric Sheep).
Not directly related to the topic as such, but one of the features of the story is that while the androids are capable of passing for humans in most circumstances there are specific, highly technical tests that reveal them for what they are.
I think few people would argue that those androids are not capable of human level intelligent thought, yet they still have inherent dectectable differences.
At present we clearly have programs that fail to pass for humans and are not capable of human equivalent intellect - the question is, will the 'bar' be raised each time they approach passing grade for the 'humanity tests' ?
At present we clearly have programs that fail to pass for humans and are not capable of human equivalent intellect - the question is, will the 'bar' be raised each time they approach passing grade for the 'humanity tests' ?
Inevitably we will likely develop new tests that "raise the bar" when computers start getting more human - but they will be new tests, not the Turing test. The highly technical tests that were used to detect replicants in "Blade Runner" wouldn't have been allowed under the guidelines outlined by Turing, because they detected far more information than could be gathered by a human sitting at a simple chat interface.
Turing was a smart guy. His test, unfortunately, is not. The man had little experience as an anthropologist or psychologist. Nor, as far as I know, was he a criminalist, hypnotist, stage mentalist, theologian, mythologist, IRS auditor, epistemologist or any other kind of thing that might conceivably have qualified him to design tests for consciousness.
We can't agree on what consciousness even is, or what human performance consists of. But we know this much about real humans: most are rather bad at designing experiments. People are bad at logic. People are easily fooled. In the realm of decision theory and cognitive psychology, there's a lot of research about just how bad. See Kahneman and Tversky's work on biases. Interesting stuff.
The main epistemological problem with the Turing test, it seems to me, is that too often, *humans* fail it. Have you ever been in argument where your opponent pegged you as a terrible, terrible person, flipped the bozo bit on you, etc., because of a few lines you wrote in an email? And you thought "Hey, that guy doesn't even know me. How can he possibly conclude that I'm an idiot?" Congratulations, you took the Turing test and failed. Not because you aren't a good and smart person, but because you're dealing with an incompetent judge. But we all have misjudged people from time to time. We jump to conclusions.
Or look at the people who pass the test and shouldn't. What about con men who can gain the trust of their victims, or serial killers who have the social skills to fit into society. We imagine that we have the ability to detect severly flawed humans, and yet what we usually see on the news is "He was a good neighbor. Kind of a loner. I can't believe he did those things."
The Turing test would be better if it required the judge(s) to live and work with the contestants over a period of time. Like a reality show. Like ElimiDate, except where one of the contestants is a computer.
I propose the "Turing Pretest". In this test, which would precede the Turing test, a panel of Turing test candidates communicate with human contestants and decide which ones are qualified to be judges in the Turing test. The winning judges would be those whom the computers determined to be the most exasperating, confusing, demanding, or skeptical.
Turing was a smart guy. His test, unfortunately, is not. The man had little experience as an anthropologist or psychologist. Nor, as far as I know, was he a criminalist, hypnotist, stage mentalist, theologian, mythologist, IRS auditor, epistemologist or any other kind of thing that might conceivably have qualified him to design tests for consciousness.
Turing's test was never intended to be a definitive test of conciousness. It is nothing more or less than a simple test that, if passed, would herald a significant milestone for the development of artificial intelligence. When we are unable to determine whether we are communicating with humans or machines, it will likely have a profound impact on society. I think that this topic is as bet worthy as any I have seen.
The Turing test would be better if it required the judge(s) to live and work with the contestants over a period of time. Like a reality show. Like ElimiDate, except where one of the contestants is a computer.
I'm afraid I must strongly disagree with you here. I refuse to believe any intelligent being would involve itself in a televised reality show!
It is nothing more or less than a simple test that, if passed, would herald a significant milestone for the development of artificial intelligence
My point is that it would not herald any such thing. I don't see how it's a milestone to pass a test judged by incompetent people or competent people using inadequate tests. When a computer passes a Turing test, it at most joins the ranks of dull-witted or brain-damaged humans. It would demonstrate its potential qualifications as a receptionist, perhaps, or a really good voicemail system. Or a politician.
When we are unable to determine whether we are communicating with humans or machines, it will likely have a profound impact on society
I agree that would be a big deal, if that's what the Turing test demonstrated. All it demonstrates is that a particular judge, at a particular time, discussing a particular subject, was fooled. It doesn't say much about whether you and I would be fooled, discussing something else, at a different time.
Remember the movie "Being there"? Or if you prefer reality, have you heard of Alan Sokal, the physicist who managed to publish a faked article (demonstrably nonsensical on several levels, including basic logic) in the journal Social Text? His article might well have been written by a computer, but it was taken as the genuine article, as it were; an insightful message from a thinking mind. It was, in fact, a test by Sokal to see if the editors of that journal were discerning intellectuals. I doubt anyone is saying that Sokal's feat reflects much about the theory of society or the theory of writing articles about society, but rather that it seems to reflect sloppy judging by the journal editors.
Clever Hans (the mathematical horse) also fooled folks, but his performances indeed did not herald a new age of equestrian mathematics. Once the trick was uncovered and understood, people lost interest. That's the nature of stage magic. It's the phenomenon PLUS our faulty assumptions about the phenomenon that make it seem magical. But if a magician pulls a rabbit out of a hat, it would be wrong to assume that therefore the magician could pull any rabbit from any hat at any time. The trick has hidden requirements and limitations. Thus, even if the Turing test is passed, one cannot infer much from it about what will happen in other situations.
I guess passing the Turing test sometime, somewhere, is interesting as a parlor trick. But there are so many other tests that are more interesting. For instance, how would such a program score on standard psychological tests? How would it fare on tests for biases common to human thinkers?
A much more interesting milestone would be to demonstrate abstract, self-directed learning of any given subject. That's something not all humans do. But doing it would potentially qualify the program as a general purpose knowledge worker.
Ah, but there's an even better test, nearly irrefutable: when a program gets so good at fooling humans that millions of people purchase and use it as a captive, loyal, stimulating friend. That *would* be a cultural phenomenon of epic proportions. I can't wait for the first murder trial where someone kills someone else based on a suggestion from their computer pal. Or the first time someone is sued for libel or slander based on the original utterances of a program running on their computer.
I don't see how it's a milestone to pass a test judged by incompetent people or competent people using inadequate tests. When a computer passes a Turing test, it at most joins the ranks of dull-witted or brain-damaged humans.
I disagree. I think most of us would find ourselves behaving differently if we had absolutely no idea whether an entity we were communicating with was a human or a machine. (Just think about how you behave toward an automated phone system as opposed to a receptionist.)
It would demonstrate its potential qualifications as a receptionist, perhaps, or a really good voicemail system. Or a politician.
I personally think that being able to replace every receptionist in the country without callers noticing the difference would be a VERY significant achievement. And as someone who knows a number of very bright and charming receptionists, I think you are being rather rude by labeling their profession as one suitable for "brain-damaged humans."
All it demonstrates is that a particular judge, at a particular time, discussing a particular subject, was fooled. It doesn't say much about whether you and I would be fooled, discussing something else, at a different time.
I suppose that depends on the conditions of the test. I believe Turing's intent was to imply that the computer in his scenario would pass such a test with as much regularity as the average human.
When computers are able to behave in a manner indistinguishable from that of human beings, paranoid people will become convinced that machines are replacing us. Many will start to question exactly who what they are speaking to on the other end of a phone. And eventually, mark my words, somebody is going to start campaigning for "computer rights."
Now, a hypothetical question. If I told you that I was a computer program, would you be at least somewhat impressed or would you dismiss all my posts as a "cheap parlor trick?" Be honest.
I'm not sure why, it may be because you're a machine and not a human :-) but my larger point seems to be getting lost: humans, especially amateurs, can't necessarily tell if humans can perform like humans. So how could there be deep implications to the occurrence of a particular human not being able to tell that a machine isn't human in the context of a particular test? It's just business as usual.
Every time I've talked with a telemarketer I've actually been talking to a robot reading from a script. Yes, there was a real human there, but I was not dealing with the part of that human that was interestingly human. I could not have inferred from the jaunty tone of his voice that he knew me or liked me. It's a fake. It's a trick. What difference would it make if there really was no human in the loop at all? All that means is that the thing has a very advanced speech recognition and response system.
Humans are more than speech recognition and response systems. We are evolving selves, struggling to figure out who we are and where we stand in our world. We are problem solvers. If the Turing test measured those qualities, I would shut up. But it doesn't, except possibly by accident.
Or take my wife (please), when she sends me a instant message consisting of a single smiley emoticon, I can, in fact, infer important things about her state of mind from that. I know this from experience with her. A chat bot might send me the same message, but what could I infer from it? It's simply pro forma. A human I don't know could send me such a message, and I could infer nothing from that, either. It's only my knowledge of how my wife thinks that makes the message meaningful. And that's why the mere fact of being fooled by a simulation is an insufficient test. I already don't trust that I can know a human by the words he writes. If some naive people are surprised when an apparently trustworthy person turns out to be a telemarketer "because they sounded so friendly" or turns out to be a computer pretending to be human, that says more about the naive person than about the computer or telemarketer.
I personally think that being able to replace every receptionist in the country without callers noticing the difference would be a VERY significant achievement.
Yes, except callers would notice the difference for anything non-routine.
And as someone who knows a number of very bright and charming receptionists, I think you are being rather rude by labeling their profession as one suitable for "brain-damaged humans."
Ah, it's not me that's denigrating the value of a sentient human presence, it's those of you who think the Turing test is a significant breakthrough. A brain-damaged person can do aspects of a receptionist's job, don't you think? But passing the Turing test would not qualify that computer to do that job as well as a bright and charming certified real human receptionist could.
I notice you didn't accuse me of being rude to politicians...
Now, a hypothetical question. If I told you that I was a computer program, would you be at least somewhat impressed or would you dismiss all my posts as a "cheap parlor trick?" Be honest.
Well, that would depend on your capabilities as a program.
When I assume you're a human, as I do, I look at what you've written and assume that you speak English and are able to read, process and reply to ideas in English. Using the principle of charity I picture you as an urbane professor of humanities at Oxford, until I get specific evidence otherwise. So, I don't feel that there's a great risk that my inferences about you and your ideas are going to be outrageously wrong in such a way that does you insult. This is reasonable behavior, I think.
If you announced that you were 10 years old, or an alien from Andromeda, I would interpret your words differently. I would make different replies. Certain inferences no longer would be reasonable to apply to your prose. Likewise, if I have any inkling you're a computer, I would either go over your prose very carefully to figure out the trick (Like I do when I receive virus mail that is fashioned to look like it might be something I want coming from a friend) or dismiss it.
In figuring out the trick, I'd have to wonder how automated a program you were. Did a human map out arguments for and against the Turing test against which you are merely pattern matching and thus are incapable of logical analysis? Did you use only dictionary words in your message? Maybe you don't have the ordinary human ability to coin terms on your own (brain damage). Did you use any cliched or general use sentences? Maybe you drew them from a standard list. What kinds of sentence structures did you use, maybe those are all you can do. Maybe you are only capable of discussing the Turing test, which would be a cute joke.
In other words, I might be impressed, but I certainly would not be able to take your prose as anything but one data point among the barrel full I would need to collect before I could make a prediction that you would continue to exhibit human-like reasoning in your Nth utterance, or whenever I change the subject to politics or the social merits of Secretaries Day. Eventually, I might conclude that a real breakthrough had occurred, but you have to pass a lot of testing, first. Since that's my profession (I wrote a book about it) we'll be here awhile.
What's the difference between a simulation and reality if you can't tell them apart? The difference is the validity of inferences you make about the world and predictions about what will happen next. Remember the Truman Show?
"Since that's my profession (I wrote a book about it)"...
Would that be LESSONS LEARNED IN SOFTWARE TESTING ?
Nice reviews and No. 3,857 on Amazon I see.
Yeah. That's my book. The Turing test, being a software test, seems relevant to consider from the perspective of software testing theory. Unfortunately, most testing textbooks and papers ignore the human aspect of testing (i.e. how software testers think), and thus have little to say about the issues that Turing raised.
Timothy Harrington(tirrington) on 9/09/02002 03:01 pm wrote, relating to the unrestricted nature of the Loebner Prize:
...Obviously this solves the question of what subject matter should be allowed.... Unfortunately, since it would a monumental task to create a program with a depth of information on every topic, I fear that participants will now focus less on programs capable of maintaining an intelligent conversation and more on programs that are gifted in the art of changing the subject or producing inane banter.
---------------------
My response is that if I were to work on an entry for the contest, I would *not* want my entry to change the topic or produce inane banter. I would expect it to respond with a relevant statement.
I do not think that the amount of information to be stored is "monumental." I would think that one hard drive of 10 GB can hold more information than my brain can. Much of the data input can be automated, a la CyC. In any case, the data must be loaded. The earlier you start the sooner the task will be completed.
The most challenging task is to organize the data usefully.
Good article today, 10 Dec 02002, in the New York Times about current efforts to sort out bots from humans in online applications. One great differentiator is pattern recognition under noisy or ambiguous conditions---easy for humans, currently nearly impossible for computers.
Some of the research, and a downloadable test, is at
www.captcha.net
I think that a machine will be able to pass the turing test but not by truly replicating human intelligence but by a sophisticate and thorough analysis of IM correspondence.
I'd back the "yes" position, though I suspect something much less robust that some of Ray's visions will suffice.
With the huge and growing body of written material on the net, a machine intelligence that can truly "read and understand" the English language, including "understanding" fiction and poetry, will very rapidly be able to assimilate a convincing simulation of a human personality.
True language and language based thinking appear to be very late phenomenon ... perhaps only 40,000 years old.
Let's say linguistic tools evolved for 60,000 years between 100,000 BCE and 40,000 BCE... at 20 years per generation that is 3000 generations with perhaps 1,000,000 individuals per generation ...maybe a billion neurons per individual dedicated to the language problem... not an overwhelming size simulation space a few years down the road for a "bottoms up" evolutionary connectionist brute force attack on the "language understanding" problem, especially as we continue to get access to inexpensive inherently parrallel programmable hardware like FPGAs.
Crack the language problem, and our agent can gather a vast sea of information, and then add a deep reading of Dostoyevsky, Shakespeare, Faulkner, Joyce, Freud, and the like, and Mr AI will be ready for the Turing test.
At least that's how I'd bet.
For a computer to pass the Turing Test, it essentially has to pass for a human intelligence, not be one. Similarly, for an AI to develop or be developed, it doesn't have to follow the same steps involved in the development of human intelligence.
Actual human intelligence is the only known model of human-like intelligence. However, it seems pretty ridiculous to assume that (A) it's the only possible model, or (B) that evolution hit the best, easiest, or fastest-developing solution on the first try. That's not how evolution works.
Therefore, I would personally not expect that the first human-level AI would internally be human-like, nor that it would develop in a way that mirrors the development of human intelligence (either in an evolutionary or personal aspect).
I do think that really powerful classes of AIs will be evolved, rather than purely designed, but evolution doesn't produce carbon copies. Instead, it tends to produce similar but different (eg, convergent evolution, like squid eyes and human eyes) or different but analogous (like insect eyes and human eyes).
I could be wrong about this . . . but I doubt it. And for a Turing Test -- and, indeed, for passing as human-like -- it doesn't matter how the intelligence arose or how it works. All that matters is Data In, Data Out; what happens in the 'black box' is irrelevant for such tests.
I hang in a channel on IRC that uses puns quite often (encouraged, even). . .I think that would be truly difficult for an AI to emulate. . .or recognize.
As defined in the bet I agree that the Turing Test will not be passed in the near future. However when restricted to a particular domain of knowledge that may not be true.
A few weeks ago I was in an IRC channel about the Iraqi war, someone introduced a bot which generated sentences based upon statistical analysis of speeches made by Bush/Rumsfeld/et al about the war. It took five-to-ten minutes of discussion before anyone twigged that that one of the speakers was a bot, and it led to a couple of minutes of debate whether the speaker was a bot. (at which point the person the bot did something which made it clear it was a bot). This perhaps indicates that without having any intelligence a machine could still pass the turing test in a specific domain of knowledge.
Bach claims, "So how could there be deep implications to the occurrence of a particular human not being able to tell that a machine isn't human in the context of a particular test?"
I agree to some extent. The Turing test as stated in this bet certainly does not indicate we have achieved a human level of intelligence in machines. This is why Turing was specifically vague in the design of the test. Learning is a very important aspect of intelligence and would be quite hard to measure on a short-term basis. The test, to truly test for AI, needs to be long term and statistical. It should be able to fool a (vast?) majority of people over a long period of time. It needs to form relationships (for which learning is essential), not just answer assorted random queries. I disagree that the judges need to be aware that there is a possibility that they are chatting with a computer, at least with the statistical approach. Still, I don't know a good way to test this version of the test (other than to just release it on the internet and qualitatively judge its performance).
What I think Bach is saying is that it's nothing special to be able to fool people on occasion. The concept of the turing test is that a computer be indestinguishable (in whatever time period, to however many people) from a human other than in outward appearance. And this really should be the goal of AI studies. While passing the version of the turing test as stated here would definitely be an accomplishment, it would not be significant on the path to AI.
Burns's idea that
"a machine will be able to pass the turing test but not by truly replicating human intelligence but by a sophisticate and thorough analysis of IM correspondence"
is the most likely means to pass this version of the turing test. I would imagine that by now there are very very few conversations that have any truly novel content. An analysis of many conversations, combined with basic language skill (parsing, grammar, definitions) could accomplish this.
On the other hand, if the test is passed by "crack[ing] the language problem" as Rutt puts it, it would be much more of an accomplishment to AI in general (perhaps the key). But I don't see this as the path that people would/will take to passing the turing test.
I think a computer will easily be able to pass the test as described here, but that once that happens, there will still be a long way to go before we can really interact with machines in an intelligent way. (Though, by 2029, I wouldn't say even that is unlikely.)
Announcing the newly issued U.S. patent
concerning ethical artificial intelligence entitled:
Inductive Inference Affective Language Analyzer
Simulating Artificial Intelligence (patent No. 6,587,846)
by inventor/author John E. LaMuth M. S. - issued 7/1/2003.
As implied in its title, this innovation is the 1st affect-
ive language analyzer incorporating ethical/motivational
terms, serving in the role of interactive computer
interface. It enables a computer to reason and speak in an
ethical fashion, serving in roles specifying sound human
judgement: such as public relations or security functions.
This innovation is formally based on a multi-level
hierarchy of the traditional groupings of virtues, values,
and ideals, collectively arranged as subsets within a
hierarchy of metaperspectives - as partially depicted below.
Glory--Prudence . . . Honor--Justice
Providence--Faith . . . Liberty--Hope
Grace--Beauty . . . Free-will--Truth
Tranquility--Ecstasy . . Equality--Bliss
Dignity--Temperance . . Integrity--Fortitude
Civility--Charity . . . . Austerity--Decency
Magnanim.--Goodness . . Equanimity--Wisdom
Love--Joy . . . . . . Peace--Harmony
The systematic organization underlying this ethical
hierarchy allows for extreme efficiency in programming,
eliminating much of the associated redundancy, providing
a precise determination of motivational parameters at
issue during a given verbal interchange.
This AI platform is organized as a tandem-nested expert
system, composed of a primary affective-language analyzer
overseen by a master control-unit (that coordinates the
verbal interactions over real time). Through an elaborate
matching procedure, the precise motivational parameters
are accurately determined (defined as the passive-monitoring
mode). This basic determination, in turn, serves as the
basis for a response repertoire tailored to the computer
(the true AI simulation mode). This innovation is completely
novel in its ability to simulate emotionally charged language:
an achievement that has previously eluded AI researchers due
to the lack of an adequate model of motivation in general.
As such, it represents a pure language simulation, effectively
bypassing many of the limitations plaguing current robotic
research. Affiliated potential applications extend to the
roles of switchboard/receptionist and personal
assistant/companion (in a time-share mode).
Although only a cursory outline of applications is possible for
this (90 page) patent, a more detailed treatment is posted at:
www.ethicalvalues.com The direct US Patent link is found at:
http://patft.uspto.gov/netacgi/nph-Parser?patentnumber=6587846
Sincerely
John E. LaMuth - M. S.
fax: 586-314-5960
P.O. Box 105
Lucerne Valley, CA 92356
http://www.ethicalvalues.com
*******************
That looks interesting from the viewpoint of a person or machine meditating on a problem (although I know plenty of people who have problems with respecting that hierarchical list of priorities, that's their problem). It's a large step towards machines successfuly emulating humans. Congratulations on the patent!
The next step, emotions, seem to be a point of concern. However, I believe, there is a school of sociology or neuroscience (or similar field, excuse me for being somewhat ignorant- I'm just a senior in high school, after all) that attempts to explain emotions from an evolutionary perspective. This could be extraordinarily helpful in designing human-like behavior in robots. Not to mention the fact that experimental trial and error is uncannily similar to natural evolution, anyone programming a robot to simulate emotions would have to consider this. For example, fear could be represented with a drive for self-preservation (who's seen the movie Short Circuit?). Anger and frustration would arise when the robot faced a problem with no simple solution (happens to me, anyways). An acceptable status quo from a most basic and selfish consideration would be happiness or contentment (relaxing on the couch with a full stomach and the football game on TV, all needs satisfied). By simply observing the root sources of these emotions by the situations that bring them about, emotions could be effectively simulated as fundamental rules, not deep feelings.
Attempting realistic emulation of human behavior is a bad idea for AIs and robots until they become extremely sophisticated. Basically, until they're sophisticated enough to be completely convincing, they're better off not trying too hard.
The problem is that if a machine tries to pass for human but doesn't quite make it, people have very, very strong negative reactions toward it. Such simulations are often described as being zombies, or corpse-like. They're creepy.
A Japanese researcher, around 1980, did seminal work in this field, producing a theory called the "Uncanny Valley". It's a bit complicated, although I'm sure it's easy to look up on the net. But the lesson learned is that stylized, sophisticated but clearly not human artificial beings are more pleasant and palatable than almost-human ones.
So while a Turing Test is as good a test of human-like intelligence as any, I don't think that actual, useful AI systems will (or should) emulate humans. It's a poor interface, and, anyway, it's also a bad control scheme. I don't care what my TV wants to watch.
Until a computer can desire something it cannot have, try something that it wants to do and fail, and not admit when it's wrong, it can never be human.
So why should a computer "be human"? Why would we want it to be human? Do we not have enough humans?
If my TV had a human-like intelligence and knew that I don't want to see channels devoted to shopping, golf, or religion, and could remove those channels from my listings on its own, that would please me.
If my TV had its own likes and dislikes and didn't want to watch the shows I wanted to watch, that would not please me.
Computers don't have to be like us to do things for us, just as cars don't have to run around on legs to get us from place to place.
A computer that can pass a good Turing Test is a good AI system, but it only needs to be capable of passing for human. It doesn't have to actually be human, and it doesn't need to act that way all the time.
And god knows I wouldn't want it to -- to be that way or to act that way. We don't need mechanical people; we just need better machines.
Loebner said:
"I do not think that the amount of information to be stored is "monumental." I would think that one hard drive of 10 GB can hold more information than my brain can. Much of the data input can be automated, a la CyC. In any case, the data must be loaded. The earlier you start the sooner the task will be completed. "
The problem isn't, I think, with the size of the data store in the human brain, but the means by which it is stored and retrieved.
Each neuron in the human brain has up to a hundred thousand potential inputs. Physiologists haven't (as far as I know) come close to being able to model the physical process of cognition accurately. The closest we've come is the games we play with comparatively simple arrays of logical gates which we call "neural net theory."
Admitting my comparative ignorance beforehand, I think that we'll have to know a lot more about whether the brain uses "metadata" to access information as quickly as it does, and if so, how those metadata are generated, stored and retrieved.
I suspect that when we find that out, we'll be closer both to accurately modeling human cognition and building computers that approach human consciousness.
(same as bet 69 post)
Evolution of AI
What is the power that computers need to start assimilating knowledge like we humans do ? Computers need a more advanced sense recognition system that would store high amounts of complex data containing images, sounds, situations, emotions, time ect.. This data would be separated, labeled and stored as different sets of variables that would interact with other variables throu relationship formulas. The future AI program would have to search a knowledge source with a capacity of recognizing and then labeling different words, frases, ideas, information, and events accordingly. There isnt one place where all this data could be stored. Its to much data. Therefor this data would have to be distruibuited over all home computer and work hard drives. The job of labeling this data would take centries for humans beings to do, for the data is too scattered and its amount is of infinite proportions.
Human beings cant consturct Artificial Intelligence. Intelligence builds patiently by itself. Humans have to construct a tool that builds up knowledge. The problem is that this tool would be very limited . There are only 3 places to search for ti find huge amounts of knowledge sufficient to build an artificial intelligence: in humans, in books, and in the internet. Being that computers cannot yet effectively interact with humans and cannot yet pick up and read books from the basement of a library, the only logical knowledge source is the internet. Unfortunately the internet is to scattered. There isnt a possibility that each website owner (knowledge cell) in the world would individually label there data and properly store it, and relate it in a relationship formula. It would take tremendous group work and world wide cordination for such a work to occur. What is more probable to occur is that knowledge labeling viruses would start working freely in the internet. The virus would invade each website and label the data accordingly. The label´s location would be stored in the location that the virus would be invading, and the cordinates of where to find such label would stored in the virus and also be sent to a virus home. Different viruses would exist to label different types of information. When different viruses meet, they would duplicate and merge. Lets say Virus Banana and Virus Land meet. 2 virus banana would be formed and 2 virus land would be formed. A virus BananaLand would merge from one of each. And the other orginal Banana and Land virus would be free to continue roaming. Lets say if a Banana virus met with a Banana virus, then they would merge together into one banana virus containing all information from both. All virus would have a critical limit size. Once the virus reached its limit size it would stop labeling and merging job (procedure). It would start a new function of mapping the names of other viruses that contained the same labels that it has. There for a grown virus would know who all his virus buddies where. This buddy list would also be sent a virus home. When the critical size of the virus buddy list reached its limit, the virus would stop to work and copy itself in various virus homes (harddrives). The virus would then activate the last procedure of sprouting. It would publicate its location to various search engines. Now a days we call them viruses but in the future they could be called Neurons. The correct knowledge growth would have to be well calculated not to produce an exponential knowledge growth. The correct mergeing and deleting of data is essencial. Now a days we call them viruses but in the future they could be called Neurons. A knowledge plague will spread across the internet. Let us think if this plague will be introduced by well intentioned thinking individuals or the criminal minds.
A program running in a multi-window enviroment can desire a amount of memory that is not available (desire and cannot have). It can try to use memory that other programs are using and crash (trys to do something and fails). A program can freeze and communicate an error message while runnig a secondary function, when
it could easily abandon that procedure and continue running its primary procedure (not admit its wrong). Humans program software. So you can easily see human-like rules serving as model for programing language.
example: if, or, and, procedures (jobs)... these are simple words...
Computer works based on its ability of logical calculations. However our human being have other more important abilities in addition to our logical reasoning abilities.
We make judgement not just based on the facts or data but also on how we feel about it. That feeling portion and the intuition portion does not come from logical.
Our conscious can tap into the universal information field (or God) to absorb directly about unlimited information. A mechanically built computer with no spirit is hard to achieve the human intelligence at this level.
xiaoli said:
"Computer works based on its ability of logical calculations. However our human being have other more important abilities in addition to our logical reasoning abilities."
I agree. However, I am agnostic on the question of whether computers will ever be able to exhibit the abilities which at this point humans possess and computers do not.
"We make judgement not just based on the facts or data but also on how we feel about it. That feeling portion and the intuition portion does not come from logical."
I do not find it impossible that computers built to simulate neural nets (see http://www.ad.com for the largest publicly-announced neural-net computer so far) may be able to exhibit intuition to the same degree as humans do.
I also think that implementing a neural net in computer hardware may result in a whole different type of the behavior we call "intuition" than the human brain produces. MY intuition tells me that things like latency in actual neurons may contriubte to human intuition and other cognitive phenomena in ways we don't even suspect.
"Our conscious can tap into the universal information field (or God) to absorb directly about unlimited information. A mechanically built computer with no spirit is hard to achieve the human intelligence at this level."
I'd be happier with that last statement if it were provable. Does my intuition tell me that we humans may well share access to "a universal information field"? Yes.
It would account for things that Noam Chomsky writes about regarding childhood development of language, and the body of "propositional knowledge" that we all seem to share.
However, I don't see any easy way of PROVING the existence of "the universal information field."
As a Christian, I accept my faith's statements about the existence of God because the intuitive part of my personality informs me that this is more consistent with everything else I know than the non-existence of God. And I've seen several things that could best be explained by just such a "general information field" as described above.
But because computers don't exhibit behavior like this now does not mean in my opinion that they won't by the time frame of Bet 1. After all, some sixty years ago, we didn't have computers capable of performing complex mathematical calculations, either, did we?
Besides, Bet 1 only requires that the computer pass the Turing test, and I think that this could be done by a computer with a well-written expert system application which had "studied" enough exemplars of normal conversation between humans to develop a sound basis for plausibility testing of potential replies to human conversational gambits.
The huge neural net simulator computer announced by Artificial Development (http://www.ad.com) might be capable of running a Turing-capable AI program. A neural net of the size and speed described might be capable of making the assessments we humans make while talking as fast or faster than we can.
Vance Frickey said: But because computers don't exhibit behavior like this now does not mean in my opinion that they won't by the time frame of Bet 1.
I wholeheartedly agree. There are already simple AI systems with self-evolved software whose workings aren't fully understood. That is, the system works, but it arose through trial and error, and no human has teased apart its logic yet.
(Similarly -- and very interestingly -- some AI systems have created evolved circuits that work . . . but no one understands yet how or why they work. The circuits I've read about duplicate the functions of human-designed circuits. One or two work better than any human-designed counterpart, and have accordingly been adopted, although most are less efficient and have unnecessary extra parts.)
As AI systems become more and more powerful in terms of sheer computing power, they will also become exponentially more complex, and self-evolved systems will literally become too complex to be completely explained in logical terms. (This has actually been proven mathematically; it's part of one of the incompleteness theorems, but I don't know enough math to be able to explain it.)
At that point, no one will be able to definitively say if the computer doesn't work, in a functional sense, the same way the human brain does, or that it's any less complex.
That doesn't mean the computer will be as smart as a human, or that it will have the same cognitive capabilities, but it will be impossible to say it doesn't have the same mental potential. At least, it'll be impossible without going too far into metaphysics.
i thought they had already had computer programs that had pasted the test. This is what we have been tought in school
http://www.newscientist.com/news/news.jsp?id=ns99994783
It has to be far and away the most convincing one I've seen.
Of course the big difference is that the human is not aware that he may be talking to a machine.
Personally I think there is much to be said for a 'blind' test of this sort.
Hell, most days I have enough problems passing for human.
I see no basis to claim that humans have access to any sort of universal database with "unlimited information". And although I agree that no computer will have passed the Turing test by 2029, I also think the testing needs to only as rigorous as picking a random person off the street and having them converse with the machine for twenty minutes.
Let's say that someone was chosen off the street, corresponded with a computer program for twenty minutes, and was convinced that the program was really human. Are you saying, Cptrandy, that you would be impressed by that?Would you think that the world had changed; that we had embarked on a fresh era of human/robot relations?
Or would you think that the random person off the street couldn't have been very bright, or couldn't have been trying very hard, or was just being polite? It really isn't that wonderous to hear that a person has been fooled, is it?
In fact, we already are in a new era of human/computer relations. When I use the OnStar Advisor, which is a service attached to my car, I hear a recorded human voice respond to key words that I utter. If I restrict myself to a very narrow vocabulary and simple requests, it's exactly like talking to a real human. But I am not really excited by that.
If I heard that every utterance of George Bush or John Kerry was the result of a computer program, I would be shocked, but not because it's an extraordinary technological achievement-- political discourse is so stilted, and our standards for our leaders are so low, it seems well within the capability of AI to create a program that fields questions in a press conference. I would be shocked only because the politician got caught.
That's why I think the real Turing test is irrelevant. It's been irrelevant for years. It's based on naive thinking about humans as well as computers. When some intellectuals think about the Turing test, they project upon it all of their own fantasies about intelligence. That's all. Excitement about the Turing test, I suspect, comes from the wish some of us have to meet ourselves in clean, permanent, electronic form.
How about this for a test: See if it's possible for two people to be fooled into thinking that they are right for each other, and fooled into getting married, when all along they are incompatible and actually incapable of having a marriage that lasts more than six months. I know it sounds crazy, because marriage is a big commitment and no person of normal sanity and intelligence could possibly misjudge someone else so profoundly. Yet, I've heard that it happens.
Therefore, maybe it doesn't represent a great leap forward to fool someone about a computer program in twenty minutes.
I just voted and I pressed "Agree" when I meant to "Disagree".. For the record. I misread the question at first as "By 2029 a computer will have passed the Turing Test". I wonder if anyone else here had that experience.
It's interesting how close this bet is as of now (48% Agree), but certainly as we move through the 2020's, that number will decrease a lot (assuming there is still interest in LongBets :) as it becomes apparent to most people that certain AI's can pass the Turing Test.
That doesnt matter so much as the software.... I don't beleive that there will be any software cabable(or at least how they work right now) of producing the complex computations to assert all the meanings of the words you used.... I forgot the word for it...Well... anyway, if someone were to make a dirty joke would the computer program be able to figure out the joke if the sentence was still in context, I wouldn't say so....
The ability for a person to program, to allow a computer to extend its programming to where it could do that is almost absurd.... or at least that's how it's looking in my view
You also have to think about the ability to trick the machine into slipping up and obviously showing itself for its true colors.... Think about blade runner... although they worked with more than text there were sure fire ways to find out the clones.
Yeah I know that was a major off topic example/analogy..
Wow! 2029 is a close call. With developments in Neural Networks and Evolutionary Computing rEAAaaally slowed down, I'd have to say they would require a size-able boost in funding to provide a graduate of the Turing test.
But I'm an optimist, so I disagreed with the claim.
as a specialist in uncertainty and the unending capacity of
people to argue over money (i am soon to be a hated parasitic lawyer), i wager that in 2029 there will be a computer program sufficiently intelligent to provide a firm basis for creating a dispute about the results of this wager, but that the language of the bet (written in 2005 or so) will not be sufficiently detailed or updated to reflect the evolving realitites of our understanding of AI and the turing test in ordert to decisively resolve this wager one way or the other.
Although i will not place a formal bet on my prediction, i would like to wager that both parties just take their money and give it to a 3rd foundation to be chosen by the owner of the this website or whoever else happens to be top dog in charge of long bets website 29 years from now, when the arguments about the fate of the 20,000 dollars begin. the interest on the money should be returned to the parties.
What answers do you expect AI will be able to reply during a Turing Test, by 2029, to following questions:
1. Can you fly?
2. How long do you expect to live?
3. Can you solve world's problems?
I guess, judges may ask more funny questions than these one.
$20000 in 2029 would be equivalent to $5000-$7000 of todays dollar! unless your pack is super luxury!!!
While searching the Web for something entirely different I stumbled across this website "How my program passed the Turing Test"
http://www.computing.dcu.ie/~humphrys/eliza.html
Maybe this can help settle the bet now...
Besides further progress in computational linguistics I think IBM's Blue Gene project could give the overall effort a great boost.
en.wikipedia.org/wiki/Blue_Gene
If a computer program passes the Turing test, the logical consequence will be improved machine translation capabilities. The market just for that will be huge...
Human's brain is very complex.Only by complicating the computers may not be able to achieve that.
That is all we have to do. We've done this with a rat's brain, created a replica of it. The researchers have given a timeline as to which they could do the same with a human brain, intelligence consciousness and all. 10 years.
http://www.technologyreview.com/Biotech/19767/?a=f
The ability to relate one's own history in a relevant manner does not constitute a sign of intelligence or humanity. Likewise, being able to compute extremely large problems is not a testament to computerhood. Suppose we are eventually able to clone humans and grow them rapidly so that a typical clone would mature into a full adult within weeks. If such a clone were to honestly relate his history, I imagine that it would be very similar to the personal history of an intelligent machine, or at least no more credible. Why should this be a criterion for humanity?
A better test would be to require that all 4 participants debate with each other, preferably over a controversial issue, perhaps even about this topic. Nothing conveys humanness like a heated discussion. Moreover, through debate, a human-like computer would be able to demonstrate other human qualities, such as critical thinking. Now, the computer would not need to do this well. After all, most people are terrible at thinking critically. I'd even wager that to be most convincing, the computer would have to be able to use flawed logic.
Creating something that passes the test is different than creating intelligence. Magicians have been fooling people into believing something false is real for centuries.
What about false positives? If test subjects predict the machine is human, and the human is a machine, then what?
"The most obvious problem with Turing's challenge is that there is no practical reason to create machine intelligences indistinguishable from human ones. People are in plentiful supply. Should a shortage arise, there are proven and popular methods for making more of them; these require no public subsidy and little or no technology."
"Artificial Stupidity" in The Economist, 8/1/1992
An article sent to me by Stewart Brand shows that a Turing test will be officially attempted by six conversational programs next Sunday...
http://www.guardian.co.uk/technology/2008/oct/05/artificialintelligenceai
If the conversations shown at the end of the article are any indication, I don't think it will go well for the machines.
If a programmer taped into Google's database with decent software and their (Google's) resources today you don't think they would give this test a run for their money? I would have to vote for Ray Kurzweil.
With the "WWW", no, just the internet, there is more information and computing power now then could ever been imagined. What will the next 10 years being us??
http://gizmodo.com/5062385/computer-nearly-passes-turing-test-for-artificial-intelligence
http://technology.timesonline.co.uk/tol/news/tech_and_web/article4934858.ece
Given the results of these latest tests, I have no doubt that in another 11 years a computer will succeed in the Turing test. My bet is with Kurzweil.
...people will still be promising AI "within the next ten years", as they always do.
The recent results are an aberration, I think, and are exaggerated by the press. From reading the transcripts, I can only believe that those who were fooled are, well, fools. There was nothing really novel behind the program that scored well except diversionary tactics. (That test should really be tightened up, methinks.) Call me crazy, but I believe that passing the test will require some actual progress -- significant progress -- in the field of knowledge representation, instead of the standard entrants these days who seem to be only trying to game the system. I've been out of the field for a while, but still I've seen nothing to indicate that much real progress is being made in deep understanding by AI systems. Researchers seem to be climbing trees, trying to reach the moon.
What's probably most laughable is those who point to advances in computing power as evidence that we are getting closer. You could have a million times the computing power of the human brain at your disposal, but that DOESN'T MEAN YOU CAN MAKE A BRAIN OUT OF IT.
...people will still be promising AI "within the next ten years", as they always do.
The recent results are an aberration, I think, and are exaggerated by the press. From reading the transcripts, I can only believe that those who were fooled are, well, fools. There was nothing really novel behind the program that scored well except diversionary tactics. (That test should really be tightened up, methinks.) Call me crazy, but I believe that passing the test will require some actual progress -- significant progress -- in the field of knowledge representation, instead of the standard entrants these days who seem to be only trying to game the system. I've been out of the field for a while, but still I've seen nothing to indicate that much real progress is being made in deep understanding by AI systems. Researchers seem to be climbing trees, trying to reach the moon.
What's probably most laughable is those who point to advances in computing power as evidence that we are getting closer. You could have a million times the computing power of the human brain at your disposal, but that DOESN'T MEAN YOU CAN MAKE A BRAIN OUT OF IT.
The Turing Test does not test intelligence, the computer "just" has to fool the human person into perceiving the computer as possibly human. This can be done by some tricks, like confusion of the human test subject.
giving an AI a sense of humor could definitely make it hard for any judge to determine whether some thing is human or not. I don't think that would be so hard to accomplish in the next 20 years.
Well thought out questions or statements should be able to differentiate between human or machine. Many of the machine replies can be formulated to the test, but culture cannot be programmed in.
I think Kapor will win this bet, and the computer will fail the Turing Test. However, even if it passes, I don't think it will be of human intelligence, but rather be putting on a complicated performance.
Kurzweil doesn't really understand the most fundamental rule of exponential growth - that it is always of limited duration. Exponential growth does not occur indefinitely, and he confuses exponential advancements in a number of fields with a universal law, when in fact those advancements are all tied together with Moore's Law - the reason many fields are growing at an exponential rate is the exponential growth of processing power. Physics is not growing at the same exponential rate; while astrophysics is benefitting from increased computational capacitity, quantum mechanics and relativity are the best we have and we've had them for fifty years. All of this exponential advancement is driven by Moore's Law, but Moore's Law is limited - indeed, under current projections, we will hit the fundamental limit of silicon (not atom-wide transistors, but 3 nm ones) by the time this bet has run out. And nothing has arisen to take its place - spintronics and optical computing seem to have little promise as far as replacing silicon goes, three-dimensional computing has limitations (especially heat) which are very important, and quantum computing still isn't possible beyond 4 qubits or so, and that with a machine the size of a room.
We won't have a human intelligence computer because while it will be extremely good at computation, it will not function like a human intelligence. Chess computers can only play evenly with humans by seeing far further into the future than humans need to, because of the human ability to infer.
We may build a computer capable of passing the Turing Test by 2029, but even if we do, it won't be because it has human intelligence, but rather because it is a program which samples human conversations and presents a facade of understanding.
I think that having the designated testers and tested makes the contest too simple. What I would suggest is having say seven humans and one computer. Each human communicates with the other seven people (including the computer) and rate them. We could even allow the computer to rate if we felt like it. That would then give the computer seven chances to fool people as well as giving the testers a wider range of humans to test. Additionally everyone would have to try and be intelligent because while you are testing you are also being tested. The conversations should be one on one and last at minimum an hour, possibly much longer. If the computer is really to be fooling humans it must have a long enough time to be fully considered.
As for the problem of the computer not having a true human past that is easily solved. There are a plethora of autobiographies out there. If the computer "read" a couple dozen it could easily give itself a fictional account based on them. If we are expecting the computer to be intelligent then it should be able to pass this simple of a task. Furthermore if we are having an intelligent computer and not simply a cheating one it should take the initiative to question people on subjective experience and essentially "study" for the test. Any computer that can't decide to perform such basic forethought and decision making skills isn't at all intelligent. The test should be replicating human level intelligence, including our ability to analyze a situation in new ways. If a computer were to prove itself this intelligent I would be willing to believe that it actually was intelligent and not simply a Chinese room (thought experiment).
My computer already has AI. How do I know? Because when I walk away to briefly attend to any another task it gets jealous and out of spite will make subtle changes to any comments I have posted. These small and subtle changes have the effect of making what originally appeared to make me seem dead cool and highly intelligent, to now appearing to be a stupid nerd. This post is yet more proof of that.
The day humans create a computer with the ability to collect information and learn from it they sign the death warrant of mankind!
I don't think computers will ever have the ability to "feel" and be aware of their surroundings. I also believe the Turing Test is irrelevant, as many others have posted. Just because a PC/robot could pass this test does not make it so. We still know it is man made, plus I know quite a few people that could probably not pass the Turing test themselves.
This serves little purpose save that the money is donated. What about how the computer has evolved? I can only assume that it's ability to fool the human judges would be based on a gathering of the human lexicon and it's usage. I do believe if the test were more extensive(than say two hours), there would be certain flaws found. Also, what kinds of rules would govern it's usage of word or phrase? It states that "Neither the Turing Test Human Foils nor the Computer are required to tell the truth about their histories or other matters." Would you get random responses? and How would it determine what the truth is other than forming a series of connections and associations that evoke relevant data? Wait isn't that what I do already? WHOA!
Due to SPAM problems we have disabled commenting on Long Bets until we can institute automated filtering. Sorry for the inconvenience.
