When will computer hardware match the human brain? by Hans Moravec

29 Oct 14

olspar7

1980s, as widely available computers reached 10 MIPS, good optical character reading (OCR) programs,
In 1992 Reddy's group demonstrated a program called Sphinx II on a 15-MIPS workstation with 100 MIPS of specialized signal-processing circuitry.
A human retina is less than a centimeter square and a half-millimeter thick. It has about 100 million neurons, of five distinct kinds.
etina seems to process about ten one-million-point images per second.
The 1,500 cubic centimeter human brain is about 100,000 times as large as the retina,
million MIPS of computer power.
The ratio of memory to speed has remained constant during computing history. The earliest electronic computers had a few thousand bytes of memory and could do a few thousand calculations per second. Medium computers of 1980 had a million bytes of memory and did a million calculations per second. Supercomputers in 1990 did a billion calculations per second and had a billion bytes of memory. The latest, greatest supercomputers can do a trillion calculations per second and can have a trillion bytes of memory.
The best evidence about nervous system memory puts most of it in the synapses connecting the neurons. Molecular adjustments allow synapses to be in a number of distinguishable states, lets say one byte's worth. Then the 100-trillion-synapse brain would hold the equivalent 100 million megabytes. This agrees with our earlier estimate that it would take 100 million MIPS to mimic the brain's function. The megabyte/MIPS ratio seems to hold for nervous systems too!
s in animals, cells in plants signal one other electrochemically and enzymatically. Some plant cells seem specialized for communication, though apparently not as extremely as animal neurons. One day we may find that plants remember much, but process it slowly (how does a redwood tree manage to rebuff rapidly evolving pests during a 2,000 year lifespan, when it took mosquitoes only a few decades to overcome DDT?).

7 more annotations...

29 Jun 14

Salvatore Larosa

Shared ICT - Software platforms and tools AI

08 Apr 14

In Search of Dead Time

artificial-intelligence singularity technology

19 Mar 12

brain ai computer Computing human singularity

04 Jan 12

BBIS ITIT

When will computer hardware match the human brain? (Received Dec. 1997)

ai technology brain computer Computing mind

28 Oct 11

Terry Bullock

technology singularity

Deep Blue, in a first for machinekind, won the first game of the 1996 match. But, Kasparov quickly found the machine's weaknesses, and drew two and won three of the remaining games.

In May 1997 he met an improved version of the machine. That February, Kasparov had triumphed over a field of grandmasters in a prestigious tournament in Linares, Spain, reinforcing his reputation as the best player ever, and boosting his chess rating past 2800, uncharted territory. He prepared for the computer match in the intervening months, in part by playing against other machines. Kasparov won a long first game against Deep Blue, but lost next day to masterly moves by the machine. Then came three grueling draws, and a final game, in which a visibly shaken and angry Kasparov resigned early, with a weak position. It was the first competition match he had ever lost.

06 Oct 11

Donielle Champlin

29 Aug 11

Rhonda Smith

29 Jun 11

ajiteapen

futurism article

Ajit Eapen

futurism article

06 Apr 11

Jason McDermott

22 Feb 11

mckenzie anderson

MIPS and Megabytes. to mimic their behavior. Note the scale. Entities rated by the computational power and memory of the smallest universal computer needed is logarithmic on both axes: each vertical division represents a thousandfold increase in processing power, and each horizontal division a thousandfold increase in memory size. Universal computers can imitate other entities at their location in the diagram, but the more specialized entities cannot. A 100-million-MIPS computer may be programmed not only to think like a human, but also to imitate other similarly-sized computers. But humans cannot imitate 100-million-MIPS computers--our general-purpose calculation ability is under a millionth of a MIPS. Deep Blue's special-purpose chess chips process moves like a 3-million-MIPS computer, but its general-purpose power is only a thousand MIPS. Most of the non-computer entities in the diagram can't function in a general-purpose way at all. Universality is an almost magical property, but it has costs. A universal machine may use ten or more times the resources of one specialized for a task. But if the task should change, as it usually does in research, the universal machine can be reprogrammed, while the specialized machine must be replaced

18 Dec 10

janice magruder

18 Oct 10

Brian Downing

technology computer

21 Sep 10

Anita Mallory

artificial intelligence

24 Jun 10

e CorText

07 Sep 09

gaia403

AI brain computing singularity Technology cognition article philosophy future science research transhumanisme dlcs

10 Aug 09

Alexandre Cabanis

evolution man robot ai brain human Computing technology prospective

10 Apr 09

Ron Hale-Evans

mentat

06 Oct 08

Ruth C

transhumanism westthought01 resource

03 Mar 08

Carlos Pereira

temp

28 Feb 08

isaac Mao

AI Brain Computing

19 Jan 08

Jason

biology computer essays philosophy science

31 Dec 07

Alan Greene

technology

<a rel="nofollow" href="http://www.transhumanist.com/">Journal of Evolution and Technology</a>. 1998. Vol. 1<hr>When will computer hardwarematch the human brain?<blockquote> <blockquote> <blockquote> <blockquote> <dl> <div align="center"><center> <dt>(Received Dec. 1997)</dt> </center></div> </dl> </blockquote> </blockquote> </blockquote></blockquote><dl> <div align="center"><center> <dt>Hans Moravec</dt> </center></div><div align="center"><center> <dt>Robotics Institute Carnegie Mellon University Pittsburgh, PA 15213-3890, USA net: <a rel="nofollow" href="http://c.gp.cs.cmu.edu:5103/prog/finger/hpm">hpm@cmu.edu </a> web: <a rel="nofollow" href="http://www.frc.ri.cmu.edu/%7Ehpm/">http://www.frc.ri.cmu.edu/~hpm/ </a></dt> </center></div><div align="center"><center> <dt> </dt> </center></div> <dd> </dd></dl><blockquote> ABSTRACT This paper describes how the performance of AI machines tends to improve at the same pace that AI researchers get access to faster hardware. The processing power and memory capacity necessary to match general intellectual performance of the human brain are estimated. Based on extrapolation of past trends and on examination of technologies under development, it is predicted that the required hardware will be available in cheap machines in the 2020s.</blockquote> <h3>Brains, Eyes and Machines</h3><dl> <dt>Computers have far to go to match human strengths, and our estimates will depend on analogy and extrapolation. Fortunately, these are grounded in the first bit of the journey, now behind us. Thirty years of computer vision reveals that 1 MIPS can extract simple features from real-time imagery--tracking a white line or a white spot on a mottled background. 10 MIPS can follow complex gray-scale patches--as smart bombs, cruise missiles and early self-driving vans attest. 100 MIPS can follow moderately unpredictable features like roads--as recent long NAVLAB trips demonstrate. 1,000 MIPS will be adequate for coarse-grained three-dimensional spatial awareness--illustrated by several mid-resolution stereoscopic vision programs, including my own. 10,000 MIPS can find three-dimensional objects in clutter--suggested by several "bin-picking" and high-resolution stereo-vision demonstrations, which accomplish the task in an hour or so at 10 MIPS. The data fades there--research careers are too short, and computer memories too small, for significantly more elaborate experiments. There are considerations other than sheer scale. At 1 MIPS the best results come from finely hand-crafted programs that distill sensor data with utmost efficiency. 100-MIPS processes weigh their inputs against a wide range of hypotheses, with many parameters, that learning programs adjust better than the overburdened programmers. Learning of all sorts will be increasingly important as computer power and robot programs grow. This effect is evident in related areas. At the close of the 1980s, as widely available computers reached 10 MIPS, good optical character reading (OCR) programs, able to read most printed and typewritten text, began to appear. They used hand-constructed "feature detectors" for parts of letter shapes, with very little learning. As computer power passed 100 MIPS, trainable OCR programs appeared that could learn unusual typestyles from examples, and the latest and best programs learn their entire data sets. Handwriting recognizers, used by the Post Office to sort mail, and in computers, notably Apple's Newton, have followed a similar path. Speech recognition also fits the model. Under the direction of Raj Reddy, who began his research at Stanford in the 1960s, Carnegie Mellon has led in computer transcription of continuous spoken speech. In 1992 Reddy's group demonstrated a program called Sphinx II on a 15-MIPS workstation with 100 MIPS of specialized signal-processing circuitry. Sphinx II was able to deal with arbitrary English speakers using a several-thousand-word vocabulary. The system's word detectors, encoded in statistical structures known as Markov tables, were shaped by an automatic learning process that digested hundreds of hours of spoken examples from thousands of Carnegie Mellon volunteers enticed by rewards of pizza and ice cream. Several practical voice-control and dictation systems are sold for personal computers today, and some heavy users are substituting larynx for wrist damage. More computer power is needed to reach human performance, but how much? Human and animal brain sizes imply an answer, if we can relate nerve volume to computation. Structurally and functionally, one of the best understood neural assemblies is the retina of the vertebrate eye. Happily, similar operations have been developed for robot vision, handing us a rough conversion factor. The retina is a transparent, paper-thin layer of nerve tissue at the back of the eyeball on which the eye's lens projects an image of the world. It is connected by the optic nerve, a million-fiber cable, to regions deep in the brain. It is a part of the brain convenient for study, even in living animals because of its peripheral location and because its function is straightforward compared with the brain's other mysteries. A human retina is less than a centimeter square and a half-millimeter thick. It has about 100 million neurons, of five distinct kinds. Light-sensitive cells feed wide spanning <cite>horizontal</cite> cells and narrower <cite>bipolar</cite> cells, which are interconnected by whose outgoing fibers bundle to form the optic nerve. Each of the million ganglion-cell axons carries signals from a <cite>amacrine</cite> cells, and finally <cite>ganglion</cite> cells, particular patch of image, indicating light intensity differences over space or time: a million edge and motion detections. Overall, the retina seems to process about ten one-million-point images per second. It takes robot vision programs about 100 computer instructions to derive single edge or motion detections from comparable video images. 100 million instructions are needed to do a million detections, and 1,000 MIPS to repeat them ten times per second to match the retina. The 1,500 cubic centimeter human brain is about 100,000 times as large as the retina, suggesting that matching overall human behavior will take about 100 million MIPS of computer power. Computer chess bolsters this yardstick. Deep Blue, the chess machine that bested world chess champion Garry Kasparov in 1997, used specialized chips to process chess moves at a the speed equivalent to a 3 million MIPS universal computer (see Figure 3-4). This is 1/30 of the estimate for total human performance. Since it is plausible that Kasparov, probably the best human player ever, can apply his brainpower to the strange problems of chess with an efficiency of 1/30, Deep Blue's near parity with Kasparov's chess skill supports the retina-based extrapolation. The most powerful experimental supercomputers in 1998, composed of thousands or tens of thousands of the fastest microprocessors and costing tens of millions of dollars, can do a few million MIPS. They are within striking distance of being powerful enough to match human brainpower, but are unlikely to be applied to that end. Why tie up a rare twenty-million-dollar asset to develop one ersatz-human, when millions of inexpensive original-model humans are available? Such machines are needed for high-value scientific calculations, mostly physical simulations, having no cheaper substitutes. AI research must wait for the power to become more affordable. If 100 million MIPS could do the job of the human brain's 100 billion neurons, then one neuron is worth about 1/1,000 MIPS, i.e., 1,000 instructions per second. That's probably not enough to simulate an actual neuron, which can produce 1,000 finely timed pulses per second. Our estimate is for very efficient programs that imitate the aggregate function of thousand-neuron assemblies. Almost all nervous systems contain subassemblies that big. The small nervous systems of insects and other invertebrates seem to be hardwired from birth, each neuron having its own special predetermined links and function. The few-hundred-million-bit insect genome is enough to specify connections of each of their hundred thousand neurons. Humans, on the other hand, have 100 billion neurons, but only a few billion bits of genome. The human brain seems to consist largely of regular structures whose neurons are trimmed away as skills are learned, like featureless marble blocks chiseled into individual sculptures. Analogously, robot programs were precisely hand-coded when they occupied only a few hundred thousand bytes of memory. Now that they've grown to tens of millions of bytes, most of their content is learned from example. But there is a big practical difference between animal and robot learning. Animals learn individually, but robot learning can be copied from one machine to another. For instance, today's text and speech understanding programs were painstakingly trained over months or years, but each customer's copy of the software is "born" fully educated. Decoupling training from use will allow robots to do more with less. Big computers at the factory--maybe supercomputers with 1,000 times the power of machines that can reasonably be placed in a robot--will process large training sets under careful human supervision, and distill the results into efficient programs and arrays of settings that are then copied into myriads of individual robots with more modest processors. Programs need memory as well as processing speed to do their work. The ratio of memory to speed has remained constant during computing history. The earliest electronic computers had a few thousand bytes of memory and could do a few thousand calculations per second. Medium computers of 1980 had a million bytes of memory and did a million calculations per second. Supercomputers in 1990 did a billion calculations per second and had a billion bytes of memory. The latest, greatest supercomputers can do a trillion calculations per second and can have a trillion bytes of memory. Dividing memory by speed defines a "time constant," roughly how long it takes the computer to run once through its memory. One megabyte per MIPS gives one second, a nice human interval. Machines with less memory for their speed, typically new models, seem fast, but unnecessarily limited to small programs. Models with more memory for their speed, often ones reaching the end of their run, can handle larger programs, but unpleasantly slowly. For instance, the original Macintosh was introduced in 1984 with 1/2 MIPS and 1/8 megabyte, and was then considered a very fast machine. The equally fast "fat Mac" with 1/2 megabyte ran larger programs at tolerable speed, but the 1 megabyte "Mac plus" verged on slow. The four megabyte "Mac classic," the last 1/2 MIPS machine in the line, was intolerably slow, and was soon supplanted by ten-times-faster processors in the same enclosure. Customers maintain the ratio by asking "would the next dollar be better spent on more speed or more memory?" The best evidence about nervous system memory puts most of it in the synapses connecting the neurons. Molecular adjustments allow synapses to be in a number of distinguishable states, lets say one byte's worth. Then the 100-trillion-synapse brain would hold the equivalent 100 million megabytes. This agrees with our earlier estimate that it would take 100 million MIPS to mimic the brain's function. The megabyte/MIPS ratio seems to hold for nervous systems too! The contingency is the other way around: computers are configured to interact at human time scales, and robots interacting with humans seem also to be best at that ratio. On the other hand, faster machines, for instance audio and video processors and controllers of high-performance aircraft, have many MIPS for each megabyte. Very slow machines, for instance time-lapse security cameras and automatic data libraries, store many megabytes for each of their MIPS. Flying insects seem to be a few times faster than humans, so may have more MIPS than megabytes. As in animals, cells in plants signal one other electrochemically and enzymatically. Some plant cells seem specialized for communication, though apparently not as extremely as animal neurons. One day we may find that plants remember much, but process it slowly (how does a redwood tree manage to rebuff rapidly evolving pests during a 2,000 year lifespan, when it took mosquitoes only a few decades to overcome DDT?). With our conversions, a 100-MIPS robot, for instance Navlab, has mental power similar to a 100,000-neuron housefly. The following figure rates various entities. <a rel="nofollow" href="http://www.transhumanist.com/volume1/All.things.150.jpg"><img src="http://www.transhumanist.com/volume1/All_things_075.jpg" height="536" style="width:245px" width="614" align="middle"></a> <cite>MIPS and Megabytes. </cite>to mimic their behavior. Note the scale. Entities rated by the computational power and memory of the smallest universal computer needed is logarithmic on both axes: each vertical division represents a thousandfold increase in processing power, and each horizontal division a thousandfold increase in memory size. Universal computers can imitate other entities at their location in the diagram, but the more specialized entities cannot. A 100-million-MIPS computer may be programmed not only to think like a human, but also to imitate other similarly-sized computers. But humans cannot imitate 100-million-MIPS computers--our general-purpose calculation ability is under a millionth of a MIPS. Deep Blue's special-purpose chess chips process moves like a 3-million-MIPS computer, but its general-purpose power is only a thousand MIPS. Most of the non-computer entities in the diagram can't function in a general-purpose way at all. Universality is an almost magical property, but it has costs. A universal machine may use ten or more times the resources of one specialized for a task. But if the task should change, as it usually does in research, the universal machine can be reprogrammed, while the specialized machine must be replaced. </dt></dl><h3>Extrapolation</h3>By our estimate, today's very biggest supercomputers arewithin a factor of a hundred of having the power to mimic a humanmind. Their successors a decade hence will be more than powerfulenough. Yet, it is unlikely that machines costing tens ofmillions of dollars will be wasted doing what any human can do,when they could instead be solving urgent physical andmathematical problems nothing else can touch. Machines withhuman-like performance will make economic sense only when theycost less than humans, say when their "brains" costabout $1,000. When will that day arrive? The expense of computation has fallen rapidly and persistentlyfor a century. Steady improvements in mechanical andelectromechanical calculators before World War II had increasedthe speed of calculation a thousandfold over hand calculation.The pace quickened with the appearance of electronic computersduring the war--from 1940 to 1980 the amount of computationavailable at a given cost increased a millionfold. Vacuum tubeswere replaced by transistors, and transistors by integratedcircuits, whose components became ever smaller and more numerous.During the 1980s microcomputers reached the consumer market, andthe industry became more diverse and competitive. Powerful,inexpensive computer workstations replaced the drafting boards ofcircuit and computer designers, and an increasing number ofdesign steps were automated. The time to bring a new generationof computer to market shrank from two years at the beginning ofthe 1980s to less than nine months. The computer andcommunication industries grew into the largest on earth. Computers doubled in capacity every two years after the war, apace that became an industry given: companies that wished to growsought to exceed it, companies that failed to keep up lostbusiness. In the 1980s the doubling time contracted to 18 months,and computer performance in the late 1990s seems to be doublingevery 12 months. <a rel="nofollow" href="http://www.transhumanist.com/volume1/power.150.jpg"><img src="http://www.transhumanist.com/volume1/power_075.jpg" height="600" style="width:245px" width="750" align="middle"></a> <cite>Faster than Exponential Growth inComputing Power. </cite>The number of MIPS in $1000 ofcomputer from 1900 to the present. Steady improvements inmechanical and electromechanical calculators before World War IIhad increased the speed of calculation a thousandfold over manualmethods from 1900 to 1940. The pace quickened with the appearanceof electronic computers during the war, and 1940 to 1980 saw amillionfold increase. The pace has been even quicker since then,a pace which would make humanlike robots possible before themiddle of the next century. The vertical scale is logarithmic,the major divisions represent thousandfold increases in computerperformance. Exponential growth would show as a straight line,the upward curve indicates faster than exponential growth, or,equivalently, an accelerating rate of innovation. The reducedspread of the data in the 1990s is probably the result ofintensified competition: underperforming machines are morerapidly squeezed out. The numerical data for this power curve arepresented in <a rel="nofollow" href="http://www.transhumanist.com/volume1/appendix.htm">the appendix</a>. At the present rate, computers suitable for humanlike robots willappear in the 2020s. Can the pace be sustained for another threedecades? The graph shows no sign of abatement. If anything, ithints that further contractions in time scale are in store. But,one often encounters thoughtful articles by knowledgeable peoplein the semiconductor industry giving detailed reasons why thedecades of phenomenal growth must soon come to an end. The keynote for advancing computation is miniaturization: smallercomponents have less inertia and operate more quickly with lessenergy, and more of them can be packed in a given space. Firstthe moving parts shrunk, from the gears in mechanicalcalculators, to small contacts in electromechanical machines, tobunches of electrons in electronic computers. Next, the switches'supporting structure underwent a vanishing act, from thumb-sizedvacuum tubes, to fly-sized transistors, to ever-diminishingflyspecks on integrated circuit chips. Similar to printedcircuits before them, integrated circuits were made by aphotographic process. The desired pattern was projected onto asilicon chip, and subtle chemistry used to add or remove theright sorts of matter in the exposed areas. In the mid-1970s, integrated circuits, age 15, hit a crisis ofadolescence. They then held ten thousand components, just enoughfor an entire computer, and their finest details were approaching3 micrometers in size. Experienced engineers wrote many articleswarning that the end was near. Three micrometers was barelylarger than the wavelength of the light used to sculpt the chip.The number of impurity atoms defining the tiny components hadgrown so small that statistical scatter would soon render mostcomponents out of spec, a problem aggravated by a similar effectin the diminishing number of signaling electrons. Increasingelectrical gradients across diminishing gaps caused atoms tocreep through the crystal, degrading the circuit. Interactionsbetween ever-closer wires were about to ruin the signals. Chipswould soon generate too much heat to remove, and require too manyexternal connections to fit. The smaller memory cells weresuffering radiation-induced forgetfulness. A look at the computer growth graph shows that the problems wereovercome, with a vengeance. Chip progress not only continued, itsped up. Shorter-wavelength light was substituted, a more preciseway of implanting impurities was devised, voltages were reduced,better insulators, shielding designs, more efficient transistordesigns, better heat sinks, denser pin patterns andnon-radioactive packaging materials were found. Where there issufficient financial incentive, there is a way. In fact,solutions had been waiting in research labs for years, barelynoticed by the engineers in the field, who were perfectingestablished processes, and worrying in print as those ran out ofsteam. As the need became acute, enormous resources wereredirected to draft laboratory possibilities into productionrealities. In the intervening years many problems were met and solved, andinnovations introduced, but now, nearing a mid-life 40, theanxieties seem again to have crested. In 1996 major articlesappeared in scientific magazines and major national newspapersworrying that electronics progress might be a decade from ending.The cost of building new integrated circuit plants wasapproaching a prohibitive billion dollars. Feature sizes werereaching 0.1 micrometers, the wavelength of the sculptingultraviolet light. Their transistors, scaled down steadily from1970s designs, would soon be so small that electrons wouldquantum "tunnel" out of them. Wiring was becoming sodense it would crowd out the components, and slow down and leaksignals. Heat was increasing. The articles didn't mention that less expensive plants could makethe same integrated circuits, if less cheaply and in smallerquantities. Scale was necessary because the industry had grown solarge and competitive. Rather than signaling impending doom, itindicated free-market success, a battle of titans driving downcosts to the users. They also failed to mention new contenders,waiting on lab benches to step in should the leader fall. The wave-like nature of matter at very small scales is a problemfor conventional transistors, which depend on the smooth flow ofmasses of electrons. But, it is a property exploited by a radicalnew class of components known as single-electron transistors andquantum dots, which work by the interference of electron waves.These new devices work better as they grow smaller. At the scaleof today's circuits, the interference patterns are so fine thatit takes only a little heat energy to bump electrons from crestto crest, scrambling their operation. Thus, these circuits havebeen demonstrated mostly at a few degrees above absolute zero.But, as the devices are reduced, the interference patterns widen,and it takes ever larger energy to disrupt them. Scaled to about0.01 micrometers, quantum interference switching works at roomtemperature. It promises more than a thousand times higherdensity than today's circuits, possibly a thousand times thespeed, and much lower power consumption, since it moves a fewelectrons across small quantum bumps, rather than pushing them inlarge masses through resistive material. In place of much wiring,quantum interference logic may use chains of switching devices.It could be manufactured by advanced descendants of today's chipfabrication machinery (Goldhaber-Gordon et al. 1997). Proposalsabound in the research literature, and the industry has theresources to perfect the circuits and their manufacture, when thetime comes. Wilder possibilities are brewing. Switches and memory cells madeof single molecules have been demonstrated, which might enable avolume to hold a billion times more circuitry than today.Potentially blowing everything else away are "quantumcomputers," in which a whole computer, not just individualsignals, acts in a wavelike manner. Like a conventional computer,a quantum computer consists of a number of memory cells whosecontents are modified in a sequence of logical transformations.Unlike a conventional computer, whose memory cells are either 1or 0, each cell in a quantum computer is started in a quantumsuperposition of both 1 and 0. The whole machine is asuperposition of all possible combinations of memory states. Asthe computation proceeds, each component of the superpositionindividually undergoes the logic operations. It is as if anexponential number of computers, each starting with a differentpattern in memory, were working on the problem simultaneously.When the computation is finished, the memory cells are examined,and an answer emerges from the wavelike interference of all thepossibilities. The trick is to devise the computation so that thedesired answers reinforce, while the others cancel. In the lastseveral years, quantum algorithms have been devised that factornumbers and search for encryption keys much faster than anyclassical computer. Toy quantum computers, with three or four"qubits" stored as states of single atoms or photons,have been demonstrated, but they can do only short computationsbefore their delicate superpositions are scrambled by outsideinteractions. More promising are computers using nuclear magneticresonance, as in hospital scanners. There, quantum bits areencoded as the spins of atomic nuclei, and gently nudged byexternal magnetic and radio fields into magnetic interactionswith neighboring nuclei. The heavy nuclei, swaddled in diffuseorbiting electron clouds, can maintain their quantum coherencefor hours or longer. A quantum computer with a thousand or morequbits could tackle problems astronomically beyond the reach ofany conceivable classical computer. Molecular and quantum computers will be important sooner orlater, but humanlike robots are likely to arrive without theirhelp. Research within semiconductor companies, including workingprototype chips, makes it quite clear that existing techniquescan be nursed along for another decade, to chip features below0.1 micrometers, memory chips with tens of billions of bits andmultiprocessor chips with over 100,000 MIPS. Towards the end ofthat period, the circuitry will probably incorporate a growingnumber of quantum interference components. As productiontechniques for those tiny components are perfected, they willbegin to take over the chips, and the pace of computer progressmay steepen further. The 100 million MIPS to match human brainpower will then arrive in home computers before 2030. <h3>False Start</h3>It may seem rash to expect fully intelligent machines in a fewdecades, when the computers have barely matched insect mentalityin a half-century of development. Indeed, for that reason, manylong-time artificial intelligence researchers scoff at thesuggestion, and offer a few centuries as a more believableperiod. But there are very good reasons why things will go muchfaster in the next fifty years than they have in the last fifty. The stupendous growth and competitiveness of the computerindustry is one reason. A less appreciated one is thatintelligent machine research did not make steady progress in itsfirst fifty years, it marked time for thirty of them! Thoughgeneral computer power grew a hundred thousand fold from 1960 to1990, the computer power available to AI programs barely budgedfrom 1 MIPS during those three decades. In the 1950s, the pioneers of AI viewed computers as locomotivesof thought, which might outperform humans in higher mental workas prodigiously as they outperformed them in arithmetic, if theywere harnessed to the right programs. Success in the endeavorwould bring enormous benefits to national defense, commerce andgovernment. The promise warranted significant public and privateinvestment. For instance, there was a large project to developmachines to automatically translate scientific and otherliterature from Russian to English. There were only a few AIcenters, but those had the largest computers of the day,comparable in cost to today's supercomputers. A common one wasthe IBM 704, which provided a good fraction of a MIPS. By 1960 the unspectacular performance of the first reasoning andtranslation programs had taken the bloom off the rose, but theunexpected launching by the Soviet Union of Sputnik, the firstsatellite in 1957, had substituted a paranoia. ArtificialIntelligence may not have delivered on its first promise, butwhat if it were to suddenly succeed after all? To avoid anothernasty technological surprise from the enemy, it behooved the USto support the work, moderately, just in case. Moderation paidfor medium scale machines costing a few million dollars, nolonger supercomputers. In the 1960s that price provided a goodfraction of a MIPS in thrifty machines like Digital EquipmentCorp's innovative PDP-1 and PDP-6. The field looked even less promising by 1970, and support formilitary-related research declined sharply with the end of theVietnam war. Artificial Intelligence research was forced totighten its belt and beg for unaccustomed small grants andcontracts from science agencies and industry. The major researchcenters survived, but became a little shabby as they made do withaging equipment. For almost the entire decade AI research wasdone with PDP-10 computers, that provided just under 1 MIPS.Because it had contributed to the design, the Stanford AI Labreceived a 1.5 MIPS KL-10 in the late 1970s from Digital, as agift. Funding improved somewhat in the early 1980s, but the number ofresearch groups had grown, and the amount available for computerswas modest. Many groups purchased Digital's new Vax computers,costing $100,000 and providing 1 MIPS. By mid-decade, personalcomputer workstations had appeared. Individual researchersreveled in the luxury of having their own computers, avoiding thedelays of time-shared machines. A typical workstation was aSun-3, costing about $10,000, and providing about 1 MIPS. By 1990, entire careers had passed in the frozen winter of 1-MIPScomputers, mainly from necessity, but partly from habit and alingering opinion that the early machines really should have beenpowerful enough. In 1990, 1 MIPS cost $1,000 in a low-endpersonal computer. There was no need to go any lower. Finallyspring thaw has come. Since 1990, the power available toindividual AI and robotics programs has doubled yearly, to 30MIPS by 1994 and 500 MIPS by 1998. Seeds long ago alleged barrenare suddenly sprouting. Machines read text, recognize speech,even translate languages. Robots drive cross-country, crawlacross Mars, and trundle down office corridors. In 1996 atheorem-proving program called EQP running five weeks on a 50MIPS computer at Argonne National Laboratory found a proof of aboolean algebra conjecture by Herbert Robbins that had eludedmathematicians for sixty years. And it is still only spring. Waituntil summer. <a rel="nofollow" href="http://www.transhumanist.com/volume1/AI.power.150.jpg"><img src="http://www.transhumanist.com/volume1/AI_power_075.jpg" height="538" style="width:245px" width="750" align="middle"></a> <cite>The big freeze. </cite>From1960 to 1990 the cost of computers used in AI research declined,as their numbers dilution absorbed computer-efficiency gainsduring the period, and the power available to individual AIprograms remained almost unchanged at 1 MIPS, barely insectpower. AI computer cost bottomed in 1990, and since then powerhas doubled yearly, to several hundred MIPS by 1998. The majorvisible exception is computer chess (shown by a progression ofknights), whose prestige lured the resources of major computercompanies and the talents of programmers and machine designers.Exceptions also exist in less public competitions, like petroleumexploration and intelligence gathering, whose high return oninvestment gave them regular access to the largest computers. <h3>The Game's Afoot</h3>A summerlike air already pervades the few applications ofartificial intelligence that retained access to the largestcomputers. Some of these, like pattern analysis for satelliteimages and other kinds of spying, and in seismic oil exploration,are closely held secrets. Another, though, basks in thelimelight. The best chess-playing computers are so interestingthey generate millions of dollars of free advertising for thewinners, and consequently have enticed a series of computercompanies to donate time on their best machines and otherresources to the cause. Since 1960 IBM, Control Data, AT&T,Cray, Intel and now again IBM have been sponsors of computerchess. The "knights" in the AI power graph show theeffect of this largesse, relative to mainstream AI research. Thetop chess programs have competed in tournaments powered bysupercomputers, or specialized machines whose chess power iscomparable. In 1958 IBM had both the first checker program, byArthur Samuel, and the first full chess program, by AlexBernstein. They ran on an IBM 704, the biggest and lastvacuum-tube computer. The Bernstein program played atrociously,but Samuel's program, which automatically learned its boardscoring parameters, was able to beat Connecticut checkerschampion Robert Nealey. Since 1994, Chinook, a program written byJonathan Schaeffer of the University of Alberta, has consistentlybested the world's human checker champion. But checkers isn'tvery glamorous, and this portent received little notice. By contrast, it was nearly impossible to overlook the epicbattles between world chess champion Garry Kasparov and IBM'sDeep Blue in 1996 and 1997. Deep Blue is a scaled-up version of amachine called Deep Thought, built by Carnegie Mellon Universitystudents ten years earlier. Deep Thought, in turn, depended onspecial-purpose chips, each wired like the Belle chess computerbuilt by Ken Thompson at AT&T Bell Labs in the 1970s. Belle,organized like a chessboard, circuitry on the squares, wiresrunning like chess moves, could evaluate and find all legal movesfrom a position in one electronic flash. In 1997 Deep Blue had256 such chips, orchestrated by a 32 processormini-supercomputer. It examined 200 million chess positions asecond. Chess programs, on unaided general-purpose computers,average about 16,000 instructions per position examined. DeepBlue, when playing chess (and only then), was thus worth about 3million MIPS, 1/30 of our estimate for human intelligence. Deep Blue, in a first for machinekind, won the first game of the1996 match. But, Kasparov quickly found the machine's weaknesses,and drew two and won three of the remaining games. In May 1997 he met an improved version of the machine. ThatFebruary, Kasparov had triumphed over a field of grandmasters ina prestigious tournament in Linares, Spain, reinforcing hisreputation as the best player ever, and boosting his chess ratingpast 2800, uncharted territory. He prepared for the computermatch in the intervening months, in part by playing against othermachines. Kasparov won a long first game against Deep Blue, butlost next day to masterly moves by the machine. Then came threegrueling draws, and a final game, in which a visibly shaken andangry Kasparov resigned early, with a weak position. It was thefirst competition match he had ever lost. The event was notable for many reasons, but one especially is ofinterest here. Several times during both matches, Kasparovreported signs of mind in the machine. At times in the secondtournament, he worried there might be humans behind the scenes,feeding Deep Blue strategic insights! Bobby Fischer, the US chess great of the 1970s, is reputed tohave played each game as if against God, simply making the bestmoves. Kasparov, on the other hand, claims to see into opponents'minds during play, intuiting and exploiting their plans, insightsand oversights. In all other chess computers, he reports amechanical predictability stemming from their undiscriminatingbut limited lookahead, and absence of long-term strategy. In DeepBlue, to his consternation, he saw instead an "alienintelligence." In this paper-thin slice of mentality, a computer seems to havenot only outperformed the best human, but to have transcended itsmachinehood. Who better to judge than Garry Kasparov?Mathematicians who examined EQP's proof of the Robbinsconjecture, mentioned earlier, report a similar impression ofcreativity and intelligence. In both cases, the evidence for anintelligent mind lies in the machine's performance, not itsmakeup. Now, the team that built Deep Blue claim no"intelligence" in it, only a large database of openingand end games, scoring and deepening functions tuned withconsulting grandmasters, and, especially, raw speed that allowsthe machine to look ahead an average of fourteen half-moves perturn. Unlike some earlier, less successful, chess programs,

18 Dec 07

Nuno Nunes

later reading intelligence artificial article toread

09 Sep 07

York Jong

5star AI MIPS brain computer megabytes memory trend

The processing power and memory capacity necessary to match general intellectual performance of the human brain are estimated. Based on extrapolation of past trends and on examination of technologies under development, it is predicted that the required hardware will be available in cheap machines in the 2020s.
Programs need memory as well as processing speed to do their work. The ratio of memory to speed has remained constant during computing history.
Dividing memory by speed defines a "time constant," roughly how long it takes the computer to run once through its memory. One megabyte per MIPS gives one second, a nice human interval.
Machines with less memory for their speed, typically new models, seem fast, but unnecessarily limited to small programs. Models with more memory for their speed, often ones reaching the end of their run, can handle larger programs, but unpleasantly slowly.
Customers maintain the ratio by asking "would the next dollar be better spent on more speed or more memory?"
The megabyte/MIPS ratio seems to hold for nervous systems too! The contingency is the other way around: computers are configured to interact at human time scales, and robots interacting with humans seem also to be best at that ratio.
faster machines, for instance audio and video processors and controllers of high-performance aircraft, have many MIPS for each megabyte. Very slow machines, for instance time-lapse security cameras and automatic data libraries, store many megabytes for each of their MIPS.
Flying insects seem to be a few times faster than humans, so may have more MIPS than megabytes. As in animals, cells in plants signal one other electrochemically and enzymatically. Some plant cells seem specialized for communication, though apparently not as extremely as animal neurons. One day we may find that plants remember much, but process it slowly
With our conversions, a 100-MIPS robot, for instance Navlab, has mental power similar to a 100,000-neuron housefly. The following figure rates various entities.
MIPS and Megabytes.
Universal computers can imitate other entities at their location in the diagram, but the more specialized entities cannot. A 100-million-MIPS computer may be programmed not only to think like a human, but also to imitate other similarly-sized computers. But humans cannot imitate 100-million-MIPS computers--our general-purpose calculation ability is under a millionth of a MIPS.
Computers doubled in capacity every two years after the war, a pace that became an industry given: companies that wished to grow sought to exceed it, companies that failed to keep up lost business. In the 1980s the doubling time contracted to 18 months, and computer performance in the late 1990s seems to be doubling every 12 months.
Faster than Exponential Growth in Computing Power. The number of MIPS in $1000 of computer from 1900 to the present. Steady improvements in mechanical and electromechanical calculators before World War II had increased the speed of calculation a thousandfold over manual methods from 1900 to 1940. The pace quickened with the appearance of electronic computers during the war, and 1940 to 1980 saw a millionfold increase. The pace has been even quicker since then, a pace which would make humanlike robots possible before the middle of the next century. The vertical scale is logarithmic, the major divisions represent thousandfold increases in computer performance. Exponential growth would show as a straight line, the upward curve indicates faster than exponential growth, or, equivalently, an accelerating rate of innovation. The reduced spread of the data in the 1990s is probably the result of intensified competition: underperforming machines are more rapidly squeezed out. The numerical data for this power curve are presented in the appendix.
Chip progress not only continued, it sped up. Shorter-wavelength light was substituted, a more precise way of implanting impurities was devised, voltages were reduced, better insulators, shielding designs, more efficient transistor designs, better heat sinks, denser pin patterns and non-radioactive packaging materials were found. Where there is sufficient financial incentive, there is a way. In fact, solutions had been waiting in research labs for years, barely noticed by the engineers in the field, who were perfecting established processes, and worrying in print as those ran out of steam. As the need became acute, enormous resources were redirected to draft laboratory possibilities into production realities.
The wave-like nature of matter at very small scales is a problem for conventional transistors, which depend on the smooth flow of masses of electrons. But, it is a property exploited by a radical new class of components known as single-electron transistors and quantum dots, which work by the interference of electron waves. These new devices work better as they grow smaller. At the scale of today's circuits, the interference patterns are so fine that it takes only a little heat energy to bump electrons from crest to crest, scrambling their operation. Thus, these circuits have been demonstrated mostly at a few degrees above absolute zero. But, as the devices are reduced, the interference patterns widen, and it takes ever larger energy to disrupt them. Scaled to about 0.01 micrometers, quantum interference switching works at room temperature.
The big freeze. From 1960 to 1990 the cost of computers used in AI research declined, as their numbers dilution absorbed computer-efficiency gains during the period, and the power available to individual AI programs remained almost unchanged at 1 MIPS, barely insect power. AI computer cost bottomed in 1990, and since then power has doubled yearly, to several hundred MIPS by 1998. The major visible exception is computer chess (shown by a progression of knights), whose prestige lured the resources of major computer companies and the talents of programmers and machine designers. Exceptions also exist in less public competitions, like petroleum exploration and intelligence gathering, whose high return on investment gave them regular access to the largest computers.
Agony to ecstasy. In forty years, computer chess progressed from the lowest depth to the highest peak of human chess performance. It took a handful of good ideas, culled by trial and error from a larger number of possibilities, an accumulation of previously evaluated game openings and endings, good adjustment of position scores, and especially a ten-million-fold increase in the number of alternative move sequences the machines can explore. Note that chess machines reached world champion performance as their (specialized) processing power reached about 1/30 human, by our brain to computer measure. Since it is plausible that Garry Kasparov (but hardly anyone else) can apply his brainpower to the problems of chess with an efficiency of 1/30, the result supports that retina-based extrapolation. In coming decades, as general-purpose computer power grows beyond Deep Blue's specialized strength, machines will begin to match humans in more common skills.

15 more annotations...

28 May 07

chinesejapanese

This paper describes how the performance of AI machines tends to improve at the same pace that AI researchers get access to faster hardware. The processing power and memory capacity necessary to match general intellectual performance of the human brain ar

computer machine intelligence Hans Moravec article cyborg brain eye sensor futurist futurology delicious import from ImportDelicious 2016 05 22

28 Mar 07

01 Feb 07

Jungmi Park

imported _Interestingresearchtopics _Human-RobotInteraction AI

21 Oct 06

Lee Davis

Transhumanism Artificial_Intelligence AI article human research technology

25 Sep 06

ruby quince

interesting future thing re AI

future

31 Jul 06

rohan_talip

artificial_intelligence brain

01 Jul 06

Grant

ai/al computer

08 Nov 05

Daniel Rourke

This paper describes how the performance of AI machines tends to improve at the same pace that AI researchers get access to faster hardware. The processing power and memory capacity necessary to match general intellectual performance of the human brain ar

singularity ai computing mind technology brain future philosophy huge-entity.com science human