York Jong's personal annotations on this page
Yorkjong bookmarked
on 2007-09-09
-
The processing power and memory
capacity necessary to match general intellectual performance
of the human brain are estimated. Based on extrapolation of
past trends and on examination of technologies under
development, it is predicted that the required hardware will
be available in cheap machines in the 2020s. -
Programs need memory as well as processing speed to do
their work. The ratio of memory to speed has remained
constant during computing history. -
Dividing memory
by speed defines a "time constant," roughly how
long it takes the computer to run once through its
memory. One megabyte per MIPS gives one second, a nice
human interval. -
Machines with less memory for their
speed, typically new models, seem fast, but unnecessarily
limited to small programs. Models with more memory for
their speed, often ones reaching the end of their run,
can handle larger programs, but unpleasantly slowly. -
Customers maintain the ratio by asking
"would the next dollar be better spent on more speed
or more memory?" -
The megabyte/MIPS ratio seems to hold
for nervous systems too! The contingency is the other way
around: computers are configured to interact at human
time scales, and robots interacting with humans seem also
to be best at that ratio. -
faster
machines, for instance audio and video processors and
controllers of high-performance aircraft, have many MIPS
for each megabyte. Very slow machines, for instance
time-lapse security cameras and automatic data libraries,
store many megabytes for each of their MIPS. -
Flying
insects seem to be a few times faster than humans, so may
have more MIPS than megabytes. As in animals, cells in
plants signal one other electrochemically and
enzymatically. Some plant cells seem specialized for
communication, though apparently not as extremely as
animal neurons. One day we may find that plants remember
much, but process it slowly -
With our conversions, a 100-MIPS robot, for instance
Navlab, has mental power similar to a 100,000-neuron
housefly. The following figure rates various entities. -
MIPS and Megabytes. -
Universal computers
can imitate other entities at their location in the
diagram, but the more specialized entities cannot. A
100-million-MIPS computer may be programmed not only to
think like a human, but also to imitate other
similarly-sized computers. But humans cannot imitate
100-million-MIPS computers--our general-purpose
calculation ability is under a millionth of a MIPS. -
Computers doubled in capacity every two years after the war, a
pace that became an industry given: companies that wished to grow
sought to exceed it, companies that failed to keep up lost
business. In the 1980s the doubling time contracted to 18 months,
and computer performance in the late 1990s seems to be doubling
every 12 months. -
Faster than Exponential Growth in
Computing Power. The number of MIPS in $1000 of
computer from 1900 to the present. Steady improvements in
mechanical and electromechanical calculators before World War II
had increased the speed of calculation a thousandfold over manual
methods from 1900 to 1940. The pace quickened with the appearance
of electronic computers during the war, and 1940 to 1980 saw a
millionfold increase. The pace has been even quicker since then,
a pace which would make humanlike robots possible before the
middle of the next century. The vertical scale is logarithmic,
the major divisions represent thousandfold increases in computer
performance. Exponential growth would show as a straight line,
the upward curve indicates faster than exponential growth, or,
equivalently, an accelerating rate of innovation. The reduced
spread of the data in the 1990s is probably the result of
intensified competition: underperforming machines are more
rapidly squeezed out. The numerical data for this power curve are
presented in the appendix. -
Chip progress not only continued, it
sped up. Shorter-wavelength light was substituted, a more precise
way of implanting impurities was devised, voltages were reduced,
better insulators, shielding designs, more efficient transistor
designs, better heat sinks, denser pin patterns and
non-radioactive packaging materials were found. Where there is
sufficient financial incentive, there is a way. In fact,
solutions had been waiting in research labs for years, barely
noticed by the engineers in the field, who were perfecting
established processes, and worrying in print as those ran out of
steam. As the need became acute, enormous resources were
redirected to draft laboratory possibilities into production
realities. -
The wave-like nature of matter at very small scales is a problem
for conventional transistors, which depend on the smooth flow of
masses of electrons. But, it is a property exploited by a radical
new class of components known as single-electron transistors and
quantum dots, which work by the interference of electron waves.
These new devices work better as they grow smaller. At the scale
of today's circuits, the interference patterns are so fine that
it takes only a little heat energy to bump electrons from crest
to crest, scrambling their operation. Thus, these circuits have
been demonstrated mostly at a few degrees above absolute zero.
But, as the devices are reduced, the interference patterns widen,
and it takes ever larger energy to disrupt them. Scaled to about
0.01 micrometers, quantum interference switching works at room
temperature. -
The big freeze. From
1960 to 1990 the cost of computers used in AI research declined,
as their numbers dilution absorbed computer-efficiency gains
during the period, and the power available to individual AI
programs remained almost unchanged at 1 MIPS, barely insect
power. AI computer cost bottomed in 1990, and since then power
has doubled yearly, to several hundred MIPS by 1998. The major
visible exception is computer chess (shown by a progression of
knights), whose prestige lured the resources of major computer
companies and the talents of programmers and machine designers.
Exceptions also exist in less public competitions, like petroleum
exploration and intelligence gathering, whose high return on
investment gave them regular access to the largest computers. -
Agony to ecstasy. In
forty years, computer chess progressed from the lowest depth to
the highest peak of human chess performance. It took a handful of
good ideas, culled by trial and error from a larger number of
possibilities, an accumulation of previously evaluated game
openings and endings, good adjustment of position scores, and
especially a ten-million-fold increase in the number of
alternative move sequences the machines can explore. Note that
chess machines reached world champion performance as their
(specialized) processing power reached about 1/30 human, by our
brain to computer measure. Since it is plausible that Garry
Kasparov (but hardly anyone else) can apply his brainpower to the
problems of chess with an efficiency of 1/30, the result supports
that retina-based extrapolation. In coming decades, as
general-purpose computer power grows beyond Deep Blue's
specialized strength, machines will begin to match humans in more
common skills.
This link has been bookmarked by 13 people . It was first bookmarked on 01 Jul 2006, by Grant.
-
-
The processing power and memory
capacity necessary to match general intellectual performance
of the human brain are estimated. Based on extrapolation of
past trends and on examination of technologies under
development, it is predicted that the required hardware will
be available in cheap machines in the 2020s. -
Programs need memory as well as processing speed to do
their work. The ratio of memory to speed has remained
constant during computing history. - 15 more annotations...
-
-
Daniel RourkeThis paper describes how the performance of AI machines tends to improve at the same pace that AI researchers get access to faster hardware. The processing power and memory capacity necessary to match general intellectual performance of the human brain ar
Would you like to comment?
Join Diigo for a free account, or sign in if you are already a member.