"A
distributed
representation is one in which meaning is not captured by a
single symbolic unit, but rather arises from the interaction of a set
of units, normally in a network of some sort."
As noted
in the same article, "The concept of distributed representation
is a product of joint developments in the neurosciences and in
connectionist work on recognition tasks (
Churchland
and Sejnowski 1992). Fundamentally, a distributed representation
is one in which meaning is not captured by a single symbolic unit,
but rather arises from the interaction of a set of units, normally in
a network of some sort."
To illustrate this concept, I
have been asking people to think of the concept 'Paris'. If 'Paris'
were represented by a simple symbol set, we would all mean the same
thing when we say 'Paris'. But in fact, we each mean a collection of
different things and none of our collections is the same. Therefore,
in our own minds, the concept 'Paris' is a loose association of a
whole bunch of different things, and hence the concept 'Paris' exists
in no particular place in our minds, but rather, is scattered
throughout our minds.
Now what the article is saying is that
human brains
are like computers - but not like the computers
as described above, with symbols and programs and all that, but like
computers when they are connected together in a network.
"The
brain as a whole operates more like a social network than a digital
computer... the computer-like features of the prefrontal cortex
broaden the social networks, helping the brain become more flexible
in processing novel and symbolic information." Understanding
'where the car is parked' is like understanding how one kind of
function applies on the brain's distributed representation, while
understanding 'the best place to park the car' is like how a
different function applies to the
same distributed
representation.
The analogy with the network of computers is a
good one (and people who develop social network software are
sometimes operating with these concepts of neural mechanisms
specifically in mind). The actual social network itself - a set of
distributed and interlinked entities, usually people, as represented
by websites or pages - constitutes a type of distributed
representation. A 'meme' - like, say, the
Friday
Five - is distributed across that network; it exists in no
particular place.
Specific mental operations, therefore, are
like thinking of functions applied to this social network. For
example, if I were to want to find 'the most popular bloggers' I
would need to apply a set of functions to that network. I would need
to represent each entity as a 'linking' entity. I would need to
cluster types of links (to eliminate self-referential links and
spam). I would then need to apply my function (now my
own view
here, and possibly O'Reilly's, though I don't read it specifically in
his article, is that to apply a function is to create
additional
neural layers that act as specialized filters - this would
contrast with, say, Technorati, which polls each individual entity
and then applies an
algorithm to it).
This theory,
stated simply, is that human thought amounts to patterns of
interactions in neural networks. More precisely, patterns of input
phenomena - such as sensory perceptions - cause or create patterns of
connections between neurons in the brain. These connections are
associative - that is, connections between two neurons form when the
two neurons are active at the same time, and weaken when they are
inactive or active at different times. See, for example, Donald
Hebb's 'The
Organization
of Behavior', which outlines what has come to be called '
Hebbian
associationism'.
