1: <HTML>
2: <HEAD><TITLE>Dynamically Reconfigurable Universal Learning Neurocomputer: Paper 0</TITLE>
3:
4: </HEAD>
5:
6: <body>
7: <font size="3">
8:
9: <font size="5"> <b><center>Can the whole brain be simpler than its "parts"?</center></b></font>
10:
11: <!--<h2><center>Can the whole brain be simpler than its "parts"?</center></h2>-->
12:
13: <p align="center"><font color="black" size="4" >Victor Eliashberg</font></p>
14: <!--<p align="center"><font color="black" size="3" > Palo Alto, CA </p>-->
15: <p align="center"><font color="black" size="3" >Avel Electronics, Palo Alto, CA </p>
16: <a name="epigr"></a>
17: <br>
18: <blockquote><blockquote><blockquote>
19: <font color="black" size="2"> I am intolerant of those who regard the whole of biological data,
20: of the phenomena of biological organization and intelligence, as not more than a grab bag from which to abstract
21: technological goodies. My intolerance is tempered only by the belief that such casual abstraction cannot succeed...
22: The point is, it is not a grab bag. The items therein show connection, and to attempt to draw out just one
23: glittering generalization entails a host of contradictory loose ends. </font>
24:
25: <font color="black" size="2"><p align="right" > George W. Zopf, Jr. Attitude and Context. 1962.</p></font>
26: </blockquote></blockquote></blockquote>
27: <br>
28: <h3><center>Abstract</center></h3>
29: <p>This is the first in a series of connected papers discussing the problem of a dynamically reconfigurable universal learning neurocomputer
30: that could serve as a computational model for the whole human brain. The whole series is entitled "The Brain Zero Project. My Brain as
31: a Dynamically Reconfigurable Universal Learning Neurocomputer." (For more information visit the website
32: <font color="blue">www.brain0.com</font>.)
33: This introductory paper is concerned with general methodology. Its main goal is to explain why it is critically important for both neural modeling and
34: cognitive modeling to pay much attention to the basic requirements of the whole brain as a complex computing system. The author argues
35: that it can be easier to develop an adequate computational model for the whole "unprogrammed" (untrained) human brain than to find adequate formal
36: representations of some nontrivial parts of brain's performance. (In the same way as, for example, it is easier to describe the behavior of a
37: complex analytical function than the behavior of its real and/or imaginary part.) The "curse of dimensionality" that plagues purely
38: phenomenological ("brainless") cognitive theories is a natural penalty for an attempt to represent insufficiently large parts of brain's
39: performance in a state space of insufficiently high dimensionality. A "partial" modeler encounters "Catch 22." An attempt to simplify
40: a cognitive problem by artificially reducing its dimensionality makes the problem more difficult.</p>
41:
42: <h3>0.0 Introduction</h3>
43:
44: <p>The goal of this paper is to create the right methodological mental set for understanding the general
45: approach to brain modeling and cognitive modeling adopted in this study. The paper consists of the following sections:</p>
46:
47: <p><a href="#sect01"><b>Section 0.1 The Power of Mental Set</b></a> illustrates the power of a wrong mental set and prepares
48: the reader for the "unsuspected or the unwanted." (See quotation from Burns (1958) at the end of this section.)
49: </p>
50: <p><a href="#sect02"><b>Section 0.2 The Whole Human Brain as a Universal Learning Neurocomputer</b></a>
51: argues that the popular belief that universal learning systems don't exist is a myth. This myth is a result of an
52: inadequate "general" definition of the concept of learning that doesn't include the "special case" of human learning.
53: </p>
54: <p><a href="#sect03"><b>Section 0.3 On the Parts and the Whole</b></a> explains why the problem of the parts of the brain and/or parts
55: of brain's performance should not be separated from the problem of the whole brain. It also formulates four general propositions
56: about the complexity of the models of the whole brain and the models of behavior of cognitive system (Man,World).
57: </p>
58: <p><a href="#sect04"><b>Section 0.4 Motivation and General Approach: the concept of E-machine</b><a> outlines the general approach
59: to brain modeling adopted in this study and gives a verbal description of the metaphor the brain as an E-machine.
60: </p>
61: <p><a href="#sect05"><b>Section 0.5 The Brain 0 Project</b></a> describes a reverse engineering methodology for the
62: development of an adequate computational model for the "untrained" human brain.
63: </p>
64: <p><a href="#sect06"><b>Section 0.6 Methodological Pitfalls</b></a> illustrates some pitfalls that await a "partial" brain
65: modeler and/or a "partial" cognitive modeler.
66: </p>
67: <p><a href="#sect07"><b>Section 0.7 On Brain Hardware, Brain Software, and the Algorithms of Thinking</b></a> presents a brief system
68: engineering analysis of the strong and the weak points of Artificial Intelligence (AI) and Artificial Neural Networks
69: (ANN) research as applied to the problem of the human brain and human behavior.
70: </p>
71: <p><a href="#sect08"><b>Section 0.8 The "Biological Grab Bag" Mentality Has not Succeeded</b></a> suggests that Zopf, Jr. (1962) was
72: right (see the <a href="#epigr">epigraph</a> above). It is important to try to understand the real brain before making
73: "glittering generalizations." The catch is that such "generalizations" tend to include everything, but the real brain.
74: </p>
75: <p><a href="#sect09"><b>Section 0.9 Summary</b></a> summarizes the main claims made in this paper.
76:
77: <p><b><a href="#references">References </a></b></p>
78:
79: <p><b><a href="#answers">Answers to Questions from Section 0.1 </a></b></p>
80: <a name="sect01"></a>
81:
82: <h3>0.1 The Power of Mental Set</h3>
83:
84: <p>Trying to describe and understand a new system we use concepts associated with the systems we already know and understand.
85: This metaphorical approach works well when the concepts we are trying to use are applicable to the new system. If not, we
86: may get caught into the pitfall of a misleading metaphor. A misleading metaphor creates a wrong mental set that prevents us
87: from understanding the new system. This powerful psychological phenomenon is well illustrated by the following old anecdote.</p>
88:
89: <p>The action takes place when streetcars were just introduced to replace horse driven cabs. A cabman attends a lecture on the
90: principles of operation of a streetcar. After the lecture he makes the following comment: "I understood almost everything.
91: The only part I missed is where they attach the horse."
92: </p>
93: <p>The cabman familiar with the concept of a horse driven vehicle had no difficulties recognizing a streetcar as a vehicle.
94: His knowledge base associated with horse driven cabs contained such useful concepts as wheels, shock absorbers, etc. that
95: are applicable to streetcars as well. However, the familiar notion of the driving force coming from a horse turned out to
96: be misleading. It created a wrong mental set that prevented the cabman from understanding the novel idea of an electrically
97: driven vehicle.</p>
98:
99: <p>The reader should be aware of the power of mental set. To experience this power, try to answer the following four questions.
100: Check how long it will take you to find the answers. If it takes too long, click on the answers.</p>
101:
102: <ul>
103: <li><a name="question1"><b>Question 1</b></a> (requires knowledge of American history)<br>
104: <font color="black"><i> Which American presidents are not buried in American soil?</i></font>
105: <a href="#answer1"> (answer)</a></li>
106: <li><a name="question2"><b>Question 2</b></a> (requires knowledge of zoology)<br>
107: <font color="black"><i> What big white animal can see backward as well as it can see forward?</i></font>
108: <a href="#answer2"> (answer)</a></li>
109: <li><a name="question3"><b>Question 3</b></a> (requires knowledge of physics)<br>
110: <font color="black"><i> A dog has a bunch of empty cans attached to his tale. How fast should the dog run to hear no sound
111: from the clinging cans?</i></font>
112: <a href="#answer3"> (answer)</a></li>
113: <li><a name="question4"><b>Question 4</b></a> (requires knowledge of mathematics)<br>
114: <font color="black"><i> There are two wallets and two pennies. How should you divide the money between wallets so one wallet would have two times
115: more money than the other?</i></font>
116: <a href="#answer4"> (answer)</a></li>
117: </ul>
118: If you didn't known all the answers in advance, you, probably, got some idea as to how a wrong mental set can influence
119: your thinking. Now, you are better prepared to discuss the problem of the whole brain. When it comes to the whole brain,
120: it is essential not to be too sure about anything. Bear with me.
121: <p>
122: In the modern era of information processing systems, a researcher trying to understand the brain has no difficulty recognizing
123: the brain as an information processing system. Depending on his/her background and interests, he/she then tries to use concepts
124: associated with the known systems for understanding the brain. Quite often, he/she gets caught into a "streetcar-horse" pitfall.
125: </p>
126: <p>The brain is such a remarkable system, that different aspects of its organization and work evoke associations with many known
127: systems. Like the magical mirror from the J.K. Rowling novel, the brain shows everyone what one wants to see. A computer
128: scientist familiar with symbol manipulation algorithms finds symbol manipulation algorithms. An expert in dynamical processes
129: finds dynamical processes. An applied mathematician interested in optimization procedures finds optimization procedures. A fuzzy
130: logic guru finds fuzzy logic. A physicist finds spin-glasses, holographic memory, and annealing. The list goes on and on.
131: Referring to this phenomenon, <a name="quot1"></a>
132: the neurophysiologist Burns (1958) wrote that (when one is dealing with the brain)
133: <font color="brown"> "it is distressingly easy to find what one is looking for and remarkably difficult to discern
134: the unsuspected or the unwanted."</font>
135: </p>
136:
137: <a name="sect02"></a>
138:
139: <h3>0.2 The Whole Human Brain as a Universal Learning Neurocomputer</h3>
140:
141: <p>There exists a large and rapidly growing set of various computational models aimed at understanding what can be loosely
142: referred to as "parts" of the brain and/or "parts" of its performance. It would take thousands of pages just to list the
143: titles of the respective publications. Important and interesting as they are, such <font color="brown">partial</font> models
144: cannot address the most fundamental question associated with the <font color="brown">whole</font> brain as a computing system:
145: <ul>
146: <font color="black"><i>
147: <li>How does the whole brain learn to perform mental computations?</i></font>
148: (Or simply speaking, how does the whole brain learn to think?)
149: </ul>
150:
151: <p>Imagine a theory of an airplane that discusses numerous parts of the airplane and tries to explain how these parts work.
152: The only question the theory doesn't address is how the whole airplane flies. I believe, this metaphor gives a fair
153: characterization of the current state of computational theory of the brain. To substantiate this claim consider the
154: following observations.</p>
155: <p>
156: A person with a sufficiently large external memory device can perform, in principle, any effective computational procedure.
157: As is well known, this observation had led famous English mathematician Alan Turing (1936) to the invention of his machine
158: and to a formalization of the intuitive notion of an algorithm.<p>
159: <p> We can extend Turing's observation and notice that a person with a good visual memory can perform, in principle,
160: any mental computations using an imaginary memory device. (See Baddeley, 1982.) Ignoring some severe, but theoretically
161: unimportant limitations on the size of the working space available via such a mechanism of mental imagery, we can conclude
162: that the human brain itself (not just a person with an external memory device) is a universal computing system.</p>
163: <p>
164: We are not born with the knowledge of all possible algorithms. Nevertheless, we can learn to perform, in principle,
165: any algorithm. By learning to perform certain computations with the use of an external memory device we automatically
166: learn to perform similar mental computations. This observation shows that the whole human brain is a universal programmable
167: computing system and that the effect of universal programming can be achieved via process we call learning.
168: This means that the whole human brain is a universal learning computing system.</p>
169: <p>There is a popular myth that universal learning systems don't exist. This myth is based on the "rigorous proof" that
170: no system can learn to simulate a grammar of Chomsky type 2 (context-free) and higher (types 1 and 0). This proof obviously
171: contradicts to the fact that we humans are universal learning systems.</p>
172: <ul>
173: <font color="black"><i>
174: <li>How can one explain this paradox?</i></font>
175: </ul>
176: The explanation is very simple. The formal definition of learning used to obtain the above rigorous proof doesn't
177: include the "special case" of human learning. (see <font color="blue">www.brain0.com</font>)
178:
179: <p><b>REMARK.</b> This paradox gives an example of a well known pitfall associated with the use of the so-called
180: "mathematical" mentality in connection with "physical" problems. The catch is that to find an adequate mathematical
181: definition of a nontrivial physical problem is as difficult as to solve this problem. For example, try to give a formal
182: definition, of the problem of simulating the behavior of electromagnetic field in a complex microwave device without the Maxwell
183: equations.</p>
184: <p><b>NOTE.</b> In this paper, I use the <font color="brown">Maxwell equation metaphor</font> several times. Interestingly enough, many aspects
185: of this metaphor turn out to be quite relevant to the issue of the general methodology of brain modeling and cognitive modeling.
186: Don't take this metaphor literally. Its goal is to obviate some subtle methodological pitfalls associated
187: with the use of "mathematical" (or "engineering") mentality in connection with the "physical" (biological) problem of the human brain
188: and human behavior.</p>
189: <ul>
190: <font color="black"><i>
191: <li>What is the nature of the brain's universality as a computing system?
192: </li>
193: <li>How can a brain-like universal computing system be implemented in a way consistent with modern neurobiological data?
194: </li>
195: <li>What data storage algorithm can be used by such a system to make it a universal learning system?
196: </li>
197: <li>What is short-term memory (working memory)?
198: </li>
199: <li>What is the explanation of mental imagery?
200: </li>
201: <li>How do we learn to perform computations with the use of an imaginary memory device after performing
202: similar computations with the real device?
203: </li>
204: <li>What is mental set?
205: </li>
206: <li>How does the brain dynamically change its attitude to adjust to a combinatorial number of different contexts?
207: </li>
208: <li>How can various psychological phenomena of short-term memory and mental set be connected with the properties of
209: protein molecules embedded in neural membranes?</i> (Modern neurobiological data point in this direction.)
210: </li></font>
211: </ul>
212: <p>The strategy of partial modeling produced no meaningful answer to this set of basic questions associated
213: with the whole human brain as a complex computing system. Therefore, the above-mentioned <font color="brown">airplane metaphor</font> may not be as
214: big an exaggeration as it may seem.</p>
215:
216: <a name="sect03"></a>
217:
218: <h3>0.3 On the Parts and the Whole</h3>
219: <ul>
220: <font color="black"><i>
221: <li>Why is everybody so busy developing "parts" for a complex system and nobody seems to care
222: about the fundamental requirements of the whole system?</i></font></ul>
223:
224: <p>The answer is mental set. It is "well known" that the whole brain is such an enormously complex and unexplored a system
225: that trying to develop its computational model at the present time would be a waste of time. Therefore, one should
226: concentrate for a while on more limited problems associated with "parts" of the brain and/or "parts" of brain's performance.
227: (<font color="brown">"A bird in the hand is worth two in the bush."</font>)</p>
228:
229: <ul><li><i>Is this a reasonable attitude?</i></li></ul>
230:
231: I argue that it is not. In real life, it is seldom possible to develop right parts for a
232: complex system without paying much attention to the requirements of the whole system. The same holds for reverse engineering.
233: It is hardly possible to reverse engineer nontrivial parts of a complex system without having a good working hypothesis as to what
234: this system is doing as a whole. Ignoring the whole leads one into the pitfall of<font color="brown"> the parts that don't fit
235: the whole.</font>
236: <ul><i>
237: <li>How complex should a meaningful computational model for the whole brain be?</li>
238: <li>Does it make sense to think about "simple" models of the whole brain? </li>
239: <li>Is it worth talking about the so called "basic mechanisms" of the brain?</li>
240: <li>Why has the traditional Artificial Intelligence (AI) research failed to explain the phenomenon of human thinking?</li>
241: <li>Doesn't this failure prove that the whole brain is indeed an enormously complex system?</li></i>
242: </ul>
243: <p>To get some answers to these important questions, we need to view the brain as a part of a system (Man,World). Let (W,D,B)
244: be a model of this system, where W is an external world, D is a set of man-like sensory and motor devices, and B is a computing
245: system simulating the work of man's nervous system. For the sake of simplicity, I will refer to system B as a model of the
246: brain or simply the brain. (At this general level, one can think of the rest of nervous system as being a part of D).</p>
247: <p>
248: It is useful to represent system (W,D,B) as a composition of two subsystems: B and (W,D); where B is the brain, and (W,D) is
249: the external world, W, as it appears to the brain via the set of devices, D. In this representation, both subsystems can be
250: treated as abstract computing systems (machines), the inputs of B being the outputs of (W,D) and vice versa. System B has
251: a finite formal representation. System (W,D) may have no finite formal representation. It doesn't matter, however, from the
252: viewpoint of simulation, since the external world W is "always there" to experiment with.</p>
253: <p>Let B(t) be a formal representation of B after time t, where t=0 corresponds to the "unprogrammed" (untrained) human
254: brain. I argue that the following general propositions are true:</p>
255: <ul>
256: <li><b>Proposition 1.</b> It is possible to find a relatively small (not too small!) formal representation of B(0).<br>
257: <b>Reason:</b> This representation is encoded in some form in the human genome. Some data suggest that it may be small enough
258: to fit into a single floppy disk. (Smaller than traditional AI programs.)<br><br>
259: <li><b>Proposition 2.</b> Any adequate formal representation of B(t) with time t greater than, say, 10 years, must be very
260: big: hundreds of gigabytes or, perhaps, terabytes. (Bigger than traditional AI programs.)<br>
261: <b>Reason:</b> This representation must include, in some form, the description of the individual experience of a person.<br><br>
262: <li><b>Proposition 3.</b> It is practically impossible to reverse-engineer B(t) (for a big t) without first reverse-engineering B(0).<br>
263: <b>Reason:</b> The main part of description of B(t) depends on the properties of the "informal" external world W. This part of
264: description of B(t) cannot be "preprogrammed" (at least for the first time).<br><br>
265: <li><b>Proposition 4.</b> It is practically impossible to develop adequate models of nontrivial parts of behavior of
266: system (W,D,B(t)) without a formal representation of B(t). That is, it is practically impossible to have adequate purely phenomenological
267: ("brainless") cognitive theories.<br>
268: <b>Reason:</b> It is like trying to simulate the behavior of the electromagnetic field in a complex microwave device without
269: the Maxwell equations describing the "basic mechanisms" of the electromagnetic field. (<a href="#sect065">See section 0.6.5</a>).<br><br>
270: </ul>
271:
272: Let us assume that propositions 1-4 are true. <i>What should be the right methodology of brain modeling and cognitive modeling?</i>
273: I argue that
274: <ol><li>The main goal of brain modeling should be reverse-engineering B(0).</li>
275: <li> The main goal of cognitive modeling should be deriving models of <font color="brown">different specific</font> cognitive phenomena from
276: <font color="brown"> the same </font> model of B(0) (or B(t) ) interacting with <font color="brown"> different</font> "external systems" (W,D).</li>
277: </ol>
278:
279: <p><b>REMARK.</b> Using the Maxwell equation metaphor, item 1 can be loosely compared with the problem of finding the Maxwell equations.
280: Item 2 can be compared with the problem of deriving different specific classical electromagnetic phenomena from the same Maxwell equations
281: coupled with different boundary conditions and sources (different external worlds).</p>
282:
283: <a name="sect04"></a>
284:
285: <h3>0.4 Motivation and General Approach: the Concept of E-machine</h3>
286:
287: <p>As a system engineer with a strong background in physics and mathematics, and some knowledge of
288: neurobiology and psychology, I became seriously interested in the problem of the whole brain as a computing system in 1966.
289: This study is an attempt to summarize and systematize some insights resulted from this long-term interest. I hope some of
290: these system engineering insights will be found thought-provoking by a broad range of readers interested in brain modeling
291: and cognitive modeling.
292: I also hope that some of these insights will be of interest to all users of the brain. After all, the whole brain is a complex
293: engineering system (designed by a very smart "system engineer"), and it is the job of a system engineer to deal with the problem
294: of reverse engineering this system. </p>
295: <p>I want to emphasize that, in this study I am not interested in brain-inspired engineering and/or brain-inspired mathematics.
296: I believe that these popular approaches ( whatever useful and important they might be from an engineering and/or a mathematical
297: viewpoint) do not help in understanding the real brain. On the contrary, they tend to create misleading metaphors that inhibit
298: one's ability to understand the brain. The truth is, that the moment one defines a brain-inspired engineering and/or mathematical
299: problem one gets a problem quite different from the original biological problem. What is even worse, this irrelevant problem
300: looks relevant and leads one into the pitfall of a misleading metaphor.</p>
301: <p>I argue that in the case of the "physical" problem of the human brain and human intelligence, the whole game of definitions
302: doesn't make much sense.<i> What does the word "define" mean? What does the word "mean" mean? What does it mean "to understand?"</i>
303: And so on. These are "physical" questions! To answer these questions one needs to understand how these words (or any other words
304: of our language) interact with our brain (more accurately, with the whole system (W,D,B(t)) mentioned in section 0.3).</p>
305:
306: <p><i>Is it valid to use words which are not defined?</i> For a person with a "physical" mentality (like me) the answer is "yes".
307: As Oliver Heaviside, the inventor of the operational calculus,
308: once said:<font color="brown"> "Shall I refuse my dinner, because I do not fully understand .. digestion?"</font>
309: For a person with an orthodox "mathematical" mentality this question is a nightmare. When it
310: comes to the problem of the whole human brain, everything becomes a "physical" problem, even the mathematics itself.</p>
311:
312: <p>Many people (especially engineers) believe that the main motivation for the study of the brain as an information processing system
313: is to learn how to build intelligent robots. Though, as an engineer, I am very interested in robots, I do not share this belief.
314: As the owners and the users of the brains we don't need any additional motivation to be interested in understanding the
315: brain.</p>
316: <p> The brain is our most important "personal computer." We need to understand this computer to become better users.
317: We need to know how to program (educate) this computer and how to fix it when something goes wrong. And with all the
318: brainwashing going on around the world, we need to know how to protect this computer against all kinds of "software viruses".</p>
319:
320: <p>In this study, I am going to make several controversial propositions about the basic computational mechanisms of the whole brain.
321: To avoid accusations that I will be making overly broad claims I provide the following</p>
322:
323: <p><b>DISCLAIMER.</b> I only claim that the ideas discussed in this study are applicable to my own brain (since this was the only
324: brain I could experiment with). I leave it to the reader to decide as to whether these ideas are also applicable to his
325: or her brain.</p>
326:
327: <p>My main claim is that the whole brain is a dynamically reconfigurable universal learning neurocomputer arranged on the principle
328: of E-machine. (Eliashberg, 1967, 1979, 1981, 1988b, 1989, 1990b). In Eliashberg (1967), some systems similar to those discussed in this
329: study were called Associative Automata. The name E-machines was introduced in (this author, 1979) to distinguish these rather specific
330: systems from other types of associative learning machines. The letter "E" implies the word "Excitation." It was meant to emphasize a
331: connection between the concept of E-machine and the old neurophysiological notion of "residual excitation" in the brain as the mechanism
332: of mental set (Vvedensky, 1901).</p>
333:
334: <p><b>NOTE.</b> There is a public company eMACHINES Inc. that makes personal computers. The products of this company have
335: nothing to do with the E-machines discussed in this study. As was mentioned above, the name E-machine was introduced in
336: 1979 (before the creation of the above company). When this company was created, I called its CEO and told him that the
337: name E-machines is used in my copyrighted manuscript. He didn't pay any attention. I understand that a trademark and a
338: copyright are two different things. At any rate, I believe that my copyright (officially registered with the Library of
339: Congress) allows me to use this name without violating anybody's trademark.</p>
340:
341: <ul><li><i> What is an E-machine?
342: </i></ul>
343: <p>In the general case, an E-machine is a complex associative learning system built from several (many) primitive associative
344: learning systems referred to as primitive E-machines or associative fields. A complex E-machine may also include blocks called
345: nuclei that provide interface among associative fields and between associative fields and the external (sensory and motor) devices.</p>
346: <p>Associative fields can form associations of different modalities and can be organized in hierarchical structures.
347: In such hierarchical structures, associative fields interact via large sets of associative fibers. A complex E-machine may also
348: include an "activating system" and "centers of emotion." By mentioning the latter mechanisms, I want to emphasize that the
349: existence of such "noncomputational" mechanisms in the brain does not contradict the general notion of the whole brain as a
350: dynamically reconfigurable universal learning neurocomputer.</p>
351:
352: A primitive E-machine has two main types of states:
353: <ol>
354: <li>The states of encoded ("symbolic") long-term memory (LTM) that represent knowledge (the brain's software). These states
355: are referred to as G-states, the letter "G" implying the notion of a modifiable synaptic Gain.</li><br><br>
356: <li>The states of distributed analog ("dynamical") short-term memory (STM) and intermediate-term memory (ITM) that serve as
357: a mechanism for the dynamic reconfiguration of the knowledge stored in LTM. These states are referred to as E-states,
358: the letter "E" implying the notion of "residual Excitation". </li>
359: </ol>
360:
361: <p>Paper 2 describes a formalism that connects the dynamics of the phenomenological E-states with the conformational dynamics of
362: ensembles of protein molecules embedded in neural membranes. (Eliashberg, 1989, 1990a.) If correct, this approach would
363: justify quite complex assumptions about the dynamic properties of E-states.
364: (The program CHANNELS.EXE (see <font color="blue">www.brain0.com</font>) illustrates some nontrivial possibilities of this formalism by allowing
365: the user to experiment with different assumptions about the microscopic structure of ionic channels.)</p>
366:
367: <p>In a primitive E-machine, G-state can have three components: G=(GX,GS,GY), where GX is the state of Input LTM (ILTM)
368: that represents the input parts of associations, GY is the state of Output LTM (OLTM) that represents the output parts of
369: associations, and GS is the state of Structural LTM (SLTM) that represents intermediate connections (and other modifiable
370: parameters). GX and GY can be characterized as encoded ("symbolic") memories, and GS, as a "connectionist" memory.
371: (Many traditional associative neural network models can be viewed as implementations of some special cases of primitive
372: E-machines.)</p>
373:
374: <p>In the general case, a primitive E-machine is a probabilistic rather than a deterministic system: it employs a random choice
375: to retrieve data from OLTM. In the case of a neural implementation, this choice is accomplished via reciprocal inhibition and
376: fluctuation phenomena. Coupled with such a random choice, the transformations of dynamical E-states provide a flexible mechanism
377: that controls the probabilities of symbolic processes in E-machines. Because of this property, in Eliashberg (1979), the
378: E-machines were characterized as "non-classical symbolic processors."</p>
379:
380: <p><b>NOTE.</b> The basic concepts associated with classical symbolic machines are useful for understanding some important
381: aspects of the behavior of E-machines (in the same way as, say, the concepts associated with linear systems are useful for
382: understanding some important aspects of the behavior of nonlinear systems.) In the general case, however, an E-machine is
383: just an E-machine, and its behavior cannot be properly characterized in either "purely symbolic" or "purely dynamic" terms.</p>
384:
385: <p>At the general information processing level, the metaphor the brain as an E-machine provides a nontrivial explanation of how
386: the brain can process symbolic information without moving symbols in a "symbolic" read/write memory. Interestingly enough,
387: the effect of such a symbolic read/write memory (working memory) is achieved via transformations of "dynamical" E-states.
388: Since the transformations of E-states are done in a massively parallel way, this general style of information processing
389: allows the brain to combine the ability to accumulate practically unlimited amount of data in the course of learning with
390: the ability to quickly process this data in the course of decision making. As will be shown, this general approach leads to
391: a rather simple (not too simple) concept of a universal learning computing system (a model of B(0) from Proposition 1).</p>
392: <p> An explicit example of a universal learning system is described in detail in Paper 3. The program
393: EROBOT.EXE (see <font color="blue">www.brain0.com</font>) allows the user to perform interesting experiments with this educational model.
394: Papers 1 and 2 (from this series of connected papers) provide a neurobiological background that allows one to understand how
395: the brain of this robot could be implemented as a neural network.</p>
396:
397: <p><b>NOTE.</b> The general idea that the brain uses a combination of symbolic and dynamical computational mechanisms is rather
398: old. Several researchers entertained different versions of this idea. (See, for example, Collins and Quilian, 1972, Anderson, 1976).
399: From time to time, this old idea is rediscovered by new enthusiasts (who soon find out that it is a hard nut to crack.) The metaphor
400: the brain as an E-machine uses a specific formalization and extrapolation of this general idea and integrates it with some other
401: basic ideas.</p>
402:
403: <p><b>DISCLAIMER.</b> To avoid misunderstanding, I want to emphasize that many "partial" ideas employed and developed in this study
404: (perhaps all such ideas) can be found in a variety of different forms elsewhere. It is a specific formalization, extrapolation and
405: integration of such partial ideas that makes a difference. <font color="brown">The whole system is more than the sum of its parts and
406: the devil is in the details.</font></p>
407:
408: <a name="sect05"></a>
409:
410: <h3>0.5 The Brain 0 Project</h3>
411:
412: <p>In section 0.3, I argued that the main goal of brain modeling should be reverse engineering the "unprogrammed" brain B(0).
413: I also argued that, without an adequate model of B(0), one cannot reverse engineer B(t) for a big t, and without B(t), it is
414: impossible to do a simulation of nontrivial cognitive phenomena in system (W,D,B(t)).Therefore, the main goal of this study is
415: to show how the metaphor the brain as an E-machine, outlined in the previous section, can be used as a tool (a working hypothesis)
416: for reverse engineering B(0). I will refer to the corresponding reverse engineering project as the <b>Brain 0 Project</b> (pronounced
417: "the brain zero project.")</p>
418:
419: <p>In what follows, I formulate five basic methodological principles (the "five commandments") that determine the general methodology
420: of this project. In section 0.6, I will demonstrate some methodological pitfalls that result from a violation of these principles.</p>
421:
422: <h4>0.5.1 Methodological Principles (the "Five Commandments")</h4>
423:
424: <p><b>Principle 1</b> One should view the problem of developing a mathematical (computational) model of B(0)
425: as a <font color="brown">"physical"</font> (biological) rather than a <font color="brown">"mathematical"</font> (or engineering) problem.
426: That is, one should not try to <font color="brown">"define"</font> this problem. One just needs to reverse engineer B(0) in whatever
427: possible way. (To define a physical problem is essentially the same as to solve this problem!) </p>
428:
429: <p><b>Principle 2</b> One should not use the criterion of <font color="brown"> efficiency</font> to evaluate the quality of
430: a model of B(0). The quality of such a model is determined only by its ability to explain psychological and neurobiological facts.</p>
431:
432: <p><b>Principle 3</b> One should try to falsify a model of B(0). If one finds a fact that seems to contradict the model one
433: needs to either show how a model can explain this fact or try to modify the model, so it is capable of explaining this fact.
434: There should be no such thing as an <font color="brown">unfalsifiable</font> model of B(0). (An unfalsifiable model is either
435: a perfect model of B(0), which is extremely unlikely, or an <font color="brown">irrelevant</font> model.)</p>
436:
437: <p><b>Principle 4</b> One should <font color="brown">reject wrong metaphors</font>. (An uncontrolled accumulation of misleading
438: metaphors is a killer for any reverse engineering project.)</p>
439:
440: <p><b>Principle 5</b> First of all, one should try to reverse-engineer <font color="brown"> the most
441: fundamental principles</font> of organization and functioning of B(0) as a computing system. These fundamental principles are
442: independent of a specific implementation of B(0). However, one should pay much attention to the specific neurobiological
443: implementation of B(0) to be able to reverse engineer the above fundamental principles.</p>
444:
445: <p>I want to emphasize the importance of principles 3 and 4 (which are not popular among brain modelers and cognitive modelers).
446: There is a tacit agreement that one can ignore these principles (at least for a while), because there is not enough reliable
447: data about the brain to separate right metaphors from wrong metaphors. I argue that this is far from the truth.
448: As the users of the brain we have an unlimited source of reliable psychological data about the whole human brain.
449: The analysis of language gives one an especially rich and important source of such data. Combined with the stream of
450: increasingly reliable neurobiological data this psychological information allows one to efficiently use principles 3 and 4.</p>
451:
452: <p>Principle 2 is also critically important. To understand why efficiency is a wrong criterion, see sections 0.6.2 and 0.6.4.
453: Also consider the following metaphor.</p>
454:
455: <p>Imagine a group of mathematicians trying to solve a system of many equation, each mathematician concentrating
456: on just a few equations. These mathematicians would be able to find many partial solutions satisfying their respective
457: subsets of equations. It is obvious, however, that not all such partial solutions would satisfy the whole system of equations.
458: Therefore, paraphrasing a well known AI thesis about sufficiency, efficiency and necessity I can say that to reverse engineer B(0),
459: it is <font color="brown">not sufficient</font> to concentrate on <font color="brown">efficient</font> solutions of just some
460: psychological and/or neurobiological equations. It is also <font color="brown">necessary </font>to try to select those solutions that
461: satisfy increasingly larger sets of such equations and reject those that don't.
462:
463: <p> The importance of the first part of Principle 5 is easy to understand. The most fundamental constraints have higher weights than any
464: other more specific constraints, so they should be satisfied first. To understand why, consider the following metaphor:<br>
465: Imagine a model of an electrical transformer that simulates the ability of this transformer to produce noise and heat, but doesn't
466: address the question as to how this transformer transforms electricity. The ability to explain how a transformer produces heat and noise
467: may be an important advantage of a model. However, it is important only in a model that also explains how a transformer
468: transforms electricity. Similarly, one should not expect too much from a computational model of the brain that simulates the brain's
469: ability to produce, say, oscillations, but doesn't address the question as to how these oscillations are connected with the brain's ability
470: to process information.</p>
471: <p>The importance of the second part of Principle 5 is less obvious. This issue is discussed in Section 0.7. The catch is that one
472: cannot easily separate the problem of the computations performed by the brain (the problem of the "algorithms of thinking" per se)
473: from the problem of brain hardware.</p>
474:
475: <a name="sect052"></a>
476: <h4>0.5.2 Iterative Reverse Engineering Strategy</h4>
477:
478: <p>To be able to deal with a large number of "equations" needed for reverse engineering B(0), I use a strategy that can be
479: loosely described as the following iterative algorithm. (The real reverse engineering process is never so straightforward,
480: so this algorithm is just a useful metaphor.)</p>
481:
482: <p><b>Step 0</b> Formulate a set of questions addressing some fundamental properties of B(0) as a computing system.
483: Invent an initial approximation for B(0), call it B<sub>0</sub>(0), that seems to answer these questions. Study B<sub>0</sub>(0) to
484: show that it does answer the above mentioned set of questions.</p>
485:
486: <p><b>Step 1</b> Expand the set of questions and improve the current model, call it B<sub>i</sub>(0), to answer this expanded set of
487: questions.
488:
489: <p><b>Step 2</b> Study B<sub>i</sub>(0) to see if it indeed answers the current set of questions. If not all questions can be answered,
490: try to adjust the model to answer all the questions.</p>
491:
492: <p><b>Step 3</b> If step 2 is a success, go to step 1. Else, go to step 0 and try to invent a better initial approximation.</p>
493:
494: <a name="sect053"></a>
495: <h4>0.5.3 Initial Approximation. Converging vs. Diverging Iterations</h4>
496:
497: <p>As is well known, the success of an iterative process depends on the quality of its initial approximation. I argue that this is
498: true for the iterative reverse engineering process described in section 0.5.2.</p>
499: <p>If the initial approximation for B(0), invented at step 0, is "supercritical" (is sufficiently close to the original)
500: one will get a "converging" process. The more one will improve such an initial approximation the more possibilities for its
501: further improvements one will find. Eventually, one will converge to an adequate model of B(0).</p>
502:
503: <p>If the initial approximation is too far from the original ("subcritical"), one will get a "diverging" process. The more
504: one will try to improve such an inadequate initial approximation the more discrepancies one will see. Soon it will become
505: obvious that this initial approximation doesn't work.</p>
506: <p>Using this metaphorical language, the metaphor the brain as an E-machine, outlined in section 0.4, can be characterized
507: as an attempt to develop a "supercritical" initial approximation for B(0).</p>
508:
509: <a name="sect054"></a>
510:
511: <h4>0.5.4 Conceptual Levels</h4>
512:
513: <p>The enormous progress in system engineering was largely a result of two critically important developments:
514: <ol>
515: <li> Development of a set of system-theoretical and system engineering concepts that allow system engineers
516: to efficiently deal with different conceptual levels of design (and understanding) of complex information processing systems.</li>
517:
518: <li>Development of computer-based tools satisfying the requirements of these different conceptual levels.</li>
519: </ol>
520: <p>Since the whole brain is a complex information processing system, I argue that the issue of conceptual levels and computer-based
521: tools, supporting these levels, is of critical importance for the development and understanding of models of B(0).</p>
522: <p>The E-machine formalism corresponds to some "intermediate" conceptual level for the representation of B(0). I will try to show
523: that at this level the models of B(0) become, in some sense, the most simple and the most understandable without losing their
524: essential properties as computing systems. From this intermediate level, one can go "up" -- toward cognitive models
525: (models of behavior of system (W,D,B(0) ), and "down" -- toward neural network models. Interestingly enough, to find a consistent
526: neurobiological interpretation of the phenomenological E-states, we will need to go "below" traditional neural network models
527: and deal with the statistical dynamics of ensembles of protein molecules.</p>
528: <p>In contrast with the "top-down" methodology of AI and the "bottom-up" methodology of ANN, the methodology of this study
529: can be best characterized as the "middle-up-down-middle" approach. (Note that this term implies a loop!)</p>
530:
531: <h4>0.5.5 The Models of B(0) Can Be and Must Be Represented Explicitly</h4>
532: <p> According to Proposition 1 from section 0.3, it is possible to find a relatively simple formal representation of B(0).
533: Therefore all models of B(0) can and must be represented explicitly, so all the claims could be verified. I argue that no special
534: language is needed for such a representation, and such popular languages as C or C++ are good enough. In this study, a combination of
535: a C-like notation and a scientific notation is used during the discussion of the models. The Microsoft Visual C++ version 6.0 is
536: used for computer simulation.</p>
537: <p> As a system engineer, I am biased against the so-called "verbal models" of the brain. It has been my experience, that a good intuition
538: about the principles of organization and functioning of a computing system can always be transformed into an adequate formalism. If,
539: by whatever reason, such a transformation is not possible, it usually means that there is something wrong with the intuition.</p>
540: <p>A model doesn't have to be (in fact, should not be) as complex as the original to serve as a useful metaphor for understanding
541: the original. However, as Einstein said, "everything should be made as simple as possible, but <font color="brown">not simpler</font>."
542: That is, <font color="brown">a model of a working system must be a working system</font>. A model of B(0) that "explains",
543: but does not work, does not explain either.</p>
544:
545: <h4>0.5.6 In the General Case, The External System (W,D) Has No Formal Representation</h4>
546: <p> Let us return to the cognitive system (W,D,B(0)) discussed in Section 0.3. Though the models of B(0) can be and must be represented
547: explicitly, in the general case, the external system (W,D) has no finite representation. Therefore, there is a good excuse for having
548: informal theories of behavior of system (W,D,B(0)), more so, system (W,D,B(t)).</p>
549: <p><b>REMARK.</b> There are informal theories and informal theories. A good informal theory should be associated with
550: a formal model of B(0) (or B(t)). Given an explicit model of B(0), all predictions of such an informal cognitive theory can be
551: verified experimentally (at least, in principle), since devices, D, have a finite representation and the "informal" external world, W,
552: is "always there" to experiment with. Simply playing with words and trying to define one set of vague verbal concepts through another set of
553: equally vague verbal concepts doesn't decrease the total level of vagueness.</p>
554:
555: <p>It is interesting to mention that to reverse engineer the main principles of organization and functioning of B(0), it may be sufficient to
556: deal with rather simple models of (W,D). Such models can often be described explicitly and used in computer simulation.
557: The program EROBOT.EXE (see <font color="blue">www.brain0.com</font>) illustrates this methodology.</p>
558:
559: <h4>0.5.7 On Engineering Expertise and Physical Mental Set</h4>
560: <p>Reverse engineering B(0) requires an unusual combination of the knowledge base of a system engineer and the mental set of a physicist.
561: The reason for this requirement can be explained as follows.</p>
562: <ul>
563: <li>First of all, B(0) is a real complex information processing system. Reverse engineering such a system requires
564: a broad system engineering expertise.</li>
565: <li>Second of all, B(0) is a system designed by Mother Nature (not by human system engineers). Deciphering such a natural system
566: requires a "physical" (rather than "engineering") mentality.
567: </li>
568: </ul>
569: <p><i> Why is traditional engineering mentality inadequate for reverse engineering B(0)?</i></p>
570: <p>
571: <p>Evolution uses design principles quite different from those employed by system engineers. We try to make our systems
572: more understandable and testable. This costs us additional resources. Evolution tends to solve its problems with
573: minimum resources. This makes its designs look clever. This also makes them difficult (for a human system engineer)
574: to understand. One needs a "physical" mental set, to crack such designs. Section 0.6 explains why.
575: </p>
576:
577: <h4>0.5.8 Asking the Right Questions</h4>
578: <p>To make a progress in reverse engineering B(0) via the metaphorical loop of Section 0.5.2, one needs to ask the right questions at
579: <b>Step 1</b> of this loop. Formulating the right questions is not a trivial task. Quite often, finding the right question is even
580: more important than finding the right answer to this question.<br><br>
581: One of the strong points of the loop of section 0.5.2 is that it helps one to formulate the right questions.
582: If the iterative process 1-2-3-1... goes in the right direction (is "converging"), the current approximation, B<sub>i</sub>(0), generates
583: a system of concepts that can be efficiently used to formulate questions needed for the synthesis of the next approximation
584: B<sub>i+1</sub>(0).</p>
585:
586: <a name="sect06"></a>
587:
588: <h3>0.6 Methodological Pitfalls</h3>
589: <ul>
590: <li><i>What can go wrong if a brain modeler or a cognitive modeler ignores the "physical" methodological principles (the "five commandments")
591: from section 0.5.1?</i></li>
592: </ul>
593: The following subsections attempt to give an answer to this question. According to the Murphy's law,
594: <font color="brown">if something can go wrong it will</font>.
595:
596: <h4> 0.6.1 Redefining a "Physical" Problem</h4>
597: <p> Referring to this pitfall, Zopf, Jr. (1962) wrote: "I do hope that we shall not fall into the trap
598: for a time occupied by a friend of mine who, speaking of a device he had built, said, in all seriousness, <font color="brown">"If the
599: conditional reflex were only like this, then what I have built is a model of the conditional reflex."</font>"</p>
600:
601: <a name="sect062"></a>
602: <h4>0.6.2 Concentrating on Efficiency and Ignoring Necessity</h4>
603:
604: <p>To obviate this pitfall, consider the following <font color="brown">digging-for-a-treasure</font> metaphor. Imagine a crowd of
605: diggers with sophisticated equipment digging in wrong places and a person with a shovel digging in the right place.
606: (Everybody, except this person, knows that one should not dig in this place!) In this (unlikely, but not impossible!)
607: situation this single person would be able to make more progress than the crowd. The point is that it doesn't matter how efficiently
608: one can dig if one is digging in a wrong place. In fact, the more efficiency the more mess.<br>
609: Using this metaphor, one can say that to find the treasure (an adequate model of B(0)) it is not sufficient to have efficient
610: equipment and good working conditions (soft soil, a lot of light, etc.) It is <font color="brown">absolutely necessary</font>
611: to dig in the right place, no matter how difficult the working conditions there might be.</p>
612:
613:
614: <h4>0.6.3 Underestimating the Power of Basic Mechanisms</h4>
615:
616: <p>The example of physics warns us that one should not underestimate the power of simple basic mechanisms of
617: Mother Nature. I argue that this warning is relevant to the problem of reverse engineering B(0).</p>
618:
619: <p>Sometimes, an adequate formalization, extrapolation and integration of a set of simple basic mechanisms produce
620: a <font color="brown">"critical mass" effect</font>. The introduction of the so-called "displacement current" in the Maxwell
621: equations gives a classical
622: example of this interesting phenomenon. All of a sudden, this simple addition to the set of known basic laws of electricity
623: and magnetism, allowed J.C. Maxwell to create his famous equations that cover the whole range of arbitrarily complex classical
624: electromagnetic phenomena.</p>
625:
626: <p>I argue that the addition of E-states to the set of traditional mechanisms of associative memory and learning
627: is capable of producing a somewhat similar "critical mass" effect. All of a sudden, it becomes possible to get a uniform
628: explanation of a broad range of psychological phenomena of working memory and mental set. Quite remarkably, it also becomes
629: possible to naturally connect these "high level" psychological phenomena with a "very low level" neurobiological data about
630: the properties of protein molecules. (There is something here to think about!)</p>
631:
632: <h4>0.6.4 Looking for the Best Solution among Wrong Solutions</h4>
633:
634: <p>Optimization is a very important word in engineering. In this competitive world, one needs to have a better product
635: than the competition to be able to sell this product. So one needs to optimize, optimize and optimize.
636: This optimization mentality is naturally transferred to brain modeling. A lot of time is spent in discussions as to
637: why a certain neural network works better than other neural networks and why a certain AI algorithm displays a better
638: performance than other AI algorithms. (Try to sell a brain modeling idea, and you are immediately asked to prove that
639: your idea can solve a certain engineering problem better than other known "brain inspired" ideas.)</p>
640: <p>Several years ago, I had a privilege to work on a government sponsored ANN project with a team of prominent researchers
641: (Prof. B. Widrow, Dr. M.E. Hoff and Dr. M. Gluck). Our goal was to design a system for the recognition of a
642: thousand or more trajectories of ballistic objects by analyzing several frames that contained positions of the
643: objects at different time moments. We got better than 90% recognition rate, whereas a previous team got less than 80%.
644: Since this was a brain-inspired engineering project (not a brain modeling project!) we could say that our "neural network"
645: was better than their "neural network."</p>
646: <p>Unfortunately, such a simple engineering criterion of quality leads one into a pitfall when applied to the problem of
647: reverse engineering the brain. Consider, for example, the popular problem of pattern recognition. There are many different
648: ANN models and AI algorithms that compete for the title of the best pattern classifier. Such systems learn from examples
649: and seem to provide good metaphors for understanding the process of pattern recognition in the brain.</p>
650: <ul><li><i>Should the best classifier also win the title of the best brain modeling metaphor for pattern recognition?</i></li></ul>
651:
652: <p>The situation is not so simple. I argue that the whole class of models that learn to perform optimal pattern classification
653: is largely irrelevant to the problem of pattern recognition in the brain. The trouble is that the human pattern recognition
654: is context dependent.</p>
655: <p>Consider the question: <i>"What is this?"</i> In the context created by this question a person will behave as a pattern
656: classifier. He/she may answer, for example, that this is a book. His/her brain was able to distinguish a book from other objects,
657: say, a box, a hard disk drive, etc. Now consider the instruction: <i>"take this."</i> In this context, it is no longer important that
658: "this" is a book. What is important is the object's size, weight, etc. The experience that the brain acquired by learning
659: to take a book is applicable to the situation when one needs to take a box or a disk drive. In other words, <font color="brown">
660: the same object
661: can be treated as a member of different classes depending on context.</font></p>
662: <p>Zopf, Jr. (1962) had emphasized the critical importance of this issue. Optimizing performance in one context may
663: completely destroy performance in a large number of other contexts.</p>
664:
665: <a name="sect065"></a>
666:
667: <h4>0.6.5 "Pure" Phenomenology </h4>
668:
669: <p>Imagine a physicist who wants to simulate the behavior of electromagnetic field in a complex microwave device, say,
670: the Stanford Linear Accelerator (SLAC). Assume that this physicist doesn't know about the existence of the Maxwell equations
671: and, even more importantly, doesn't believe that the complex behavior he observes may have something to do with such simple
672: equations. (In the AI jargon this physicist would be called a "scruffy." If he believed in the existence of the basic equations
673: he would be called a "neat.").</p>
674: <p>So this "scruffy physicist" sets out to do a purely phenomenological computer simulation of the observed complex behavior
675: per se. Anyone who has been involved in a computer simulation of the behavior of electromagnetic field in a linear accelerator
676: can easily predict the results of this gedanken experiment. (I spent a few years doing such a simulation as a computational
677: physicist at the medical division of Varian Associates.)</p>
678: <p>In the best case scenario, the above mentioned scruffy physicist comes up with a computer program (with a large number of
679: empirical parameters) that would be able to simulate the behavior of electromagnetic field in a very narrow range. This computer
680: program would have <font color="brown">no extrapolating power </font> and would not be accepted by the SLAC community as a theory of a linear accelerator.</p>
681: <p>Note that it would be impossible to reverse engineer the Maxwell equations (a metaphorical counterpart of B(0)) from
682: the analysis of the behavior of electromagnetic field in such a complex "external world" as SLAC. I argue that, similarly, it is
683: impossible to reverse engineer B(0) from the analysis of such complex cognitive phenomena in system (W,D,B(t)) as playing chess,
684: solving complex mathematical problems, story telling, etc. No wonder AI got into trouble with the "physical" problem of
685: the human brain and human intelligence.</p>
686:
687: <h4>0.6.6 "Brainless" Theories of Language </h4>
688: <p> I argue that an adequate information processing (computational) theory of language can only be derived from an adequate
689: computational model of B(0). It takes a lot of time (t > 1year) for the "language-ready" B(0) to change into a B(t) that has some
690: rudimentary language. According to Proposition 2 from Section 0.3, B(t) has a much more complex representation than B(0).
691: To understand how B(t) uses language one needs to understand the process of learning that transforms B(0) into B(t).
692: The metaphor the brain as an E-machine offers some nontrivial insights into this problem. It suggests that
693:
694: <ul><li> The grammar of a natural language can be learnt</li>
695: <li>There is no such thing as a representation of a grammar independent of the representation of many other activities of B(t)</li>
696: <li>Any sufficiently expressive "free" motor channel can be used to represent a natural language.</li>
697: </ul>
698:
699: <p>I argue that an attempt to artificially separate language from other human activities leads one into the pitfall of a wrong
700: dimensionality similar to the attempt to separate the description of the behavior of electric field from the description of the
701: behavior of the whole electromagnetic field. This pitfall of a wrong dimensionality can be formulated as the following
702: "catch 22"</a>: <font color="brown"> An attempt
703: to artificially reduce the dimensionality of a "physical" problem, to simplify this problem, makes the problem more
704: difficult.</font></p>
705:
706:
707: <h4>0.6.7 Inadequate "Physical" Formalism</h4>
708:
709: <p>One of the great intellectual achievements of the last century was the discovery of universal languages. For example,
710: any computable function can be described in terms of any universal computing system, say, a universal Turing machine.</p>
711: <p>This important mathematical result, taken out of its proper theoretical context, and improperly applied to
712: a "physical" problem, may lead one into a pitfall. The catch is that not all mathematically equivalent formal representations
713: of a real "physical" system are equally useful for our brain. Only a handful of such representations qualify as adequate
714: "physical" representations.</p>
715: <p>An adequate "physical" formalism uses variables and functions that make "physical" sense. As a result of that, it generates
716: a system of intuitive concepts (a metaphorical language) that allows our brain to efficiently deal with (mentally simulate)
717: the "physical" system in question, and to understand the implications of the formalism. In turn, an adequate metaphorical
718: language provides heuristic considerations for the development of an adequate formalism. And so on.</p>
719: <p>It is easy to verify that these two critically important components (an adequate formalism and an adequate system of
720: intuitive concepts) are present in any good science and any good engineering. To clearly see the pitfall associated with
721: an inadequate "physical" formalism, consider the following two extreme (intentionally exaggerated) examples.</p>
722:
723: <p>At one extreme, imagine a program for a universal Turing machine that simulates the Maxwell equations (with some boundary
724: conditions) with a great accuracy (say, 128 bit). Ignoring the "practical" considerations of time and efficiency, this program
725: would, eventually, produce correct simulation results. This "correct" representation, however, would not qualify as an
726: adequate "physical" formalism because it would not generate an adequate metaphorical language. Accordingly, our brain would
727: not be able to deal with this representation.</p>
728:
729: <p>At another extreme, imagine a system of millions of nonlinear differential equations (with some initial conditions) describing
730: the "low level" dynamics of the circuitry of a personal computer running a complex piece of software, say, the Microsoft Office.
731: The main problem here is not in the differential equations themselves (which, in fact, would not be too difficult to describe),
732: but in their initial conditions which, in this example, represent the Microsoft Office. The same set of differential equations
733: with different initial conditions would represent Linux and any other possible computer program. This "correct" representation,
734: however, would not qualify as an adequate "physical" formalism, because our brain would be unable to efficiently deal with such a
735: representation.
736:
737: <h4>0.6.8 Confusing the Problem of Reverse Engineering the Human Brain with the Problem of Understanding the Human Mind</h4>
738:
739: <p>I want to emphasize that the problem of reverse engineering B(0) is quite different from the problem of understanding the
740: behavior of system (W,D,B(0)) or, even more so, system (W,D,B(t)) with a large t. The former problem is complex, but finite
741: (and solvable). The latter problem is infinite. Given an adequate model of B(0) we will always be able to improve our
742: understanding of the behavior of (W,D,B(t)), with bigger and bigger t, but will never be able to say that we have reached a
743: "complete" understanding.</p>
744: <p>It is important to emphasize that the human mind and human intelligence are not the properties of the human brain alone, but
745: the properties of the whole system (W,D,B(t)), where the external world W includes other people.
746: We are all parts of the same giant system (People,World), and we cannot understand ourselves without understanding the whole system.
747: While the generations of people come and go, our world becomes more and more complex. New brains B(0) are exposed to this more
748: complex world and become more complex systems B(t) which makes our world even more complex. And so on. The whole system
749: (W,D,B(t)) is developing, not just the brain.</p>
750: <p>The mystery of the human brain can be eventually solved, but the mystery of the human mind will remain forever.
751: I feel good about it! The last thing we want is to completely understand ourselves. Fortunately, this is impossible, in principle.
752: Hopefully, by reverse engineering B(0) we will learn how to better use and educate our brain and how to cure it.</p>
753: <p>Of course, we can also learn how to build intelligent robots, but I don't worry about that right now. If we learn how
754: to use our brains, we will figure out how to deal with robots. We should realize, however, that if we ever create really
755: intelligent robots (which is possible in principle), we will create a mystery of robot's mind. It is the "physical" world W
756: where the main mystery is. A robot will become intelligent in the human sense, only if it is exposed to the
757: same rich and mysterious world W as we are.</p>
758:
759: <p><b>Note:</b> It is interesting to mention that people who limit their exposure to the rich external world
760: (like some fanatics do) do not become intelligent people. Their brains do not develop in a right way, and this problem
761: is difficult to fix at a later time. I argue that the human LTM is pretty much a one-time-programmable memory, so you don't
762: want to put a wrong "BIOS" in this memory. (This "BIOS" is formed at a very young age, and, unfortunately, not many people
763: pay enough attention to this critically important problem.)<p>
764:
765: <h4>0.6.9 Creating a Jigsaw Puzzle</h4>
766: <p>Imagine a round cardboard cut into many intricate pieces to create a jigsaw puzzle. The shape of each piece can be
767: quite difficult to describe. However, assembling these pieces together in a right way produces the shape that is easy to describe.</p>
768: <p>I argue that this metaphor is relevant to brain modeling and cognitive modeling. Due to many different reasons, the problem of
769: the whole brain as a complex computing system is divided (rather artificially) into many intricate pieces. Researchers dealing with
770: these separate pieces encounter many complex issues that often have more to do with how the problem is cut into pieces, than with the
771: complexity of the brain itself. Assemble these pieces together in a right way, and a simpler and more meaningful picture appears.
772: The artificially created issues become non-issues.</p>
773: <p> Concentrating on parts per se (especially inadequate parts), it is easy to lose the big picture. When one is too preoccupied
774: with the trees, <font color="brown">one can't see the forest for the trees.</font></p>
775:
776: <a name="sect07"></a>
777:
778: <h3>0.7 On Brain Hardware, Brain Software, and the Algorithms of Thinking</h3>
779:
780: <p>Through the history of brain modeling and cognitive modeling, this topic has been a subject of a heated debate between AI
781: and ANN researchers. Let us forget for a while about the all important issue of "physical" vs. "mathematical" mentality
782: and compare the strong and the week points of AI and ANN from a system engineering viewpoint.</p>
783:
784: <p>AI has concentrated on the problem of complex algorithms underlying human thinking. It proposed that the problem of the algorithms
785: of thinking per se can be (and should be) separated from the problem of brain hardware. AI methodologists used the following reasoning.</p>
786: <p>Imagine a system engineer trying to understand the work of a complex computer with sophisticated software by studying the
787: wiring diagram of this computer and the transient processes in its gates and flip-flops. A brain researcher trying to understand
788: the work of the brain by studying its neural circuitry is trapped in a similar methodological pitfall. This type of reasoning
789: has practically killed all brain modeling research in the seventies. (The time known as the "Dark Ages" of brain modeling.)</p>
790:
791: <p>This metaphor has a strong point and a weak point. Its strong point is that it emphasizes a critical importance of brain software
792: (whatever representation such a software may have in the brain's memory). Its weak point is in suggesting that the general
793: relationship between brain hardware and the complex computations underlying mental processes (thinking) is similar to that in
794: a conventional (von Neumann) computer. This part of the metaphor is misleading.</p>
795: <p>The brain is a slow and massively parallel computer. From the basic timing considerations one can conclude that some interesting
796: algorithms of thinking require only a few of the massively parallel computational "steps" performed by brain hardware.
797: (Feldman's 100-step principle.) Traditional AI algorithms are quite long and violate this principle. Such algorithms cannot
798: be performed by a brain-like hardware.</p>
799:
800: <p>The traditional neural network research has concentrated on the problem of brain hardware. It's gone to another extreme and
801: dismissed the whole idea of algorithms of thinking and brain software. It assumed that, somehow, the distributed modifiable connections
802: can do the job. That is, one doesn't need to understand how these distributed sets of connections work. One just needs to figure
803: out the right weight modification (learning) procedure and everything will happen automatically. Let us do experiments and the
804: miracle will happen. (The good old Perceptron mentality.)</p>
805:
806: <p>It should be mentioned, that this approach does work in many practically interesting cases, so small miracles do happen. However,
807: <font color="brown">a big miracle is not going to happen.</font> The spoiler is the general level of computing power. The popular feedforward neural networks
808: have the general level of computing power similar to that of combinatorial machines. (The analog vs. digital distinction is of
809: no principal importance at this level of generality.) With feedback, such systems can reach the general level of computing power
810: of state-machines (Chomsky type 3).</p>
811: <p>To go above type 3, one needs a working memory, and the problem of working memory is a stumbling block for the connectionist
812: approach. A system with a lower general level of computing power cannot simulate the work of a system with a higher level of computing
813: power. This is a fundamental restriction that cannot be defeated by a smart design. (As mentioned in section 0.1, the human brain
814: has the highest general level of computing power (Turing machines or Chomsky type 0).
815:
816: <p><b>REMARK.</b> An attempt to simulate the work of the human brain using a computational model that has a level of computing
817: power lower than the brain resembles an attempt to design a Perpetual Motion machine in violation of the energy conservation law.
818: It doesn't matter how smart a learning algorithm is. A system cannot learn to do, what it cannot do in principle.</p>
819:
820: <p>Let us summarize the results of this system engineering discussion:
821:
822: <ul><li>Traditional AI has concentrated on the problem of the algorithms of thinking and ignored the problem of brain hardware.
823: It developed models of the algorithms of thinking that cannot be implemented in brain hardware.</li>
824: <li>Traditional neural network research has concentrated on the problem of brain hardware and ignored the problem of the
825: algorithms of thinking. It developed models of brain hardware that don't have enough computing power to perform
826: nontrivial algorithms of thinking.</li></ul></p>
827:
828: <p>To avoid these pitfalls, a meaningful model of B(0) should satisfy the following requirements:
829: <ul>
830: <li>Be implementable as a neurobiologically meaningful brain hardware.</li>
831: <li>Have enough computing power to perform psychologically meaningful algorithms of thinking.</li>
832: <li>Use a data storage procedure that would allow it to form almost all its software in the process of learning
833: (starting with a relatively small initial "firmware").</li>
834: </ul></p>
835:
836: <a name="sect08"></a>
837:
838: <h3>0.8 The "Biological Grab Bag" Mentality Has Not Succeeded</h3>
839:
840: <p>I began this paper with the <a href="#epigr">epigraph</a> containing two quotations from George W. Zopf, Jr.'s (1962) article entitled
841: "Attitude and context". I assembled these two quotations together, so they look like a single one. (I first read this article
842: in Russian translation in 1967. It had a considerable influence on my general methodological mental set.)</p>
843: <p>Forty years has passed since this entertaining, but rather deep article was presented at the Illinois Symposium on the
844: Principles of Self-organization. Nowadays, it is difficult to find people who have read this article. The truth is, however,
845: that Zopf, Jr. was right. It is becoming increasingly obvious to any unbiased person that the "biological grab bag" mentality
846: has not succeeded in understanding the fundamental biological phenomenon of the human brain and human intelligence.</p>
847:
848: <p>Let us admit that Mother Nature is smarter than human engineers and mathematicians, and let us try to understand how the
849: real biological brain works before making broader generalizations. Otherwise, we will keep producing "glittering generalizations" that
850: include everything but the brain.</p>
851:
852: <p>It is appropriate to end this methodological discussion with another quotation from the above mentioned article.
853: Wrote Zopf, Jr.: <font color="brown">"And for a bedtime story tonight I suggest Hans Christian Andersen's "The Emperor's New Clothes."
854: Fellow emperors - and weavers - may we become as little children."</font></p>
855:
856: <a name="sect09"></a>
857: <h3>0.9 Summary</h3>
858: <ul>
859: <li>
860: Like the magical mirror in the J.K. Rowling novel the brain shows everyone what one wants to see. Referring to this phenomenon,
861: the neurophysiologist Burns (1958) wrote that (when one is dealing with the brain) "it is distressingly easy to find what one is
862: looking for and remarkably difficult to discern the unsuspected or the unwanted." (<a href="#sect01">Section 0.1.</a>)
863: </li><br><br>
864:
865: <li>
866: The methodology of "partial" brain modeling and "partial" cognitive modeling has failed to address the most fundamental
867: question associated with the whole human brain as a computing system: <i>"How does the whole brain learn to perform mental
868: computations?"</i> This approach ignores the basic requirements of the whole brain as a complex computing system and
869: leads one into the pitfall of the "parts that don't fit the whole." (<a href="#sect02">Section 0.2.</a>)
870: </li><br><br>
871:
872: <li>
873: The fact that a person with a good visual memory can learn to perform, in principle, any mental computations means that the whole human
874: brain is a universal learning computing system. That is, the popular belief that universal learning systems
875: don't exist is a myth. This myth is a result of an inadequate "definition" of the concept of learning that does not include the
876: case of human learning. (<a href="#sect02">Section 0.2.</a>)
877: </li><br><br>
878:
879: <li>To understand the brain, one should view it as a part of a cognitive system (Man,World). Let (W,D,B) be a model of this system, where
880: W is an external world, D is the set of man-like sensory and motor devices, and B is a computing system simulating the work of man's
881: nervous system. For the sake of simplicity, B is referred to as a model of the brain or, simply the brain. Let B(t) be a formal representation
882: of B at the time t, where t=0 correspond to the "untrained" ("unprogrammed") brain, B(0). (<a href="#sect03">Section 0.3.</a> )
883: </li><br><br>
884:
885: <li>
886: It is convenient to view system (W,D,B) as a composition of two subsystems: (W,D) and B, where (W,D) is the external world as it appears
887: to the brain B via the set of devices D. System B has a finite formal representation. In the general case, system (W,D) doesn't have
888: a finite formal representation. However, the external world W is "always there" to experiment with, so, given a model of B, and a model
889: of D, one can simulate the whole system. (<a href="#sect03">Section 0.3.</a>)
890: </li><br><br>
891: <li>
892: It is possible to find a relatively small (not too small) formal representation of B(0). This representation may fit into a single floppy
893: disk. (<a href="#sect03">Section 0.3.</a>)
894: </li><br><br>
895:
896: <li>
897: Any formal representation of B(t) with a big t (say t> 10 years) must be very big. It is practically impossible to reverse engineer
898: B(t) without first reverse engineering B(0). (<a href="#sect03">Section 0.3.</a>)
899: </li><br><br>
900:
901: <li>
902: It is practically impossible to simulate some nontrivial parts of the behavior of system (W,D,B(t)) without having an adequate formal
903: representation of B(t). All purely phenomenological ("brainless") computational theories of cognitive phenomena fell pray to this pitfall.
904: (<a href="#sect03">Section 0.3</a> and <a href="#sect06">Section 0.6.</a>)
905: </li><br><br>
906:
907: <li>
908: The main goal of brain modeling should be reverse engineering B(0). The main goal of cognitive modeling should be understanding
909: how B(0) changes into different B(t)'s, with bigger and bigger t, as a result of interaction with different external systems (W,D).
910: (<a href="#sect03">Section 0.3.</a>)
911: </li><br><br>
912:
913: <li>
914: To reverse engineer B(0) one should invent a "supercritical" initial approximation for B(0) and use iterative reverse engineering
915: strategy described in <a href="#sect05">Section 0.5.</a>
916: </li><br><br>
917:
918: <li>
919: The metaphor the brain as an E-machine discussed in this study is an attempt to introduce a "supercritical" initial approximation
920: for B(0) that can be efficiently used in the above mentioned reverse engineering process. (<a href="#sect04">Section 0.4.</a>)
921: </li><br><br>
922:
923: <li> The main goal of this study is to initiate a project aimed at reverse engineering the structure of a complex E-machine that
924: could serve as a computational model of B(0). This project is referred to as the Brain 0 project. (<a href="#sect05">Section 0.5.</a>)
925: </li><br><br>
926:
927: <li>
928: Traditional AI research has concentrated on the problem of the "algorithms of thinking," but ignored the problem of "brain hardware."
929: It developed models of the algorithms of thinking that can not be performed by a brainlike hardware. (<a href="#sect07">Section 0.7.</a>)
930: </li><br><br>
931:
932: <li>
933: Traditional ANN research has concentrated on the problem of brain hardware, but ignored the problem of the algorithms of thinking.
934: It developed models of brain hardware that don't have enough computing power to perform nontrivial algorithms of thinking.
935: (<a href="#sect07">Section 0.7.</a>)
936: </li><br><br>
937: To avoid these pitfalls a model of B(0) must
938: <ul>
939: <li>
940: be implementable as a neurobiologically meaningful brain hardware
941: </li>
942:
943: <li>
944: have enough computing power to perform psychologically meaningful algorithms of thinking
945: </li>
946:
947: <li>
948: be able to create the main part (almost all) its software in the course of learning
949: (starting with a relatively small initial "firmware")
950: </li>
951: </ul>
952: </li>
953: </ul>
954: <a name="references"></a>
955: <h3>References</h3>
956: <font size=2>
957: Anderson, J.R. (1976). Language, Memory, and Thought. Hillsdale, New Jersey: Lawrence Erlbaum Associates, Publishers.
958: <br><br>
959: Baddeley, A.D. (1982). Your memory. A user's guide. Macmilan Publishing Co., Inc.
960: <br><br>
961: Burns, B.D., (1958). The Mamalian Cerebral Cortex. Arnold.
962: <br><br>
963: Changeux, J.P. (1993). Chemical Signaling in the Brain. Scientific American, Novemeber, 58-62.
964: <br><br>
965: Chomsky, N. (1956). Three models for the description of language. <i>I.R.E. Transactions on Information Theory.</i> JT-2, 113-124.
966: <br><br>
967: Collins, A.M., & Quillian, M.R., (1972). How to make a language user. <i>In E. Tulving & W. Donaldson (Eds.) Organization and memory.</i>
968: New York: Academic Press.
969: <br><br>
970: Eliashberg, V. (1967). On a class of learning machines. <i>Proceedings of the Conference on Automation in the Pulp & Paper industry,
971: April 1967, Leningrad USSR. Proc of VNIIB, #54,</i> 350-398. (in Russian)
972: <br><br>
973: Eliashberg, V. (1979). The concept of E-machine and the problem of context-dependent behavior. Txu 40-320, US Copyright Office.
974: <br><br>
975: Eliashberg, V. (1981). The concept of E-machine: On brain hardware and the algorithms of thinking. <i>Proceedings of the Third Annual
976: Meeting of the Cognitive Science Society,</i> 289-291.
977: <br><br>
978: Eliashberg, V. (1982). The states of residual excitation in neural networks as the mechanism of short-term memory and mental set. Abstract.
979: <i>Neuroscience,</i> Supplement to volume 7, p. S63.
980: <br><br>
981: Eliashberg, V. (1988a). Neuron layer with reciprocal inhibition as a mechanism of random choice. <i>Proceedings of the IEEE ICNN 88.</i>
982: <br><br>
983: Eliashberg, V. (1988b). The E-machines: Associative neural networks as nonclassical symbolic processors. Abstract, <i>The First Annual
984: Meeting of INNS.</i>
985: <br><br>
986: Eliashberg, V. (1989). Context-sensitive associative memory: "Residual excitation" in neural networks as the mechanism of STM and
987: mental set. <i>Proceedings of IJCNN-89, June 18-22, 1989, Washington, D.C. </i>vol. I, 67-75.
988: <br><br>
989:
990: Eliashberg, V. (1990a). Molecular dynamics of short-term memory. <i>Mathematical and Computer modeling in Science and Technology.</i>
991: vol. 14, 295-299.
992: <br><br>
993: Eliashberg, V. (1990b). Universal learning neurocomputer. <i>Proceeding of the Fourth Annual parallel processing symposium. California
994: state university, Fullerton. April 4-6, 1990.</i>, 181-191.
995: <br><br>
996: Eliashberg, V. (1993). A relationship between neural networks and programmable logic arrays. 0-7803-0999-5/93, IEEE, 1333-1337.
997: <br><br>
998: Rosenblatt, F. (1962). Principles of neurodynamics. Washington D.C.: Spartan Books.
999: <br><br>
1000: Turing, A.M. (1936). On computable numbers, with an application to the Entscheidungsproblem. <i>Proc. London Math. Society,</i> ser. 2, 42
1001: <br><br>
1002: Vvedensky, N.E. (1901). Excitation, inhibition, and narcosis. <i>In Complete collection of works.</i> USSR, 1953.
1003: <br><br>
1004: Zopf, G.W., Jr. (1962). Attitude and Context. <i>In "Principles of Self-organization",</i> Pergamon Press, 325-346.
1005: </font>
1006: <br><br><br>
1007: <a name="answers"></a>
1008: <font size="3">
1009: <h3>Answers to Questions from Section 0.1</h3>
1010: <a name="answer1"></a>
1011: <p><b>Answer to Question 1:</b>
1012: </p>
1013: <p> All presidents who are alive. The word <font color="blue">"buried"</font> makes you think about dead presidents.
1014: The particle <font color="blue"> "not"</font> is ignored.
1015: </p>
1016: <br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br>
1017: <br><br><br><br><br><br><br><br><br><br>
1018: <a name="answer2"></a>
1019: <p><b> Answer to Question 2:</b>
1020: </p>
1021: <p>
1022: A blind white horse. The word <font color="blue">"see"</font> prevents you from thinking about the animals that cannot see at all.
1023: </p>
1024: <br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br>
1025: <br><br><br><br><br><br><br><br><br><br>
1026: <a name="answer3"></a>
1027: <p><b> Answer to Question 3:</b></p>
1028: <p>The speed=0. The dog should sit quiet. The words <font color="blue">"run"</font> and <font color="blue">"fast"</font> create
1029: a misleading mental set. You imagine a supersonic dog crossing the sound barrier.
1030: </p>
1031: <br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br>
1032: <br><br><br><br><br><br><br><br><br><br>
1033: <a name="answer4"></a>
1034: <p><b> Answer to Question 4:</b>
1035: </p>
1036: <p>Put one penny in each wallet, and put one wallet inside the other. The idea of <font color="blue">"dividing money"</font> makes
1037: it difficult to think of this solution.</p>
1038: <br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br>
1039: <br><br><br><br><br><br><br><br><br><br>
1040:
1041: </font>
1042:
1043: </font>
1044: </BODY>
1045: </HTML>
1046:
1047: