A Back-Propagation Programmed Network That Simulates Response Properties of Parietal Neurons(REVIEW)

The posterior parietal lobe; more specifically the Brodmann area 7a has been implicated in computing spatial representation and location of external stimuli and objects. Three different populations of neurons in this region have been classified according to the stimulus they respond to. 15% of the neurons here where found to be sensitive to the position of the eye in its sockets; 21% where found to be sensitive to visual stimuli alone; whilst the 3rd and evidently more complex class of neurons constituted 57% of the neural population of this region and were found to be sensitive to both sensory information i.e visual stimuli and orbital eye-position. This third class of neurons (referred to as spatial neurons in this paper) had been experimentally observed to have spatial gain field [6]; that is to say that they had a variant responses to the exact same retinotopic stimuli (stimuli that fell on the same retinal patch) as a function of eye position. This meant that a particular spatial neuron would have different responses to the same retinotopic stimulus depending on the position of the eye its orbit.

This paper by Andersen and Zipser (1988) was an effort to solve the computational conundrum surrounding the spatial neurons of area 7a. If one; with their head fixed, moves their eyes away from a stationary object that they had foveated, the brain is able to correctly interpret the object as being stationary despite the fact that the motion of the eyes alters the retinal location of the visual target. This should mean that the brain should be somehow integrating the retinal location stimuli as well as that of the eye-position/motion; carrying out a kind of sensory and motor integration. This integration would have a normalizing effect that counteracts the motion of the eyes such that the brain can correctly interpret the external object/target as stationary or non-moving. Thus it would be expected that the neurons generating the normalizing signal would have an invariant electrical activity/response with varying eye position to a stationary object. Area 7a neurons where thought to have been the ones to generate this spatial representation signal until it was discovered that none of the neurons in area 7a generated an invariant signal with changes in eye position but that the contrary was the case as seen by the spatial gain fields property of more than half of the neurons in this region[1,6]; showing a tuning for target retinal location that was perhaps yet to be translated to actual spatial representation.

The aim of Andersen and Zisper’s paper was to build a back-propagation programmed neural network whose goal was to carry out the functions of the spatially tuned neurons of area 7a; that of computationally integrating eye position data and retinal location data. The two data inputs would be extracted from the responses of the two other classes of neurons found in area 7a that respond to singular data of either one of retinal location or eye position. The data to be fed into the neural network had actually been obtained in a previous experiment (R. A. Andersen et tal, 1985) [1] that Andersen was also a part of. The eye position responses could be modelled by monotonic functions that either increased or decreased with vertical or horizontal eye motion whilst the visual response could be modelled by a gaussian function with 1/e widths of 15 degrees. These two were fed into the input layers of the network.

The “teacher” of the network would be the expected spatial coordinates of the head-centred location of an external target; that is to say that the network would be required; by the teaching signal, to be able to give reliable and accurate spatial coordinates of the external target and thus compensating for any changes in the retinotopic coordinate projections of the visual target that are a result of eye movement. This aforementioned teaching signal was imposed on the output layer. Two teaching formats where employed in this paper, were the network was required either to regurgitate the correct eye position by decoding the integrated signal of eye position and retinal position data carried out by the hidden layers; or vice versa for the retinal/visual stimuli.

The inputs were fed into the input layer of the network and the accompanying expected output was imposed at the output layer. All the input units in the input layer were connected to all the units in the hidden layer; and in turn all the units of the hidden layer were connected to all the units of the output. The sigmoid function was used as a transfer function between the layers in order to allow for the non-linear combination/computation of the eye position vectors with those of retinal location as a means of mimicking the computational properties of spatial neurons that combine these two signals in a non-linear way [1]. The connection weights of the entire network was initially randomised and produced large output errors initially, as expected. Back-propagation was used to programme the network using the back-propagation procedure. In the procedure, the obtained output of the network would be compared to and subtracted from the expected output in order to generate the error. The error quantity would then be used to change the connection weights of the network appropriately following a gradient descent mechanism. Over many teaching iterations the error inherent in the network got progressively smaller until the experimenters stopped after a 1, 000 training sessions; at which point the network had achieved the desired error margin.

After the network had been trained it was striking to find that the firing rates of the hidden layer varied in such a way as to give rise to spatial gain fields that were very similar to those of the spatial neurons of area 7a. This suggested that as required, the hidden layer of the neural network was carrying out computations of the kind that the spatial neurons of area 7a did, producing a response to retinotopically identical stimuli that varied as a function of eye position. By constraining the neural network with parameters of inputs from neurons and expected outputs on the other end, a sort of “black box” was created in the middle in which the neural network was coerced to device/evolve an algorithm that in a way revealed how posterior parietal cortex neurons and the rest of the connected brain carry out neuronal computations involved in creating accurate spatial representations that coded for the location of external objects. The revelations of the network suggested that the spatial neurons of area 7a are likely to be performing intermediary computations within a larger neural network. Thus in the end, the trained neural network that Andersen and Zipser created performed a higher order brain function that decoded the integrated signal from the two way inputs originating from neurons encoding visual stimuli and those encoding eye-position during reception of the visual stimuli.

One of the constraints of the model that may have accounted for some inconsistencies between model and experimental responses may been the apparent fact that this research sought to extract information from narrow parameters that otherwise lay in a more expansive interconnection of other computations. To obtain these results, the researchers had to fix the head of a macaque monkey and allow only for its eyes to rotate in order to fixate on a stationary stimulus (the visual target). Thus the units of the biological neural network mechanism responsible for computing and carrying out the narrow functions in this experiment would usually otherwise have to contend with and compute far more data sets than they did in this experiment. This would account for some of the inconsistencies where the experimental data showed spatial gains and retinal receptive fields that had a greater level of complexity, showing uneven receptive fields with multiple peaks. Also, given that the biological neural network responsible for carrying out this function in the brain is an ever self-improving and therefore ever-changing system, it can be surmised that it can never really compute the exact same data inputs in exactly the same way as the first time [5], because the biological network would be seeing a gradually asymptotic change of itself. This is in contrast with the Artificial Neural Network of the kind created by Andersen and Zipser which obtained results used in this paper from a finite number of learning iterations of just a thousand.

This experiment brings us to question what mechanisms the brain might actually be using to carry out this computation of translating spatial coordinates to retinal frame coordinates and decoding it in time for the brain to make sense of the information. Whilst the Andersen and Zipser network used back-propagation to adjust its weights and program itself, it does not definitively imply that this is the mechanism that the biological brain employs. Thus what the Artificial Neural Network accomplished brings to mind arguments surrounding the “Turing Test” idea where it is argued that a computer or network may arrive at the same results as that of a biological network but that would not mean that the two networks process and manipulate data in the same way. This is especially the case here where back-propagation requires that there be retrograde movement of signals in neuronal axons. Neurons only propagate signals in one direction, this contradicting disagreement with the back-propagation learning rule used in this paper’s neural network led it to being dubbed “unbiological” [3] and that it does not accurately depict the actual brain process. By taking into account neuronal properties and their interactions with one another in the brain, more biologically plausible learning rules have been suggested. One of these learning rules was posited by P. Mazzoni et tal (1991) [3], which uses the Associative Reward Penalty mechanism to change the connection weights of the network accordingly. This learning mechanism is more biologically plausible because it has much more biological correlates with the actual brain. It includes Hebbian-like adjustment of connection weights that follows along the biological observation that “neurons that fire together wire together.” The network also showed that only a single projection from the reinforcement brain region was necessary to travel back to area 7a region to help set the pace for the appropriate changing of weights. This is more biologically plausible given that a signal from a cluster of neurons travelling back to an entire cortical region to regulate the activity there is the modus operandi of most brain feedback loops. Back-propagation on the other hand would have required the existence of an intricate and complex system of biological feedback loops that fed into specific individual neuronal units to cause the required change in activity and connection weights.

The downside of the ARP learning rule suggested by P. Mazzoni and colleagues is that the learning rule trains the artificial neural network much slower than back-propagation. This is because the ARP learning rule causes the ANN to be more random in its pursuit of the required output. The back-propagation learning rule produces a monotonic decrease in the error, causing it to achieve required accuracies much faster than the ARP learning rule which bounces back and forth more randomly in a slower progressing harmonic asymptote. But as Francis Crick said in his 1989 commentary; “what is really required is a brain-like algorithm which produces results of the same general character as back-propagation” [4]. This is especially more the case when the idea is to understand better how the biological brain works so that we are in a better position to remedy neurological complications that occur when the brain’s processes go awry. However if the intention is just to create an intelligent network that accomplishes brain-like functions, then the underlying learning mechanism employed by the network need not be anything like its biological counterpart, let alone be scrutable or comprehensible to the experimenters themselves.

References:

1. Richard A. Andersen, Greg K. Essick and Ralph M. Siegel (1985) Encoding of Spatial Location By Posterior Parietal Neurons. Science. 230(4724):456-458

2. AF Fuchs and DA Robinson (1966) A Method for Measuring Horizontal and Vertical Eye Movement Chronically in the Monkey. J Appl Physiol. 21(3):1068-1070

3. P. Mazzoni et tal (1991) A More Biologically Plausible Learning Rule for Neural Networks. PNAS. 88: 4433-4437

4. F. Crick (1989) The Recent Excitement About Neural Networks. Nature. 337(6203)129-132

5. Mark A. Gluck and David E. Rumelhart (1990) Neuroscience and Connectionist Theory. Hillsdale, New Jersey 07642. Lawrence Erlbaum Associates, Inc., Publishers. 369

Richard A. Andersen and Vernon B. Mountcastle (1982) The Influence of Angle of Gaze Upon The Excitability of the Light-Sensitive Neurons of the Posterior Parietal Cortex. The Journal of Neuroscience. 3(3)532-548