1458 Part G Human-Centered and Life-Like Robotics consistent environmental cues other than the T-shape of ferred direction.To unlearn-90,say,the array must the maze.If anything,the lesioned animals learn this reduce the peak there,while at the same time building problem faster than normals.After the criterion was a new peak at the new direction of+90.If the old peak reached,probe trials with an eight-arm radial maze were has mass p(t)and the new peak has mass g(t),then as interspersed with the usual T-trials.Animals from both p(t)declines toward 0 while g(t)increases steadily from groups consistently chose the side to which they were 0,the center of mass will progress from-90°to+90°， trained on the T-maze.However,many did not choose fitting the behavioral data. the 90 arm but preferred either the 45 or 135 arm, The determination of movement direction was mod- suggesting that the rats eventually solved the T-maze by eled by rat-ification of the Arbib and House [62.25] learning to rotate within an egocentric orientation sys-model of frog detour behavior.There,prey was repre- tem at the choice point through approximately 90.This sented by excitation coarsely coded across a population. leads to the hypothesis of an orientation vector being while barriers were encoded by inhibition whose ex- stored in the animal's brain but does not tell us where tent closely matched the retinotopic extent of each or how the orientation vector is stored.One possible barrier.The sum of excitation was passed through model would employ coarse coding in a linear array of a winner-takes-all circuit to yield the choice of move- cells,coding for turns from-180 to +180.From the ment direction.As a result,the direction of the gap behavior,one might expect that only the cells close to closest to the prey,rather than the direction of the prey the preferred behavioral direction are excited,and that itself,was often chosen for the frog's initial movement. learning marches this peak from the old to the new pre- The same model serves for behavioral orientation once Hippocampal formation place Sensory inputs Prefrontal Motor world graph outputs Dynamic remapping Goal object Affordances Parietal Premotor Consequences action selection Internal state Caudoputamen Nucleus accumbens Hypothalamu Incentives drive states Actor-critic Sensory inputs Part G162.2 Fig.62.2 The TAM-WG model has at its basis a system,TAM(the taxon affordance model),for exploiting affordances. This is elaborated by a system,WG(the world graph),which can use a cognitive map to plan paths to targets which are not currently visible.Note that the model processes two different kinds of sensory inputs.At the bottom right are those associated with,e.g.,hypothalamic systems for feeding and drinking,and that may provide both incentives and rewards for the animal's behavior,contributing both to behavioral choices.and to the reinforcement of certain patterns of behavior.The nucleus accumbens and caudo-putamen mediate an actor-critic style of reinforcement learning based on the hypothalamic drive of the dopamine system.The sensory inputs at the top left are those that allow the animal to sense its relation with the external world,determining both where it is(the hippocampal place system)as well as the affordances for action (the parietal recognition of affordances can shape the premotor selection of an action).The TAM model focuses on the parietal-premotor reaction to immediate affordances;the WG(world graph)model places action selection within the wider context of a cognitive map.(after Guazzelli et al.[62.41])
1458 Part G Human-Centered and Life-Like Robotics consistent environmental cues other than the T-shape of the maze. If anything, the lesioned animals learn this problem faster than normals. After the criterion was reached, probe trials with an eight-arm radial maze were interspersed with the usual T-trials. Animals from both groups consistently chose the side to which they were trained on the T-maze. However, many did not choose the 90◦ arm but preferred either the 45◦ or 135◦ arm, suggesting that the rats eventually solved the T-maze by learning to rotate within an egocentric orientation system at the choice point through approximately 90◦. This leads to the hypothesis of an orientation vector being stored in the animal’s brain but does not tell us where or how the orientation vector is stored. One possible model would employ coarse coding in a linear array of cells, coding for turns from −180◦ to +180◦. From the behavior, one might expect that only the cells close to the preferred behavioral direction are excited, and that learning marches this peak from the old to the new preHippocampal formation place Hypothalamus drive states Prefrontal world graph Premotor action selection Affordances Actor-critic Dopamine neurons Parietal Caudoputamen Nucleus accumbens Sensory inputs Sensory inputs Motor outputs Goal object Consequences Internal state Incentives Dynamic remapping Fig. 62.2 The TAM-WG model has at its basis a system, TAM (the taxon affordance model), for exploiting affordances. This is elaborated by a system, WG (the world graph), which can use a cognitive map to plan paths to targets which are not currently visible. Note that the model processes two different kinds of sensory inputs. At the bottom right are those associated with, e.g., hypothalamic systems for feeding and drinking, and that may provide both incentives and rewards for the animal’s behavior, contributing both to behavioral choices, and to the reinforcement of certain patterns of behavior. The nucleus accumbens and caudo-putamen mediate an actor–critic style of reinforcement learning based on the hypothalamic drive of the dopamine system. The sensory inputs at the top left are those that allow the animal to sense its relation with the external world, determining both where it is (the hippocampal place system) as well as the affordances for action (the parietal recognition of affordances can shape the premotor selection of an action). The TAM model focuses on the parietal–premotor reaction to immediate affordances; the WG (world graph) model places action selection within the wider context of a cognitive map. (after Guazzelli et al. [62.41]) ferred direction. To unlearn −90◦, say, the array must reduce the peak there, while at the same time building a new peak at the new direction of +90◦. If the old peak has mass p(t) and the new peak has mass q(t), then as p(t) declines toward 0 while q(t) increases steadily from 0, the center of mass will progress from −90◦ to +90◦, fitting the behavioral data. The determination of movement direction was modeled by rat-ification of the Arbib and House [62.25] model of frog detour behavior. There, prey was represented by excitation coarsely coded across a population, while barriers were encoded by inhibition whose extent closely matched the retinotopic extent of each barrier. The sum of excitation was passed through a winner-takes-all circuit to yield the choice of movement direction. As a result, the direction of the gap closest to the prey, rather than the direction of the prey itself, was often chosen for the frog’s initial movement. The same model serves for behavioral orientation once Part G 62.2
Neurorobotics:From Vision to Action 62.2 Neuroethological Inspiration 1459 we replace the direction of the prey (frog)by the di- be shared by brain theorists.cognitive scientists.connec- rection of the orientation vector(rat),while the barriers tionists,ethologists,kinesiologists-and roboticists.In correspond to the presence of walls rather than alley particular,schema theory can provide a distributed pro- ways. gramming environment for robotics [see,e.g.,the robots To approach the issue of how a cognitive map can schemas (RS)language of Lyons and Arbib [62.47]. extend the capability of the affordance system,Guazzelli and supporting architectures for distributed control as et al.[62.43]extended the Lieblich and Arbib [62.44]in Metta et al.[62.48]].Schema theory becomes specif- approach to building a cognitive map as a world graph,ically relevant to neuroroborics when the schemas are a set of nodes connected by a set of edges,where the inspired by a model constrained by data provided by, nodes represent recognized places or situations,and the e.g.,human brain mapping,studies of the effects of brain links represent ways of moving from one situation to lesions,or neurophysiology. another.A crucial notion is that a place encountered in A perceptual schema not only determines whether different circumstances may be represented by multiple an object or other domain of interaction is present in the nodes,but that these nodes may be merged when the environment but can also provide important parameters similarity between these circumstances is recognized.to motor schemas (see below)for the guidance of ac- They model the process whereby the animal decides tion.The activity level of a perceptual schema signals where to move next,on the basis of its current drive the credibility of the hypothesis that what the schema state(hunger,thirst,fear,etc.).The emphasis is on spatial represents is indeed present,whereas other schema maps for guiding locomotion into regions not necessarily parameters represent relevant properties such as size, current visible,rather than retinotopic representations of location,and motion of the perceived object.Given a per- immediately visible space,and yields exploration and ceptual schema we may need several schema instances, latent learning without the introduction of an explicit each suitably tuned,to subserve perception of several exploratory drive.The model shows:(1)how a route,instances of its domain,e.g.,several chairs in a room. possibly of many steps,may be chosen that leads to Motor schemas provide the control systems which the desired goal;(2)how short cuts may be chosen; can be coordinated to affect a wide variety of actions. and (3)through its account of node merging why,in open fields,place cell firing does not seem to depend on Recognition Visual direction. criteria input The overall structure and general mode of operation Visual Visual of the complete model is shown in Fig.62.2,which gives input input Visual a vivid sense of the lessons to be learned by studying Activation location 4 not only specific systems of the mammalian brain but of visual Target Size Orientation also their patterns of large-scale interaction.This model search locaton recognition recognition is but one of many inspired by the data on the role of the Size Orientation Visual. hippocampus and other regions in rat navigation.Here, Activation Visual and kinesthetic and of reaching kinesthetic input tactile input we just mention as pointers to the wider literature the papers by Girard et al.[62.45]and Meyer et al.[62.461, which are part of the Psikharpax project,which is doing Fast phase Hand Hand movement preshape rotation for rats what Rana computatrix did for frogs and toads. art 62.2.4 Schemas Slow phase Actual and Coordinated Control Programs movement grasp Hand reaching Grasping Schema theory complements neuroscience's well- established terminology for levels of structural analysis Fig.62.3 Hypothetical coordinated control program for reaching (brain region,neuron,synapse)with a fiunctional vo- and grasping.The perceptual schemas (top)provide parameters for cabulary,a framework for analysis of behavior with no the motor schemas (bottom)for the control of reaching (arm trans- necessary commitment to hypotheses on the localization port and reaching)and grasping (controlling the hand to conform of each schema (unit of functional analysis),but which to the object).Dashed lines indicate activation signals which estab- can be linked to a structural analysis whenever appropri- lish timing relations between schemas;solid lines indicate transfer ate.Schemas provide a high-level vocabulary which can of data.(After Arbib [62.42])
Neurorobotics: From Vision to Action 62.2 Neuroethological Inspiration 1459 we replace the direction of the prey (frog) by the direction of the orientation vector (rat), while the barriers correspond to the presence of walls rather than alley ways. To approach the issue of how a cognitive map can extend the capability of the affordance system, Guazzelli et al. [62.43] extended the Lieblich and Arbib [62.44] approach to building a cognitive map as a world graph, a set of nodes connected by a set of edges, where the nodes represent recognized places or situations, and the links represent ways of moving from one situation to another. A crucial notion is that a place encountered in different circumstances may be represented by multiple nodes, but that these nodes may be merged when the similarity between these circumstances is recognized. They model the process whereby the animal decides where to move next, on the basis of its current drive state (hunger, thirst, fear, etc.). The emphasis is on spatial maps for guiding locomotion into regions not necessarily current visible, rather than retinotopic representations of immediately visible space, and yields exploration and latent learning without the introduction of an explicit exploratory drive. The model shows: (1) how a route, possibly of many steps, may be chosen that leads to the desired goal; (2) how short cuts may be chosen; and (3) through its account of node merging why, in open fields, place cell firing does not seem to depend on direction. The overall structure and general mode of operation of the complete model is shown in Fig. 62.2, which gives a vivid sense of the lessons to be learned by studying not only specific systems of the mammalian brain but also their patterns of large-scale interaction. This model is but one of many inspired by the data on the role of the hippocampus and other regions in rat navigation. Here, we just mention as pointers to the wider literature the papers by Girard et al. [62.45] and Meyer et al. [62.46], which are part of the Psikharpax project, which is doing for rats what Rana computatrix did for frogs and toads. 62.2.4 Schemas and Coordinated Control Programs Schema theory complements neuroscience’s wellestablished terminology for levels of structural analysis (brain region, neuron, synapse) with a functional vocabulary, a framework for analysis of behavior with no necessary commitment to hypotheses on the localization of each schema (unit of functional analysis), but which can be linked to a structural analysis whenever appropriate. Schemas provide a high-level vocabulary which can be shared by brain theorists, cognitive scientists, connectionists, ethologists, kinesiologists – and roboticists. In particular, schema theory can provide a distributed programming environment for robotics [see, e.g., the robots schemas (RS) language of Lyons and Arbib [62.47], and supporting architectures for distributed control as in Metta et al. [62.48]]. Schema theory becomes specifically relevant to neurorobotics when the schemas are inspired by a model constrained by data provided by, e.g., human brain mapping, studies of the effects of brain lesions, or neurophysiology. A perceptual schema not only determines whether an object or other domain of interaction is present in the environment but can also provide important parameters to motor schemas (see below) for the guidance of action. The activity level of a perceptual schema signals the credibility of the hypothesis that what the schema represents is indeed present, whereas other schema parameters represent relevant properties such as size, location, and motion of the perceived object. Given a perceptual schema we may need several schema instances, each suitably tuned, to subserve perception of several instances of its domain, e.g., several chairs in a room. Motor schemas provide the control systems which can be coordinated to affect a wide variety of actions. Visual location Fast phase movement Hand preshape Hand rotation Slow phase movement Actual grasp Recognition criteria Activation of visual search Activation of reaching Visual input Size recognition Visual input Size Orientation recognition Visual input Orientation Hand reaching Grasping Visual and kinesthetic input Visual, kinesthetic and tactile input Target location Fig. 62.3 Hypothetical coordinated control program for reaching and grasping. The perceptual schemas (top) provide parameters for the motor schemas (bottom) for the control of reaching (arm transport ≈ and reaching) and grasping (controlling the hand to conform to the object). Dashed lines indicate activation signals which establish timing relations between schemas; solid lines indicate transfer of data. (After Arbib [62.42]) Part G 62.2
1460 Part G Human-Centered and Life-Like Robotics The activity level of a motor schema instance may signal schemas for grasping in a localized area (AIP)of pari- its degree of readiness to control some course of action. etal cortex and motor schemas for grasping in a localized What distinguishes schema theory from usual control area(F5)of premotor cortex;see Fig.62.4.]The notion theory is the transition from emphasizing a few basic of schema is thus recursive-a schema may be analyzed controllers (e.g.,for locomotion or arm movement)to as a coordinated control program of finer schemas,and a large variety of motor schemas for diverse skills(peel-so on until such time as a secure foundation of neural ing an apple,climbing a tree,changing a light bulb),with specificity is attained. each motor schema depending on perceptual schemas to Subsequent work has refined the scheme of Fig.62.3, supply information about objects which are targets for for example,Hoff and Arbib's [62.52]model uses the interaction.Note the relevance of this for robotics-the time needed for completion of each of the movements robot needs to know not only what the obiect is but also transporting the hand and preshaping the hand-to how to interact with it.Modern neuroscience (see the explain data on how the reach to grasp responds to per- works by Ungerleider and Mishkin [62.49]and Goodale turbation of target location or size.Moreover,Hoff and and Milner [62.50])has indeed established that the mon-Arbib [62.53]show how to embed an optimality princi- key and human brain each use a dorsal pathway(via the ple for arm trajectories into a controller which can use parietal lobe)for the how and a ventral pathway(via the feedback to resist noise and compensate for target per- inferotemporal cortex)for the what.Moreover,coupling turbations,and a predictor element to compensate for between these two streams mediates their integration in delays from the periphery.The result is a feedback sys- normal ongoing behavior. tem which can act like a feedforward system described A coordinated control program interweaves the by the optimality principle in familiar situations,where activation of various perceptual,motor,and coordinat- the conditions of the desired behavior are not perturbed ing schemas in accordance with the current task and and accuracy requirements are such that normal errors sensory environment to mediate complex behaviors.Fig- in execution may be ignored.However,when perturba- ure 62.3 shows the original coordinated control program tions must be corrected for or when great precision is (Arbib [62.42],inspired by the data of Jeannerod and required,feedback plays a crucial role in keeping the Biguer [62.511).As the hand moves to grasp a ball,it is behavior close to that desired,taking account of delays preshaped so that,when it has almost reached the ball,it in putting feedback into effect.This integrated view of is of the right shape and orientation to enclose some part feedback and feedforward within a single motor schema of the ball prior to gripping it firmly.The outputs of three seems to us of value for neurorobotics as well as the perceptual schemas are available for the concurrent acti- neuroscience of motor control. vation of two motor schemas,one controlling the arm to It is standard to distinguish a forward or direct model transport the hand towards the object and the other pre- which represents the path from motor command to motor shaping the hand.Once the hand is preshaped,it is only output,from the inverse model which models the reverse the completion of the fast initial phase of hand transport pathway,i.e.,going from a desired motor outcome to that wakes up the final stage of the grasping schema to a set of motor commands likely to achieve it.As we shape the fingers under control of tactile feedback.[This have just suggested,the action plan unfolds as if it were model anticipates the much later discovery of perceptual feedforward or open-loop when the actual parameters of Part G62.2 Cerebellum Efferent Cerebral Spinal Skeleto- feedback motor muscular cortex Motor plan cord system Afferent feedback Fig.62.4 Simplified control loop relating cerebellum and cerebral motor cortex in supervising the spinal cord's control of the skeletomuscular system
1460 Part G Human-Centered and Life-Like Robotics The activity level of a motor schema instance may signal its degree of readiness to control some course of action. What distinguishes schema theory from usual control theory is the transition from emphasizing a few basic controllers (e.g., for locomotion or arm movement) to a large variety of motor schemas for diverse skills (peeling an apple, climbing a tree, changing a light bulb), with each motor schema depending on perceptual schemas to supply information about objects which are targets for interaction. Note the relevance of this for robotics – the robot needs to know not only what the object is but also how to interact with it. Modern neuroscience (see the works by Ungerleider and Mishkin [62.49] and Goodale and Milner [62.50]) has indeed established that the monkey and human brain each use a dorsal pathway (via the parietal lobe) for the how and a ventral pathway (via the inferotemporal cortex) for the what. Moreover, coupling between these two streams mediates their integration in normal ongoing behavior. A coordinated control program interweaves the activation of various perceptual, motor, and coordinating schemas in accordance with the current task and sensory environment to mediate complex behaviors. Figure 62.3 shows the original coordinated control program (Arbib [62.42], inspired by the data of Jeannerod and Biguer [62.51]). As the hand moves to grasp a ball, it is preshaped so that, when it has almost reached the ball, it is of the right shape and orientation to enclose some part of the ball prior to gripping it firmly. The outputs of three perceptual schemas are available for the concurrent activation of two motor schemas, one controlling the arm to transport the hand towards the object and the other preshaping the hand. Once the hand is preshaped, it is only the completion of the fast initial phase of hand transport that wakes up the final stage of the grasping schema to shape the fingers under control of tactile feedback. [This model anticipates the much later discovery of perceptual Efferent feedback Cerebral motor cortex Cerebellum Motor plan ∑ Afferent feedback + – Skeletomuscular system Spinal cord Fig. 62.4 Simplified control loop relating cerebellum and cerebral motor cortex in supervising the spinal cord’s control of the skeletomuscular system schemas for grasping in a localized area (AIP) of parietal cortex and motor schemas for grasping in a localized area (F5) of premotor cortex; see Fig. 62.4.] The notion of schema is thus recursive – a schema may be analyzed as a coordinated control program of finer schemas, and so on until such time as a secure foundation of neural specificity is attained. Subsequent work has refined the scheme of Fig. 62.3, for example, Hoff and Arbib’s [62.52] model uses the time needed for completion of each of the movements – transporting the hand and preshaping the hand – to explain data on how the reach to grasp responds to perturbation of target location or size. Moreover, Hoff and Arbib [62.53] show how to embed an optimality principle for arm trajectories into a controller which can use feedback to resist noise and compensate for target perturbations, and a predictor element to compensate for delays from the periphery. The result is a feedback system which can act like a feedforward system described by the optimality principle in familiar situations, where the conditions of the desired behavior are not perturbed and accuracy requirements are such that normal errors in execution may be ignored. However, when perturbations must be corrected for or when great precision is required, feedback plays a crucial role in keeping the behavior close to that desired, taking account of delays in putting feedback into effect. This integrated view of feedback and feedforward within a single motor schema seems to us of value for neurorobotics as well as the neuroscience of motor control. It is standard to distinguish a forward or direct model which represents the path from motor command to motor output, from the inverse model which models the reverse pathway, i. e., going from a desired motor outcome to a set of motor commands likely to achieve it. As we have just suggested, the action plan unfolds as if it were feedforward or open-loop when the actual parameters of Part G 62.2
Neurorobotics:From Vision to Action 62.2 Neuroethological Inspiration 1461 the situation match the stored parameters,while a feed- presses recently attended locations from the saliency back component is employed to counteract disturbances map).Because it includes a detailed low-level vision (current feedback)and to learn from mistakes (learning front-end,the model has been applied not only to lab- from feedback).This is obtained by relying on a forward oratory stimuli,but also to a wide variety of natural model that predicts the outcome of the action as it un- scenes,predicting a wealth of data from psychophysical folds in real time.The accuracy of the forward model can experiments. be evaluated by comparing the output generated by the When specific objects are searched for,low-level system with the signals derived from sensory feedback visual processing can be biased both by the gist (e.g. (Miall et al.[62.541).Also.delays must be accounted outdoor suburban scene)and also for the features of that for to address the different propagation times of the neu-object.This top-down modulation of bottom-up process- ral pathways carrying the predicted and actual outcome ing results in an ability to guide search towards targets of the action.Note that the forward model in this case of interest (Wolfe [62.57]).Task affects eye movements is relatively simple,predicting only the motor output (Yarbus [62.581),as do training and general expertise in advance:since motor commands are generated inter- Navalpakkam and Itti [62.59]propose a computational nally it is easy to imagine a predictor for these signals model which emphasizes four aspects that are impor- (known as an efference copy).The inverse model,on the tant in biological vision:determining the task relevance other hand,is much more complicated since it maps sen-of an entity,biasing attention for the low-level visual sory feedback(e.g.,vision)back into motor terms.These features of desired targets,recognizing these targets concepts will prove important both in our study of the using the same low-level features,and incrementally cerebellum(Sect.62.3)and mirror systems(Sect.62.4).building a visual map of task relevance at every scene location.It attends to the most salient location in the 62.2.5 Salience and Visual Attention scene,and attempts to recognize the attended object through hierarchical matching against object representa- Discussions of how an animal (or robot)grasps an ob-tions stored in long-term memory.It updates its working ject assume that the animal or robot is attending to the memory with the task relevance of the recognized en- relevant object.Thus,whatever the subtlety of process- tity and updates a topographic task-relevance map with ing in the canonical and mirror systems for grasping,its the location and relevance of the recognized entity,for success rests on the availability of a visual system cou-example,in one task the model forms a map of likely pled to an oculomotor control system that bring foveal locations of cars from a video clip filmed while driv- vision to bear on objects to set parameters needed for ing on a highway.Such work illustrates the continuing successful interaction.Indeed,the general point is that interaction between models based on visual neurophys- attention greatly reduces the processing load for animal iology and human psychophysics with the tackling of and robot.The catch,of course,is that reducing comput- practical robotic applications. ing load is a Pyrrhic victory unless the moving focus of Orabona et al.[62.60]implemented an extension attention captures those aspects of behavior relevant for of the Itti-Koch model on a humanoid robot with the current task-or supports necessary priority inter- moving eyes,using log-polar vision as in Sandini and rupts.Indeed,directing attention appropriately is a topic Tagliasco [62.61],and changing the feature construction for which there is a great richness of both neurophys- pyramid by considering proto-objectelements(blob-like iological data and robotic application (see Deco and structures rather than edges).The inhibition-of-return Rolls [62.55]and Choi,et al.[62.41]). mechanism has to take into account a moving frame Part In their neuromorphic model of the bottom-up guid- of reference,the resolution of the fovea is very different ance of attention in primates,Itti and Koch [62.56] from that at the periphery of the visual field,and head and decompose the input video stream into eight feature body movements need to be stabilized.The control of channels at six spatial scales.After surround sup-movement might thus have a relationship with the struc- pression,only a sparse number of locations remain ture and development of the attention system.Rizzolatti active in each map,and all maps are combined into et al.[62.62]proposed a role for the feedback projec- a unique saliency map.This map is scanned by the fo- tions from premotor cortex to the parietal lobe,assuming cus of attention in order of decreasing saliency through that they form a tuning signal that dynamically changes the interaction between a winner-takes-all mecha- visual perception.In practice this can be seen as an im- nism (which selects the most salient location)and an plicit attention system which selects sensory information inhibition-of-return mechanism(which transiently sup- while the action is being prepared and subsequently ex-
Neurorobotics: From Vision to Action 62.2 Neuroethological Inspiration 1461 the situation match the stored parameters, while a feedback component is employed to counteract disturbances (current feedback) and to learn from mistakes (learning from feedback). This is obtained by relying on a forward model that predicts the outcome of the action as it unfolds in real time. The accuracy of the forward model can be evaluated by comparing the output generated by the system with the signals derived from sensory feedback (Miall et al. [62.54]). Also, delays must be accounted for to address the different propagation times of the neural pathways carrying the predicted and actual outcome of the action. Note that the forward model in this case is relatively simple, predicting only the motor output in advance: since motor commands are generated internally it is easy to imagine a predictor for these signals (known as an efference copy). The inverse model, on the other hand, is much more complicated since it maps sensory feedback (e.g., vision) back into motor terms. These concepts will prove important both in our study of the cerebellum (Sect. 62.3) and mirror systems (Sect. 62.4). 62.2.5 Salience and Visual Attention Discussions of how an animal (or robot) grasps an object assume that the animal or robot is attending to the relevant object. Thus, whatever the subtlety of processing in the canonical and mirror systems for grasping, its success rests on the availability of a visual system coupled to an oculomotor control system that bring foveal vision to bear on objects to set parameters needed for successful interaction. Indeed, the general point is that attention greatly reduces the processing load for animal and robot. The catch, of course, is that reducing computing load is a Pyrrhic victory unless the moving focus of attention captures those aspects of behavior relevant for the current task – or supports necessary priority interrupts. Indeed, directing attention appropriately is a topic for which there is a great richness of both neurophysiological data and robotic application (see Deco and Rolls [62.55] and Choi, et al. [62.41]). In their neuromorphic model of the bottom-up guidance of attention in primates, Itti and Koch [62.56] decompose the input video stream into eight feature channels at six spatial scales. After surround suppression, only a sparse number of locations remain active in each map, and all maps are combined into a unique saliency map. This map is scanned by the focus of attention in order of decreasing saliency through the interaction between a winner-takes-all mechanism (which selects the most salient location) and an inhibition-of-return mechanism (which transiently suppresses recently attended locations from the saliency map). Because it includes a detailed low-level vision front-end, the model has been applied not only to laboratory stimuli, but also to a wide variety of natural scenes, predicting a wealth of data from psychophysical experiments. When specific objects are searched for, low-level visual processing can be biased both by the gist (e.g., outdoor suburban scene) and also for the features of that object. This top-down modulation of bottom-up processing results in an ability to guide search towards targets of interest (Wolfe [62.57]). Task affects eye movements (Yarbus [62.58]), as do training and general expertise. Navalpakkam and Itti [62.59] propose a computational model which emphasizes four aspects that are important in biological vision: determining the task relevance of an entity, biasing attention for the low-level visual features of desired targets, recognizing these targets using the same low-level features, and incrementally building a visual map of task relevance at every scene location. It attends to the most salient location in the scene, and attempts to recognize the attended object through hierarchical matching against object representations stored in long-term memory. It updates its working memory with the task relevance of the recognized entity and updates a topographic task-relevance map with the location and relevance of the recognized entity, for example, in one task the model forms a map of likely locations of cars from a video clip filmed while driving on a highway. Such work illustrates the continuing interaction between models based on visual neurophysiology and human psychophysics with the tackling of practical robotic applications. Orabona et al. [62.60] implemented an extension of the Itti–Koch model on a humanoid robot with moving eyes, using log-polar vision as in Sandini and Tagliasco [62.61], and changing the feature construction pyramid by considering proto-object elements (blob-like structures rather than edges). The inhibition-of-return mechanism has to take into account a moving frame of reference, the resolution of the fovea is very different from that at the periphery of the visual field, and head and body movements need to be stabilized. The control of movement might thus have a relationship with the structure and development of the attention system. Rizzolatti et al. [62.62] proposed a role for the feedback projections from premotor cortex to the parietal lobe, assuming that they form a tuning signal that dynamically changes visual perception. In practice this can be seen as an implicit attention system which selectssensory information while the action is being prepared and subsequently exPart G 62.2
1462 Part G Human-Centered and Life-Like Robotics ecuted (see Flanagan and Johansson [62.63],Flanagan tor and parietal neurons suggest a premotor mechanism et al.[62.64],and Mataric and Pomplun [62.651).The of attention that deserves exploration in further work in early responses,before action onset,of many premo- neurorobotics. 62.3 The Role of the Cerebellum Although cerebellar involvement in muscle control was grasp the object.Thus analysis of how various compo- advocated long ago by the Greek gladiator surgeon nents of cerebral cortex interact to support forward and Galen of Pergamum (129-216/17 CE),it was the publi- inverse models which determine the overall shape of cation by Eccle et al.[62.66]of the first comprehensive the behavior must be complemented by analysis of how account of the detailed neurophysiology and anatomy the cerebellum handles control delays and nonlineari- of the cerebellum (/to [62.671)that provided the inspi-ties to transform a well-articulated plan into graceful ration for the Marr-Albus model of cerebellar plasticity coordinated action.Within this perspective,cerebellar (Marr [62.68];Albus [62.69])that is at the heart of structure and function will be very helpful in the control most current modeling of the role of the cerebellum of a new class of highly antagonistic robotic systems as in control of motion and sensing.From a robotics well as in adaptive control. point of view,the most convincing results are based on Albus'[62.70]cerebellar model articulation con- 62.3.1 The Human Control Loop troller(CMAC)model and subsequent implementations by Miller [62.71].These models,however,are only Lesions and deficits of the cerebellum impair the co- remotely based on the structure of the biological cerebel- ordination and timing of movements while introducing lum.More detailed models are usually only applied to excessive.undesired motion:effects which cannot be two-degree-of-freedom robotic structures,and have not compensated by the cerebral cortex.According to main- been generalized to real-world applications(see Peters stream models,the cerebellum filters descending motor and van der Smagt [62.72]).The problem may lie with cortex commands to cope with timing issues and com- viewing the cerebellum as a stand-alone dynamics con- munication delays which go up to 50 ms one way for arm troller.An important observation about the brain is that control.Clearly,closed-loop control with such delays is schemas are widely distributed,and different aspects of not viable in any reasonable setting,unless augmented the schemas are computed in different parts of the brain. with an open-loop component,predicting the behavior of Thus,one view is that (1)the cerebral cortex has the the actuator system.This is where the cerebellum comes necessary models for choosing appropriate actions and into its own.The complexity of the vertebrate muscu- getting the general shape of the trajectory assembled loskeletal system,clearly demonstrated by the human to fit the present context,whereas (2)the cerebellum arm using a total of 19 muscle groups for planar mo- provides a side-path which (on the basis of extensive tion of the elbow and shoulder alone (see Nijhof and learning of a forward motor model)provides the ap- Kouwenhoven [62.73])requires a control mechanism propriate corrections to compensate for control delays,coping with this complexity,especially in a setting with Part muscle nonlinearities,Coriolis and centrifugal forces long control delays.One cause for this complexity is occasioned by joint interactions,and subtle adjustments that animal muscles come in antagonistic pairs (e.g., 9 of motor neuron firing in simultaneously active motor flexing versus extending a joint).Antagonistic control 23 pattern generators to ensure their smooth coordination.of muscle groups leads to energy-optimal(Damsgaard Thus,for example,a patient with cerebellar lesions may et al.[62.741)and intrinsically flexible systems.Contact be able to move his arm to successfully reach a target,with stiff or fast-moving objects requires such flexibil- and to successfully adjust his hand to the size of an ob-ity to prevent breakage.In contrast,classical (industrial) ject.However,he lacks the machinery to perform either robots are stiff,with limb segments controlled by lin- action both swiftly and accurately,and further lacks the ear or rotary motors with gear boxes.Even so,most ability to coordinate the timing of the two subactions. laboratory robotic systems have passively stiff joints, His behavior will thus exhibit decomposition of move- with active joint flexibility obtainable only by using fast ment-he may first move the hand till the thumb touches control loops and joint torque measurement.Although the object,and only then shape the hand appropriately to it may be debatable whether such robotic systems re-
1462 Part G Human-Centered and Life-Like Robotics ecuted (see Flanagan and Johansson [62.63], Flanagan et al. [62.64], and Mataric and Pomplun [62.65]). The early responses, before action onset, of many premotor and parietal neurons suggest a premotor mechanism of attention that deserves exploration in further work in neurorobotics. 62.3 The Role of the Cerebellum Although cerebellar involvement in muscle control was advocated long ago by the Greek gladiator surgeon Galen of Pergamum (129–216/17 CE), it was the publication by Eccle et al. [62.66] of the first comprehensive account of the detailed neurophysiology and anatomy of the cerebellum (Ito [62.67]) that provided the inspiration for the Marr–Albus model of cerebellar plasticity (Marr [62.68]; Albus [62.69]) that is at the heart of most current modeling of the role of the cerebellum in control of motion and sensing. From a robotics point of view, the most convincing results are based on Albus’ [62.70] cerebellar model articulation controller (CMAC) model and subsequent implementations by Miller [62.71]. These models, however, are only remotely based on the structure of the biological cerebellum. More detailed models are usually only applied to two-degree-of-freedom robotic structures, and have not been generalized to real-world applications (see Peters and van der Smagt [62.72]). The problem may lie with viewing the cerebellum as a stand-alone dynamics controller. An important observation about the brain is that schemas are widely distributed, and different aspects of the schemas are computed in different parts of the brain. Thus, one view is that (1) the cerebral cortex has the necessary models for choosing appropriate actions and getting the general shape of the trajectory assembled to fit the present context, whereas (2) the cerebellum provides a side-path which (on the basis of extensive learning of a forward motor model) provides the appropriate corrections to compensate for control delays, muscle nonlinearities, Coriolis and centrifugal forces occasioned by joint interactions, and subtle adjustments of motor neuron firing in simultaneously active motor pattern generators to ensure their smooth coordination. Thus, for example, a patient with cerebellar lesions may be able to move his arm to successfully reach a target, and to successfully adjust his hand to the size of an object. However, he lacks the machinery to perform either action both swiftly and accurately, and further lacks the ability to coordinate the timing of the two subactions. His behavior will thus exhibit decomposition of movement – he may first move the hand till the thumb touches the object, and only then shape the hand appropriately to grasp the object. Thus analysis of how various components of cerebral cortex interact to support forward and inverse models which determine the overall shape of the behavior must be complemented by analysis of how the cerebellum handles control delays and nonlinearities to transform a well-articulated plan into graceful coordinated action. Within this perspective, cerebellar structure and function will be very helpful in the control of a new class of highly antagonistic robotic systems as well as in adaptive control. 62.3.1 The Human Control Loop Lesions and deficits of the cerebellum impair the coordination and timing of movements while introducing excessive, undesired motion: effects which cannot be compensated by the cerebral cortex. According to mainstream models, the cerebellum filters descending motor cortex commands to cope with timing issues and communication delays which go up to 50 ms one way for arm control. Clearly, closed-loop control with such delays is not viable in any reasonable setting, unless augmented with an open-loop component, predicting the behavior of the actuator system. This is where the cerebellum comes into its own. The complexity of the vertebrate musculoskeletal system, clearly demonstrated by the human arm using a total of 19 muscle groups for planar motion of the elbow and shoulder alone (see Nijhof and Kouwenhoven [62.73]) requires a control mechanism coping with this complexity, especially in a setting with long control delays. One cause for this complexity is that animal muscles come in antagonistic pairs (e.g., flexing versus extending a joint). Antagonistic control of muscle groups leads to energy-optimal (Damsgaard et al. [62.74]) and intrinsically flexible systems. Contact with stiff or fast-moving objects requires such flexibility to prevent breakage. In contrast, classical (industrial) robots are stiff, with limb segments controlled by linear or rotary motors with gear boxes. Even so, most laboratory robotic systems have passively stiff joints, with active joint flexibility obtainable only by using fast control loops and joint torque measurement. Although it may be debatable whether such robotic systems rePart G 62.3