Action Prediction

Measuring eye movements remotely via the participant's webcam promises to be an attractive methodological addition to in-person eye-tracking in the lab. However, there is a lack of systematic research comparing remote web-based eye-tracking with in-lab eye-tracking in young children. We report a multi-lab study that compared these two measures in an anticipatory looking task with toddlers using WebGazer.js and jsPsych. Results of our remotely tested sample of 18-27-month-old toddlers (N = 125) revealed that web-based eye-tracking successfully captured

In recent years, deep learning classification methods, specially Convolutional Neural Networks (CNNs), combined with multi-modality image fusion schemes have achieved remarkable performance. Hence, in this paper, we focus on improving the late-fusion scheme for pedestrian classification on the Daimler stereo vision data set. We propose cross training method in which a CNN for each independent modality (Intensity, Depth, Flow) is trained and validated on different modalities, in contrast to classical training method in which the training and validation of each CNN is on same modality. The CNN outputs are then fused by a Multi-layer Perceptron (MLP) before making the recognition decision.

We combined psychophysical and transcranial magnetic stimulation studies to investigate the dynamics of action anticipation and its underlying neural correlates in professional basketball players. Athletes predicted the success of free shots at a basket earlier and more accurately than did individuals with comparable visual experience (coaches or sports journalists) and novices. Moreover, performance between athletes and the other groups differed before the ball was seen to leave the model's hands, suggesting that athletes predicted the basket shot's fate by reading the body kinematics. Both visuo-motor and visual experts showed a selective increase of motor-evoked potentials during observation of basket shots. However, only athletes showed a time-specific motor activation during observation of erroneous basket throws. Results suggest that achieving excellence in sports may be related to the fine-tuning of specific anticipatory 'resonance' mechanisms that endow elite athletes' brains with the ability to predict others' actions ahead of their realization.

Studies demonstrate that elite athletes are able to extract kinematic information of observed domain-specific actions to predict their future course. Little is known, however, on the perceptuo-motor processes and neural correlates of the athletes' ability to predict fooling actions. Combining psychophysics and transcranial magnetic stimulation, we explored the impact of motor and perceptual expertise on the ability to predict the fate of observed actual or fake soccer penalty kicks. We manipulated the congruence between the model's body kinematics and the subsequent ball trajectory and investigated the prediction performance and corticospinal reactivity of expert kickers, goalkeepers, and novices. Kickers and goalkeepers outperformed novices by anticipating the actual kick direction from the model's initial body movements. However, kickers were more often fooled than goalkeepers and novices in cases of incongruent actions. Congruent and incongruent actions engendered a comparable facilitation of kickers' lower limb motor representation, but their neurophysiological response was correlated with their greater susceptibility to be fooled. Moreover, when compared with actual actions, motor facilitation for incongruent actions was lower among goalkeepers and higher among novices. Thus, responding to fooling actions requires updation of simulative motor representations of others' actions and is facilitated by visual rather than by motor expertise.

Congenital or acquired cerebellum alterations are associated with a complex pattern of motor, cognitive and social disorders. These disturbances may reflect the involvement of the cerebellum in generating and updating the internal models that subserve-the prediction of sensory events. Here, we tested whether the cerebellar involvement in using contextual expectations to interpret ambiguous sensory sceneries is specific for social actions or also extends to physical events. We applied anodic, cathodic and sham cerebellar transcranial Direct Current Stimulation (ctDCS) to modulate the performance of an adult sample in two tasks requiring the prediction of social actions or moving shapes. For both tasks, in an earlier implicit-learning phase (familiarization), we manipulated the probability of co-occurrence between a particular action/shape and contextual elements, which could provide either strongly or moderately informative expectations. The use of these expectations was then tested when participants had to predict the unfolding of temporally occluded videos, in situations of perceptual uncertainty (testing). Results showed that in the testing, but not in the familiarization phase, cathodic as compared to anodic and sham ctDCS hindered participants' sensitivity in predicting actions embedded in strongly, but not moderately, informative contexts. Conversely, anodic as compared to sham ctDCS boosted the prediction of actions embedded in moderately, but not strongly, informative contexts. We observed no ctDCS effects for the shape prediction task, thus pointing to a specific involvement of the cerebellum in forming expectations related to social events. Our results encourage the exploration of rehabilitative effects of ctDCS in patients with social perception deficits.

Late fusion schemes with deep learning classification patterns set up with multi-modality images have an essential role in pedestrian protection systems since they have achieved prominent results in the pedestrian recognition task. In this paper, the late fusion scheme merged with Convolutional Neural Networks (CNN) is investigated for pedestrian recognition based on the Daimler stereo vision data sets. An independent CNN-based classifier for each imaging modality (Intensity, Depth, and Optical Flow) is handled before the fusion of its probabilistic output scores with a Multi-Layer Perceptron which provides the recognition decision. In this paper, we set out to prove that the incremental cross-modality deep learning approach enhances pedestrian recognition performances. It also outperforms state-of-the-art pedestrian classifiers on the Daimler stereovision data sets.

Origins of Mindreading Abilities in Children and Monkeys Judith Burkart ([email protected]), Anthropological Institute, University of Zurich, Switzerland Adolf Heschl ([email protected]), Zoological Institute, University of Graz, Austria Luca Surian & Stefania Caldi ([email protected]), Dipartimento di Psycologia, Universita di Trieste, Trieste, Italy Claudia Thoermer ([email protected]) & Beate Sodian ([email protected]); LMU, Munich, Germany Henrike Moll ([email protected]) & Michael Tomasello (tomas@)eva.mpg.de), Max-Planck Institute for Evolutionary Anthropology, Leipzig, Germany went to the same location as before and therefore to a new goal object. Infants looked at the two test trials differently in the two conditions, suggesting an implicit mindreading ability when interpreting the animation events. One central issue in research on precursors to a “theory of mind” and representation of intentional relations is how the ep...

We devise an algorithm using a Bayesian optimization framework in conjunction with contextual visual data for the efficient localization of objects in still images. Recent research has demonstrated substantial progress in object localization and related tasks for computer vision. However, many current state-of-the-art object localization procedures still suffer from inaccuracy and inefficiency, in addition to failing to provide a principled and interpretable system amenable to high-level vision tasks. We address these issues with the current research. Our method encompasses an active search procedure that uses contextual data to generate initial bounding-box proposals for a target object. We train a convolutional neural network to approximate an offset distance from the target object. Next, we use a Gaussian Process to model this offset response signal over the search space of the target. We then employ a Bayesian active search for accurate localization of the target. In experiments, we compare our approach to a state-of-theart bounding-box regression method for a challenging pedestrian localization task. Our method exhibits a substantial improvement over this baseline regression method.

Bayesian accounts of autism suggest that this disorder may be rooted in an impaired ability to estimate the probability of future events, possibly owing to reduced priors. Here, we tested this hypothesis within the action domain in children with and without autism using a behavioural paradigm comprising a familiarization and a testing phase. During familiarization, children observed videos depicting a child model performing actions in diverse contexts. Crucially, within this phase, we implicitly biased action-context associations in terms of their probability of co-occurrence. During testing, children observed the same videos but drastically shortened (i.e. reduced amount of kinematics information) and were asked to infer action unfolding. Since during the testing phase movement kinematics became ambiguous, we expected children's responses to be biased to contextual priors, thus compensating for perceptual uncertainty. While this probabilistic effect was present in controls, no ...

Background Previous research studies have assessed the relationship between attention to social information and peripheral (e.g., plasma and salivary) oxytocin (OT) levels in typically developing (TD) children and children with autism spectrum disorder (ASD). A relationship between them was observed in TD children, but not in children with ASD. However, this relationship remains unexamined in other age groups. To clarify whether this lack of association is maintained throughout development in individuals with ASD, we aimed to assess the relationship between salivary OT levels and attention to social information in adolescents and adults with and without ASD. Methods We recruited male adolescents and adults with ASD (n = 17) and TD participants (n = 24). Using the all-in-one eye-tracking system Gazefinder, we measured the percentage fixation time allocated to social information. We also measured the salivary OT levels and Autism Spectrum Quotient (AQ) of participants. Subsequently, w...

Background Previous research studies have assessed the relationship between attention to social information and peripheral (e.g., plasma and salivary) oxytocin (OT) levels in typically developing (TD) children and children with autism spectrum disorder (ASD). A relationship between them was observed in TD children, but not in children with ASD. However, this relationship remains unexamined in other age groups. To clarify whether this lack of association is maintained throughout development in individuals with ASD, we aimed to assess the relationship between salivary OT levels and attention to social information in adolescents and adults with and without ASD. Methods We recruited male adolescents and adults with ASD (n = 17) and TD participants (n = 24). Using the all-in-one eye-tracking system Gazefinder, we measured the percentage fixation time allocated to social information. We also measured the salivary OT levels and Autism Spectrum Quotient (AQ) of participants. Subsequently, w...

Autism is associated with difficulties in making predictions based on contextual cues. Here, we investigated whether the distribution of autistic traits in the general population, as measured through the Autistic Quotient (AQ), is associated with alterations of context-based predictions of social and non-social stimuli. Seventy-eight healthy participants performed a social task, requiring the prediction of the unfolding of an action as interpersonal (e.g., to give) or individual (e.g., to eat), and a non-social task, requiring the prediction of the appearance of a moving shape as a short (e.g., square) or a long (e.g., rectangle) figure. Both tasks consisted of (i) a familiarization phase, in which the association between each stimulus type and a contextual cue was manipulated with different probabilities of co-occurrence, and (ii) a testing phase, in which visual information was impoverished by early occlusion of video display, thus forcing participants to rely on previously learned context-based associations. Findings showed that the prediction of both social and non-social stimuli was facilitated when embedded in high-probability contexts. However, only the contextual modulation of non-social predictions was reduced in individuals with lower 'Attention switching' abilities. The results provide evidence for an association between weaker context-based expectations of non-social events and higher autistic traits.

This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Significance A major challenge in studying intention reading is high motor variability. Analyses conducted across trials provide insights into what happens on average; however, they may obscure how individual observers read intention information in individual movements. We combined motion tracking, psychophysics, and computational analyses to examine intention reading in autism spectrum disorders (ASDs) with single-trial resolution. Results revealed that a sizeable fraction of ASD observers can identify intention-informative variations in ASD (but not in typically developing) movement kinematics, but they are nonetheless unable to extract the encoded intention information. This approach not only enhances our basic understanding of mind reading in ASD but also provides potential avenues for the rational design of training procedures to improve the reading of others’ actions.

In this paper, a deep neural network (Behavior-CNN) is proposed to model pedestrian behaviors in crowded scenes, which has many applications in surveillance. A pedestrian behavior encoding scheme is designed to provide a general representation of walking paths, which can be used as the input and output of CNN. The proposed Behavior-CNN is trained with real-scene crowd data and then thoroughly investigated from multiple aspects, including the location map and location awareness property, semantic meanings of learned filters, and the influence of receptive fields on behavior modeling. Multiple applications, including walking path prediction, destination prediction, and tracking, demonstrate the effectiveness of Behavior-CNN on pedestrian behavior modeling.

The ability to localize moving joints of a person in action is crucial for interacting with other people in the environment. However, it remains unclear how the visual system encodes the position of joints in a moving body. We used a paradigm based on a well-known phenomenon, the flash-lag effect, to investigate the mechanisms underlying joint localization in bodily movements. We first found that observers perceived a strong flash-lag effect in biological motion: when a briefly-flashed dot was presented physically in perfect alignment with a continuously moving limb, the flash dot was perceived to lag behind the position of the moving joint. Importantly, our study revealed that for familiar forward walking actions, the strength of the flash-lag effect for a joint depends on body orientation. Specifically, observing a walker with a natural body orientation (i.e., upright) yielded a significantly stronger flash-lag effect for the critical foot joint than did viewing an inverted walker. In contrast, the hand joint showed a weaker flash-lag effect in the upright walker than the inverted walker. These findings suggest that the impact of body orientation on encoding joint locations depended on body part. Furthermore, we found that action familiarity modulates the impact of body orientation on the flash-lag effect. Body orientation impacted location encoding in familiar forward walking actions, but not in unfamiliar actions (e.g., backward walking, jumping-jack). Simulation results showed that generic motion mechanisms, such as the temporal averaging model, cannot fully account for these empirical findings regarding the flash-lag effect in biological motion. The present study provides compelling evidence that action processing interacts with position processing to localize the moving joints in whole-body actions, and that this influence depends on body orientation and familiarity of actions.

Recent evidence suggests that young infants, as well as nonhuman apes, can anticipate others' behavior based on their false beliefs. While such behaviors have been proposed to be accounted by simple associations between agents, objects, and locations, human adults are undoubtedly endowed with sophisticated theory of mind abilities. For example, they can attribute mental contents about abstract or non-existing entities, or beliefs whose content is poorly specified. While such endeavors may be human specific, it is unclear whether the representational apparatus that allows for encoding such beliefs is present early in development. In four experiments we asked whether 15-month-old infants are able to attribute beliefs with underspecified content, update their content later, and maintain attributed beliefs that are unknown to be true or false. In Experiment 1, infants observed as an agent hid an object to an unspecified location. This location was later revealed in the absence or presence of the agent, and the object was then hidden again to an unspecified location. Then the infants could search for the object while the agent was away. Their search was biased to the revealed location (that could be represented as the potential content of the agent's belief when she had not witnessed the re-hiding), suggesting that they (1) first attributed an underspecified belief to the agent, (2) later updated the content of this belief, and (3) were primed by this content in their own action even though its validity was unknown. This priming effect was absent when the agent witnessed the re-hiding of the object, and thus her belief about the earlier location of the object did not have to be sustained. The same effect was observed when infants searched for a different toy (Experiment 2) or when an additional spatial transformation was introduced (Experiment 4), but not when the spatial transformation disrupted belief updating (Experiment 3). These data suggest that infants' representational apparatus is prepared to efficiently track other agents' beliefs online, encode underspecified beliefs and define their content later, possibly reflecting a crucial characteristic of mature theory of mind: using a metarepresentational format for ascribed beliefs. ☆ This paper is a part of special issue "Special Issue in Honour of Jacques Mehler, Cognition's founding editor".

Human action-anticipation methods predict what is the future action by observing only a few portion of an action in progress. This is critical for applications where computers have to react to human actions as early as possible such as autonomous driving, human-robotic interaction, assistive robotics among others. In this paper, we present a method for human action anticipation by predicting the most plausible future human motion. We represent human motion using Dynamic Images [1] and make use of tailored loss functions to encourage a generative model to produce accurate future motion prediction. Our method outperforms the currently best performing action-anticipation methods by 4% on JHMDB-21, 5.2% on UT-Interaction and 5.1% on UCF 101-24 benchmarks.

Brain imaging studies have shown that observation of both bodily movements and abstract motion displays complying with human kinematics activate the observer's motor cortex. However, it is unknown whether the same processes are active in the two conditions. Here, we addressed this issue using transcranial magnetic stimulation (TMS) to directly compare cortico-spinal excitability during observation of actions and motion stimuli that complied with or violated normal human kinematics. We found that kinematics significantly modulated the motor-evoked potentials (MEPs) produced by TMS during observation of both human and abstract motion stimuli. However, only the temporal unfolding of cortico-spinal excitability during observation of human movements significantly correlated with instantaneous stimulus velocity. This correlation was present for normal movements and also for a subset of the movements having unnatural kinematics. Furthermore, bodily movements for which we found no correlation between MEPs and stimulus velocity produced significantly higher MEPs. Our novel results suggest a dissociation in how human movements and abstract motion displays engage the observer's motor system. Specifically, while both stimulus types significantly activate the observer's motor cortex, only bodily movements produce patterns of cortico-spinal excitability that closely follow the velocity profile of the observed movement. This internal "re-enactment" of observed bodily movements seems to be only partially attuned to normal human kinematics.

Background: Previous research studies have assessed the relationship between attention to social information and peripheral (e.g., plasma and salivary) oxytocin (OT) levels in typically developing (TD) children and children with autism spectrum disorder (ASD). A relationship between them was observed in TD children, but not in children with ASD. However, this relationship remains unexamined in other age groups. To clarify whether this lack of association is maintained throughout development in individuals with ASD, we aimed to assess the relationship between salivary OT levels and attention to social information in adolescents and adults with and without ASD. Methods: We recruited male adolescents and adults with ASD (n = 17) and TD participants (n = 24). Using the all-inone eye-tracking system Gazefinder, we measured the percentage fixation time allocated to social information. We also measured the salivary OT levels and Autism Spectrum Quotient (AQ) of participants. Subsequently, we confirmed group differences and conducted a correlation analysis to investigate the relationships between these three measures. Results: Salivary OT levels did not show any significant difference between the ASD and TD groups and were negatively correlated with the AQ in the whole-group analysis, but not in within-group analysis. Individuals with ASD had significantly lower percentage fixation times than did TD individuals for eye regions in human faces with/ without mouth motion, for upright biological motion, and for people regions in the people and geometry movies. The percentage of fixation for geometric shapes in the people and geometry movies was significantly higher in the ASD than in the TD group. In the TD group, salivary OT levels were positively correlated with percentage fixation times for upright biological motion and people and negatively correlated with inverted biological motion and geometry. However, no significant correlations were found in the ASD group. Conclusions: Our exploratory results suggest that salivary OT levels in adolescents and adults with ASD are less indicative of attention to social stimuli than they are in TD adolescents and adults. It is suggested that their association

Al lm ma a M Ma at te er r S St tu ud di io or ru um m-U Un ni iv ve er rs si it tà à d di i B Bo ol lo og gn na a DOTTORATO DI RICERCA IN NEUROSCIENZE COGNITIVE Ciclo XXIV Settore Concorsuale di afferenza: AREA 11 Settore scientifico-disciplinare: M-PSI/02

The motor system can no longer be considered as a mere passive executive system of motor commands generated elsewhere in the brain. On the contrary, it is deeply involved in perceptual and cognitive functions and acts as an “anticipation device”. The present thesis investigates the anticipatory motor mechanisms occurring in two particular instances: i) when processing sensory events occurring within the peripersonal space (PPS); and ii) when perceiving and predicting others’actions. The first study provides evidence that PPS representation in humans modulates neural activity within the motor system, while the second demonstrates that the motor mapping of sensory events occurring within the PPS critically relies on the activity of the premotor cortex. The third study provides direct evidence that the anticipatory motor simulation of others’ actions critically relies on the activity of the anterior node of the action observation network (AON), namely the inferior frontal cortex (IFC)....

Influential theories suggest that humans predict others' upcoming actions by using their own motor system as an internal forward model. However, evidence that the motor system is causally essential for predicting others' actions is meager. Using transcranial direct current stimulation (tDCS), we tested the role of the inferior frontal cortex (IFC), in action prediction (AP). We devised a novel AP task where participants observed the initial phases of right-hand reaching-to-grasp actions and had to predict their outcome (i.e., the goal/object to be grasped). We found that suppression by cathodal (inhibitory) tDCS of the left IFC, but not the left superior temporal sulcus or the right IFC, selectively impaired performance on the AP task, but not on a difficulty-matched control task. Remarkably, anodal (excitatory) tDCS of the left IFC brought about a selective improvement in the AP task. These findings indicate that the left IFC is necessary for predicting the outcomes of observed human right-hand actions. Crucially, our study shows for the first time that down-and up-regulating excitability within the motor system can hinder and enhance AP abilities, respectively. These findings support predictive coding theories of action perception and have implications for enhancement of AP abilities.

I am indebted to many people for their support, friendship and guidance over the last three years. First, and foremost, I am extremely grateful to my supervisor Associate Professor Jason Low. Thank you for your persistent faith in my abilities and your enduring patience as my understanding grew, often slowly and through error. I am certain this dissertation would not have been possible without your unwavering encouragement in the face of adversity. I would also like to extend my gratitude to Dr. Stephen Butterfill for his insight and helpful feedback. In addition, I extend my thanks to Dr. Hannes Rakoczy for his evaluation of the present experiment, and also to Dr. Alia Martin for her thoughts about how to more clearly organize these experiments for presentation. Next, I must recognize my colleagues in the ToM lab. I am forever grateful to Katheryn Edwards, an intellectual giant whose counsel and comradery steadied my resolve on numerous occasions. I also wish to thank Schyana Sivanantham, Claudie Peloquin, Radhika Patel-Cornish, Tui Shaw, and Cong Fan for tolerating my unconventional, borderline-obsessive occupation of our shared workspace. Outside of Victoria University, I am grateful to my dear friends and creative collaborators Kalani Muller and Pani Stavrou-Wilson, whose respective artwork is displayed herein in the form of the procedural figures (see figures 2, 3, and 9) and the state-analysis figures (see figures 8, 10 11 and 14). I

Bayesian accounts of autism suggest that this disorder may be rooted in an impaired ability to estimate the probability of future events, possibly owing to reduced priors. Here, we tested this hypothesis within the action domain in children with and without autism using a behavioural paradigm comprising a familiarization and a testing phase. During familiarization, children observed videos depicting a child model performing actions in diverse contexts. Crucially, within this phase, we implicitly biased action-context associations in terms of their probability of co-occurrence. During testing, children observed the same videos but drastically shortened (i.e. reduced amount of kinematics information) and were asked to infer action unfolding. Since during the testing phase movement kinematics became ambiguous, we expected children's responses to be biased to contextual priors, thus compensating for perceptual uncertainty. While this probabilistic effect was present in controls, no ...

After viewing an image representing an action on an object, we recognize the forward states of the seen action faster than the backward states. The present study exploits a variant of a new experimental paradigm to investigate cognitive mechanisms underlying this effect. Participants viewed a series of still photos of unfolding actions on objects, each followed by a photo depicting either one of three (instead of two of the original paradigm) different and temporally distant moments after the image or one moment before the image, along with photos of different actions. Experiment 1 revealed the classical forward effects in this new context: when the task was to judge whether the action in the second photo was the same as in the first photo, evaluations were faster for all forward photos than for backward photos. In Experiment 2, we examined the role of participants’ attention to the object alongside the role of attention to action kinematics in triggering these “forward effects” by ...

The ability to predict, anticipate and reason about future outcomes is a key component of intelligent decision-making systems. In light of the success of deep learning in computer vision, deep-learning-based video prediction emerged as a promising research direction. Defined as a self-supervised learning task, video prediction represents a suitable framework for representation learning, as it demonstrated potential capabilities for extracting meaningful representations of the underlying patterns in natural videos. Motivated by the increasing interest in this task, we provide a review on the deep learning methods for prediction in video sequences. We firstly define the video prediction fundamentals, as well as mandatory background concepts and the most used datasets. Next, we carefully analyze existing video prediction models organized according to a proposed taxonomy, highlighting their contributions and their significance in the field. The summary of the datasets and methods is accompanied with experimental results that facilitate the assessment of the state of the art on a quantitative basis. The paper is summarized by drawing some general conclusions, identifying open research challenges and by pointing out future research directions.

It has been argued that infants possess a rich, sophisticated theory of mind (ToM) that is only revealed with tasks based on spontaneous responses. A mature (ToM) implies the understanding that mental states are person specific. Previous studies on infants' understanding of motivational mental states, such as goals and preferences have revealed that, by 9 months of age, infants do not generalize these motivational mental states across agents. However, it remains to be determined if infants also perceive epistemic states as person specific. Therefore, the goal of the present study was to use a switch agent paradigm with the classic false belief violation-of-expectation task. Results revealed that 16-month-old infants attributed true and false beliefs to a naïve agent-they did not perceive beliefs as person specific. These findings indicate that the mechanisms that underlie infants' implicit attribution of beliefs differ from those assumed for explicit reasoning about beliefs.

Experiments 1a, b and d from Chapter Two have been published in a peer-reviewed journal: Schenke, K. C., Wyer, N. A., & Bach, P. (2016). The Things You Do: Internal Models of Others’ Expected Behaviour Guide Action Observation. PLoS One, 11(7), e0158910. http://dx.doi.org/10.1371/journal.pone.0158910.

Perception of the final position of a moving object or creature is distorted forward along its actual or implied motion path, thus enabling anticipation of its forthcoming position. In a previous research, we demonstrated that viewing static snapshots that imply body actions activates the human motor system. What remains unknown, however, is whether extrapolation of dynamic information and motor activation are higher for upcoming than past action phases. By using single-pulse transcranial magnetic stimulation, we found that observation of start and middle phases of grasp and flick actions engendered a significantly higher motor facilitation than observing their final postures. Differential motor facilitation during start and end postures was independent of finger configuration at the different hand apertures. Subjective ratings showed that modulation of motor facilitation was not due to the amount of implied motion per se but to the forward direction of the motion path toward upcoming phases. Thus, motor facilitation proved maximal for the snapshots evoking ongoing but incomplete actions. The results provide compelling evidence that the frontal component of the observation-execution matching system is preferentially activated by the anticipatory simulation of future action phases and thus plays an important role in the predictive coding of others' motor behaviors.

Action observation is often conceptualized in a bottom-up manner, where sensory information activates conceptual (or motor) representations. In contrast, here we show that expectations about an actor's goal have a top-down predictive effect on action perception, biasing it toward these goals. In 3 experiments, participants observed hands reach for or withdraw from objects and judged whether a probe stimulus corresponded to the hand's final position. Before action onset, participants generated action expectations on the basis of either object types (safe or painful, Experiments 1 and 2) or abstract color cues (Experiment 3). Participants more readily mistook probes displaced in a predicted position (relative to unpredicted positions) for the hand's final position, and this predictive bias was larger when the movement and expectation were aligned. These effects were evident for low-level movement and high-level goal expectancies. Expectations bias action observation toward the predicted goals. These results challenge current bottom-up views and support recent predictive models of action observation.

Prior research has often conceptualized this capacity in terms of a motoric matching of observed actions to an action in one's motor repertoire, but has ignored the role of object information. In this manuscript, we set out an alternative conception of intention understanding, which places the role of objects as central to our observation and comprehension of the actions of others. We outline the current understanding of the interconnectedness of action and object knowledge, demonstrating how both rely heavily on the other. We then propose a novel framework, the affordance-matching hypothesis, which incorporates these findings into a simple model of action understanding, in which object knowledge-what an object is for and how it is used-can inform and constrain both action interpretation and prediction. We will review recent empirical evidence that supports such an object-based view of action understanding and we relate the affordance matching hypothesis to recent proposals that have re-conceptualized the role of mirror neurons in action understanding.

The ability to predict, anticipate and reason about future outcomes is a key component of intelligent decision-making systems. In light of the success of deep learning in computer vision, deep-learning-based video prediction emerged as a promising research direction. Defined as a self-supervised learning task, video prediction represents a suitable framework for representation learning, as it demonstrated potential capabilities for extracting meaningful representations of the underlying patterns in natural videos. Motivated by the increasing interest in this task, we provide a review on the deep learning methods for prediction in video sequences. We firstly define the video prediction fundamentals, as well as mandatory background concepts and the most used datasets. Next, we carefully analyze existing video prediction models organized according to a proposed taxonomy, highlighting their contributions and their significance in the field. The summary of the datasets and methods is accompanied with experimental results that facilitate the assessment of the state of the art on a quantitative basis. The paper is summarized by drawing some general conclusions, identifying open research challenges and by pointing out future research directions.

Significance A major challenge in studying intention reading is high motor variability. Analyses conducted across trials provide insights into what happens on average; however, they may obscure how individual observers read intention information in individual movements. We combined motion tracking, psychophysics, and computational analyses to examine intention reading in autism spectrum disorders (ASDs) with single-trial resolution. Results revealed that a sizeable fraction of ASD observers can identify intention-informative variations in ASD (but not in typically developing) movement kinematics, but they are nonetheless unable to extract the encoded intention information. This approach not only enhances our basic understanding of mind reading in ASD but also provides potential avenues for the rational design of training procedures to improve the reading of others’ actions.

Face perception and emotion categorization are widely investigated under laboratory conditions that are devoid of real social interaction. Using mobile eye-tracking glasses in a standardized diagnostic setting while applying the Autism Diagnostic Observation Schedule (ADOS-2), we had the opportunity to record gaze behavior of children and adolescents with and without Autism Spectrum Conditions (ASCs) during social interaction. The objective was to investigate differences in eye-gaze behavior between three groups of children and adolescents either (1) with ASC or (2) with unconfirmed diagnosis of ASC or (3) with neurotypical development (NTD) during social interaction with an adult interviewer in a diagnostic standard situation using the ADOS-2. In a case control study, we used mobile eye-tracking glasses in an ecologically valid and highly standardized diagnostic interview to investigate suspected cases of ASC. After completion of the ASC diagnostic gold standard including the ADOS-...

Chapter Introduction 7 Chapter 2 Means selection information overrides outcome selection information in goal attribution 41 Chapter 3 Evidence for a unitary goal concept in 12-month-old infants 61 Chapter 4 Outcome potential influences twelve-month-olds' interpretation of a novel action as goal-directed 79 Chapter 5 Explicit learning of arbitrary and non-arbitrary action-effect relations in adults and 4-year-olds Chapter 6 Explicit learning of arbitrary and non-arbitrary action-effect relations in adults and 4-year-olds Chapter 7 From outcome prediction to action selection: Developmental change in the role of action-effect bindings Chapter 8 The developing cognitive substrate of sequential action control in 9-to 12month-olds: Evidence for concurrent activation models Chapter 9 Summary and conclusions References Nederlandse samenvatting Curriculum Vitae

In joint action, multiple people coordinate their actions to perform a task together. This often requires precise temporal and spatial coordination. How do co-actors achieve this? How do they coordinate their actions toward a shared task goal? Here, we provide an overview of the mental representations involved in joint action, discuss how co-actors share sensorimotor information and what general mechanisms support coordination with others. By deliberately extending the review to aspects such as the cultural context in which a joint action takes place, we pay tribute to the complex and variable nature of this social phenomenon.

Our behavior is frequently influenced by those around us. However, the majority of social cognition research is conducted using socially isolated paradigms, without the presence of real people (i.e., without a "social presence"). The current study aimed to test the influence of social presence upon a measure of mentalizing behavior in adults. Study 1 used a first-order theory of mind task; and study 2 used a second-order theory of mind task. Both studies included two conditions: live, where the task protagonists were physically present acting out the task, or recorded, where the same task protagonists demonstrated the task in a video recording. In both experiments, participants were affected by the social presence and demonstrated significantly different patterns of behavior in response to the presence of real people. This study, therefore, highlights the critical importance of understanding the effect of a social presence in mentalizing research, and suggests that the inclusion of a social presence needs to be given strong consideration across social cognition paradigms.

The delivery of Ambient Assisted Living services, specifically relating to the smart-home paradigm, assumes that people can be provided with help, automatically and in real time, in their homes as and when required. Nevertheless, the deployment of a smart-home can lead to high levels of expense due to configuration requirements of multiple sensing and actuating technology. In addition, the vast amount of data produced leads to increased levels of computational complexity when trying to ascertain the underlying behavior of the inhabitant. This contribution presents a methodology based on feature selection which aims to reduce the number of sensors required whilst still maintaining acceptable levels of activity recognition performance. To do so, a smart-home dataset has been utilized, obtaining a configuration of sensors with the half sensors with respect to the original configuration.

Understanding irrational actions may require the observer to make mental state inferences about why an action was performed. Individuals with autism spectrum conditions (ASC) have well documented difficulties with mentalizing; however the degree to which rationality understanding is impaired in autism is not yet clear. The present study uses eye-tracking to measure online understanding of action rationality in individuals with ASC. Twenty adults with ASC and 20 typically developing controls, matched for age and IQ watched movies of rational and irrational actions while their eye movements were recorded. Measures of looking time, scan path and saccade latency were calculated. Results from looking time and scan path analyses demonstrate that participants with ASC have reduced visual attention to salient action features such as the action goal and the hand performing the action, regardless of action rationality. However, when participants with ASC do attend to these features, they are able to make anticipatory goal saccades as quickly as typically developing controls. Taken together these results indicate that individuals with autism have reduced attention to observed actions, but when attention is maintained, goal prediction is typical. We conclude that the basic mechanisms of action understanding are intact in individuals with ASC although there may be impairment in the top-down, social modulation of eye movements.

Recognizing human activities in partially observed videos is a challenging problem and has many practical applications. When the unobserved subsequence is at the end of the video, the problem is reduced to activity prediction from unfinished activity streaming, which has been studied by many researchers. However, in the general case, an unobserved subsequence may occur at any time by yielding a temporal gap in the video. In this paper, we propose a new method that can recognize human activities from partially observed videos in the general case. Specifically, we formulate the problem into a probabilistic framework: 1) dividing each activity into multiple ordered temporal segments, 2) using spatiotemporal features of the training video samples in each segment as bases and applying sparse coding (SC) to derive the activity likelihood of the test video sample at each segment, and 3) finally combining the likelihood at each segment to achieve a global posterior for the activities. We further extend the proposed method to include more bases that correspond to a mixture of segments with different temporal lengths (MSSC), which can better represent the activities with large intra-class variations. We evaluate the proposed methods (SC and MSSC) on various real videos. We also evaluate the proposed methods on two special cases: 1) activity prediction where the unobserved subsequence is at the end of the video, and 2) human activity recognition on fully observed videos. Experimental results show that the proposed methods outperform existing state-of-the-art comparison methods.

Origins of Mindreading Abilities in Children and Monkeys Judith Burkart ([email protected]), Anthropological Institute, University of Zurich, Switzerland Adolf Heschl ([email protected]), Zoological Institute, University of Graz, Austria Luca Surian & Stefania Caldi ([email protected]), Dipartimento di Psycologia, Universita di Trieste, Trieste, Italy Claudia Thoermer ([email protected]) & Beate Sodian ([email protected]); LMU, Munich, Germany Henrike Moll ([email protected]) & Michael Tomasello (tomas@)eva.mpg.de), Max-Planck Institute for Evolutionary Anthropology, Leipzig, Germany went to the same location as before and therefore to a new goal object. Infants looked at the two test trials differently in the two conditions, suggesting an implicit mindreading ability when interpreting the animation events. One central issue in research on precursors to a “theory of mind” and representation of intentional relations is how the ep...

Recent findings suggest that infants understand others' preferential choice and can use the perspectives and beliefs of others to interpret their actions. The standard interpretation in the field is that infants understand preferential choice as a dispositional state of the agent. It is possible, however, that these social situations trigger the acquisition of more general, not person-specific knowledge. In a looking-time study we showed an Agent A demonstrating a choice, that only could have been interpreted as preferential based on the perspective (and thus the belief) of the agent, not the observer. Then we introduced a new agent (Agent B), who chose consistently or inconsistently with Agent A; also varying whether Agent B was an adult or a child. Results show that infants expected Agent B (both the adult and the child) to choose as Agent A, but only in the condition where according to Agent A's knowledge two objects were present in familiarization(confirming previous evidence on the importance of contrastive choice). We interpret these results in the following way: (1) infants do not encode the perspectives of other agents as person-specific sources of knowledge and (2) they learn about the object, rather than the agent's disposition towards that object. We propose that early theory of mind processes lack the binding of belief content to the belief holder. However, such limitation may in fact serve an important function, allowing infants to acquire information through the perspectives of others in the form of universal access to general information.