Quantitative video analysis for online behavioral analysis in ASD

  • Awarded: 2017
  • Award Type: Explorer
  • Award #: 553169

In spite of significant, recent advances in the molecular genetics and neuroscience of mental health disorders, behavioral ratings based on clinical observations are still the gold standard for screening, diagnosing and assessing outcomes in neurodevelopmental disorders, including autism spectrum disorder (ASD). Such behavioral ratings are subjective, require significant clinician expertise and training, and typically do not capture data from the children in their natural environments. Further, these behavioral measures are not scalable for large population screening, low resource settings or longitudinal monitoring, all of which are critical for outcome evaluation in multisite studies and for understanding and evaluating symptoms in the general population. The development of innovative and efficient approaches to behavioral screening, diagnosis and monitoring is thus a significant unmet need in the area of healthcare, including ASD. It is critical to develop validated (and if so required, FDA approved), low-cost and scalable tools for behavioral analysis of (childhood) mental health conditions.

Guillermo Sapiro’s team has conducted studies of an ASD digital screening and monitoring tool based on computer vision1, demonstrating proof of concept and feasibility. These early measures of attention were based on automated coding head turns. In the proposed project, Sapiro aims to enhance his group’s ability to measure attention by developing robust, online gaze-analysis capability. Gaze patterns are a sensitive index of ASD symptoms; however, current studies rely on expensive eye-tracking technology used in controlled laboratory settings. Sapiro’s team will build upon their preliminary work to develop computational and machine-learning tools for gaze analysis from cameras embedded in standard phones/tablets/computers, along with tools to integrate gaze analysis with other video-recorded behaviors, such as emotions. This type of integrated stimuli sensing-analysis approach is needed for truly scalable online behavioral sensing.

The group will then validate these new tools in a large population of infants and young children with ASD, ascertained through Duke Primary Care in a study supported by the newly awarded National Institutes of Health Autism Center of Excellence at Duke University. The tools developed will be available for deployment in other environments, since they are all are comprised solely of software. The development and validation of this simple, automated behavioral-analysis tool will help systematize behavioral ratings and allow broader, more uniform screening for conditions such as ASD.



1.Hashemi J., et al. Autism Res. Treat. 2014:935686 (2014) PubMed
Subscribe to our newsletter and receive SFARI funding announcements and news