ANIMAFEST SCANNER X | SYMPOSIUM FOR CONTEMPORARY ANIMATION STUDIES | PANEL 2 – DIGITAL TECHONOLOGIES AND ORIGINALITY

CONVERTING VIDEO LECTURES INTO EFFECTIVE AND INCLUSIVE ANIMATED PRESENTATIONS Nicoletta Adamo (Professor, Purdue University, USA) Christos Mousas (Assistant Professor, Purdue University, USA)

PANEL 2: DIGITAL TECHONOLOGIES AND ORIGINALITY
TEATAR &TD
06/06 TUE 14:50-15:20

Our project aims to develop, validate, study, and make publicly available an AI-based framework that takes existing online instructional video lectures on STEM topicscreated by human instructors and transform them into instructionally effective animated agent-based presentations.In particular, the transformed presentations are delivered by animated pedagogical agents who display emotionally and socially sensitive voice and gesture, react to the emotional state of the learner, and represent diversity in gender, ethnicity, and age. More specifically, our framework converts the gestures extracted from instructional video lectures into animation data that can be mapped to a rigged 3D character that can be rendered with various genders, ethnicities, and ages. Further, the framework allows for synthesizing affect-expressive body gestures based on a novel hybrid method that combines rule-based, speech prosody analysis and video analysis approaches. While rule-based approaches have been used extensively to generate affective body gestures, speech-driven gesture animation has been studied in the recent literature mostly without specific emphasis on affect-expressive models. Videos have been used to extract the gesturing style of the speaker and generate gesture animations consistent with the performer’s style. The proposed framework builds on our recent work on rule-based affective gesture generation and also exploits the affective content of the speech and the expressive content of the video to enhance the expressiveness of the animated agent’s synthesized gestures. This allows for creating a particular agent type with a unique personality and movement style, thus avoiding the pitfall
of nonspecific behavior. (The project is funded by NSF- RETLL - Collaborative Research: Using Artificial Intelligence to Transform Online Video Lectures into
Effective and Inclusive Agent-Based Presentations – Award #2201019.)

Nicoletta Adamo is Professor of Computer Graphics Technology and Purdue University Faculty Scholar. She is an award-winning animator and graphic
designer and creator of several 2D and 3D animations that aired on national television. Her area of expertise is in character animation and character design and her research interests focus on the application of 3D animation technology to education, HCI (Human Computer Interaction), affective computing and visualization. Prof. Adamo is the co-founder and director of the IDEAlaboratory [https:// polytechnic.purdue.edu/facilities/idea-laboratory].

Christos Mousas is an Assistant Professor in the Department of Computer Graphics Technology and director of the Virtual Reality Lab at Purdue University, Polytechnic Institute. His research revolves around virtual reality, virtual humans, computer graphics, and human-computer interaction. From 2015-2016 he was a Postdoctoral Researcher in the Visual Computing Lab of the Department of Computer Science at Dartmouth College. He holds a Ph.D. (2014) in Informatics and an M.Sc. (2011) in Multimedia Applications and Virtual Environments, both from the Department of Informatics of the University of Sussex and a five-year integrated Masters (B.Sc. & M.Sc.; 2009) in Audiovisual Science and Art from the Department of Audio & Visual Arts of Ionian University. He is a member of ACM and IEEE.