FACES

Feasibility, Acceptance, and Data Quality of New Multimodal Surveys

FACES_LOGO

The FACES project (Feasibility, Acceptance, and Data Quality of New Multimodal Surveys) aims to create a multimodal data space for survey research that can expand and replace face-to-face interviews in the future through the use of virtual reality (VR) and artificial intelligence (AI). This multi-interface system for online surveys is designed to offer a high degree of variability in terms of avatars, situational parameters, interfaces and AI technologies for the automatic processing of speech and behavioural data.

www.lifbi.de/

Background

Interview-based surveys face challenges such as increasing costs and decreasing response rates. Recent innovations in the fields of VR and AI offer new approaches to address these issues. Avatar-based interviews open up additional degrees of freedom through the choice of avatars and the variability of interaction situations. However, systematic studies on the effects and acceptance of such systems and on their potential to reduce interviewer effects are still lacking. The project goes beyond existing studies by comprehensively investigating self-avatar and other-avatar effects and their influence on classical interviews for the first time. In addition, scenarios with different degrees of immersion (from fully immersive VR interviews to video-based interviews) are compared.

Approach

In a first step, an open-source system for avatar-based and video-based interviews is being developed. This system will be used in a small preliminary study to examine the effects of different avatar and situational characteristics in experiments. Scenarios with different degrees of immersion (from fully immersive VR interviews to video-based interviews) will also be compared.

Based on the results, promising feature combinations will be tested in interviews with former NEPS participants.

Three central questions will be investigated:

  • What are the advantages of avatar-based interviews compared to video-based interviews in terms of acceptance, feasibility and data quality?
  • Which combinations of features reduce interviewer effects and how do they interact?
  • How can the results be integrated into a theory for training virtual interviewers?

Project Profile

Project lead and application:

Project members:

Publications

Patrick Schrottenbacher, Alexander Mehler, Vivienne Bernhardt, Leon Rohe and Giuseppe Abrami. 2026. ReEmote: Towards Emotion Representation in VR Through Va.Si.Li-Lab. Proceedings of XR Salento 2026. accepted.
BibTeX 
@inproceedings{Schrottenbacher:et:al:2026:a,
  author    = {Schrottenbacher, Patrick and Mehler, Alexander and Bernhardt, Vivienne
               and Rohe, Leon and Abrami, Giuseppe},
  title     = {ReEmote: Towards Emotion Representation in {VR} Through {Va.Si.Li}-Lab},
  booktitle = {Proceedings of XR Salento 2026},
  year      = {2026},
  publisher = {Springer International Publishing},
  keywords  = {VR, XR, affective computing, virtual humans, emotion detection, FACES},
  abstract  = {Human social interactions are inherently multimodal, shaped not
               only by what speakers convey but also by cues such as facial expressions,
               posture, and gestures. Together, these channels shape both participants'
               perceptions and behaviors, further reinforcing conversational
               feedback loops. This multimodal system extends to VR, where avatars
               serve as proxies for human interaction, making both visual and
               auditory fidelity essential for engaging. To properly utilize
               the emotional expression space that virtual environments allow,
               we introduce ReEmote. ReEmote extends the capabilities of Va.Si.Li-Lab,
               a collaborative, multi-user VR platform built on Ubiq. While Va.Si.Li-Lab
               supports user emotional expression through facial and hand tracking,
               ReEmote extends this by introducing schema-based emotion mappings
               that affect both avatars and their environments. This fosters
               immersive, emotionally aware environments that are beneficial
               for human and chatbot agent interactions, where human users and
               virtual agents share an emotional expression space. By enabling
               richer emotional dynamics, ReEmote opens up new ways of designing
               affective and engaging virtual experiences.In this paper, we describe
               the design choices behind ReEmote and present an evaluation of
               the graphical validity of the emotion representation introduced
               by ReEmote. Our results indicate that emotions can be validly
               represented through avatar facial expressions that users can quickly
               identify as Ekman's basic emotions.This opens up several possibilities
               for extending emotion-related text-to-speech (TTS) applications
               in Extended Reality (XR) with ReEmote. The paper also outlines
               use cases for XR-based TTS applications.},
  note      = {accepted}
}