Research

See also Corpora for web-based research results.

Call for papers
HCI 2021 Session: “Semantic, artificial and computational interaction studies: Towards a behavioromics of multimodal communication”
Organized by Andy Lücking, Alexander Mehler, Cornelia Ebert
In natural language interactions, the whole human body acts as a “semiotic display” (Goodwin 2003). The field of HCI was among the first that paid this social fact its due attention. Notwithstanding a few exceptions, theoretical linguistics only recently begun to investigate the communicative bandwidth of speech and gesture. We use “gesture” in a broad sense here, covering manual gestures, facial expresssions, head movements, shrugs, laughter, or body orientation. Simultaneously, due to the digital turn, the traditional methodological triad of “armchair, laboratory and field” (Clark & Bangerter 2004) is expanded by data analytics, that is in the context of multimodal interaction, a statistical means to describe the form of communication. This session aims at bringing these branches together. Potential aims include to delineate experimental studies, computational methods, and resource building and exploitation that connects armchair, laboratory, field and data — a joint methodological endeavour that might be called “behavioromics”.

Possible questions in this regard include: Can HCI, health or ergonomic applications be used as experimental gesture study settings? Which computational methods enable big gesture data analysis? Are there already resources that can be used in this regard? What phenomena are studied in semantics and dialogue theory at all? Finally: how do gesture data analyses, digital human modelling applications and semantic/pragmatic gesture research combine methodologically?

The session cover, but is not limited to, topics such as the following:

– phenomena under discussion: past, present, future
– big gesture data
– creation and exploitation of multimodal corpora
– avatars as experimental setting
– cross-modal tracking
– data-based multimodal analysis
– detecting gestalts
– automatic annotation
– multimodal networks

We want to emphasize that conceptual contributions are highly welcome! By this we mean contributions that reflect on, or propose steps towards, the individual thematic priority with respect to “behavioromics”.

Last but not least, the conference session provides a platform for bringing together semanticists, computer scientists and researchers from related fields that deal with multimodal interaction. Since we all work on virtually the same topic but from different angles, there should be opportunities to get in touch: to see what others are doing and to approach some of the above-outlined, methodological challenges.

References for quotations:

Clark, Herbert H. and Adrian Bangerter (2004). “Changing Ideas about Reference.” In: Experimental Pragmatics. Ed. by Ira A. Noveck and Dan Sperber. Palgrave Studies in Pragmatics, Language and Cognition. Basingstoke and New York: Palgrave Macmillan. Chap. 2, pp. 25–49.

Goodwin, Charles (2003). “Pointing as Situated Practice.” In: Pointing: Where Language, Culture, and Cognition Meet. Ed. by Sotaro Kita. Mahwah, New Jersey: Lawrence Erlbaum Associates, Inc. Chap. 2, pp. 217–241.

Important dates:
December 01, 2020 1-2 page abstract upload via the CMS
February 14, 2021 full paper is due
Current Projects
BRIDGE
Berufspraktische Bildungsprozesse im Recht- und Lehramtsreferendariat sowie der Medizin unter Nutzung digitaler Medien (BRIDGE)
Homepage: BRIDGE
Supported by: BMBF
Managed by: Alexander Mehler
Duration: 01.07.2020 – 31.06.2023
Minderheitensprachen in Multimodalen Netzwerken
Part of: LOEWE-Schwerpunkt “Minderheitenstudien: Sprache und Identität”
Homepage: Minderheitenstudien
Press: Press
Supported by: BMBF
Managed by: Alexander Mehler
Duration: 01.01.2020 – 31.12.2023
PLATO: Positive Learning in the Age of InformaTiOn
Homepage: PLATO
Supported by: Rhine-Main Universities (RMU)
Managed by: Alexander Mehler, Olga Zlatkin-Troitschanskaia
Duration: 15.01.2019 – 14.01.2020
Fachinformationsdienst Biodiversitätsforschung
Homepage: BIOfid; TTLab BIOfid
Supported by: Deutsche Forschungsgemeinschaft
Managed by: Alexander Mehler, Angela Hausinger, Volker Mosbrugger
Duration: July 2017 – June 2020
Public History
Homepage: Stolperwege
Supported by: Forschungszentrum für Historische Geisteswissenschaften
Förderfonds Lehre
Managed by: Alexander Mehler, Christoph Cornelißen, Nicole Kramer, Giuseppe Abrami
Duration: Since 2015
CEDIFOR
Homepage: Centrum für Digitale Forschung in den Geistes-, Sozial- und Bildungswissenschaften
Supported by: BMBF
Managed by: Goethe-Universität Frankfurt, TU Darmstadt, Deutsches Institut für Internationale Pädagogische Forschung (DIPF)
Duration: 2014-2019
Forschungsfeld Digital Humanities
Homepage: Forschungsfeld 5: Digital Humanities
Supported by: Forschungszentrum Historische Geisteswissenschaften
Managed by: Jost Gippert, Alexander Mehler, Bernhard Jussen
Duration: Since 2010
Completed Projects
CompHistSem
Homepage: Computational Historical Semantics
Supported by: Gottfried Wilhelm Leibniz Award, BMBF, LOEWE
Managed by: Alexander Mehler, Bernhard Jussen
Duration: Since 2013 – 2016
WikiNect: kinetische Mesuemswikis
Homepage: WikiNect
Supported by: Förderfonds Lehre
Managed by: Alexander Mehler, Andy Lücking
Duration: 2012 – 2015
Pilotprojekt 1: Academic Trails in Multimedia Spaces
Homepage: Academic Trails in Multimedia Spaces
Supported by: BMBF
Managed by: Anne Bohnenkamp-Renken, Alexander Mehler
Duration: May to October 2015
Pilotprojekt 2: Historische Semantik, Kookkurrenzanalyse und Kollationierung
Homepage: Historische Semantik, Kookkurrenzanalyse und Kollationierung
Supported by: BMBF
Managed by: Bernhard Jussen, Alexander Mehler
Duration: January to October 2015
Corpusmanagement: Historische Semantik
Supported by: Landes-Offensive zur Entwicklung Wissenschaftlich-ökonomischer Exzellenz (LOEWE)
Managed by: Alexander Mehler, Bernhard Jussen
Duration: 2011-2013
Corpusmanagement: Hofmannsthal-Ausgabe und Faust-Illustration
Supported by: Landes-Offensive zur Entwicklung Wissenschaftlich-ökonomischer Exzellenz (LOEWE)
Managed by: Alexander Mehler, Anne Bohnenkamp-Renken
Duration: 2011-2013
Corpusmanagement: Parallelkorpora und sprachliche Beziehungen
Supported by: Landes-Offensive zur Entwicklung Wissenschaftlich-ökonomischer Exzellenz (LOEWE)
Managed by: Alexander Mehler, Jost Gippert
Duration: 2011-2013
Corpusmanagement: Historisch-kritische Apparate
Supported by: Landes-Offensive zur Entwicklung Wissenschaftlich-ökonomischer Exzellenz (LOEWE)
Managed by: Alexander Mehler, Jost Gippert
Duration: 2011-2013
Sprachliche Netzwerke: texttechnologische Repräsentation, computerlinguistische Synthese und physikalische Modellierung
Homepage: Linguistic Networks
Supported by: BMBF – Förderschwerpunkt Wechselwirkungen zwischen Natur- und Geisteswissenschaften
Managed by: Alexander Mehler, Philippe Blanchard, Barbara Job, Hans-Jürgen Eikmeyer
Duration: 2008-2011
Automatische Anreicherung von OAI-Metadaten mit Hilfe computerlinguistischer Verfahren und Entwicklung von Services für die inhaltsorientierte Vernetzung von Repositorien
Supported by: DFG-Schwerpunktprogramm Literaturversorgungs- und Informationssysteme
Managed by: Michael Höppner, Alexander Mehler, Gerhard Heyer
Duration: 2009-2011
Knowledge Enhanced Embodied Cognitive Interaction Technologies (Know CIT)
Supported by: Excellence Cluster 277 Cognitive Interaction Technology
Managed by: Alexander Mehler, Ipke Wachsmuth
Duration: 2008-2011
Entwicklung, Erprobung und Evaluation eines Softwaresystems von inhaltsorientierten P2P-Agenten für die thematische Strukturierung und Suchoptimierung in digitalen Bibliotheken
Supported by: DFG-Schwerpunktprogramm Literaturversorgungs- und Informationssysteme
Managed by: Alexander Mehler, Gerhard Heyer
Duration: 2007-2009
Multimodal alignment corpora: statistical modeling and information management
Homepage: X1: Multimodal alignment corpora
Supported by: Sonderforschungsbereichs 673 Alignment in Communication
Managed by: Alexander Mehler, Hannes Rieser, Hans-Jürgen Eikmeyer
Duration: 2006-2010
Dialogue games and group dynamics
Homepage: A3: Dialogue games and group dynamics
Supported by: Sonderforschungsbereichs 673 Alignment in Communication
Managed by: Gerhard Jäger, Alexander Mehler
Duration: 2006-2010
Induktion von Dokumentgrammatiken zur Repräsentation logischer Hypertext-Dokumentstrukturen
Homepage: DFG-Research Group Text Technological Modelling of Information
Supported by: DFG-Projekt innerhalb der Forschergruppe 437 Texttechnologische Informationsmodellierung
Managed by: Alexander Mehler
Duration: 2005-2008