WikiNect

WikiNect: kinetische Museumswikis
Prof. Dr. Alexander Mehler
Institut für Informatik – AG Texttechnologie

Lehrveranstaltungen

Projektbeschreibung:

Das zugleich kultur- und informationswissenschaftliche Projekt WikiNect zielt auf die Konzeptionierung, Modellierung und Erprobung einer geisteswissenschaftlich motivierten Technologie, welche das Prinzip des Game with a Purpose mit dem Prinzip der wikibasierten Kollaboration und der bewegungsunterstützten Texttechnologie im Anwendungskontext von Museen vereint:

  • Games with a Purpose [1] wurden dazu entwickelt, nicht automatisierbare Aufgaben der Informationsanreicherung von Objekten (Bilder, Videos etc.) derart in webbasierte Spiele zu integrieren, dass die Aufgabenerledigung als Nebenprodukt des Spiels abfällt. Der Spielcharakter birgt einen Unterhaltungswert für die beteiligten Spieler und motiviert diese zur Teilnahme.
  • Wikis [2] wurden zu dem Zweck entwickelt, Webusern die Möglichkeit zur kooperativen Dokumentation und Strukturierung von Wissens- oder allgemeiner von Interessensgebieten zu geben. Auf diese Weise ist mit der Wikipedia [3] eine weltberühmte online Enzyklopädie ebenso entstanden wie eine Vielzahl von Spezialwikis, die in der Mediengeschichte ihresgleichen suchen.
  • Bewegungsunterstützte Texttechnologien entwickeln sich derzeit als neues Paradigma der Human Computer Interaction. Sie machen die Nutzung klassischer Schnittstellen (wie Maus, Tastatur etc.) obsolet und versprechen die Nutzbarmachung von Gesten und Körperbewegungen als Mittel zur Steuerung von Programmen.

In dem Projekt WikiNect: kinetische Museumswikis soll ein Konzept für die Integration dieser drei Paradigmen entwickelt und in Form eines Prototyps erprobt werden, wobei der Brückenschlag zwischen konzeptuellem Entwurf und praktischer Erprobung durch ein informationswissenschaftliches Modell erfolgen soll. Im Anwendungskontext von Museen ergibt sich die kulturwissenschaftliche Frage, wie die verschiedenen Besucher die jeweilige Ausstellung erlebt haben und bewerten, eine Frage, die mit herkömmlichen Mitteln (Umfrage, Fragebogen) nur allzu aufwändig und kaum interaktiv beantwortet werden kann. In dem Projekt soll die Integration der Methoden 1.-3. exemplarisch durchgeführt werden, um dieses Defizit zu beheben:

  • Das spielerische Szenario ermuntert die Besucher zur unmittelbaren Mitarbeit an dem Museumswiki.
  • Das Wikiprinzip erlaubt die kooperative Weiterentwicklung des Wikis über den Museumsbesuch hinaus.
  • Schließlich stellen bewegungsunterstützte Technologien eine nahezu voraussetzungslose gestische Interaktionsschnittstelle auch für computerunerfahrene Besucher bereit.

Es geht also darum, Museumsbesuchern eine Technologie an die Hand zu geben, mit der sie ihren Museumsbesuch thematisieren, resümieren und bewerten können. Besucher sollen auf diese Weise Trajektorien durch den virtuellen Raum der Ausstellungsobjekte bewegungsgesteuert erzeugen können, die sie überall bewerten und kontextualisieren können. Durch die computergestützte Erfassung dieser Trajektorien entstehen Besuchergeschichten, die im Web nach dem Wiki-Prinzip kollaborativ weiterverarbeitet und angereichert werden können. Die wissenschaftliche Fragestellung des Projekts betrifft konkret die interdisziplinäre Ausgestaltung eines Konzepts, das diese Idee für den Anwendungskontext Museumsbesuch spezifiziert, informationswissenschaftlich modelliert und anhand eines Technologie-Prototypen praktisch erprobt.

Projektseminar:

Leitung und Betreuung: Alexander Mehler, Giuseppe Abrami, Andy Lücking

Publications
  • [PDF] A. Mehler, T. vor der Brück, and A. Lücking, “Comparing Hand Gesture Vocabularies for HCI,” in Proceedings of HCI International 2014, 22 – 27 June 2014, Heraklion, Greece, Berlin/New York: Springer, 2014.
    [Abstract] [BibTeX]

    HCI systems are often equipped with gestural                    interfaces drawing on a predefined set of admitted                    gestures. We provide an assessment of the fitness of                    such gesture vocabularies in terms of their                    learnability and naturalness. This is done by example                    of rivaling gesture vocabularies of the museum                    information system WikiNect. In this way, we do not                    only provide a procedure for evaluating gesture                    vocabularies, but additionally contribute to design                    criteria to be followed by the gestures.
    @InCollection{Mehler:vor:der:Brueck:Luecking:2014,
      Author         = {Mehler, Alexander and vor der Brück, Tim and
                       Lücking, Andy},
      Title          = {Comparing Hand Gesture Vocabularies for HCI},
      BookTitle      = {Proceedings of HCI International 2014, 22 - 27 June
                       2014, Heraklion, Greece},
      Publisher      = {Springer},
      Address        = {Berlin/New York},
      abstract       = {HCI systems are often equipped with gestural
                       interfaces drawing on a predefined set of admitted
                       gestures. We provide an assessment of the fitness of
                       such gesture vocabularies in terms of their
                       learnability and naturalness. This is done by example
                       of rivaling gesture vocabularies of the museum
                       information system WikiNect. In this way, we do not
                       only provide a procedure for evaluating gesture
                       vocabularies, but additionally contribute to design
                       criteria to be followed by the gestures.},
      keywords       = {wikinect},
      pdf            = {https://www.texttechnologylab.org/wp-content/uploads/2015/08/Comparing-Gesture-Vocabularies-1_1.pdf},
      website        = {{http://link.springer.com/chapter/10.1007/978-3-319-07230-2_8#page-1}},
      year           = 2014
    }
  • [PDF] [DOI] A. Mehler, A. Lücking, and G. Abrami, “WikiNect: Image Schemata as a Basis of Gestural Writing for Kinetic Museum Wikis,” Universal Access in the Information Society, pp. 1-17, 2014.
    [Abstract] [BibTeX]

    This paper provides a theoretical assessment of                    gestures in the context of authoring image-related                    hypertexts by example of the museum information system                    WikiNect. To this end, a first implementation of                    gestural writing based on image schemata is provided                    (Lakoff in Women, fire, and dangerous things: what                    categories reveal about the mind. University of Chicago                    Press, Chicago, 1987). Gestural writing is defined as a                    sort of coding in which propositions are only expressed                    by means of gestures. In this respect, it is shown that                    image schemata allow for bridging between natural                    language predicates and gestural manifestations.                    Further, it is demonstrated that gestural writing                    primarily focuses on the perceptual level of image                    descriptions (Hollink et al. in Int J Hum Comput Stud                    61(5):601–626, 2004). By exploring the metaphorical                    potential of image schemata, it is finally illustrated                    how to extend the expressiveness of gestural writing in                    order to reach the conceptual level of image                    descriptions. In this context, the paper paves the way                    for implementing museum information systems like                    WikiNect as systems of kinetic hypertext authoring                    based on full-fledged gestural writing.
    @Article{Mehler:Luecking:Abrami:2014,
      Author         = {Mehler, Alexander and Lücking, Andy and Abrami,
                       Giuseppe},
      Title          = {{WikiNect}: Image Schemata as a Basis of Gestural
                       Writing for Kinetic Museum Wikis},
      Journal        = {Universal Access in the Information Society},
      Pages          = {1-17},
      abstract       = {This paper provides a theoretical assessment of
                       gestures in the context of authoring image-related
                       hypertexts by example of the museum information system
                       WikiNect. To this end, a first implementation of
                       gestural writing based on image schemata is provided
                       (Lakoff in Women, fire, and dangerous things: what
                       categories reveal about the mind. University of Chicago
                       Press, Chicago, 1987). Gestural writing is defined as a
                       sort of coding in which propositions are only expressed
                       by means of gestures. In this respect, it is shown that
                       image schemata allow for bridging between natural
                       language predicates and gestural manifestations.
                       Further, it is demonstrated that gestural writing
                       primarily focuses on the perceptual level of image
                       descriptions (Hollink et al. in Int J Hum Comput Stud
                       61(5):601–626, 2004). By exploring the metaphorical
                       potential of image schemata, it is finally illustrated
                       how to extend the expressiveness of gestural writing in
                       order to reach the conceptual level of image
                       descriptions. In this context, the paper paves the way
                       for implementing museum information systems like
                       WikiNect as systems of kinetic hypertext authoring
                       based on full-fledged gestural writing.},
      doi            = {10.1007/s10209-014-0386-8},
      issn           = {1615-5289},
      keywords       = {wikinect},
      pdf            = {https://www.texttechnologylab.org/wp-content/uploads/2015/08/art_10.1007_s10209-014-0386-8.pdf},
      website        = {http://dx.doi.org/10.1007/s10209-014-0386-8},
      year           = 2014
    }
  • A. Mehler, A. Lücking, T. vor der Brück, and G. Abrami, WikiNect – A Kinetic Artwork Wiki for Exhibition Visitors, 2013.
    [Poster][BibTeX]

    @Misc{Mehler:Luecking:vor:der:Brueck:2013:a,
      Author         = {Mehler, Alexander and Lücking, Andy and vor der
                       Brück, Tim and Abrami, Giuseppe},
      Title          = {WikiNect - A Kinetic Artwork Wiki for Exhibition
                       Visitors},
      HowPublished   = {Poster Presentation at the Scientific Computing and
                       Cultural Heritage 2013 Conference, Heidelberg},
      keywords       = {wikinect},
      month          = {11},
      poster         = {https://www.texttechnologylab.org/wp-content/uploads/2015/08/SCCHPoster2013.pdf},
      url            = {http://scch2013.wordpress.com/},
      year           = 2013
    }
  • [PDF] A. Mehler and A. Lücking, “Pathways of Alignment between Gesture and Speech: Assessing Information Transmission in Multimodal Ensembles,” in Proceedings of the International Workshop on Formal and Computational Approaches to Multimodal Communication under the auspices of ESSLLI 2012, Opole, Poland, 6-10 August, 2012.
    [Abstract] [BibTeX]

    We present an empirical account of multimodal                    ensembles based on Hjelmslev’s notion of selection.                    This is done to get measurable evidence for the                    existence of speech-and-gesture ensembles. Utilizing                    information theory, we show that there is an                    information transmission that makes a gestures’                    representation technique predictable when merely                    knowing its lexical affiliate – in line with the                    notion of the primacy of language. Thus, there is                    evidence for a one-way coupling – going from words to                    gestures – that leads to speech-and-gesture alignment                    and underlies the constitution of multimodal ensembles.
    @InProceedings{Mehler:Luecking:2012:d,
      Author         = {Mehler, Alexander and Lücking, Andy},
      Title          = {Pathways of Alignment between Gesture and Speech:
                       Assessing Information Transmission in Multimodal
                       Ensembles},
      BookTitle      = {Proceedings of the International Workshop on Formal
                       and Computational Approaches to Multimodal
                       Communication under the auspices of ESSLLI 2012, Opole,
                       Poland, 6-10 August},
      Editor         = {Gianluca Giorgolo and Katya Alahverdzhieva},
      abstract       = {We present an empirical account of multimodal
                       ensembles based on Hjelmslev’s notion of selection.
                       This is done to get measurable evidence for the
                       existence of speech-and-gesture ensembles. Utilizing
                       information theory, we show that there is an
                       information transmission that makes a gestures’
                       representation technique predictable when merely
                       knowing its lexical affiliate – in line with the
                       notion of the primacy of language. Thus, there is
                       evidence for a one-way coupling – going from words to
                       gestures – that leads to speech-and-gesture alignment
                       and underlies the constitution of multimodal ensembles.},
      keywords       = {wikinect},
      pdf            = {https://www.texttechnologylab.org/wp-content/uploads/2016/06/Mehler_Luecking_FoCoMC2012-2.pdf},
      website        = {http://www.researchgate.net/publication/268368670_Pathways_of_Alignment_between_Gesture_and_Speech_Assessing_Information_Transmission_in_Multimodal_Ensembles},
      year           = 2012
    }
  • [PDF] A. Mehler and A. Lücking, “WikiNect: Towards a Gestural Writing System for Kinetic Museum Wikis,” in Proceedings of the International Workshop On User Experience in e-Learning and Augmented Technologies in Education (UXeLATE 2012) in Conjunction with ACM Multimedia 2012, 29 October- 2 November, Nara, Japan, 2012, pp. 7-12.
    [Abstract] [BibTeX]

    We introduce WikiNect as a kinetic museum information                    system that allows museum visitors to give on-site                    feedback about exhibitions. To this end, WikiNect                    integrates three approaches to Human-Computer                    Interaction (HCI): games with a purpose, wiki-based                    collaborative writing and kinetic text-technologies.                    Our aim is to develop kinetic technologies as a new                    paradigm of HCI. They dispense with classical                    interfaces (e.g., keyboards) in that they build on                    non-contact modes of communication like gestures or                    facial expressions as input displays. In this paper, we                    introduce the notion of gestural writing as a kinetic                    text-technology that underlies WikiNect to enable                    museum visitors to communicate their feedback. The                    basic idea is to explore sequences of gestures that                    share the semantic expressivity of verbally manifested                    speech acts. Our task is to identify such gestures that                    are learnable on-site in the usage scenario of                    WikiNect. This is done by referring to so-called                    transient gestures as part of multimodal ensembles,                    which are candidate gestures of the desired                    functionality. 
    @InProceedings{Mehler:Luecking:2012:c,
      Author         = {Mehler, Alexander and Lücking, Andy},
      Title          = {WikiNect: Towards a Gestural Writing System for
                       Kinetic Museum Wikis},
      BookTitle      = {Proceedings of the International Workshop On User
                       Experience in e-Learning and Augmented Technologies in
                       Education (UXeLATE 2012) in Conjunction with ACM
                       Multimedia 2012, 29 October- 2 November, Nara, Japan},
      Pages          = {7-12},
      abstract       = {We introduce WikiNect as a kinetic museum information
                       system that allows museum visitors to give on-site
                       feedback about exhibitions. To this end, WikiNect
                       integrates three approaches to Human-Computer
                       Interaction (HCI): games with a purpose, wiki-based
                       collaborative writing and kinetic text-technologies.
                       Our aim is to develop kinetic technologies as a new
                       paradigm of HCI. They dispense with classical
                       interfaces (e.g., keyboards) in that they build on
                       non-contact modes of communication like gestures or
                       facial expressions as input displays. In this paper, we
                       introduce the notion of gestural writing as a kinetic
                       text-technology that underlies WikiNect to enable
                       museum visitors to communicate their feedback. The
                       basic idea is to explore sequences of gestures that
                       share the semantic expressivity of verbally manifested
                       speech acts. Our task is to identify such gestures that
                       are learnable on-site in the usage scenario of
                       WikiNect. This is done by referring to so-called
                       transient gestures as part of multimodal ensembles,
                       which are candidate gestures of the desired
                       functionality. },
      keywords       = {wikinect},
      pdf            = {https://www.texttechnologylab.org/wp-content/uploads/2015/08/UXeLATE2012-copyright.pdf},
      website        = {http://www.researchgate.net/publication/262319200_WikiNect_towards_a_gestural_writing_system_for_kinetic_museum_wikis},
      year           = 2012
    }

 

Student-Results
  • B. T. H. H. N. R. B. S. N. U. T. W. T. K. L. V. S. A. D. A. C. H. A. K. S. O. S. Asir Abdulsamet; Creech and C. Schöneberger, WikiNect Prototype, Summer Term 2013, 2013.
    [BibTeX]

    @unpublished{prototype2013,
      author={Asir, Abdulsamet; Creech, Brandin; Timo, Homburg; Hoxha, Neritan; Röhrl, Benjamin; Stender, Nicole; Uslu, Tolga; Wiegand, Thomas; Kastrati, Liridon; Valipour, Siavash; Akemlek, Deniz; Auth, Christian; Hemati, Abdul; Korchi, Said; Omari, Süleyman and Schöneberger, Christian},
      title={WikiNect Prototype, Summer Term 2013},
      year={2013},
      keywords={wikinect}
    }
  • T. J. A. K. F. R. R. S. K. Abrami Giuseppe; Göb and S. Voinea, WikiNect Prototype, Winter Term 2014 / 2015, 2014.
    [BibTeX]

    @unpublished{prototype2014,
      author={Abrami, Giuseppe; Göb, Tobas; Jungmann, Arthur; Knöller, Fabian; Rosenstein, René; Sicker, Kai and Voinea, Sebastian},
      title={WikiNect Prototype, Winter Term 2014 / 2015},
      year={2014},
      keywords={wikinect}
    }
  • E. Hanssen and K. Zacharias, WikiNect: Implementing of a Pointing-Gesture-Prototype, Winter Term 2014 / 2015, 2014.
    [BibTeX]

    @unpublished{prototype2014:b,
      author={Hanssen, Eric and Zacharias, Konrad},
      title={WikiNect: Implementing of a Pointing-Gesture-Prototype, Winter Term 2014 / 2015},
      year={2014},
      keywords={wikinect}
    }
  • H. P. P. Engler Lukas; Ghavami and S. Voinea, WikiNect Prototype, Summer Term 2014, 2014.
    [BibTeX]

    @unpublished{prototype2014:a,
      author={Engler, Lukas; Ghavami, Hesamedin; Pech, Patrick and Voinea, Sebastian},
      title={WikiNect Prototype, Summer Term 2014},
      year={2014},
      keywords={wikinect}
    }
  • J. Bassl and A. Tekin, WikiNect Prototype, Winter Term 2013 / 2014, 2014.
    [BibTeX]

    @unpublished{prototype2014:a,
      author={Bassl, Jonas and Tekin, Altug},
      title={WikiNect Prototype, Winter Term 2013 / 2014},
      year={2014},
      keywords={wikinect}
    }
  • M. R. Inceoglu, WikiNect, 2013.
    [BibTeX]

    @unpublished{inceoglu:2013,
      author={Inceoglu, Mehmet R.},
      title={WikiNect},
      year={2013},
      howpublished={Bachelor Thesis},
      keywords={wikinect}
    }
  • S. Voinea, Implementierung des Kinoogle-Gestenalphabetes mit Kinect, 2015.
    [BibTeX]

    @unpublished{voinea:2015,
      author={Voinea, Sebastian},
      title={Implementierung des Kinoogle-Gestenalphabetes mit Kinect},
      year={2015},
      howpublished={Bachelor Thesis},
      keywords={wikinect}
    }

 


[1] [doi] L. von Ahn, “Games with a Purpose,” Computer, vol. 39, iss. 6, pp. 92-94, 2006.
[Bibtex]
@article{Ahn:2006,
 author = {Ahn, Luis von},
 title = {Games with a Purpose},
 journal = {Computer},
 issue_date = {June 2006},
 volume = {39},
 number = {6},
 month = jun,
 year = {2006},
 issn = {0018-9162},
 pages = {92--94},
 numpages = {3},
 url = {http://dx.doi.org/10.1109/MC.2006.196},
 doi = {10.1109/MC.2006.196},
 acmid = {1155342},
 publisher = {IEEE Computer Society Press},
 address = {Los Alamitos, CA, USA},
 keywords = {invisible computing, invisible computing, online games, online games},
}
[2] B. Leuf and W. Cunningham, The Wiki Way: Quick Collaboration on the Web, Boston, MA, USA: Addison-Wesley Longman Publishing Co., Inc., 2001.
[Bibtex]
@book{Leuf:2001,
 author = {Leuf, Bo and Cunningham, Ward},
 title = {The Wiki Way: Quick Collaboration on the Web},
 year = {2001},
 isbn = {0-201-71499-X},
 publisher = {Addison-Wesley Longman Publishing Co., Inc.},
 address = {Boston, MA, USA}
}
[3] A. Thilo, “Ich habe einen Traum: “Die Bildung wird revolutioniert” (\”Uber Jimmy Wales Co-founder of WikiPedia),” Die Zeit, vol. 51, 2008.
[Bibtex]
@article{Thilo2008,
    author  = {Thilo, Andrea},
    title   = {Ich habe einen Traum: “Die Bildung wird revolutioniert” ({\''U}ber Jimmy Wales Co-founder of  WikiPedia)},
    journal = {Die Zeit},
    volume  = {51},
    year    = {2008},
    month={12}
    pages   = {}
 }