CORE B05

Modelling the Information Landscape (IL) for Assessing and Analyzing Domain-Specific and Generic Critical Online Reasoning

Project B05 investigates how linguistic features function as cues in the online information landscape (IL) and how they relate to students’ performance in critical online reasoning (COR) tasks. Although previous research has shown that linguistic cues influence text readability, source credibility, and performance in domain-specific knowledge tests in offline contexts, their applicability to online environments remains insufficiently examined. B05 addresses this gap by modeling the linguistic characteristics of texts that students engage with while solving COR tasks.

The main objective is to develop a theoretically grounded model of linguistic features that predicts COR processes and performance. The study analyzes differences in linguistic predictors between generic and domain-specific COR tasks across four domains (economics, medicine, social sciences, and physics) and across three cognitive facets of COR: online information acquisition, critical information evaluation, and reasoning through evidence, argumentation, and synthesis. It further examines the levels at which these features operate, ranging from individual texts to domains, genres, and the IL as a whole.

Methodologically, B05 integrates qualitative and quantitative approaches. Linguistic features related to evidentiality, information sources, and text organization are first identified qualitatively, then operationalized quantitatively, expanded using machine learning, and evaluated for predictive validity. This integration follows a computational hermeneutic approach in which quantitative modeling is grounded in and interpretable through prior linguistic analysis.

The project yields machine learning models that enable automated analysis of fine-grained linguistic features across multiple texts within the IL. Within the CORE research unit, B05 contributes detailed linguistic data that complement analyses of text, performance, media and content characteristics, narrative structures, and multimodal data in related projects.

Team TTLab

Team JGU

  • Principal Investigator: Prof. Dr. Walter Bisang
  • Patryk Czerwinski

Publications

Manuel Schaaf, Kevin Bönisch and Alexander Mehler. May, 2026. GhostWriter: Hidden AI-Generated Texts over Multiple Languages, Domains and Generators. Proceedings of the Fifteenth Language Resources and Evaluation Conference (LREC 2026), 10497–10516.
BibTeX
@inproceedings{Schaaf:et:al:2026,
  title     = {GhostWriter: Hidden AI-Generated Texts over Multiple Languages,
               Domains and Generators},
  author    = {Schaaf, Manuel and Bönisch, Kevin and Mehler, Alexander},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation
               Conference (LREC 2026)},
  month     = {May},
  year      = {2026},
  pages     = {10497--10516},
  keywords  = {Corpus, Natural Language Generation; Validation of LRs, AI-generated Text Detection, core, core_b05},
  address   = {Palma, Mallorca, Spain},
  publisher = {European Language Resources Association (ELRA)},
  editor    = {Piperidis, Stelios and Bel, Núria and van den Heuvel, Henk and Ide, Nancy
               and Krek, Simon and Toral, Antonio},
  doi       = {10.63317/57fd7juh5zek},
  abstract  = {The advent of Transformer-based Large Language Models (LLMs) has
               led to an unprecedented surge of AI-generated text (AIGT) across
               online platforms and academic domains. While these models exhibit
               near-human fluency and stylistic coherence, their widespread adoption
               has raised concerns about authorship integrity, research quality,
               and the recursive contamination of training corpora with synthetic
               data. These developments underscore the need for reliable AIGT
               detection methods and benchmark datasets, particularly for malicious
               or deceptive *ghostwriting* scenarios where AIGT is intentionally
               crafted to evade detection. To address this, we present **GhostWriter**,
               a large-scale, bilingual (German and English), multi-generator,
               and multi-domain dataset for AIGT detection. The dataset comprises
               human- and AI-authored texts produced under domain-specific *ghostwriting*
               conditions, including examples intentionally embedded within otherwise
               human-written texts to obscure their AI origin. With **GhostWriter**,
               we (i) aim to expand the resources available for German AIGT datasets,
               (ii) emphasize mixed or fused synthesizations—since most existing
               corpora are limited to the document level—and (iii) introduce
               specifically crafted malicious ghostwriting scenarios across multiple
               domains and generators.}
}
Walter Bisang and Alexander Mehler. 2026. Linguistic Features as Predictors of Students' Performance in Domain-Specific Critical Online Reasoning Tasks. International Test Commission Conference (ITC) 2026. accepted.
BibTeX
@inproceedings{Bisang:Mehler:2026,
  title     = {Linguistic Features as Predictors of Students' Performance in
               Domain-Specific Critical Online Reasoning Tasks},
  author    = {Bisang, Walter and Mehler, Alexander},
  booktitle = {International Test Commission Conference (ITC) 2026},
  eventdate = {2026-06-30/2026-07-03},
  location  = {Auckland, New Zealand},
  note      = {accepted},
  year      = {2026},
  keywords  = {core,core_b05}
}
Cedric Borkowski, Giuseppe Abrami, Dawit Terefe, Daniel Baumartz and Alexander Mehler. 2026. DUUIgateway: A Web Service for Platform-independent, Ubiquitous Big Data NLP. SoftwareX, 34:102549.
BibTeX
@article{Borkowski:et:al:2026,
  title     = {{DUUIgateway}: A Web Service for Platform-independent, Ubiquitous Big Data NLP},
  journal   = {SoftwareX},
  volume    = {34},
  pages     = {102549},
  year      = {2026},
  issn      = {2352-7110},
  doi       = {https://doi.org/10.1016/j.softx.2026.102549},
  url       = {https://www.sciencedirect.com/science/article/pii/S2352711026000439},
  author    = {Borkowski, Cedric and Abrami, Giuseppe and Terefe, Dawit and Baumartz, Daniel
               and Mehler, Alexander},
  keywords  = {duui, neglab, core, core_b05, core_c08, new-data-spaces, circlet},
  abstract  = {Distributed processing of unstructured text data is a challenge
               in the rapidly changing and evolving natural language processing
               (NLP) landscape. This landscape is characterized by heterogeneous
               systems, models, and formats, and especially by the increasing
               influence of AI systems. While many of these systems handle text
               data, there are also unified systems that process multiple input
               and output formats, while allowing for distributed corpus processing.
               However, there are hardly any user-friendly interfaces that allow
               existing NLP frameworks to be used flexibly and extended in a
               user-controlled manner. Due to this gap and the increasing importance
               of NLP for various scientific disciplines, there has been a demand
               for a web and API based flexible software solution for deploying,
               managing and monitoring NLP systems. Such a solution is provided
               by Docker Unified UIMA-gateway. We introduce DUUIgateway and evaluate
               its API and user-driven approach to encapsulation. We also describe
               how these features improve the usability and accessibility of
               the NLP framework DUUI. We illustrate DUUIgateway in the field
               of process modeling in higher education and show how it closes
               the latter gap in NLP by making a variety of systems for processing
               text and multimodal data accessible to non-experts.}
}
Alexander Mehler, Walter Bisang, Maxim Konca, Patryik Czerwinski, Jeremias Josef Graf and Jana Fritsch. 2026. Linguistic Features of Student Responses as Indicators of Performance in Critical Online Reasoning Tasks. Zeitschrift für Erziehungswissenschaft.
BibTeX
@article{Mehler:et:al:2026:a,
  title     = {Linguistic Features of Student Responses as Indicators of Performance
               in Critical Online Reasoning Tasks},
  author    = {Alexander Mehler and Walter Bisang and Maxim Konca and Patryik Czerwinski
               and Jeremias Josef Graf and Jana Fritsch},
  journal   = {Zeitschrift für Erziehungswissenschaft},
  issn      = {1862-5215},
  url       = {http://dx.doi.org/10.1007/s11618-026-01388-6},
  doi       = {10.1007/s11618-026-01388-6},
  year      = {2026},
  publisher = {Springer Science and Business Media LLC},
  keywords  = {core,core_b05}
}
Daniel Baumartz, Maxim Konca, Alexander Mehler, Patrick Schrottenbacher and Dominik Braunheim. 2024. Measuring Group Creativity of Dialogic Interaction Systems by Means of Remote Entailment Analysis. Proceedings of the 35th ACM Conference on Hypertext and Social Media, 153––166.
BibTeX
@inproceedings{Baumartz:et:al:2024,
  author    = {Baumartz, Daniel and Konca, Maxim and Mehler, Alexander and Schrottenbacher, Patrick
               and Braunheim, Dominik},
  title     = {Measuring Group Creativity of Dialogic Interaction Systems by
               Means of Remote Entailment Analysis},
  year      = {2024},
  isbn      = {9798400705953},
  publisher = {Association for Computing Machinery},
  address   = {New York, NY, USA},
  url       = {https://doi.org/10.1145/3648188.3675140},
  doi       = {10.1145/3648188.3675140},
  abstract  = {We present a procedure for assessing group creativity that allows
               us to compare the contributions of human interlocutors and chatbots
               based on generative AI such as ChatGPT. We focus on everyday creativity
               in terms of dialogic communication and test four hypotheses about
               the difference between human and artificial communication. Our
               procedure is based on a test that requires interlocutors to cooperatively
               interpret a sequence of sentences for which we control for coherence
               gaps with reference to the notion of entailment. Using NLP methods,
               we automatically evaluate the spoken or written contributions
               of interlocutors (human or otherwise). The paper develops a routine
               for automatic transcription based on Whisper, for sampling texts
               based on their entailment relations, for analyzing dialogic contributions
               along their semantic embeddings, and for classifying interlocutors
               and interaction systems based on them. In this way, we highlight
               differences between human and artificial conversations under conditions
               that approximate free dialogic communication. We show that despite
               their obvious classificatory differences, it is difficult to see
               clear differences even in the domain of dialogic communication
               given the current instruments of NLP.},
  booktitle = {Proceedings of the 35th ACM Conference on Hypertext and Social Media},
  pages     = {153–-166},
  numpages  = {14},
  keywords  = {Creative AI, Creativity, Generative AI, Hermeneutics, NLP, core, core_b05, core_c08},
  location  = {Poznan, Poland},
  series    = {HT '24}
}