Prof. Dr. Walt Detmar Meurers
Creation and Analysis of a Reading Comprehension Exercise Corpus: Towards Evaluating Meaning in Context


Niels Ott, Ramon Ziai, and Detmar Meurers


In Thomas Schmidt and Kai Wörner (eds.) Multilingual Corpora and Multilingual Corpus Analysis. Hamburg Studies in Multilingualism (HSM). Benjamins. 47-69.


We discuss the collection and analysis of a cross-sectional and longitudinal learner corpus consisting of answers to reading comprehension questions written by adult second language learners of German. We motivate the need for such task-based learner corpora and identify the properties which make reading comprehension exercises a particularly interesting task.


In terms of the creation of the corpus, we introduce the web-based WELCOME tool we developed to support the decentralized data collection and annotation of the richly structured corpus in real-life language teaching programs. On the analysis side, we investigate the binary and the complex content-assessment classification scheme used by the annotators and the inter-annotator agreement obtained for the current corpus snapshot, at the halfway point of our four-year effort. We present results showing that for such task-based corpora, meaning assessment can be performed with reasonable agreement and we discuss several sources of disagreement.



Electronically available:

  • Draft: pdf (676.939 bytes)


Note: The electronic versions of the publications linked on this page are the last versions I had the copyright for. Where a publisher copyedited and/or typeset the papers, the electronic copies linked here are NOT identical to the officially published version, which should be used for any quotes, references to page numbers, etc.



Bibtex entry:

  author    = {Ott, Niels   and Ziai, Ramon and Meurers, Detmar},
  title     = {Creation and Analysis of a Reading Comprehension 
               Exercise Corpus: Towards Evaluating Meaning in Context},
  editor    = {Thomas Schmidt and Kai Wörner},
  booktitle = {Multilingual Corpora and Multilingual Corpus Analysis},
  series    = {Hamburg Studies in Multilingualism (HSM)},
  year      = {2012},
  pages     = {47--69},
  address   = {Amsterdam},
  publisher = {Benjamins},
  url       = {}