Linguistic Modeling and its Interfaces
Oberseminar, Detmar Meurers, Summer Semester 2015
This series features presentations and discussions of current issues in linguistic modeling and its interfaces. This includes linguistic modeling in computational linguistics, language acquisition research, Intelligent Computer-Assisted Language Learning, and education, as well as theoretical linguistic research with a focus on the interfaces of syntax and information structure. It is open to anyone interested in this interdisciplinary enterprise.
Development of a cross-platform serious game for children with dyslexia.
Abstract: One of the major causes of dyslexia, of which about 4 – 8 % of the population are
affected, is a deficient phonological awareness - the ability to deal with the sound system of a
language and to detect, distinguish and manipulate segments of a language, like syllables, rimes, or
even single sounds. Despite research results that a shortcoming in syllable stress detection in the
context of words or sentences is a very strong predictor of dyslexia, currently no mobile serious
games are known of that explicitly focus on the improvement of this deficit. In this work, I present
the iterative and user-centered development of the prototype of a mobile serious game for iOS und
Android. The mobile serious game is designed to support primary school-aged children to
improve their phonological awareness outside the classroom or learning therapy. It
represents an absolute novelty in the area of mobile serious games for children with
dyslexia as its focus is on the stress of single syllables. By integrating an intelligent
tutoring system, which is based on principles of the cognitive architecture ACT-R, the
mobile serious game can adapt to the performance of a child dynamically and offers the
possibility to optimize the learning curve and simultaneously maintain motivation and
fun.
A Readable Read: Automatic Assessment of Language Learning Materials based on Linguistic Complexity
Abstract: Corpora and web texts can become a rich language learning resource if we have a means of assessing whether they are linguistically appropriate for learners at a given proficiency level. In this paper, we aim at addressing this issue by presenting the first approach for predicting linguistic complexity for Swedish second language learning material on a 5-point scale. After showing that the traditional Swedish readability measure, Läsbarhetsindex (LIX), is not suitable for this task, we propose a supervised machine learning model, based on a range of linguistic fea- tures, that can reliably classify texts according to their difficulty level. Our model obtained an accuracy of 81.3% and an F-score of 0.8, which is comparable to the state of the art in English and is considerably higher than previously reported results for other languages. We further studied the utility of our features with single sentences instead of full texts since sentences are a common linguistic unit in language learning exercises. We trained a separate model on sentence-level data with five classes, which yielded 63.4% accuracy. Although this is lower than the document level performance, we achieved an adjacent accuracy of 92%. Furthermore, we found that using a combination of different features, compared to using lexical features alone, resulted in 7% improvement in classification accu- racy at the sentence level, whereas at the document level, lexical features were more dominant. Our models are intended for use in a freely acces- sible web-based language learning platform for the automatic generation of exercises, and they will be available also in the form of web-services.
Personalized, Adaptive Learning based on Cognitive Models Increases Learning Efficiency
Abstract: It is well known that the schedule of practice partly determines the efficiency of learning sessions, and thus the retention of the learned materials. Even the classical Leitner method for learning factual information is based on this principle, as better encoded items are practiced less often. By calculating the optimal distance between repetitions of a to be learned item, large learning gains can be obtained. Most methods that aim for optimizing the schedule of practice adapt the schedule based on whether a learner provides a correct, or an incorrect answer. However, even when an item is correctly answered, the speed by which an answer is given could be used to assess how well an item is encoded in memory. In this talk, I will present an adaptive learning system that is based on computational cognitive models of the human long-term memory system. This system keeps track of the internal activation of each to-be-learned item, and updates the internal activation after each presentation. Based on this activation value, the system determines which item needs to be practiced at what point in time, or whether the learner is ready for the presentation of new items. We have tested this system in multiple experiments demonstrating typical learning gains of 10%, and it is now used by a large Dutch publishing house in their online systems associated with all their secondary education learning materials. I will also discuss recent work with this system that suggests that the internal parameters of the system are better predictors of how well an item is mastered than the score on a test, that these parameters are stable over time and relatively stable over materials, and how these parameters correlate with other, more traditional measures of learning aptitude.
References:
Automatic Identification of Translationese: Highlighting the nature of translation
Abstract: Translated texts differ from texts originally written in the same (target) language. Several Translation Studies hypotheses aim at explaining these differences. We use computational methodology, specifically supervised and unsupervised classification, to distinguish between translated and original texts. This facilitates a close inspection of the specific features along which the two types of texts differ.
This enterprise yields several findings:
We show that the import of these results is not only theoretical; they have implications for natural
language processing applications, in particular statistical machine translation.
On the heterogeneity of projective content
Abstract: Tonhauser et al (2013) motivate, based on data from English and Paraguayan Guaraní, that projective content is heterogeneous: this work distinguished four class es of projective content based on two properties, namely whether the content imposes a strong constraint on prior context and whether the content shows local effect. Our goal in this presentation is to provide further empirical evidence for the heterogene ity of projective content. We present preliminary results from experiments that compa re the degree of projectivity for a wide range of types of projective content, includ ing the prejacent of “only”, the pre-state of “stop”, contents of the complements of a range of “factive” verbs, and conventional implicatures. We show that the degree of projectivity of a projective content is correlated with the degree of not-at-issuene ss of that content: the more the content is not-at-issue, the more projective it is.
Reference:
Matching Text Features and Reader Skills via Conjoint IRT
Abstract: Computational linguistics and psychometrics choose different pathways to
explain text difficulty: While the first discipline focuses on factors at the text level
(e.g., surface features like sentence length, …), psychology emphasizes the individual
determinants of reading comprehension (e.g., vocabulary, reading fluency, …). Both
aspects are necessary to explain and forecast the performance of groups and individuals.
Item-Repsonse-Models (IRT) have the potential to better interlink both theoretical
backgrounds within a common approach, by representing text difficulty and reader aptitude on
the same scale and modelling the probability of success with logistic regressions. This
talk shows an advanced IRT methodology -– so called Conjoint IRT (Klein Entink,
2009) – that can incorporate both, accuracy and reading speed. It further shows how to
match reader competencies, text difficulty and time consume, as well as the modelling of
these parameters with finite Taylor polynomials on the basis of surface features of the
texts.
Measuring Linguistic Complexity for Adaptive Generation
Abstract: In this talk I will share the results of some early experiments on ranking sentences by
their difficulty using psycholinguistic metrics which correlate with reading times. Preliminary
results indicate that measures like surprisal (Hale 2001; Levy 2008) and dependency length
(Gibson 2000; Gildea & Temperley 2007) do improve model accuracy beyond simple–but
frustratingly good–baseline systems. I will also discuss ongoing work which uses better feature
extraction and corpora (and compares to other systems like, e.g., Vajjala & Meurers
2014) as well as the relation of this work to the new project “Adapting Information
Density to Changing Situations and Individual Users” at Universität des Saarlandes (SFB
1102).
Phonetic representations for short-answer scoring in language assessment
Abstract: Content-oriented scoring of short responses is a meaning focused task in which, for the most part, evaluation should not be affected by low-level input flaws. Responses which scoring systems process are, however, often ill-formed. This is especially true in the context of computer-based foreign language assessment in which form errors, such as misspellings or ungrammaticality, are likely to occur in productions of learners at low proficiency levels.
In this presentation I’ll talk about ongoing work on addressing misspellings in computer-based
evaluation of learners’ responses to listening comprehension items in DaF placement tests. Based
on a corpus of responses by test takers of different language proficiencies, I’ll illustrate the extent of
the misspellings problem and present an evaluation of an off-the-shelf spell-checking tool. As a way
of addressing misspellings, I’ll introduce work in progress on scoring based on phonetic
representations - specifically, encodings borrowed from historical linguistics - and discuss prospects
for scoring based on string similarity; here, computed using phonetically transcribed
strings.
Pilán, I., S. Vajjala & E. Volodina (2015). A Readable Read: Automatic Assessment of Language Learning Materials based on Linguistic Complexity. In Proceedings of CICLING 2015- Research in Computing Science Journal Issue (to appear).
_________________________________________________________________________________
Last updated: July 14, 2015