The ELWIS project: General Information
Project Objectives and Issues
ELWIS is an acronym for German Korpusgestützte
Entwicklung lexikalischer Wissensbasen (= corpus-based
development of lexical knowledge bases).
The project is funded by the Ministry of Science and Research
has been in existence since the beginning of 1992 at the Department of
Linguistics at the University of Tütbingen, Germany.
Research in the ELWIS framework deals with crucial issues
related to a theoretically sound description of lexical units
for NLP systems using corpus-based methods and tools.
We distinguish two interrelated areas:
Both areas were (and still are) investigated in several national and
international projects, but German has only rarely been the object
of such studies. The ELWIS project not only evaluates possibilities of
transfer methods developed for other languages to German, but also
develops and tests new methods and approaches for description.
- Exploitation of existing lexical knowledge
sources (text corpora, machine-readable dictionaries, computer
lexicons) for efficient construction of lexical knowledge bases.
- adequate, theoretically sound descriptions of
lexical knowledge for different types of applications in computational
The linguistic phenomena under investigation are inflectional and
derivational morphology, word formation, syntactic and semantic
valency, collocations and morphosyntactic properties of idioms..
The ELWIS Scenario
The project's different activities yield a prototypical instance of
the reusability scenario shown in figure 1.
Figure 1: ELWIS scenario
The lexical database is the central part of the scenario: on
the one hand it is the target component for the research experiments
carried out in the field of lexical acquisition; on the other hand the
lexical knowledge stored in the database can be automatically compiled
into lexical knowledge bases with alternative knowledge representation
formalisms. The lexical database in itself is realised by the use of
a relational database management system.
Specific methods and tools are used to extract lexical information
from machine readable resources such as machine readable
dictionaries, existing computer lexicons and machine
readable text corpora and to store the information in the
database. In most cases these resources need additional treatment:
German machine readable dictionaries are available as typesetting
files only and have to be converted into a more appropriate format.
Quality and quantity of information obtainable through automatic
analysis of text corpora is improved considerably if the texts are
linguistically annotated (POS, syntactic and semantic tagging). This
requires the availability of annotation tools for the language under
inspection; the tools in turn can make use of the lexical information
stored in the database.
The information stored in the lexical database will finally be
integrated in lexical knowledge bases by use of compiler
programmes. This two-stage approach was chosen because it has been
shown that direct construction of intelligent knowledge bases
from text corpora and dictionary resources is not feasible. While
these types of systems (i.e., DATR, TFS) were never really tested for
large amounts of data, relational database management systems have
been sufficiently proven to safely administer large amounts of data.
In addition this two-tiered architecture allows to test the
theoretical generalisations expressed in the knowledge bases against
large amounts of data for adequacy of description. This allows
efficient testing of competing theories within a single formalism or
across different ones.
Results and work in progress
Collection of lexical and linguistic resources
One major part of the project is the accumulation of lexical and
linguistic resources for German. The following resources are
- Text corpora
- in machine readable form were initally not general available in
sufficient quantities. In addition to various texts which has
assembled from a variety of sources (e. g.: Mannheimer Korpus I
from the Institut für deutsche Sprache at Mannheim, Literary texts
from the Oxford Text Archive, one year of the newspapers
Frankfurter Rundschau and Donaukurier from the European
Data Initiative), all messages from all German news groups accessible
in the Internet are archived within the project. This type of text is
not only suitable for the construction of large text corpora due to
its very machine readable nature, moreover, the richness of topics
covered by the texts, and the fact that most of the texts are unedited
written language is highly unusual; this type of language is
frequently used and rarely studied. Moreover, the texts are highly
idiosyncratic with respect to mark up and orthography and are thus
challenging automatic procedures of text preparation and
- Machine readable dictionaries,
- i.e., dictionaries made for humans available in machine readable
format. The Duden editorial provided us with typesetting files
containing parts of the Duden Stilwöterbuch and the
Other sources available within the project are machine readable
versions of Gerhard Augst's Morpheminventar A-Z and Verben
in Feldern. Valenzwörterbuch zur Syntax und Semantik deutscher
- Computer lexicons,
- i.e., description of lexical units stored in computer files or
databases which are in most cases designed as lexical components of
NLP systems. ELWIS investigates possibilities and limitations of the
exploitation of two existing computer lexicons for German, namely the
Saarbrücker deutsches Analysewörterbuch (SADAW) and the
German part of the lexical data from the Centre for Lexical
Another major work package within the project is the acquisition and
adaption of available software tools, as well as the development of
own software tools required for ELWIS's different fields of study:
Apart from the above mentioned tools developed within the project,
ELWIS has assembled a large variety of tools for NLP processing
from external sources.
- is a robust probabilistic part-of-speech tagger for German. It has
been developed by combining well-tested techniques for English with
training data for German which was made available by the Institut
für deutsche Sprache. The output of the tagger, POS-tagged
German texts, will eventually be used as base for the development of
lemmatization and noun phrase extraction tools for huge amounts of
- is a flexible dictionary entry parser which was developed for the
conversion of dictionaries stored on typesetting files. The parser
consults a user-supplied dictionary grammar specifying the
hierarchical structure of the dictionary entries by means of
context-free rewrite rules. The grammar formalism provides schemes for
expressing (multiple) optional rules, sets of terminals, and different
homonym and sense counters. Switches and configuration options allow
the adaption to user-specific needs, e.g. the display of the generated
parse trees is user-configurable and supports SGML and LaTeX output,
as well as hierarchical attribute-value structures. Lexparse was used
for the parsing of the Duden dictionaries and is currently used
for the parsing of other dictionaries.
- INserter SYSTem, is a unique system for automatic insertion
(i.e., classification) of lexical items under appropriate nodes in
hierarchies as used in modern formalisms for lexical information such
the DATR formalism. It allows efficient (a) testing of generalizations
when designing a lexical hierarchy; (b) transferring large numbers of
lexical items from flat data structures to a finished lexical
hierarchy when using it to build a large lexicon. INSYST was developed
and used to test competing DATR-theories for German noun inflection
within the ELWIS project.
- facilitates the extraction of collocations from texts by means of
statistical calculations: frequencies of clusters of words are
compared with frequencies of individual words in the cluster, pairs of
such frequencies having a high relation are said to have high
mutual information-values and the pertaining word clusters are
automatically selected as candidates for collocations.
- is a LEXical EXPANDer programme that generates full forms from
the project's lemmatized core database. The database contains a
complete description of the morphological relevant features for each
lemma. Lexpand takes a lemma and its features as its input and
generates all inflected forms for the lemma. Underlying the programme
is a thorough rule system of inflectional morphology for German.
Research is concerned with the following issues:
- Lexical database design:
- For the development of the lexical database, a conceptual schema
(i.e., an entity-relationship-model) was developed by the use of
appropriate methods of semantic data modeling. This conceptual schema
was then transferred into a schema of relations and implemented. It
contains a theoretically sound description of the inflectional
morphology and valency patterns for German, and can easily be adapted
for descriptions of lexical semantics.
- Methodology for the development of dictionary grammars:
- based on meta-lexicographical findings on the structure of
dictionaries, a method was explicated for the development of grammars
for the dictionary entry parser Lexparse. The method was employed for
the development of dictionary grammars for the two Duden
- Tag Sets for German Corpus Tagging:
- for the part-of-speech tagger developed in the project a revised
set of tags of 42 syntactic categories for German was developed. This
tag set is based on an earlier set developed in the 1970s at the
University of Saarbrücken and was revised on a strictly
distributional basis. The set will serve as a starting point for noun
phrase extraction and other syntactic tools.
- Methods for the automatic extraction of idioms and collocations:
- automatic extraction of idioms and collocations was the target of
a series of experiments: it was shown that corpus-based statistical
methods which have been tested for English can not simply be applied
to German data. A number of additional problems must be solved first.
A combination of corpus- and dictionary-based methods seems to be more
appropriate for the extraction of this type of lexical entries.