Language Resource Management -- Morpho-syntactic Annotation Framework (MAF)

11.3.1 Structural ambiguities over word formsISO: Structural ambiguities over word forms¶

A general and very generic answer is to describe the possible readings as paths through an Directed Acyclic Graph (DAG) whose edges are labeled by a word form. Such DAGs forms a sub-part of Finite State Automata and also cover the notion of word lattice used in parsing and speech recognition communities. They are powerful enough to represent ambiguities between several decompositions into compound forms. They can also be used to denote simpler cases of lexical ambiguities.

For instance, the French textual sequence “fer à cheval” (horse shoe) can still be decomposed into several readings (“'’, “[iron] [on horse]”, “'’), giving the following DAG:

Figure 3

Figure 3. DAG for “fer à cheval”

The linguistic units “fer à cheval”, “fer”, “à”, “cheval”, and “à cheval” correspond to minimal syntagmatic units that can be annotated.

Additional information could be added to edges such as probabilities.

11.3.2 Structural ambiguities over tokensISO: Structural ambiguities over tokens¶

Structural ambiguities may also arise over sequences of tokens, resulting from ambiguities in the tokenization of the annotated document, e.g. speech documents.

Structural ambiguities over tokens are represented by transitions labeled by tokens. The attributes tinit and tfinal on elements fsm are used to state the initial and final states for the token paths.

The two levels of structural ambiguities are represented by two lattices that form a kind of chart. It is not mandatory but advised that the two lattices share their states, whenever possible.

A validity condition has to be expressed between the two levels of structural ambiguity:

the tokens covered by word forms along a word form path belong to some token path.

11.4.1 Non ambiguous linear representationISO: Non ambiguous linear representation¶

When there is no ambiguity, MAF allows to replace the global lattice notation by a much simpler linear notation where the token, wordForm and wfAlt elements are implicitly chained following their appearance order, as illustrated by the following example:

11.4.2 Mixed linear and lattice representationISO: Mixed linear and lattice representation¶

Ambiguities are generally localized and it is tempting to also localize the use of the lattice notation only where it is needed. MAF allows to insert local lattice fsm in a linear flow of token, wordForm and wfAlt elements.

11.5.1 Separating tokens and word formsISO: Separating tokens and word forms¶

All tokens embedded within a word form may be extracted and moved just before the word form (and before an enclosing wfAlt) , not changing the relative order between tokens.

becomes

Note: There is no clear semantic to handle tokens embedded in word forms, themselves embedded in transitions. This case should be avoided.

11.5.2 Wrapping into local latticesISO: Wrapping into local lattices¶

Tokens and word forms outside transitions are embedded into local lattices, wfAlt elements being considered as word forms.

becomes

Lattice states are local to each lattice.

11.5.3 Merging local latticesISO: Merging local lattices¶

Two adjacent lattices may be merged by renaming the intermediary states in order to avoid name clashes and in such a way that the word form (resp. token) final state of the first lattice equals the word form (resp. token) initial state of the second lattice. Whenever possible, it is recommended, when merging, to rename the lattice states in such a way that the final (resp. final) states for tokens and word form coincide.

The previous example becomes:

and then

11.5.4 Removing wfAltISO: Removing wfAlt¶

A transition over a lexical ambiguity, materialized by a wfAlt element, may be expanded into two equivalent simpler transitions.

The ordering of transitions inside lattices is not pertinent. On the other hand, the ordering of word forms and tokens outside lattices is pertinent. The relative ordering of local lattices is also pertinent.