Ling 567 - Knowledge Engineering
Lab 3
Prescott Klassen and Jason Shaw
January 23, 2010

Summary

We had two overarching goals for Lab 3 that tied together the assigned requirements for language features from Lab 2 and Lab 3. Our first goal was to tackle the morphophonological complexity of our language by decomposing our "tokenized" nouns and verbs from Lab 2 into discrete stems and morphemes. This required digging deep into our reference grammar to find keys to regularize stems, prefixes, and suffixes and create matching noun/verb types and inflectional rules in our implementation. It also required us to be realistic about what was possible given time constraints and limit our coverage of language features and complex phenomena. Our second goal was to establish a baseline process for incremental testsuite and grammar refactoring given the tools we have to work with.

Refactoring Nouns and Verbs

In Lab 2, all nouns were typed based on a combined Gender, Person, Number, and Obviation fully inflected surface form and lexical entries entered to match. In our Lab 3 implementation noun types now conform to the 8 classes defined in our reference grammar, and lexical entries are organized by their noun class and recorded in their regularized stem form. Inflection rules are used to match the appropriate affixes of the surface forms in the testsuite based on Gender, Person, Number, and Obviation. Given the time constraints of this week, we decided on partial coverage of the 7 classes defined by Valentine in our reference grammar, implementing 5 of those classes as types divided into animate and inanimate subtypes, as well as inflection rules for singular and plural. 

Verbs have been refactored in a similar way, using classes from our reference grammar for types and inflectional patterns for our lexical rules. Verbs were considerably more complex to tackle than nouns and we chose to focus on solving for the classes in an incremental order:

1. VAI class - Intransitive verb with animate subject
2. VII class - Intransitive verb with inanimate subject
3. VTI class - Transitive verb with inanimate object
4. VTA class - Transitive verb with animate object

The above classes are further complicated by the fact they do not require an explicit subject or object (for transitive classes) and provide subject and object marking through inflection on the verb. We made the decision to limit our grammar and testsuite, where we could, to indicative and independent order forms.

Our work on demonstrative and personal pronouns from Lab 2 is unchanged.
Refactoring our Test Suite and Tools pipeline
In order to leverage our new approach to the morphophonological complexity of Nishnaabemwin in our grammar our test suite we changed its format to:

- standard orthography
- morpheme-segmented (and regularized) representation
- gloss line (one gloss per morpheme)
- free translation

This change required an extensive rewriting of test sentences and was done incrementally based on decisions about the grammar. For example, in many of our sample sentences, 3rd person singular present tense in the standard orthography has no visible inflection for person/number/gender or tense (morphophonological reasons for the deletion of person/number/gender and historical reasons for the leaving off of present tense). We decided to make these inflections explicit in our regularized morpheme delimited test sentences.

Due to the complexity of the verbal inflection pipeline (suffixes and prefixes appear in a regular order on verbs--with optional slots) our development process required that changes in the grammar implementation were tested as they were made. The process was as follows:

1. Make a refactoring decision, discuss and vet with partner
2. Change the grammar in the online questionnaire
3. Download a new version of the grammar, unpack and upload into svn
4. Change test sentences that relate to the grammar change into the new regularized form
5. Check the changed test sentences into svn
6. Load the new grammar into the LKB in emacs
7. test the sentence using c-c-p in the LKB
8. test free-form combinations of prefix/suffix noun/verb agreement by c-c-p in the LKB
9. Debug
10. Go back to step 1

Due the radical changes to the Lab 2 grammar and testsuite, and the inordinate number of combinations of prefix/suffix noun/verb, taking the extra step of using make_item and running the entire testsuite through itsdb would take too much time (and add other confounding factors like errors in the testsuite breaking make_item or the tsdb). Errors were frequent especially when creating the verbal inflection pipeline.


Tense/Aspect

Nishnaabemwin distinguishes three tenses: past, present, and future.  The tense prefixes for past and future are obligatory when a non-present meaning is expressed.  The present tense prefix is omitted sometimes, especially among older speakers, but since we cannot use the customization system to make a single instance of a morpheme type optional, we have left all tense marking obligatory.  This is also a good approach because the younger generation of speakers, to compensate for an overall pattern of syncope in the language, have begun using the present tense prefix in all situations.  Our choice reflects a more up-to-date version of the language.

There are number of other time-related constructions, some of which can be classified as aspect.  Some of the meanings that can be expressed with aspect include habituality, beginning of an action, and end of an action.  There are several other "semi-aspectual" prefixes available, but many of these are more semantically contentful, including "next", "try", and others.  We have chosen to omit these from our selection of aspect markers in the lexicon.
There are also words in Nishnaabemwin which can refer to a recent past.  These are used in ways similar to the English perfect tenses, and as such we have chosen to leave them out of our grammar at this time.

Here is an example sentence in which we previously did not analyze the morphemes in the verb.  Now that we have included tense, we can be less naive:

#2.4 vai
Source: a:548
Vetted: s
Judgment: g
Phenomena: wo
Nenbozh giizegzi
Nenbozh w-gii-zegzi
Nenabush 3-PAST-be.frightened
'Nenabush was frightened'

The prefix gii- is a past tense marker, and immediately precedes the verb.  The w- prefix is explained in the section on agreement.  On other occasions, we have had to add a present tense morpheme in the segmented line because speakers sometimes omit it:

#2.2 vii
Source: a:542
Vetted: s
Judgment: g
Phenomena: wo
Baatiinad wiiyaas
W-doo-baatiina wiiyaas
3-PRES-be.abundant.SG meat
'There is plenty of meat'

If an aspectual marker is present, it intervenes between the tense marker and the verb.  The aspect marker is always immediately adjacent to the verb:

#2.5 vai
Source: a:935
Vetted: s
Judgment: g
Phenomena: wo
Ntaagshkigwaaso nnahaangnikwem
W-doo-ntaa-gshkigwaaso nnahaangnikwem
3-PRES-HAB-be.good.at.sewing my.daughter.in.law
'My daughter-in-law is good at sewing'

The aspect marker "ntaa" in this sentence indicates habituality.  A different ordering is ungrammatical:

#2.5X aspect marker must be adjacent to verb
Source: a:935
Vetted: s
Judgment: u
Phenomena: wo
Ntaagshkigwaaso nnahaangnikwem
W-ntaa-doo-gshkigwaaso nnahaangnikwem
3-HAB-PRES-be.good.at.sewing my.daughter.in.law
'My daughter-in-law is good at sewing'

Tense and aspect information are always the closest prefixes to the verb if present.  Therefore, we created a category of verb inflection slot called verb-aspect, which accepted a verb as input, was optional, and attached before the input.  Each possible aspectual morpheme was listed as a possible morpheme for this slot.  Each morpheme constrained the aspect feature of the verb to be habitual, begin, or end, as appropriate.  Next, we created an inflection slot called verb-tense that took verb-aspect as input, was obligatory, and accepted as morphemes the tense markers gii (past), doo (present), and wii (future).  Each of these constrains the tense feature on the verb to be past, present, or future, as appropriate.

Agreement

Agreement in Nishnaabemwin is extremely complex.  Besides agreement between nouns and any demonstratives which modify them, there is extensive agreement between a verb and its arguments.  The agreement that is required in any given sentence depends on the class of verb that is present.  We will consider each class in turn.

VAI:
These verbs are intransitive with animate subjects.  Nouns in Nishnaabemwin can fall into one of eight categories, and the interaction of several morphological slots determines which category the subject of a VAI verb falls into.  One slot is a prefix which must agree in person with the subject.  One slot is a suffix which indicates whether a 1st/2nd-person subject is plural, or whether a 3rd-person subject is an unspecified actor.  Finally, the last suffix in the construction indicates whether a 3rd-person subject is plural or obviative.  These three slots enforce agreement between the verb and subject in person, number, and obviation.  Gender (animacy) is enforced by default, since all subjects of VAI verbs are animate.

VII:
These verbs are intransitive with inanimate subjects.  Under certain circumstances, the subjects of these verbs can be 3rd-person animate, but generally speaking, there is agreement in both number and obviation (gender agreement is enforced by the verb type, and person agreement is a consequence of inanimate nouns always being 3rd person).  Here the system is simple: four distinct suffixes, one for each combination of singular/plural and proximate/obviative.

VTI:
As with the subjects of VII verbs, the inherent inanimacy of objects of VTI verbs reduces the need for morphological agreement.  As with every other verb class, there are personal prefixes which enforce agreement in person between the subject and verb.  Two distinct morphemes indicate the numbers of the subject and object, and allow for obviation of the subject.

VTA:
These verbs are transitive with animate objects.  This is the most complex kind of agreement in Nishnaabemwin because the verb agrees with both the subject and object in all of the agreement features mentioned above.  The agreement is not straightforward, however -- it can be a bit of a logic puzzle to identify the agreement features of the subject and object using the verb morphology.  It is in this class of verb that the direct-inverse nature of Nishnaabemwin comes into play: the personal prefix, for example, agrees in person not with the subject, but with the higher-ranking verb argument.  The theme suffix indicates whether the actor or goal is higher-ranked, and also depends on the person of the lower-ranked argument.  Additional suffixes refine the agreement in number, gender, and obviation.  Due to the extreme morphological complexity of this verb class, we have demonstrated agreement phenomena in each of the others, and left the implementation of this class of verbs to another day.  More information about this phenomenon is in the section on unimplemented phenomena.

Here is an example of agreement between the subject and the personal prefix:

#2.12 vai
Source: a:613
Vetted: s
Judgment: g
Phenomena: wo, pn
Giin ggiimmaazhiiwii
Giin g-gii-mmaazhiiwii
you 2-PAST-do.poorly
'You did poorly'

The prefix g- indicates a second-person actor for VAI verbs.  The same prefix is used for transitive verbs involving a second-person actor or goal.  For third-person prefixes, we have adopted the convention of the author of our reference grammar in using a prefix w-, called "structural w".  For some verb classes, a prefix was formerly present but has since been syncopated.  Prefixes are not optional -- if a prefix is absent, then the verb is unambiguously in the third person -- so "no prefix" is equivalent to a phonologically empty prefix, and we denote this with w-.  Therefore, in any examples, if a sentence is not overtly prefixed in the orthography, we have added a "w-" to our segmented line to represent this empty prefix.

Here is an ungrammatical example in which the person prefix does not agree with the subject of the sentence:

#2.23 vta
Source: author
Vetted: f
Judgment: u
Phenomena: agr
Giin giwi kwewag nwiignawenmaag
giin giwi kwe-wag n-wii-gnawenim-aa-ag
you that.3.PL woman-PL 1-FUT-take.charge.of-DIR.THM-3.PL.GL
'You will take charge of those women'

A second person is involved, so the prefix should be "g-", not "n-".  Finally, let's look another kind of agreement on the verb.  Here is an example of a transitive verb agreeing with its object in number:

#3.12
Source: a:13.1/648
Vetted: s
Judgment: g
Phenomena: vti, past
Aankonaanyan ngiimiijnan
Aankonaanh-an n-gii-miijid-am-an
cracker-3.INAN.PL 1-PAST-eat-THM1.VTI-3.INAN.PL.GL
"I ate crackers"

Here, the object, crackers, has a plural suffix.  Thus, the verb must agree with it in number, and this is accomplished with the last suffix, "-an", which indicates a third-person plural goal.

Implementing these verb classes in the grammar was a major challenge and the source of most of our troubles in getting test sentences to parse.  Each verb class has a different series of inflections to account for.  Because of the complications associated with the direct-inverse properties of VTA verbs, we limited our implementation this week to VAI, VII, and VTI verbs.  For VAI and VTI verbs, the person is not known ahead of time, so there is a personal prefix slot which attaches to the verb-tense slot and constrains the person of the subject accordingly.  For VTI, another suffix attaches to verb-neg and constrains the number of the actor.  Finally, a final slot attaches at the end constraining the number of the goal.  For VAI, the actor plurality slot is the same, and the final suffix constrains obviation of the actor.  VII verbs are the simplest, with one suffix attaching to verb-neg and constraining the actor in both number and obviation.

Negation

Sentential negation in Nishnaabemwin is represented by a negative adverb, "gaa wii", and by negative inflection "-sii" on the verb.  In intransitive verbs, this negative suffix immediately follows the verb, while in transitive verbs it follows the theme suffix.  The adverb "gaa wii" (sometimes simply "gaa") always precedes the verb, though it is not necessarily adjacent.  Occasionally, in casual speech, the adverb can be omitted, but generally speaking, both elements are required in sentential negation. Due to this complexity we chose to implement only the negative verbal inflection "-sii" in our grammar implementation.  Here is an example of sentential negation:

#3.4
Source: a:16.305/838
Vetted: s
Judgment: g
Phenomena: vai, neg, past
Gaa wii giinaagzisii
Gaawii w-gii-naagzi-sii
not 3-PAST-be.visible-NEG
"He was not visible"

Note that this would be ungrammatical if only the negative adverb "gaa wii" were used:

#3.24a Verb must show negative inflection
Source: author (cf a:16.305/838)
Vetted: s
Judgment: u
Phenomena: vai, neg, past
Gaa wii giinaagzi
Gaawii w-gii-naagzi
not 3-PAST-be.visible
"He was not visible"

Since negation appears either directly after the verb, or directly after a theme suffix if one is present, we established a slot called verb-neg that optionally placed the negative inflection morpheme after these two slots.

Argument Optionality

Due to its extremely rich verbal inflectional morphology, Nishnaabemwin has pervasive argument dropping.  Personal pronouns are rarely used except for emphasis, because all agreement information can be obtained from the verb, uniquely specifying whatever personal pronoun may have been used.  Both subjects and objects can be dropped in all contexts, and sentences -- even those involving transitive verbs -- are frequently composed of a single word:

#3.3
Source: a:12.346/629
Vetted: s
Judgment: g
Phenomena: vta, past
Ngiimbwaachhig
N-gii-mbwaachh-igw
1-PAST-visit-INV.THM
"He visited me"

Argument dropping is always allowed.  This was represented by answer appropriately in the questionnaire: subject dropping can occur with any verb and object dropping is always allowed.  In such situations, agreement morphology on the verb is not lost.  We take this to be subject/object marking, and therefore defined subject and object marking to be required in all situations, regardless of any argument dropping.

Demonstratives/Definiteness

Nishnaabemwin does not have definite and indefinite articles, as English does.  Instead, a variety of devices are used to communicate the definiteness of discourse referents.  The clearest way of marking indefiniteness is to use the word "bezhig", which means 'one'.  A phrase of the form bezhig + N serves to introduce the noun into the discourse.  Nouns may also be introduced by being used simply as bare nouns, without any sort of numerical or pronominal modification.  Somewhat strangely, mass nouns may be introduced as indefinites with demonstrative pronouns.  The most common way to identify a noun as definite in Nishnaabemwin is to use a demonstrative pronoun.  Demonstratives can even be used with proper names.  Demonstratives must agree in number, gender, and obviation with the nouns they modify.  Once a noun has been introduced into the discourse, it may be referred to simply by a bare demonstrative, or it may be dropped altogether and understood from the verbal morphology.

Here is an example of indefiniteness:

#2.17 vai
Source: author
Vetted: f
Judgment: g
Phenomena: wo, pn, indef
Giimaajaad bezhig nishnaabe
W-gii-maajaad bezhig nishnaabe
3-PAST-leave one Indian
'An Indian left'

In this example a demonstrative pronoun "maaba" is used to mark definiteness of the subject:

#2.9 vai
Source: author
Vetted: f
Judgment: g
Phenomena: wo, pn
Maaba nishnaabe moonshkwed
Maaba nishnaabe w-doo-moonshkwed
this.3.SG Indian 3-PRES-weed
'This Indian is weeding'

Our analysis of determiners and definiteness has not changed from last week, since they were accounted for then as part of "the rest of the NP".  There are eight determiner types defined in the lexicon, each varying in person, number, gender, and obviation.  Their presence in the grammar and their position relative to the known are encoded in the Word Order section -- our language does have determiners and the order is Det-Noun.  While each of the determiner types in the lexicon are really demonstrative pronouns, Nishnaabemwin does not have articles analogous to English "a" or "the", and the demonstrative pronouns are used in many of the same capacities.  Very frequently they are simply markers of definiteness.

Unimplemented Phenomena

As mentioned previously, we have not implemented the morphology for VTA verbs.  There are several inflectional slots for this class, and each one only provides partial constraints on multiple features.  For example, the personal prefix, which is straightforward in other verbs, only indicates that at least one of the arguments has a particular value for person.  A prefix "g-" means that either the subject is second person, or the object is second person.  Which is the case depends on the value of suffixes on the verb.  When only first and second persons are involved, the theme suffix unambiguously specifies the person of both subject and object, but if a third person is involved, only the relative position of subject and object on the direct-inverse hierarchy is specified.  If only third persons are involved, this can help resolve the feaures of gender and obviation, but if a first or second person is involved, no such information is included.  Due to these complications and more, we elected to leave this class of verbs unimplemented for this week.  There are many VTA verbs in our test suite, mostly because they are the most common class of verbs.  But sentences with these verbs in them will not parse.

Test Suite Results

We had 63 test sentences in our test suite. After running the sentences through [incr(tsdb)] we obtained the following results:


14 of 63 parsed
41 parsing errors were recorded (the majority of these were due to unrecognized tokens due to our incomplete implementation of VTA verbs in our grammar)

The Coverage Report was as follows:

Total items: 63
Positive items: 37
word string: 2.86
lexical items: 5.85
distinct analyses: 1.00
total results: 10
ovreall coverage: 27.0 %

The Overgeneration Report was as follows:

Total items: 63
negative items: 26
word string: 2.62
lexical items: 5.22
distinct analyses: 1.00
total results: 4
overall coverage: 15.4 %




