Assignment 1

Morphological Structure Discovery in a Corpus of the Punjabi Language, Using Undivide++

Salman Mohammad

10630

Corpus taken from: corpora.heliohost.org

Observations

Part A

Corpus Pruning and Segmentation of Words by Hand

The original punjabi language corpus
of size ~200k words was obtained from Heliohost. Due to presence of a lot of stray characters, it had to be pruned using sed/awk and grep to a wordlist
containing ~74k words. Following cleaning operations were done:
Remove entries with alphanumeric characters.
Remove words consisting of two or less characters.
Remove other characters e.g. /,-, ø, : etc.

The corpus was formatted apropriately so as to be accepted by the Undivide++ executable. A hand annotated test set of 475 sufficiently long words, taken from the cleaned corpus, was created employing personal knowledge and understanding of the Punjabi language. It can be found here
.

Part B

Tests and Analysis with different Prameters

The parameters given while running the executable were:
./a.out wordlist.pun 1 1 0 1 0

Varying the runtime arguments given to undivide++ only slightly affected the resultant Precision, Recall and F-score values. Amongst internal parameters, promote_long_segmentation seemed to have the most pronounced effect. Low_frequency_dropouts and corresponding learning parameters ere kept to a minimum (1) as higher values were not giving better results. This could have been because of the small size of the corpus due to which many 'good' words were appearing only once or twice.


Precision, Recall and F-score for this analysis were defined as Precision = (H) / (H+I) Recall = (H) / (H+D) where H = number of correct breaks, I and D denote number of breaks to be inserted and deleted from the output segmentation of the word to achieve correct segmentations.

Undivided++ Parameters Used

The parameters in Undivided++ which gave the maximum f score were:
#define SMALL_ROOT_LENGTH 3
#define LOW_FREQUENCY_DROPOUTS 1
#define LOW_FREQUENCY_DROPOUTS_LEARNING 1
#define SUFFIX_CUTOFF_THRESHOLD 70
#define PREFIX_CUTOFF_THRESHOLD 50
#define COMPOSITE_SUFFIX_THRESHOLD 0.65
#define WRFR_SUFFIX_THRESHOLD 6
#define WRFR_PREFIX_THRESHOLD 1.5
#define SLS_NORMALIZATION_CONSTANT 5
#define ALLOMORPH_REPLACEMENT_THRESHOLD 3
#define ALLOMORPH_DELETION_THRESHOLD 3
#define ALLOMORPH_ADDITION_THRESHOLD 3
#define PROMOTE_LONG_SEGMENTATION 1
#define PROMOTE_LONG_SEGMENTATION_LENGTH 15
#define INDUCE_OUTOFVOCABULARY_ROOTS 1
#define INDUCE_OUTOFVOCABULARY_ROOTS_THRESHOLD 5



Analysis

1. Over-segmentation - Many long words are segmented excessively. Turning on the insert_outofvocabulary_roots (inceased recall) and promote_long_segmentation (decreased precision, more segments) options further encourages this. e.g. [ ਉਘੇੜਿਆ:ਉ+ਘ+ੇ+ੜ+ਿਆ ] , the correct segmentation should be [ ਉਘੇੜਿਆ:ਉਘੇੜ+ਿਆ ].Some english language(transliterated) words were also correctly segmented, because they are used with significant frequency in the Punjabi parlance, others were segemented brutally at every possible position.

2. Under_segmentation And Words of a Different Origin Words like ਕਾਮਯਾਬ => ਕਾਮ+ਯਾਬ have their origin in persian/arabic and due to limited corpus size, not many such words were available to aide training. Therfore a lot of such words remained unsegmented. Setting promote_long_segmentation to zero aggravates this.



3. In morphologically analysing breakup of words, whenever an extra syllable is to be added/removed due to the word-construction paradigms of the Punjabi language, it almost invariably fails. e.g. ਉਚਿਆਇਆ is a word formed from ਉਚਾ+ਿਆ+ਿਆ. Due to punjabi's word formation rules the first 'ਾ' vanishes. But our program has no way to know how to bring back that 'ਾ' into its segmentation [ ਉਚਿਆਇਆ:ਉ+ਚਿ+ਆਇਆ], the correct hand segmentation being [ ਉਚਿਆਇਆ:ਉਚ+ਿਆ+ਇਆ ] or [ ਉਚਿਆਇਆ:ਉਚ+ਾ+ਿਆ+ਇਆ ].



Part C

Code and Observation Data

The data generated for this analysis can be accessed over here in ods format, and code used for pruning/manipulating the files would be uploaded here.