#Arpit Jain
#10145
1.The folder contains the folder FinalDistribution,file segregate.py and file final.py.
2.Run segregate.py as python3 segregate.py
We obtain the file output.txt containing the words extracted from the corpus.
3.Move this file onto the directory FinalDistribution.
Run the code for UnsupervisedWordSegmentation.cpp with the following parameters:
#define SMALL_ROOT_LENGTH 3
#define LOW_FREQUENCY_DROPOUTS 1
#define LOW_FREQUENCY_DROPOUTS_LEARNING 1
#define SUFFIX_CUTOFF_THRESHOLD 70	//CRITICAL	
#define PREFIX_CUTOFF_THRESHOLD 50	//CRITICAL
#define COMPOSITE_SUFFIX_THRESHOLD 0.65
#define WRFR_SUFFIX_THRESHOLD 10
#define WRFR_PREFIX_THRESHOLD 1.5
#define SLS_NORMALIZATION_CONSTANT 5
#define ALLOMORPH_REPLACEMENT_THRESHOLD 3
#define ALLOMORPH_DELETION_THRESHOLD 3	
#define ALLOMORPH_ADDITION_THRESHOLD 3 
#define PROMOTE_LONG_SEGMENTATION 0
#define PROMOTE_LONG_SEGMENTATION_LENGTH 15
#define INDUCE_OUTOFVOCABULARY_ROOTS 0
#define INDUCE_OUTOFVOCABULARY_ROOTS_THRESHOLD 5

We remove the words with frequency less than 3.The threshold have been shifted too as per the language corpus requirements.Long segmentation is set to 0 to improve precision.
The parameters passed are 1 1 0 1 0.
3.Type this onto the terminal to come back to the original directory:    cd ..
4.Type in python3 final.py to check for the words common to the set assigned to us(handsegmented corpus).This is saved onto a new file called new.txt . This file is then used to compare the morphology between the segmented dataset and handsegmented dataset.
Precision and recall values are calculated accordingly.F-score is also calculated using the values of precision and recall.
