CS671 - HW1 
Umair Z Ahmed (13111166)
--------------------------

1. Creating training corpus: The wordlist "wordlist_hindi.txt" was obtained after running the script "freq.py" on the training corpus of "Hi_Blogs.txt" and "Hi_Newspapers.txt" (taken from the helios-corpus). The script also uses "list 32.txt" test corpus to simply place the test corpus words at the beginning, for increased readability. In order to re-extract devanagiri words & compute their frequencies,
	(a) Copy the files "Hi_Blogs.txt" and "Hi_Newspapers.txt" from Helios corpus into current (./code) directory (didn't upload them along with other files since they are huge in size)
	(b) Run the script freq
		> python freq.py
	(c) The output is written into "wordlist_hindi.txt", in the format required by UnDivide++: 'Frequency Hindi_Word'. All non-hindi words and short (<=3 size) words have been removed.
	
2. Running UnDivide++: To run the sementation tool UnDivide++,
	(a) First, compile the code 
		> g++ UnsupervisedWordSegmentation.cpp 
	(b) Once compiled, run the executable on the "wordlist_hindi.txt" corpus, with the pre-set parameters
		> ./a.out wordlist_hindi.txt 1 1 0 1 0
	(c) The output of UnDivide++ is written in the file "./Output/finalSegmentation.txt". It contains all the words from training corpus along with their sementation.

3. Evaluating the sementation: The file "testCorpus.txt" contains the hand-annotated sementation for the word list - "list 32.txt", after removing out-of-corpus words (words which didn't occur in training corpus) and non-hindi words. In order to run evaluation of UnDivide++ sementation on this hand-annotated sementation,
	(a) Run the script	evaluate
		> python evaluate.py
	(b) The output is written into the file "evalResult.txt", where the comparison between hand-annotated segmentation "testCorpus.txt" and the segmentation produced by UnDivide++ "./Output/finalSegmentation.txt" for each word can be viewed.
		The precision, recall, f-score and accuracies are printed onto the console itself. 
		