The script final.py calculates all the relevant parameters required in verifying the morphological segmentation done by Undivide++
Instructions
1.Put the FinalSegmentation.txt,your_hand_annotated file in the folder containing this script
undivide++_output=FinalSegmentation.txt
your_hand_annotated=man.txt
Run the script
Tweak the parameters in Undivide++ and again run the script to see the magic :)

Scripts for cleaning corpus
*******************************
There is a small script for cleaning the corpus-cleaner py
Script has different regular expressions which should be uncommented and run sequentially and output file should be used as input for next regular expression
**************************************
tr,uniq,sort utilities were used in these manner
tr -s '[:space:;—”“’‘:।!–]' '\n'<Hi_Blogs.txt >output1.txt
tr -d '([a-zA-Z]*)|([0-9]*)|([0-9]*\/[0-9]*\/[0-9]*)'<output1 >output2.txt
tr -d '([\.\?,\|\-])' <output2.txt | grep -v '^$'>output3.txt
uniq -c c.txt| sort -r>d.txt
sort refined8.txt |uniq -c |sort -n>d.txt

