Concatenate both files from blogs and newspaper
perl -0777 -lape's/\s+/\n/g' input.txt | sort | uniq -c | sort -nr > out.txt
python format.py > output.txt
python out.py > final.txt
g++ UnsupervisedWordSegmentation.cpp
./a.out final.txt 1 1 1 1 0
python found_corpus_words.py (To filter out the words in the list actually present in the corpus along with its morph parse)
