Home Work 2B
Spell Checker (Hindi)
Mohit Sharma (11434)
The corpus was pruned with words with incorrect spellings in the list provided removed.Determined the unigram frequency, mapped to probablities with smoothing functions appropriately applied. Then we constructed an edit-distance function based on the pair of correct and incorrect words provided. This was then used to make the spell-checker and we achieved the accuracies mentioned in a section below relative to suggestions of a-spell. Then we created bi-gram model based on the corpus for the "hi-spell2.txt". We used probabilities based on both unigram and bigram for getting top 3 suggestions for words in "hi-spell2.txt". In the end, we used Hindi phonemes transition data provided in aspell folder to further improve accuracies.
(1) Given List of Incorrect words and Ground Truth
Unigram model
Accuracy (Without Phonetics): 46.1%
Accuracy (With phonetics): 47.6%
Bigram model
Accuracy (without phonetics) : 46.7%
Accuracy (with phonetics) : 48.1%
(2)For the given paragraph based text "hi-spell2.txt", please find the top-3 suggestions by my spell-checker here.
Codes:
Unigram for List
n1.zip
Bigram for List
n2.zip
Unigram+Bigram Suggestions for hi-spell2.txt
n3.zip