Assignment 2

Observations

Part A

Given a statement, we needed to find out the word boundary segmentation of that statement

The results obtained after finding out the unigram data was

	UNIGRAM
HITS	2127
MISSES	655
PRECISION	0.764086956522
RECALL	0.239130434783

Discussion of Error Cases

The errors were mostly occuring when there was a "matra" in the middle of the word. That made it segment there. We need to remove words which start with matra.

Part B

Code for calculating word boundary segmentation

For the code for handling the corpus (pruning the corpus), the link is Download all code