Measuring Similarity Between Documents
CS671(NLP)
Pranjal Singh:10511
Mohit Sharma: 11434
Advisor: Prof. Amitabha Mukerjee

---

Proposal                        Slides                        Poster                        Report                        Code                       

---

Abstract

In todays world with ever increasing volume of text resources over internet and digital libraries,organizing these documents has become a practical need. Clustering is an important technique which automatically organizes large number of objects into small number of coherent groups. This leads to efficient and effective use of these documents for information retrieval and other such NLP tasks. Clustering algorithms require a metric to quantify how different two given documents are. This difference is often measured by some distance measure such as Euclidean distance, Cosine similarity to name a few. In this work we experiment with five well known distance measures and compare their performance on seven datasets using k-means clustering algorithm. The work shows that some distance measures clearly outperform others. Also, which measure to use in a specific scenario depends on what type of clusters does one want for the task at hand.