Home | Students | Course Content | Homework | Projects |Announcements

HW3: How systems learn: Intro to Cognitive Science SE367 2011jul-dec

Assignment 3: How systems learn

READING:
check up any book on Linear Algebra for your fundamentals of matrix operations, least-squares estimation, pseudo-inverse etc. for information theory and entropy, the first chapter of the book "Elements of Information Theory" by Cover and Thomas gives a very succinct introduction.

you are strongly encouraged to use MATLAB for problem 1 at least. however, you can also use any other language you prefer.

1. Linear Regression
consider the CPI data in the file cpi-sort.txt (collected from class and by Ankur and friends).

Assuming that the JEE rank is the only factor, plot a regression function that minimizes error, assuming a hypotheses that are
 
a) zero-th order
b) linear 
c) quadratic
d) cubic
e) starting with data row 1, select every 4th from the 39 points, so you have
   only 10 points.  fit these with a polynomial of order 9.
 
f) consider both the JEE rank and the age as determining variables for the
   CPI.  Using a linear and a quadratic function, do you find any reduction
   in error from the uni-variate cases (b,c)?
 
Turn in:
webpage at your-area/hw3/1.html, containing
i) the equation of each hypothesis function in a-d
ii) the graphs for each hypothesis 
iii) the residual error in each case
iv) give a comparative table of the values of the coefficients in a-d. 
    What do you observe about the coefficients as the order increases,
    particularly in case (e)? 
v) results of (f)
vi) Given these investigations, which of these functions you would rather
    choose and why.
2. Probabilities, Entropy:
Consider this fragment of text:
Long years ago we made a tryst with destiny, and now the time comes when we shall redeem our pledge, not wholly or in full measure, but very substantially. At the stroke of the midnight hour, when the world sleeps, India will awake to life and freedom. A moment comes, which comes but rarely in history, when we step out from the old to the new, when an age ends, and when the soul of a nation, long suppressed, finds utterance.
Write a program that will take a string such as the above and compute a histogram with the frequency of each alphabet. You should then

 
a) Estimate the probability of each letter a-z based on its frequency in
   this text. 
b) Compute the entropy of this distribution? 
c) Choose a longer text and see if the distributions and
   entropies tally. 
Submission: via webpage in your-area/hw3/2.html.

Due date: Wednesday 17 August

Submissions

Y8021ABHISHEK KAR Home Work
Y8040AGRIM GUPTA Home Work
Y8148ASMITA BHATTACHARYA Home Work
Y8167BHUWAN DHINGRA Home Work
Y8214HEMANGINI PARMAR Home Work
Y8469SHANTANU SARASWAT Home Work
Y8487SHOURYA SONKAR ROY BURMANHome Work
Y8560VIDUR KUMAR Home Work
Y9103ANKUR AGRAWAL Home Work
Y9111ANUJ GUPTA Home Work
Y9189DEBOSMITA CHAUDHURI Home Work
Y9259JAYANT SHARMA Home Work
Y9277KARTHIKEYAN .M Home Work
Y9424PRANJAL SAXENA Home Work
Y9500RUHI DANG Home Work
Y9574SHUBHAM TULSIANI Home Work
Y9588SNEHA AGARWAL Home Work
Y9591SONAL KUMARI Home Work
Y9605SUMIT VERMAHome Work
Y6104Apurva Gupta Home Work