\documentclass[11pt]{article}
\usepackage{amsmath}
%\usepackage{fullpage}
%\usepackage{epic}
%\usepackage{eepic}
%\usepackage{psfig}

%\newcommand{\proof}[1]{
%{\noindent {\it Proof.} {#1} \rule{2mm}{2mm} \vskip \belowdisplayskip}
%}


%\newtheorem{lemma}{Lemma}[section]
%\newtheorem{theorem}[lemma]{Theorem}
%\newtheorem{claim}[lemma]{Claim}
%\newtheorem{definition}[lemma]{Definition}
%\newtheorem{corollary}[lemma]{Corollary}

%Theorems and likes
\newtheorem{assumption}{Assumption}[section]
\newtheorem{theorem}{Theorem}[section]
\newtheorem{fact}{Fact}[section]
\newtheorem{claim}{Claim}[section]
\newtheorem{lemma}{Lemma}[section]
\newtheorem{example}{Example}[section]
\newtheorem{definition}{Definition}[section]
\newtheorem{corollary}{Corollary}[section]
\newtheorem{exercise}{Exercise}[section]


\newcommand{\bproof}{\noindent{\it Proof}}
%\newcommand{\eproof}{\hspace*{\fill}$\Box$~~~~~\bigskip}
\newcommand{\eproof}{\hspace*{\fill}\rule{2mm}{2mm}~~~~~\bigskip}
\newenvironment{proof}{\bproof: }{\eproof}

% symbols and notation
\newcommand{\defeq}{\stackrel{\rm def}{=}}


\setlength{\oddsidemargin}{0in} \setlength{\topmargin}{0in}
\setlength{\textwidth}{6in} \setlength{\textheight}{8in}

\begin{document}

\setlength{\fboxrule}{.5mm}\setlength{\fboxsep}{1.2mm}
\newlength{\boxlength}\setlength{\boxlength}{\textwidth}
\addtolength{\boxlength}{-4mm}
\begin{center}\framebox{\parbox{\boxlength}{\bf
CS 681: Computational Number Theory and Algebra \hfill Lecture 3
\\\\
Madhusudan's decoding of Reed-Solomon Code and Divide and Conquer Tool
\\\\
Lecturer: Manindra Agrawal \hfill Scribe: Sudeepa Roy
%\\
\begin{flushright}
%date
August 5, 2005
\end{flushright}
}}\end{center} \vspace{5mm}

\section{Introduction }
In the last lecture we studied Berlekamp-Welsh decoding of
Reed-Solomon code. In this lecture we shall first look at another decoding
algorithm for Reed-Solomon code, by P. Madhusudan (1994). Then we will discuss some applications of our first tool to design algorithms - \textit{Divide and Conquer Technique}.


\section{Madhusudan's Decoding Algorithm of Reed-Solomon Code }

First we state the main features of these two decoding algorithms of Reed-Solomon code.\\\\
    \textbf{Berlekamp-Welsh Algorithm}    
    	%\begin{description}    
    		\begin{itemize}
					\item needs solving system of linear equations
					\item corrects upto $\frac{1}{2}(n-k)$ errors
				\end{itemize}
			%\end{description}
    \textbf{Madhusudan's Algorithm} 
    	%	\begin{description}    
    		\begin{itemize}
					\item corrects upto $n - 2~ \sqrt[ ]{nk}$ errors
					\item needs more algebraic operations
				\end{itemize}
Let us restate the notations used for Reed-Solomon code from lecture 0.
\begin{itemize}
	\item The data to be stored on a CD is divided into chunks of $b \times k$ bits and each chunk is coded separately.
	\item $F$ is finite field of size $2^b$.
	\item Each chunk is again divided into $k$ blocks of $b$ bits each, say $d_0, d_1, \cdots, d_{k-1}$.
	\item Each $d_i$ is treated as an element of field $F$.
	\item Let $e_0, e_1, \cdots, e_{n-1}$ be $n$ distinct elements of $F$.
	\item Let $f_j = P(e_j)$.
\end{itemize}
Then the original codeword corresponding to $d_0d_1\cdots d_{k-1}$ is
\begin{center}
 $f_0f_1 \cdots f_{n-1}$
\end{center}
Input to the decoding algorithm for a chunk assuming that at least $t$ of the $f_j$s should remain unchanged is
\begin{center}
$\hat{f_0}\hat{f_1} \cdots \hat{f}_{n-1}$
\end{center}
Let $Q(x, y)$ be a polynomial such that $Q(e_j, \hat{f_j}) = 0$ for $j = 0~ to~ n-1$. Also let $D_x$ and $D_y$ be the degrees of $x$ and $y$ respectively in $Q$.\\
Then we can write $Q(x, y)$ as
\begin{center}
$Q(x,y) = \sum\limits_{i=0}^{D_y}~ \sum\limits_{j=0}^{D_x}~ \alpha_{ij}~ x^i y^j$	
\end{center}
In the above equation, there are $(1 + D_x)(1 + D_y)$ different $\alpha_{ij}$s and $n$ equations $Q(e_j, \hat{f_j}) = 0$ for $j = 0~ to~ n-1$. So if $(1 + D_x)(1 + D_y) > n $ then $Q$ exists and can be computed easily.\\
Now let us consider another polynomial
\begin{center}
$R(x) = Q(x, P(x))$
\end{center}
As $\deg P$ is $k-1$, so
\begin{center}
$\deg R \leq D_x + (k-1)D_y$\\
\end{center}
But $R(x)$ is zero on at least $t$ distinct values since for at least $t$ $j$s,
\begin{center}
$R(e_j) = Q(e_j, P(e_j)) = Q(e_j, f_j) = Q(e_j, \hat{f_j}) = 0$
\end{center}
Hence if $\deg R \leq D_x + (k-1)D_y < t$, then $R(x)$ must be the zero polynomial or, $R(x) = 0$ for all $x$.
Then

$R(x) = Q(x, P(x)) = 0$

$\Rightarrow Q(x, y)$ becomes $0$ when $y = P(x)$

$\Rightarrow Q(x, y) = 0$ $(mod~ y = P(x))$

$\Rightarrow (y = P(x))~ |~ Q(x, y)$\\\\
So the algorithm is
\begin{itemize}
	\item Factor $Q(x, y)$ into irreducible factors
	\item Collect all factors of the form $y - P'(x)$
	\item Use \textit{domain knowledge} to identify right $P(x)$
\end{itemize}
where the \textit{domain knowledge} is knowledge about some typical pattern followed by valid video data so that we can get the correct original video data from a list of candidates.\\
If we choose $D_x = \sqrt{kn}$ and $D_y = \sqrt{\frac{n}{k}}$, then
\begin{center}
$(1 + D_x)(1 + D_y) > \sqrt{kn} \cdot \sqrt{\frac{n}{k}} = n$
\end{center}
So condition for existence of polynomial $Q$ holds.\\
Now for $R(x)$ to be a zero polynomial,

$D_x + (k-1)D_y < t$

$\Rightarrow t > \sqrt{kn} + (k - 1)\sqrt{\frac{n}{k}}~ \geq~ 2\sqrt{kn}$ \\
Hence at least $2\sqrt{kn}$ data should remain unchanged, or in other words, the algorithm can correct upto $n - 2\sqrt{kn}$ errors.\\\\
This algorithm takes more than real time but was improved later. Further, in a true sense, it is not a decoding algorithm as it does not produce a single decoded output but a list of candidate outputs. Then we have to extract the correct output applying knowledge about video data.


\section{Divide and Conquer Technique}
Divide and Conquer is the first tool for designing efficient algorithms in Number Theory and Algebra that we will study. As this is a well known tool so we will study only some applications of this technique.

\subsection{Matrix Multiplication}

\begin{description}
	\item[Problem] Given two $n \times n$ matrices $A$ and $B$, compute $A \times B$.
\end{description}
Time complexity of standard algorithm of matrix multiplication is $O(n^3)$.\\
But by using divide and conquer technique the time complexity can be reduced. For simplcity let us assume $n = 2^m$ (else blow up the size filling rest of the entries with zeros).\\
Let 
\begin{center}
$A = $
$\begin{bmatrix}
	A_{11} & A_{12} \\
	A_{21} & A_{22}
\end{bmatrix}$,
$B = $
$\begin{bmatrix}
	B_{11} & B_{12} \\
	B_{21} & B_{22}
\end{bmatrix}$
\end{center}
Then
\begin{center}
$A \times B = $
$\begin{bmatrix}
	A_{11}B_{11}+A_{12}B_{21} & A_{11}B_{12}+A_{12}B_{22} \\
	A_{21}B_{11}+A_{22}B_{21} & A_{21}B_{12}+A_{22}B_{22}
\end{bmatrix}$
\end{center}
where each of $A_{ij}$~s and $B_{ij}$~s are $2^{m-1} \times 2^{m-1}$ matrices.\\
The most simple way would be to compute each of the individual products of $A_{ij}$ and $B_{kl}$ matrices and then performing the sum. If we denote time complexity of multiplication of two $n \times n$ matrices as $T(n)$ then the recursive formula of $T(n)$ will be
\begin{center}
$T(n) = 8T(\frac{n}{2}) + O(n^2)$
\end{center}
where the $O(n^2)$ term comes due to addition of $\frac{n}{2} \times \frac{n}{2}$ matrices.
\begin{claim}
All the terms in $A \times B$ can be computed using only 7 $\frac{n}{2} \times \frac{n}{2}$ matrix multiplications [Strassen's Algorithm].
\label{claim-strassen}
\end{claim}
\begin{exercise}
Prove Claim \ref{claim-strassen}.
\end{exercise}
Then the improved recursive relation for time complexity of matrix multiplication of two $n \times n$ matrices will be
\begin{center}
$T(n) = 7T(\frac{n}{2}) + O(n^2)$
\end{center}
Solving this recursive relation we get
\begin{center}
$T(n) = O(n^{\log_2 7}) = O(n^{2.71})$
\end{center}
Time complexity of Strassen's algorithm was still improved further by taking $n$ as powers of larger integers than 2. The best known algorithm for matrix multiplication has time complexity as $O(n^{2.36})$ though it is strongly believed by the community that the best possible time complexity is $\theta(n^2)$!

\subsection{Extension of Matrix Multiplication}

Advantage of better time complexity of Matrix Multiplication using Divide and Conquer can be extended to other problems like finding inverse of a matrix, finding the value of determinant and solving a system of linear equations. Here we give one relevant example of reduction of Matrix Inversion to Matrix Multiplication problem.

\begin{description}
	\item[Problem] Given an $n \times n$ matrix $A$, compute the matrix $A^{-1}$.
\end{description}
Let
\begin{center}
 $A = $
$\begin{bmatrix}
	A_{11} & A_{12} \\
	A_{21} & A_{22}
\end{bmatrix}$
\end{center}
Let
\begin{center}
 $A^{-1} = $
$\begin{bmatrix}
	B_{11} & B_{12} \\
	B_{21} & B_{22}
\end{bmatrix}$
\end{center}
Then
\begin{center}
$A \times A^{-1} = $
$\begin{bmatrix}
	A_{11}B_{11}+A_{12}B_{21} & A_{11}B_{12}+A_{12}B_{22} \\
	A_{21}B_{11}+A_{22}B_{21} & A_{21}B_{12}+A_{22}B_{22}
\end{bmatrix}$
$= I =$
$\begin{bmatrix}
	1 & 0 \\
	0 & 1
\end{bmatrix}$
\end{center}
Equating corresponding terms we need to solve four equations to get the values of $B_{ij}$ matrices.
\begin{exercise}
Fill in the details to complete the above reduction.
\end{exercise}
\end{document}