Given two highly correlated sources, X and Y, with respective entropies
H(X) and H(Y), and joint entropy H(X,Y), the entropy reached by jointly
encoding the two sources is called the joint entropy of the two sources,
denoted by H(X,Y). A surprising fact,
established -at least in theoratical sense- by Slepian and Wolf is that it is
possible to encode the two highly correlated sources, X and Y, separately, and
still reach the joint entropy, H(X,Y), by decoding them at a joint decoder.
[1]
Thus, in a system with two entirely independent encoders, each coding either
X or Y, and a joint decoder, it is possible to reach the same entropy as a
joint encoder/joint decoder system would reach. This theorem, if it can be translated into practice by efficient coding algorithms, may have a significant
impact in image communications because it can lower the complexity of
source coders used in mobile communications by allowing independence between
them. The diagram below illustrates this separate encoding (followed by
joint decoding) of two highly correlated images, known as Distributed
Source Coding. [4]
Figure 1) Distributed Source Coding of Highly Correlated
Sources (figure by S. Pradhan)
 
Recently, significant work has been done to turn the Slepian-Wolf Theorem into practical schemes. The main strategy is to make use of error correcting codes to separate inputs into cosets and transmit the syndromes of the cosets. One of the limitations of these schemes, however, is that they assume that the probabilities of the binary inputs 0 and 1 are equal.
 
This project aims to use this concept of distributed source coding using syndromes in the presence of unequal probabilities of binary inputs, 0 and 1. The major strategy is to use a variable length code at the encoder (to equate the probabilities of 0's and 1's) and a maximum likelihood detector at the decoder.
The underlying idea is that the problem of unequal probabilities of 0's and 1's
is essentially a decoder problem, which can be solved at that stage.
[1]
ABSTRACT
INTRODUCTION
PRIOR WORK
BASIC SCHEME
METHODOLOGY
RESULTS
CONCLUSIONS
REFERENCES