Credits: 3 (3-0-0)
Prerequisites: ELL205
Description
Introduction to entropy, relative entropy, mutual information, fundamental inequalities like Jensen’s inequality and log sum inequality. Proof of asymptotic equipartition property and its usage in data compression. Study of entropy rates of the stochastic process following Markov chains. Study of data compression: Kraft inequality and optimal source coding. Channel capacity: symmetric channels, channel coding theorem, Fano’s inequality, feedback capacity. Differential entropy. The Gaussian channel: bandlimited channels, channels with colored noise, Gaussian channels with feedback. Detailed study of the rate-distortion theory: rate distortion function, strongly typical sequences, computation of channel capacity. Joint source channel coding/separation theorem. There are no laboratory or design activities involved with this course.
Prerequisite Tree
flowchart TD
ELL714-348[ELL714]
ELL714-348 --> ELL205-348[ELL205]
classDef empty height:17px, fill:transparent, stroke:transparent;
classDef trueEmpty height:0px, width:0px;