Malaviya National Institute of Technology, Jaipur
Department of Mathematics
Comprehensive Course
Title of the Course:-Information Theory
Course Code
:-MAT-702
Course Credits
:-04
Entropy as a measure of uncertainty and information Shannon's entropy and entropies
of order Algebraic properties and possible interpretations, analytical properties and
inequality, joint and conditional entropies. Mutual information. Csiszar’s f-divergence
measures and their properties
Noiseless coding, unique decipherability, condtion of existence of instantaneous codes, its
extension to uniquely decipherable codes, Noiseless coding theorem, construction of optimal
codes. Discrete memoryless channels. Models for communication channel, channel capacity.
Classification of channels, Calculation of channel capacity, decoding schemes. Fundamental
Theorems, Exponential error bound weak converse of Fundamental theorm, Extension of
definition of entropies to continuous memory less channels. and properties.
Error correcting codes-Minimum distance, principle and error correcting properties.
Hamming bounds, parity check coding. Upper and Lower bounds of parity check codes.
Books Recommended:
1. Information Theory - Robert B Ash.
2. Introduction to Information Theory- F. M Reza
3. Introduction to Coding & Information Theory- Steven Romann.
4. Error correcting codes - W.W. Peterson and E. J. Weldon.