Course Overview

Advanced study of information theory fundamentals. Key areas included:

Information Measures

• Entropy
• Mutual information
• Relative entropy
• Channel capacity

Source Coding

• Huffman coding
• Arithmetic coding
• Universal source coding
• Rate-distortion theory

Channel Coding

• Error-correcting codes
• Linear codes
• Cyclic codes
• Convolutional codes

Communication Theory

• Channel models
• Noise and capacity
• Shannon's theorems
• Error detection

Applications

• Data compression
• Cryptography
• Machine learning
• Network coding