lec(1)Introduction to -Information Theory(1) .pptx

Introduction to Information Theory Father of Digital Communication The roots of modern digital communication stem from the ground-breaking paper “A Mathematical Theory of Communication” by Claude Elwood Shannon in 1948. Communication Channel Includes And even this…  Shannon’s Definiti...

Introduction to Information Theory Father of Digital Communication The roots of modern digital communication stem from the ground-breaking paper “A Mathematical Theory of Communication” by Claude Elwood Shannon in 1948. Communication Channel Includes And even this…  Shannon’s Definition of Communication “The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point.” “Frequently the messages have meaning” “... [which is] irrelevant to the engineering problem.” And he thought about this problem for a while… He later on found a solution and published in this 1948 paper. In his 1948 paper he build a rich theory to the problem of reliable communication, now called “Information Theory” or “The Shannon Theory” in honor of him.  Shannon’s Vision Source Channel Data Encoding Encoding Channel Source Channel User Decoding Decoding  Example: Disk Storage Data Zip Add CRC Channel Verify User Unzip CRC In terms of Information Theory Terminology Source Zip = Encoding Data Compression = Source Unzip Data Decompression Decoding Channel Error Protection Add CRC = Encoding Verify Channel CRC = Decoding Error Correction  Example: VCD and DVD MPEG RS Moive Encoder Encoding CD/DVD MPEG RS TV Decoder Decoding RS stands for Reed-Solomon Code. Example: Cellular Phone Speech CC Encoding Encoding Channel Speech CC Decoding Decoding GSM/CDMA CC stands for Convolutional Code.  Example: WLAN IEEE 802.11b CC Data Zip Encoding Channel CC User Unzip Decoding IEEE 802.11b CC stands for Convolutional Code.