Near-instant communication has become such a given that we no longer wonder how this technology works, let alone that we try to understand it. Our interconnectedness through technological devices has become so abstract that many might even believe it is too difficult to understand.

But the fundamental ideas of instant communication are surprisingly easy to understand. It’s a field that’s called Information Theory. Much of it is based on two equations written by the American mathematician Claude Shannon. This post will explain both equations. But before we do that, let’s first have a look at how Shannon explained the transmission of information:

Imagine that you wish to send a picture to your friend. The computer, tablet, or cell phone is the source. Then, your picture goes through a transmitter that prepares the pic by compressing and modulating it. What’s compression? Here’s an example: a picture consists of millions of bits (a bit is either a 0 or a 1). Each bit holds a tiny bit of information of the picture, but sometimes one bit holds exactly the same information as another bit. A dot of red in the picture might have thousands of bits containing exactly the same information (if there is no difference in hue or intensity). Compressing the picture means removing these redundant bits, and so also reducing the file size.

After the compression, the bits are turned from numbers into tones. These tones will be transmitted down a channel (mostly a telephone line). This is called modulation, and it is the second part of the encoding process. After this, the picture goes through the channel and arrives at a decoder, which restores the picture back to its original state. It reconstructs the picture in the receiver, which could be any technological device.

That is how information is transferred. It gives the bigger picture in which Shannon’s two equations operate.

Shannon’s First Equation

To understand how information can be measured, we need to look at Shannon’s first equation, also called the entropy equation:

Shannon's Entropy Equation
Shannon’s Entropy Equation

It looks complex, but all it really says is that the amount of information depends on the surprise (the probability p) that the message holds. Shannon realized that information is proportional to how much you don’t know. The more you know, the less information you need to receive/download. The log stands for the logarithm with base 2. A logarithm with base 2 of 5, for example, is simply 2 to the 5th, or 32.

The log is useful because a bit can only be in two states, either a 0 or a 1. To give an example: a flipped coin with equal probability will give 1 bit of information. This means that it has two possible outcomes. Either 0 or 1, heads or tail. Two coins then give 2 bits of information or four possible outcomes (00, 11, 01, 10).

Shannon’s entropy equation was important because it allowed us to measure information through bits. The bit lied at the basis of the kilobyte, megabyte, gigabyte, and so on… This was important for the development of information theory and ultimately information technology.

Shannon’s Second Equation

Shannon second equation, also called the capacity equation, was even more influential than his first. It introduced the concepts of hertz and bandwidth. This equation was the reason why we used Morse instead of spoken word during the Second World War.

Shannon's Capacity Equation
Shannon’s Capacity Equation

C is measured in bits per second and tells us the amount of information that can be transmitted through a phone line or any other medium. It depends heavily on bandwidth (B, the range of frequencies that can get through) and the signal-to-noise ratio (S/N). This equation can be applied to any situation. To give an example: You’re at a party and the music (N) is so loud that you need to shout (increase the signal S) to make yourself understandable. You’ll need to shout even louder for someone who can’t hear well (whose bandwidth is restricted).

Bandwidth can be measured in hertz or in bits per second. A standard telephone line can transfer 5,000 hertz. Each up and down movement of a wave constitutes two bits. So to know the number of bits per second, you just have to multiply the hertz by two: 10,000 bits per second in this case. Now imagine a signal that is seven times stronger than the noise (crackle/hiss) of the line. The logarithm with base 2 of 8 is 3 (because 2 to the 3rd is 8). 30,000 bits per second can be transmitted through this line. Then imagine you wish to send a compressed picture of 1 million bits. It will take 33.3 seconds for the line to transfer this picture (1 million divided by 30,000). An eternity, today.

The equation explains why we communicated via morse rather than speech until after the Second World War. Communication lines back then were much noisier and human speech, with its subtle tones and intricacies, was lost in the crackle and hiss. Morse was cleverly encoded to allow it to pass through lines with heavy noise. Since then, communication lines have improved drastically. Instead of the regular phone line of 5,000 hertz, we now have coaxial cables of 200 million hertz and fiber-optical cables that don’t even transmit electrical pulses, but laser-generated light.

Shannon’s two equations are the pillars of information theory, modern technology, and connectivity. The bit has now evolved into a tool to measure stored information. Hard disks and RAM, vital hardware in modern computers, became conceptually possible because of both equations. In essence, Shannon’s equations provided a framework for technology to thrive in.