# A Mathematical Theory of Communication

INFORMATION

SOURCE

TRANSMITTER

RECEIVER

DESTINATION

SIGNAL

RECEIVED

SIGNAL

MESSAGE

MESSAGE

NOISE

SOURCE

Fig. 1 â€" Schematic diagram of a general communication system.

a decimal digit is about 3 3 bits. A digit wheel on a desk computing machine has ten stable positions and

therefore has a storage capacity of one decimal digit. In analytical work where integration and differentiation

are involved the base e is sometimes useful. The resulting units of information will be called natural units.

Change from the base a to base b merely requires multiplication by logb a.

By a communication system we will mean a system of the type indicated schematically in Fig. 1. It

consists of essentially ï¬ve parts:

1. An information source which produces a message or sequence of messages to be communicated to the

receiving terminal. The message may be of various types: (a) A sequence of letters as in a telegraph

of teletype system; (b) A single function of time f (t) as in radio or telephony; (c) A function of

time and other variables as in black and white television â€" here the message may be thought of as a

function f (x; y; t) of two space coordinates and time, the light intensity at point (x; y) and time t on a

pickup tube plate; (d) Two or more functions of time, say f (t), g(t), h(t) â€" this is the case in â€œthree-

dimensionalâ€ sound transmission or if the system is intended to service several individual channels in

multiplex; (e) Several functions of several variables â€" in color television the message consists of three

functions f (x; y; t), g(x; y; t), h(x; y; t) deï¬ned in a three-dimensional continuum â€" we may also think

of these three functions as components of a vector ï¬eld deï¬ned in the region â€" similarly, several

black and white television sources would produce â€œmessagesâ€ consisting of a number of functions

of three variables; (f) Various combinations also occur, for example in television with an associated

audio channel.

2. A transmitter which operates on the message in some way to produce a signal suitable for trans-

mission over the channel. In telephony this operation consists merely of changing sound pressure

into a proportional electrical current. In telegraphy we have an encoding operation which produces

a sequence of dots, dashes and spaces on the channel corresponding to the message. In a multiplex

PCM system the different speech functions must be sampled, compressed, quantized and encoded,

and ï¬nally interleaved properly to construct the signal. Vocoder systems, television and frequency

modulation are other examples of complex operations applied to the message to obtain the signal.

3. The channel is merely the medium used to transmit the signal from transmitter to receiver. It may be

a pair of wires, a coaxial cable, a band of radio frequencies, a beam of light, etc.

4. The receiver ordinarily performs the inverse operation of that done by the transmitter, reconstructing

the message from the signal.

5. The destination is the person (or thing) for whom the message is intended.

We wish to consider certain general problems involving communication systems. To do this it is ï¬rst

necessary to represent the various elements involved as mathematical entities, suitably idealized from their

2