Prev
Unit 5 · Lesson 5-1
The Sampling Theorem
1h 50m1300 XP
Next

The Sampling Theorem

Impulse-train sampling, spectral copies, the Nyquist rate, who found the theorem, bandlimited signals in practice and bandpass sampling

1979 — digital audio on video tape
In 1979 Sony introduces the PCM-1600, a box that turns stereo audio into numbers and records them on a U-matic video cassette — at the time the only affordable recorder with room for that much data. To reuse the video machinery unchanged, the samples are packed into the picture: three samples per video line, 245 active lines per field, 60 fields per second. 3×245×60=44 1003 \times 245 \times 60 = 44\,100 samples per second. On European equipment, 3×294×503 \times 294 \times 50 gives the same number (44,100 Hz). Such adaptors were the cheapest way to carry digital audio from a studio to a disc factory, so when Sony and Philips fix the compact disc in the Red Book standard of 1980, the number stays: 44.1 kHz, a little more than twice the 20 kHz limit of human hearing. Why twice? And why should any list of numbers, however long, be enough to rebuild a continuous sound — between the samples as well as at them? This lesson is the answer, and its fine print.
Sampling keeps the values x[n]=x(nT)x[n] = x(nT) of a continuous signal every TT seconds: the sampling period TT, the sampling frequency fs=1/Tf_s = 1/T, or ωs=2π/T\omega_s = 2\pi/T in radians per second. Between the samples the signal could, in principle, do anything. The sampling theorem says that if the signal contains no frequencies above some limit, it cannot do anything: the samples pin it down completely.
Derivation
Sampling copies the spectrum
Model sampling as multiplication by an impulse train p(t)=∑nδ(t−nT)p(t) = \sum_n \delta(t - nT):
xp(t)=x(t) p(t)=∑n=−∞∞x(nT) δ(t−nT).x_p(t) = x(t)\,p(t) = \sum_{n=-\infty}^{\infty} x(nT)\,\delta(t - nT).
The train is periodic with period TT, and its Fourier coefficients are all equal: ak=1T∫−T/2T/2δ(t) e−jkωst dt=1Ta_k = \frac1T\int_{-T/2}^{T/2}\delta(t)\,e^{-jk\omega_s t}\,dt = \frac1T. So
p(t)=1T∑k=−∞∞ejkωst,xp(t)=1T∑k=−∞∞x(t) ejkωst.p(t) = \frac1T\sum_{k=-\infty}^{\infty} e^{jk\omega_s t}, \qquad x_p(t) = \frac1T\sum_{k=-\infty}^{\infty} x(t)\,e^{jk\omega_s t}.
Multiplying by ejkωste^{jk\omega_s t} shifts a spectrum by kωsk\omega_s (Lesson 4-2), so
Xp(jω)=1T∑k=−∞∞X(j(ω−kωs)).X_p(j\omega) = \frac1T\sum_{k=-\infty}^{\infty} X\big(j(\omega - k\omega_s)\big).
The sampled signal's spectrum is the original one, scaled by 1/T1/T, together with copies centred on every multiple of the sampling frequency.
Xp(jω)=1T∑k=−∞∞X(j(ω−kωs)),ωs=2πTX_p(j\omega) = \frac{1}{T}\sum_{k=-\infty}^{\infty} X\big(j(\omega - k\omega_s)\big), \qquad \omega_s = \frac{2\pi}{T}

Sampling every T seconds copies the spectrum to every multiple of the sampling frequency.

💡The sampling theorem
Let x(t)x(t) be bandlimited: X(jω)=0X(j\omega) = 0 for ∣ω∣>ωM|\omega| > \omega_M. If it is sampled faster than twice its highest frequency, ωs>2ωM(fs>2B in hertz),\omega_s > 2\omega_M \qquad (f_s > 2B \text{ in hertz}), the copies do not overlap, and an ideal low-pass filter with gain TT and a cut-off anywhere between ωM\omega_M and ωs−ωM\omega_s - \omega_M returns exactly X(jω)X(j\omega). The samples determine the signal completely, between the sampling instants as well as at them. 2B2B is the signal's Nyquist rate; half the sampling frequency, fs/2f_s/2, is the sampler's Nyquist frequency.
📐Compact discs and telephones
  • Audio CD. Hearing reaches about 20 kHz, so the Nyquist rate is 40 kHz. At 44.1 kHz the Nyquist frequency is 22.05 kHz, which leaves a 2.05 kHz gap above the audio band for the anti-aliasing filter to fall in (Lesson 5-2).
  • Telephone. The classic voice channel passes about 300–3400 Hz and is sampled at 8 kHz: a Nyquist frequency of 4 kHz, 600 Hz above the band. Eight thousand samples a second of 8 bits each make the 64 kbit/s channel on which digital telephony was built.
Who found it? The theorem has many parents. E. T. Whittaker published the interpolation formula in 1915; Kinnosuke Ogura gave what a 2011 historical review counts as the first proof, in 1920; Harry Nyquist's telegraph papers of 1924 and 1928 tied signalling speed to bandwidth; Karl Küpfmüller found a similar result in 1928; Vladimir Kotelnikov proved it in 1933, and in Russian it carries his name. Claude Shannon, who had derived it by 1940, published it in 1949 and made it common property among communication engineers (Nyquist–Shannon sampling theorem).
💡Bandlimited signals in practice
No signal that starts and stops is exactly bandlimited: a finite duration forces a spectrum that never dies out completely, as the rectangle's sinc showed in Lesson 4-1. In practice "bandlimited" means "with negligible energy above some frequency" — negligible next to the noise and the converter's resolution — and engineers make it true on purpose, with an anti-aliasing filter in front of every sampler. The theorem is exact; real signals satisfy it to within a stated error.
📐Sampling a band, not a baseband
A signal confined to 20–25 kHz has a bandwidth BB of only 5 kHz. Sampling above twice its top frequency, 50 kHz, works — but so does far less, because the copies only need to miss each other. They do when 2fHn≤fs≤2fLn−1\frac{2f_H}{n} \le f_s \le \frac{2f_L}{n - 1} for some whole number nn up to fH/Bf_H/B. Here n=4n = 4 allows 12.5–13.33 kHz, and at 12.8 kHz the 5 kHz band lands intact, reversed, at 0.6–5.6 kHz. Bandpass sampling lets a radio receiver digitise a narrow channel at a high frequency with a slow converter.
[I]n spite of its evident importance, [it] seems not to have appeared explicitly in the literature of communication theory.
– Claude Shannon on the sampling theorem, "Communication in the Presence of Noise" (1949), quoted in Nyquist–Shannon sampling theorem
🕰The rate Nyquist never stated
Nyquist's 1924 and 1928 papers were about telegraph speed, and neither states the sampling theorem. The terms Nyquist interval and Nyquist rate very likely began in Shannon's 1949 paper, where he named the critical interval 1/2W1/2W after Nyquist, in recognition of his work on telegraphy (Nyquist–Shannon sampling theorem).
⚠Exactly twice is not enough
The inequality is strict. Sample sin⁡(2π⋅5000 t)\sin(2\pi\cdot 5000\,t) at exactly 10 kHz, starting at t=0t = 0, and every sample lands on a zero crossing: sin⁡(πn)=0\sin(\pi n) = 0. In general, Acos⁡(πn+ϕ)=Acos⁡ϕ (−1)nA\cos(\pi n + \phi) = A\cos\phi\,(-1)^n: a tone at exactly fs/2f_s/2 leaves its amplitude and phase tangled together. Sample strictly faster than 2B2B — in practice, comfortably faster.
💡Where you will use this
Lesson 5-2 is what happens when the condition fails, and Lesson 5-3 builds the reconstruction. The DTFT of a sampled signal (Lesson 6-1) is exactly the periodic spectrum derived here, and every digital filter of Unit 9 works on samples taken under this theorem. Bandpass sampling returns in the radio receivers of Unit 10.
📚Further reading
  • Oppenheim and Willsky, Signals and Systems, section 7.1 — the sampling theorem through impulse-train sampling.
  • Claude E. Shannon, "Communication in the Presence of Noise", Proceedings of the IRE 37 (1949), 10–21 — the paper that made the theorem common property.
  • 44,100 Hz on Wikipedia — how a video format set the CD's sampling rate.
↓But there is a problem
The theorem's condition is not a formality. Sample too slowly and high frequencies are not simply lost: they come back disguised as low ones, and nothing after the sampler can tell the disguise from the real thing. You have seen it in every western, when the stagecoach speeds up and its wheels begin to turn backwards.