Prev
Unit 2 · Lesson 2-2
Continuous-Time Convolution
2h 5m1300 XP
Next

Continuous-Time Convolution

The convolution integral, the RC circuit, step responses, how widths and areas combine, and why deconvolution is hard

Low Earth orbit — 1990
The Hubble Space Telescope goes up on the Space Shuttle mission STS-31 in 1990. Its main mirror was believed to be one of the most precisely figured mirrors ever made, smooth to about 10 nanometres. Within weeks of the launch, the images showed that something was badly wrong: the mirror had been ground to the wrong shape, its outer edge too flat by about 2,200 nanometres — a fiftieth of the thickness of a human hair. Light from the edge of the mirror focused at a different point from light from its centre. Every star came out as a sharp core inside a wide halo. The error was well characterised and stable. That made it a convolution: every image was the true sky convolved with the telescope's known blur, its point-spread function. Until a servicing mission, STS-61, fitted corrective optics in 1993, astronomers partially undid the blur with image-processing techniques such as deconvolution (Hubble Space Telescope). This lesson is the continuous-time convolution integral — the mathematics of every blur, every circuit's smoothing and every sensor's lag.
In continuous time the impulse response is again the output to an impulse, h(t)=S{δ(t)}h(t) = S\{\delta(t)\}, and the derivation of Lesson 2-1 goes through with sums becoming integrals. Approximate the input by a staircase of narrow pulses, each of width Δ\Delta, take the response to each pulse from the impulse response, add them, and let Δ→0\Delta \to 0.
y(t)=∫−∞∞x(τ) h(t−τ) dτ=(x∗h)(t)y(t) = \int_{-\infty}^{\infty} x(\tau)\,h(t-\tau)\,d\tau = (x * h)(t)

The convolution integral. τ is a dummy variable of integration; t is the time at which the output is wanted.

Derivation
From the staircase to the integral
Write the input as a staircase of pulses of width Δ\Delta. Each pulse is x(kΔ) δΔ(t−kΔ) Δx(k\Delta)\,\delta_\Delta(t - k\Delta)\,\Delta, where δΔ\delta_\Delta is the unit-area pulse of Lesson 1-2:
x(t)≈∑kx(kΔ) δΔ(t−kΔ) Δ.x(t) \approx \sum_k x(k\Delta)\,\delta_\Delta(t - k\Delta)\,\Delta .
Let hΔ(t)h_\Delta(t) be the response to δΔ(t)\delta_\Delta(t). By time invariance, δΔ(t−kΔ)→hΔ(t−kΔ)\delta_\Delta(t - k\Delta) \to h_\Delta(t - k\Delta), and by linearity
y(t)≈∑kx(kΔ) hΔ(t−kΔ) Δ.y(t) \approx \sum_k x(k\Delta)\,h_\Delta(t - k\Delta)\,\Delta .
As Δ→0\Delta \to 0, δΔ→δ\delta_\Delta \to \delta so hΔ→hh_\Delta \to h, the samples kΔk\Delta become a continuous variable τ\tau, Δ\Delta becomes dτd\tau, and the sum becomes the integral:
y(t)=∫−∞∞x(τ) h(t−τ) dτ.y(t) = \int_{-\infty}^{\infty} x(\tau)\,h(t - \tau)\,d\tau .
The same properties carry over: convolution is commutative, associative and distributive; x∗δ=xx * \delta = x; the system is causal iff h(t)=0h(t) = 0 for t<0t < 0, and BIBO stable iff ∫∣h(t)∣ dt<∞\int |h(t)|\,dt < \infty.
📐An RC circuit switched on
A resistor RR in series with a capacitor CC, input the source voltage, output the capacitor voltage. Its impulse response is h(t)=1RC e−t/RC u(t),h(t) = \frac{1}{RC}\,e^{-t/RC}\,u(t), an exponential decay with time constant τc=RC\tau_c = RC (Lesson 2-3 derives it). Apply a unit step, x(t)=u(t)x(t) = u(t). For t>0t > 0 only 0≤τ≤t0 \le \tau \le t contributes: y(t)=∫0t1RC e−(t−τ)/RC dτ=[e−(t−τ)/RC]τ=0τ=t=1−e−t/RC.y(t) = \int_0^t \frac{1}{RC}\,e^{-(t - \tau)/RC}\,d\tau = \left[e^{-(t-\tau)/RC}\right]_{\tau=0}^{\tau=t} = 1 - e^{-t/RC}. The capacitor charges towards the input, reaching 63.2 % after one time constant and 99.3 % after five. With R=10R = 10 kΩ and C=1C = 1 µF, RC=10RC = 10 ms: the output follows a switch-on within about 50 ms.
💡The step response
The response to a unit step is the running integral of the impulse response, and the impulse response is the derivative of the step response: s(t)=∫−∞th(τ) dτ,h(t)=dsdt.s(t) = \int_{-\infty}^{t} h(\tau)\,d\tau, \qquad h(t) = \frac{ds}{dt}. This is how impulse responses are measured in practice: nobody can apply an infinite spike, but anyone can switch something on. Record the step response and differentiate it — carefully, because differentiation amplifies noise.
📐Two rectangles make a triangle
Convolve a unit-height rectangle on [0,1][0, 1] with itself. Slide the flipped copy across: the overlap grows linearly from t=0t = 0 to t=1t = 1, then shrinks to zero at t=2t = 2. The result is a triangle of height 1 on [0,2][0, 2]. Convolve rectangles of widths 2 and 3 instead, and the overlap grows for 2 units, stays constant while the narrow one is inside the wide one, and shrinks for 2 units: a trapezoid of total width 5. Two rules hold for every convolution of pulses, and they are worth memorising:
  • Widths add: if xx lives on [a1,b1][a_1, b_1] and hh on [a2,b2][a_2, b_2], the output lives on [a1+a2, b1+b2][a_1 + a_2,\ b_1 + b_2].
  • Areas multiply: ∫y dt=(∫x dt)(∫h dt)\int y\,dt = \left(\int x\,dt\right)\left(\int h\,dt\right).
Undoing a convolution. If we know y=x∗hy = x * h and hh, can we recover xx? In principle yes — Unit 4 shows that convolution becomes multiplication of spectra, so deconvolution is a division. In practice the division is by numbers close to zero wherever the blur wiped out fine detail, and any noise at those frequencies is multiplied enormously. Practical deconvolution therefore regularises: it gives up the detail that the noise has buried and restores only what the data can still support. That is what the Hubble astronomers did, and why it was only a partial rescue.
Equipped with his five senses, man explores the universe around him and calls the adventure Science.
– Edwin Hubble, "The Exploration of Space", Harper's Magazine (May 1929), quoted on Wikiquote
🛰Spectacles for a telescope
Because the Hubble error was known precisely, engineers could build corrective optics with exactly the same error in the opposite sense — in effect, spectacles for the telescope. Astronauts fitted them on STS-61 in 1993, and the optics reached their intended quality. Until then, deconvolution had to do what the glass could not.
⚠Convolution is not multiplication
(x∗h)(t)(x * h)(t) is an integral over all of xx and all of hh; it is not x(t) h(t)x(t)\,h(t). And inside the integral, τ\tau is the dummy variable while tt is fixed: a common slip is to integrate over tt and end up with a number instead of a function.
💡Where you will use this
Lesson 2-3 derives impulse responses from differential equations. Lesson 4-2 proves the convolution theorem, which turns this integral into a product of Fourier transforms; Lesson 7-2 does the same with Laplace transforms. Deconvolution and its noise problem return with the sampling and filtering of Units 5 and 9.
📚Further reading
  • Oppenheim and Willsky, Signals and Systems, sections 2.2–2.3 — the convolution integral and the properties of LTI systems.
  • Lathi and Green, Linear Systems and Signals, section 2.4 — graphical convolution with many worked cases.
  • Hubble Space Telescope on Wikipedia — the flawed mirror and its correction.
↓But there is a problem
Convolution needs hh. We found the RC circuit's impulse response by quoting it, and a step response can be measured. But where do impulse responses come from in the first place? Most physical systems are described by differential equations — Newton's law for a mass on a spring, Kirchhoff's laws for a circuit. In 1935, earthquake magnitude itself was defined by the swing of one particular damped oscillator. Its equation tells us its impulse response directly.