Prev
Unit 2 · Lesson 2-3
Differential and Difference Equations
2h 10m1300 XP
Next

Differential and Difference Equations

Linear constant-coefficient equations, zero-input and zero-state responses, characteristic roots, first- and second-order systems and block diagrams

Pasadena — 1935
Charles Richter, working with Beno Gutenberg at Caltech, needs a way to compare the sizes of the small local earthquakes his network records. In a 1935 paper he defines a magnitude: the logarithm of the largest swing of one particular instrument's trace, corrected for distance. A quake that drives the trace 0.001 mm at 100 km is magnitude 0; each factor of ten in the swing adds one (Richter scale). The instrument was the Wood–Anderson torsion seismometer, built by Harry Wood and John Anderson to catch the short-period waves of nearby quakes. It is a small mass on a taut wire with a natural period of 0.8 seconds, a nominal magnification of 2,800, and a damping constant of 0.8 (Wood–Anderson seismometer). Those three numbers are the coefficients of a second-order differential equation, and the whole magnitude scale is defined by how that equation responds. This lesson is about such equations: where impulse responses come from.
Most physical LTI systems obey a linear constant-coefficient differential equation,
∑k=0Nak dkydtk=∑k=0Mbk dkxdtk,\sum_{k=0}^{N} a_k\,\frac{d^k y}{dt^k} = \sum_{k=0}^{M} b_k\,\frac{d^k x}{dt^k},
and their discrete-time cousins a difference equation,
∑k=0Nak y[n−k]=∑k=0Mbk x[n−k].\sum_{k=0}^{N} a_k\,y[n - k] = \sum_{k=0}^{M} b_k\,x[n - k].
Newton's second law gives one for every mass, spring and damper; Kirchhoff's laws give one for every circuit of resistors, capacitors and inductors; a digital filter is a difference equation by construction. The order NN counts the independent energy stores — a capacitor, a moving mass, a delay element.
💡Zero-input plus zero-state
The total response splits into two parts:
  • the zero-input response, what the system does from its initial conditions with no input — a pendulum released from an angle, a capacitor discharging;
  • the zero-state response, what the input does to a system starting from rest.
Only the zero-state part is linear in the input and given by convolution, yzs=x∗hy_{zs} = x * h. An LTI description assumes the system starts at rest; Lesson 7-2 handles initial conditions with the Laplace transform.
Derivation
The first-order system and its impulse response
The RC circuit, a thermometer and a leaky tank all obey
τ dydt+y=x(t).\tau\,\frac{dy}{dt} + y = x(t).
Multiply by the integrating factor et/τ/τe^{t/\tau}/\tau:
ddt(et/τ y)=1τ et/τ x(t).\frac{d}{dt}\left(e^{t/\tau}\,y\right) = \frac{1}{\tau}\,e^{t/\tau}\,x(t).
Starting from rest, integrate from −∞-\infty to tt:
y(t)=∫−∞t1τ e−(t−τ′)/τ x(τ′) dτ′.y(t) = \int_{-\infty}^{t} \frac{1}{\tau}\,e^{-(t - \tau')/\tau}\,x(\tau')\,d\tau' .
That is a convolution, y=x∗hy = x * h, with
h(t)=1τ e−t/τ u(t).h(t) = \frac{1}{\tau}\,e^{-t/\tau}\,u(t).
The equation is the impulse response, written differently. For the RC circuit, τ=RC\tau = RC.
💡Characteristic roots
Try y=eλty = e^{\lambda t} in the homogeneous equation ∑aky(k)=0\sum a_k y^{(k)} = 0: every derivative brings down a λ\lambda, so aNλN+⋯+a1λ+a0=0.a_N\lambda^N + \dots + a_1\lambda + a_0 = 0 . Its roots — the characteristic roots — are the system's natural modes eλte^{\lambda t}. A real negative root is an exponential decay; a complex pair λ=−σ±jωd\lambda = -\sigma \pm j\omega_d is a decaying oscillation e−σtcos⁡(ωdt+ϕ)e^{-\sigma t}\cos(\omega_d t + \phi); any root with a positive real part is an instability. The impulse response is a sum of these modes. Unit 7 will call the same numbers poles.
Second-order systems. A mass on a spring with a damper, an RLC circuit and a seismometer pendulum all reduce to
y¨+2ζωn y˙+ωn2 y=ωn2 x,\ddot y + 2\zeta\omega_n\,\dot y + \omega_n^2\,y = \omega_n^2\,x ,
with natural frequency ωn\omega_n and damping ratio ζ\zeta. The characteristic roots are λ=−ζωn±ωnζ2−1\lambda = -\zeta\omega_n \pm \omega_n\sqrt{\zeta^2 - 1}:
  • ζ<1\zeta < 1, underdamped: complex roots, a decaying oscillation at the damped frequency ωd=ωn1−ζ2\omega_d = \omega_n\sqrt{1 - \zeta^2};
  • ζ=1\zeta = 1, critically damped: a double real root, the fastest response with no overshoot;
  • ζ>1\zeta > 1, overdamped: two real roots, a sluggish approach.
The step response of an underdamped system overshoots its final value by e−πζ/1−ζ2e^{-\pi\zeta/\sqrt{1 - \zeta^2}} of the step.
h(t)=ωn1−ζ2 e−ζωnt sin⁡(ωdt) u(t),ωd=ωn1−ζ2h(t) = \frac{\omega_n}{\sqrt{1-\zeta^2}}\,e^{-\zeta\omega_n t}\,\sin(\omega_d t)\,u(t), \qquad \omega_d = \omega_n\sqrt{1-\zeta^2}

The impulse response of an underdamped second-order system (ζ < 1).

📐The Wood–Anderson seismometer
Period 0.80.8 s means ωn=2π/0.8=7.854\omega_n = 2\pi/0.8 = 7.854 rad/s, and the damping constant is ζ=0.8\zeta = 0.8. The characteristic roots are λ=−ζωn±j ωn1−ζ2=−6.283±4.712 j rad/s.\lambda = -\zeta\omega_n \pm j\,\omega_n\sqrt{1 - \zeta^2} = -6.283 \pm 4.712\,j \ \text{rad/s}. The instrument rings at ωd=4.71\omega_d = 4.71 rad/s, but its envelope decays as e−6.28te^{-6.28 t} — to a thousandth in about 1.1 s. A step would overshoot by e−π⋅0.8/0.6=e−4.19=1.5 %e^{-\pi \cdot 0.8/0.6} = e^{-4.19} = 1.5\,\%. Heavy damping was the point: a lightly damped pendulum would keep swinging at its own period and write its own resonance over the earthquake's record. The magnification of 2,800 applies to ground displacements faster than the natural frequency — the instrument's displacement response is a second-order high-pass, which the lab simulates.
💡Difference equations run themselves
Solve a difference equation for its newest output and it becomes a recipe: y[n]=1a0(∑k=0Mbk x[n−k]−∑k=1Nak y[n−k]).y[n] = \frac{1}{a_0}\left(\sum_{k=0}^{M} b_k\,x[n-k] - \sum_{k=1}^{N} a_k\,y[n-k]\right). Start from rest, feed in δ[n]\delta[n], and run the recipe: out comes h[n]h[n], one sample at a time. That is exactly what scipy.signal.lfilter(b, a, x) does, and what every digital filter chip does in real time. The first-order recursion y[n]=a y[n−1]+x[n]y[n] = a\,y[n-1] + x[n] gives h[n]=anu[n]h[n] = a^n u[n] — the discrete version of the exponential decay.
Block diagrams. A differential equation can be drawn with three blocks: integrators (or delays z−1z^{-1} in discrete time), gains, and adders. The diagram for τy˙+y=x\tau\dot y + y = x is an integrator whose input is (x−y)/τ(x - y)/\tau — a feedback loop around an integrator. Analog computers were wired this way, and digital filters are still drawn this way. Unit 9 uses the diagrams to choose how to implement a filter.
It is written in the language of mathematics, and its characters are triangles, circles, and other geometric figures
– Galileo Galilei, Il Saggiatore (1623), as translated in Richard H. Popkin, The Philosophy of the Sixteenth and Seventeenth Centuries (1966), on Wikiquote
🕰A scale defined by an instrument
Richter's local magnitude is ML=log⁡10A−log⁡10A0(δ)M_L = \log_{10}A - \log_{10}A_0(\delta), where AA is the maximum excursion of the Wood–Anderson trace and A0A_0 depends on the epicentral distance. Because the definition is tied to one instrument's response, later seismometers have their records converted — by simulating the Wood–Anderson equation — before an MLM_L can be read from them.
⚠Initial conditions are not inputs
A system that is not at rest when the input starts has a zero-input response on top of the convolution. Leaving it out is the classic mistake in circuit problems with a pre-charged capacitor. Superposition holds for the zero-state part only: doubling the input does not double a response that includes initial conditions.
💡Where you will use this
Characteristic roots become the poles of Lessons 7-3 and 8-2. Second-order specifications — overshoot, settling time, damping — are the language of feedback design (Lesson 7-4). Difference equations are every digital filter (Unit 9), and the recursion you write in the lab is what lfilter runs.
📚Further reading
  • Oppenheim and Willsky, Signals and Systems, section 2.4 — causal LTI systems described by differential and difference equations, and block diagrams.
  • Lathi and Green, Linear Systems and Signals, chapter 2 — zero-input and zero-state responses in detail.
  • Wood–Anderson seismometer and Richter scale on Wikipedia.
↓But there is a problem
We can now get hh from an equation and the output from a convolution. But convolution is laborious: an integral for every instant of the output. Is there an input that an LTI system passes through unchanged in shape — scaled, perhaps delayed, but still the same kind of signal? If so, we could describe any system by what it does to that one kind of input, and replace convolution by multiplication. There is, and you have met it already.