Systems and Their Properties

Memory, causality, stability, invertibility, time invariance and linearity, and how systems connect

London — 10 June 2000

The Millennium Bridge, the first new river crossing in central London for a century, opens to the public. About 90,000 people walk across it that day, with up to 2,000 on the deck at a time. Within minutes the deck begins to sway from side to side. The engineers' models had treated the crowd as a load: an input applied to the bridge. But walkers sway slightly with every step, and on a deck that moves sideways they fall into step with the motion to keep their balance — which pushes the deck further in the same rhythm. The sway changed the walking, and the walking grew the sway: a positive feedback phenomenon, now called synchronous lateral excitation. The bridge closed on 12 June, two days after opening. It reopened in February 2002, after 37 viscous dampers had been fitted to soak up the energy (Millennium Bridge, London). The structure itself behaved as its designers expected. What surprised them was the system they had not drawn: the one in which the output fed back into the input.

A system is anything that turns an input signal into an output signal:

x(t) \to y(t)

, or

x[n] \to y[n]

in discrete time. An RC circuit turns an input voltage into a capacitor voltage; a thermometer turns the true temperature into a lagging reading; a moving-average filter turns noisy data into smooth data; an aircraft turns stick deflections into a flight path. We draw a system as a box with an arrow in and an arrow out, and much of engineering is the art of drawing the right boxes.

We cannot write down the response of an arbitrary system to an arbitrary input. But a handful of properties, each testable, tell us how much a system lets us predict.

💡Six properties

Memoryless: the output now depends only on the input now. A resistor ( $v = Ri$ ) is memoryless; a capacitor, which integrates current, is not.
Causal: the output now depends only on the input now and in the past. Every real-time physical system is causal; a filter run on stored data need not be.
BIBO stable: every bounded input gives a bounded output.
Invertible: different inputs give different outputs, so the input can be recovered from the output.
Time-invariant: delaying the input by $t_0$ just delays the output by $t_0$ — the system behaves the same today as tomorrow.
Linear: superposition holds. If $x_1 \to y_1$ and $x_2 \to y_2$ , then $a x_1 + b x_2 \to a y_1 + b y_2$ for all constants $a, b$ .

📐Six systems, tested

| System | Memoryless | Causal | Stable | Invertible | Time-invariant | Linear | |---|---|---|---|---|---|---| |

y(t) = x^2(t)

| yes | yes | yes | no (sign lost) | yes | no | |

y(t) = x(t - 1)

| no | yes | yes | yes | yes | yes | |

y(t) = t\,x(t)

| yes | yes | no | no | no | yes | |

y[n] = x[n] + x[n-1]

| no | yes | yes | no | yes | yes | |

y[n] = \sum_{k \le n} x[k]

| no | yes | no | yes | yes | yes | |

y(t) = x(2t)

| no | no | yes | yes | no | yes | Three entries deserve a word.

y = t\,x(t)

is unstable because the bounded input

x = 1

gives

y = t

. The two-point sum is not invertible because

x[n] = (-1)^n

produces

y = 0

, exactly like

x = 0

. And

y(t) = x(2t)

is not causal because at

t = 1

it needs the input at

t = 2

— the future.

Derivation

Testing time invariance: y(t) = t x(t)

Feed the system a delayed input

x_1(t) = x(t - t_0)

. Its response is

y_1(t) = t\,x_1(t) = t\,x(t - t_0).

Now delay the original output instead:

y(t - t_0) = (t - t_0)\,x(t - t_0).

The two differ by

t_0\,x(t - t_0)

, so delaying the input does not simply delay the output: the system is time-varying. The factor

t

is a clock built into the system, and a clock is exactly what time invariance forbids.

Linearity, by contrast, holds:

t\,[a x_1 + b x_2] = a\,t x_1 + b\,t x_2

. Linearity and time invariance are independent properties, and a system can have either one without the other.

Two consequences of linearity are worth memorising. Setting

a = 0

in the superposition test shows that a linear system must map zero input to zero output. So

y = 2x + 3

is not linear, even though its graph is a straight line: its output at zero input is 3. (It is affine, or incrementally linear — linear plus a constant offset, and we can always subtract the offset.)

And superposition is a promise about every input. Test a system with three inputs and it may pass by luck; a single counterexample proves it nonlinear. The lab below tests systems with random inputs for exactly this reason.

💡Interconnections

Systems combine in three basic ways:

Series (cascade): the output of one is the input of the next.
Parallel: both get the same input and their outputs add.
Feedback: the output is fed back and combined with the input. With a forward system $G$ and a return path $H$ subtracted at the input, $y = G\{x - H\{y\}\}$ .

Feedback is the most powerful connection and the most dangerous. Negative feedback can make an amplifier precise and an unstable aircraft flyable (Lesson 7-4). Positive feedback, like the Millennium Bridge's crowd, can turn a stable structure into a swaying one.

Essentially, all models are wrong, but some are useful.

– George E. P. Box and Norman R. Draper, Empirical Model-Building and Response Surfaces (1987), p. 424, quoted on Wikiquote

⚠Common confusions

A straight-line graph is not enough for linearity: $y = 2x + 3$ fails because zero input gives a non-zero output.
Time-varying is not the same as nonlinear: $y = t\,x(t)$ is linear but time-varying; $y = x^2$ is time-invariant but nonlinear.
Stability is about every bounded input. An accumulator responds boundedly to a zero-mean input, but a step drives it to infinity, so it is unstable.

💡Why linearity and time invariance together are special

Linearity says the response to a sum is the sum of responses. Time invariance says the response to a shifted input is the shifted response. Lesson 1-2 wrote every input as a sum of shifted impulses. Put the three together: know the response to one impulse, and you know the response to everything. That observation is the whole of Unit 2.

💡Where you will use this

Every property here has a quick test in terms of the impulse response (Lessons 2-1 and 2-2), a test in terms of the transfer function's poles (Lessons 7-3 and 8-2), and a role in design: causality decides which filters can run in real time (Lesson 4-4), stability whether a feedback loop is safe (Lesson 7-4), invertibility whether a blur can be undone (Lesson 2-2).

📚Further reading

Oppenheim and Willsky, Signals and Systems, sections 1.5–1.6 — basic system properties and interconnections.
Haykin and Van Veen, Signals and Systems (2nd ed., Wiley, 2003), section 1.8 — properties of systems with many examples.
Millennium Bridge, London on Wikipedia — the opening, the sway and the damper retrofit.

↓But there is a problem

Linear and time-invariant: the response to a sum is the sum of the responses, and the response to a delayed input is the delayed response. Now suppose you had measured the response to one tiny input — a single impulse. Every input is a row of shifted, scaled impulses. Add up the matching row of shifted, scaled responses, and you would have the response to anything. That sum has a name, and it is the subject of the next unit.