Abstract—This difference between digital and analog systems affects the security of digital systems, particularly the Internet of Things.
Keywords—Dan Geer; security; Internet of Things; cybersecurity; system security
Digital systems are brittler than analog systems; they contain no natural stasis to return to when bumped. An off-by-one transient can cascade, whereas an off-by-a-millivolt transient will be corrected by nature. Digital engineers must build sanity checks into their systems; analog engineers need only build tolerances. Digital systems keep themselves on point by checking against internally constructed norms; analog systems are kept in equilibrium by the laws of physics. This difference between digital and analog systems affects digital systems’ security, particularly the Internet of Things.
Initially, it would seem that brittleness is digital systems’ Achilles’ heel. If a system goes off the rails at the slightest touch, life becomes a lot easier for attackers who seek to bring the system down. At the same time, life becomes much more difficult for attackers who want to silently alter system behavior. A system going off the rails is a sure sign that it's been fiddled with.
Therefore, a design tradeoff in constructing internal checks is between system robustness (against attacks to bring it down) and system correctness (against attacks on its functionality). The harder the system is to bring down, the easier it is to alter it undetected. As everybody gets spattered with mud when a system crashes, this tradeoff is usually made, without hesitation, in favor of more internal checks.
Most internal checks are against absolutes. If a denominator is zero, the division isn't performed. If the denominator isn't supposed to be zero, an error condition is raised. This brings the system's unexpected behavior to the designer's attention, who can arrange for a considered response. The challenges of exception-based programming are widely recognized and need not be examined here. Suffice it to say that if your exception-handling code's primary goal is to fix up, make good, and press on, you've made it much easier for attackers who simply want the system to misbehave occasionally.
The interest here is rather in another kind of internal check—one against relatives. Relative checks are against a system's own historic record: Has this person ever before purchased lingerie in Romania? The exception is probabilistic rather than absolute, and the historic record is used to estimate the probability. If the exception raised by the check is highly likely, then it can be ignored, with the understanding that some risk has been incurred by doing so. (The estimation of this risk is by no means an easy task, which is why it is rarely undertaken. Indeed, even acknowledgment of the risk is rare.) If, on the other hand, the probability of the exception based on the historic record is low, then the exception is turned into an absolute exception and the previous paragraph applies.
It's in the matter of relative checks that the difference between the two systems comes into stark relief. An analog system doesn't modify the criteria it checks itself against as it operates. These are the laws of physics; they're both external to the system and inviolate. By contrast, a digital system's relative checks are against its own historic record, which the system changes as it runs. Should this historic record become slowly contaminated, decisions about what is and isn't an exception to a relative check become quietly skewed. Synchronization primitives will ensure that a unitary change propagates. In the unlikely event that the contamination of the historic record is eventually recognized, there's little alternative to wiping the record clean and starting fresh. (Do, please, read Laney Salisbury and Aly Sujo's Provenance, Penguin, 2010.)
This brings us to Orwell's point, which is cautionary advice to digital system security designers because it's also an invitation to those who seek to ever so slowly and quietly change the trajectory of the system or to divert it, from time to time, to their own use. Like changing a constant in router code.