Digital Privacy, July 2022

Threat Modeling, Deciding What You Actually Protect

Most privacy advice is offered without any statement of who it protects against, which makes it impossible to evaluate. A measure that defeats an advertising network is irrelevant against a determined adversary with legal authority, and a measure appropriate to the second is disproportionate against the first.

Threat modelling is the step that turns an unbounded problem into a decidable one. It is four questions, and it can be done in an afternoon.

The four questions

What do you have that is worth protecting. What are the consequences if it is exposed. Who would want it and what can they actually do. How much effort are you willing to spend.

The fourth question is the one usually omitted, and leaving it out is why so much advice goes unfollowed. A measure requiring constant vigilance will be abandoned within weeks, and a measure abandoned provides no protection while creating a false impression that it did.

Adversaries differ enormously in capability

Grouping them by what they can actually do produces very different conclusions from grouping them by how alarming they sound.

AdversaryCapabilityProportionate response
Advertising networksCorrelate behaviour across sites and appsBlocking, compartmentalisation, fewer accounts
Data brokersAggregate public and purchased recordsRemoval requests, minimise what is public
Opportunistic criminalsCredential stuffing, phishing, device theftPassword manager, second factor, encryption
Someone who knows youPhysical access, knows your answersDevice locks, account recovery hygiene
Employer or platformFull visibility of their own systemsSeparation, do not mix personal and work
State with legal authorityCompel providers, intercept, seizeMinimise what is collected at all

Each row implies a different defence, and several of them are mutually irrelevant. Effort spent on the bottom row while ignoring the third is a common and expensive error, because the third row is what actually happens to most people.

Assets are broader than data

The first question tends to be answered in terms of files and messages, which is too narrow.

Accounts are assets, because control of an email account usually confers control of everything that resets through it. Access is an asset, since a work login is valuable to someone targeting the employer rather than the person. Location is an asset where physical safety is the concern. Reputation and relationships are assets that cannot be restored by any technical measure.

Listing these explicitly changes the conclusions, because the measures protecting an account differ entirely from those protecting a document.

Likelihood and consequence are separate axes

A risk that is likely but trivial deserves less attention than one that is unlikely and severe, and the two are routinely conflated.

Being tracked across websites is near certain and has modest direct consequences. Having an account taken over is less likely and considerably worse. Being targeted individually by a well resourced adversary is unlikely for most people and severe for a few, and the honest answer is that some people are in that category and most are not.

Where the model changes the answer

Two people can receive opposite correct advice from the same question, which is why generic recommendations fail.

Someone whose concern is commercial profiling is well served by blocking trackers, using separate identities for separate purposes, and reducing what they publish. Someone at risk from an individual who knows them personally has a different problem entirely, where account recovery questions, shared devices, and location sharing matter far more than advertising trackers do.

Neither set of measures helps much with the other problem, and applying both is usually more effort than either person will sustain.

The mistakes that recur

Three errors account for most poor models, and each is easy to check for.

Modelling the most dramatic adversary rather than the most likely one, which produces measures that are burdensome and misdirected. Protecting the data while ignoring the account that controls it, which leaves the strongest lock on the wrong door. And adopting measures without considering maintenance, so the model describes a posture that lasted three weeks.

Revisit it occasionally

A threat model is a snapshot of a situation, and situations change. A new job, a change in relationship, a move, or public attention can all move someone between categories.

Reviewing it once a year is sufficient for most people, and the review is short, because the four questions are the same ones. What changes is the answers.

Note: the most common failure is not choosing the wrong measures. It is choosing measures that are too demanding to maintain, then abandoning them, and being less protected than a modest approach would have delivered.