Simpson's paradox is a statistical paradox described by E. H. Simpson in 1951: an association in sub-populations may be reversed in the population. It appears that two sets of data separately support a certain hypothesis, but, when considered together, they support the opposite hypothesis.
As an example, suppose we have two people, Ann and Bob. In the first test, Ann and Bob are let loose on Wikipedia, and Ann edits 100 articles, improving 60 of them, while Bob edits 10 articles, improving 9 of them. In the second test, Ann and Bob are again let loose on Wikipedia, and this time Ann edits 10 articles, improving 1 of them, while Bob edits 100 articles, improving 30 of them.
Now we can summarise, introduce some notation (that will be useful later) and generate the paradox
- In the first test, Ann improved 60% of the articles she edited (SA(1) = 60%), while Bob's success rate was 90% (= SB(1)) Success is associated with Bob.
- In the second test Ann managed 10% (SA(2)) while Bob achieved 30% (SB(2)). On both occasions Bob's edits were more successful than Ann's. Success is again associated with Bob.
- But if we combine the two tests, we see that Ann and Bob both edited 110 articles, and that Ann improved 61 (SA = 61/110) while Bob improved only 39 (SB = 39/110).
- SB < SA. Success is now associated with Ann. Bob is better on every test but worse overall!
SA = 100/110SA(1) + 10/110SA(2).
SB = 10/110SB(1) + 100/110SB(2).
By more extreme reweighting A's overall score can be pushed up to 60% and B's down to 30%.
The arithmetic allows us to see through the paradox but there is still the conflict between the individual performances and the overall performance: who is better, A or B? Ann and Bob's creator thought Ann was better--her overall success rate is higher. But it is possible to retell the story so that it appears obvious that B is better. A and B are now hospitals and the two tests have become two types of patient: mild and severe. The numerical data is as before: B is better at curing both types of patient but its overall success rate is worse because almost all (100/110) of its patients are severe cases while almost all of A's are mild (100/110). The association of success with A is misleading, even spurious.
In this retelling has something been added, or has a tacit assumption of the Ann and Bob story been changed? These issues are discussed in the modern literature on Simpson's paradox. Although statisticians have known about the Simpson's paradox phenomenon for over a century, there has lately been a revival of interest in it and philosophers, computer scientists, epidemiologists, economists and others have discussed it too.
