The Absence of Evidence is evidence of absence.

In probability theory, absence of evidence is always evidence of absence. If E is a binary event and P(H|E) > P(H), “seeing E increases the probability of H”; then P(H|~E) < P(H), “failure to observe E decreases the probability of H”. P(H) is a weighted mix of P(H|E) and P(H|~E), and necessarily lies between the two.

Think of probability as a spectrum between 0% and 100%. If you look at a thing and find evidence of it every time, you approach 100% probability. But if you do not, you become less certain, and it approaches 0% probability in the absence of evidence.

This is also improtant if there are multiple hypothesis to explain an event. At some point, the lack of evidence for a hypothesis means you must reject the hypothesis as probably wrong.

Say there is evidence for Hypothesis 1, but no evidence for hypothesis 2. The collection of evidence increases the probability of H1 at the expense of H2.

Also important is the Conservation of Expected Evidence

Here’s a simple example.
Take a black jar. You are told there are 100 balls randomly distributed inside and some of them are black.

You update your prior beliefs. There is a set of black balls and a set of non-black balls. “Some” black balls must mean at least three but not a majority, but it’s unknown how many. Black balls may be at least 3% and may go up to say 49% (You can update this belief incrementally be picking out the balls and counting the number of each color.

So you reach in an take out a red ball. Very well. Now you know one of the colors in the non-black set.

Here’s what you have to do. You have to track the frequency of observed balls (evidence) and maintain/update your prior belief. Reach in and pull out another ball. This one is blue. Another ball – red. So it’s 2/3rds Red, 1/3rd blue. Another ball – green. It’s a 2-1-1 ratio of red-blue-green but so far, no black balls.

Every time you measure more evidence you incrementally update the posterior and prior probabilities and beliefs.

As you continually count more and more balls without finding black balls, the probability that there are many black balls decreases towards 3%. The black balls must be rare if you haven’t found any yet so likely cannot be 49% of the balls. Your belief in some black balls without evidence is taking a hit everytime you pull out a non-black ball. Lack of evidence does not disprove black balls; this could be a “black swan” case – an elusive rarity. The absence of evidence indicates that black balls must be very rare and you become more confident that only a few balls may be black.

After you count 97 balls, the last 3 must be black or else the prior belief was wrong. What happens when you pull out the 98th ball and it is … blue?

Some individuals will triumphantly hold out the blue ball and declare it black. People do this all the time in politics. It should disprove the prior belief in black balls but cognitive bias makes us insist that the prior belief is still true.

What if there are no black balls at all? Then the absence of black balls disproved the belief in black balls in a jar.

But, you think, that’s in a jar. You can literally count every ball so you can reach 100% certainty that there are no black balls.

What if you cannot count every jar? What if someone says there are a special kind of black ball in the world. People keep looking for it but find no evidence that it exists. You cannot get 100% exact certainty, but no evidence decreases the probability that these black balls are common and increases doubt that they even exist.

Advertisements