Friday, August 12, 2016

Rough Jottings on Absence of Evidence

The phrase 'absence of evidence is not evidence of absence' is a fairly widespread one, but its history is very difficult to trace. You can find the principle being used here and there in reasoning, of course, but the actual phrase doesn't seem to pop up before the 1990s -- and its popularity is almost certainly due to Carl Sagan (although he got it from Martin Rees). From him it spread quickly, and it tends to come up quite often in certain fields. It's fairly easy to find in geology, medicine, and criminology, in particular. Discussion for and against also shows up a lot in apologetical circles, whether the apologetics is of Christians or of skeptics out to convince the masses -- I'm fairly sure that this is just because both of these groups have read a lot of Carl Sagan.

Both groups are mixed, but the Christians tend generally to be for it and the skeptics against it, if you're interested. Reading some of the skeptical discussions makes for hilarity, at times, because they are so often vehemently committed to the phrase encapsulating a fallacy, often calling it the fallacy of appeal to ignorance, or the argument from ignorance, because they associate it with UFOers, that they cannot believe that he endorses it, even though anyone with basic reading skills can see that he does.

A distinction is sometimes made between 'Absence of evidence is not evidence of absence' and 'Absence of proof is not proof of absence'. This is a post-Sagan distinction (Sagan uses 'evidence' and 'proof' interchangeably when the phrase comes up), and has come about very certainly because of the spread of accounts of evidence based on interpretations of probability theory (rather than on practices of inquiry), which as generally proposed require that absence of evidence be evidence of absence. And in fact the distinction doesn't salvage anything unless one has a very specific kind of account of proof in mind; in a lot of areas what is called 'proof' just is clear preponderance of evidence, and (on the other side) it's not actually difficult to find or rig situations in which an absence of proof would indeed be a proof of absence.

There's little serious philosophical examination of the maxim, although one can find some. Elliott Sober has argued (in "Absence of evidence and evidence of absence: evidential transitivity in connection with fossils, fishing, fine-tuning and firing squads") that while the Absence Maxim can be true if it just means that not looking for evidence is not evidence of evidence, it is not strictly true in general, although it is often close to true: in such cases the maxim should in strictness be 'Absence of evidence often only very weakly provides evidence of absence', but we often treat very weak evidence as if it were nonevidence. (Contrary to Michael Strevens's response (PDF), I don't think it is quite accurate to treat this as conveying "an underlying meaning that contradicts its apparent meaning". Sober is certainly right that we often treat making a very small difference as making no difference; and it is undeniable, I think, that most of the time when people say, "There is no evidence for p," what they mean is, "Any evidence for p is so weak as to be negligible." On Sober's account, we could add 'for practical purposes' to the Absence Maxim and get something at least in the neighborhood of right; and this seems very different from an actual contradiction.)

Let's call 'Absence of evidence is not evidence of absence' the Absence Maxim to simplify things. There are a number of distinct but related issues tangled up with the Absence Maxim that have never really been untangled; I don't hope to do so here, but it might be worthwhile to point out a few things that do seem constantly to arise.

(1) The first and most obvious thing is that the Absence Maxim gets its plausibility from the way it is structured. To be absent is simply to be not present, presence has to be presence of something, and evidence is always evidence for soemthing, so we could rephrase it as 'Evidence-for-X is not present' is not 'There is evidence for X-is-not-present'. If one were to deny this, there would be obvious questions, since the kind of shifting around of operators and scope that this involves usually suggests an equivocation. One would at least need a principled reason for denying the Maxim that shows that you aren't just trading on ambiguity. One obvious worry is how you get the existence of something (evidence for X-is-not-present) entirely from the nonexistence of something else (evidence for X). Where does the existential operator pop out from?

This, I take it is related to the issue raised in the following video by Ian Goddard, which argues that it is important to make a distinction between absence of evidence and negative evidence:



There is little doubt that you can have logical systems in which denial of the Absence Maxim is required under some interpretation (Bayesian interpretations of probability theory are common examples); but, as one can make all sorts of formal systems with all sorts of arbitrary assumptions, and interpret them all sorts of different ways, this does not get us very far on its own. The point is that any formal representation of a denial of the Absence Maxim raises questions that would require principled answers, and that these answers are not necessarily immediately obvious. And they will not fall out of the formalism itself at all.

(2) We do clearly recognize cases where absence of evidence is indeed evidence of absence -- call these negation-as-failure cases. If I thoroughly search my room for a dog and discover no evidence at all that there is a dog, then that is indeed evidence that there is no dog in my room.

One of the things that characterizes obvious negation-as-failure cases is that they are cases in which either (A) we can reasonably guarantee that our search has been exhaustive (the closed world assumption), or (B) we can reasonably conclude that further search is increasingly unlikely to turn up anything fundamentally different from what we have already discovered, so that we can discount as unpromising whatever we have not yet searched (which is a practical closed world assumption). There are many cases, however, in which we cannot guarantee any kind of closed world assumption -- for instance, if I have been searching for a dog in the city for two minutes and have not yet found it, this search was neither exhaustive nor reasonably extensive, and most people, I imagine, would regard someone as an idiot if they concluded from such a cursory search that there was no dog in the city. And as with cursory searches, so with searches that are incomplete for other reasons than merely trying.

We are sometimes in situations in which we already know that our bank of available evidence is missing a lot. Indeed, we are often in these situations -- they are the norm in most historical contexts. We already know that our fossil record does not record everything; it also does not provide a representative sample, because it only gives us things that were in fact in situations in which they, in particular, could be fossilized. Happily, there is more than one way a thing can be fossilized, but each way has its own specific conditions, and if they are not met, we don't get the fossil. Let's suppose that our fossil record is indeed very, very patchy. One way to interpret this kind of situation is to say that the fossil record, considered on its own, only provides us any evidence at all about organisms actually showing up in it, because we already know that the field of organisms and organism-types that won't show up can be massive in comparison to the field of those that do. Unless you were independently trying to fit evidence to a principle like Bayes' Theorem, there doesn't seem to be any specific reason to take an organism's not showing up in the record as evidence that it didn't exist -- most organisms that existed don't, so there's nothing about the fossil record itself that requires that you take things it doesn't show not to exist. In general this will be the case with patchy evidence; by the very fact of its being patchy, there's nothing about the collection of evidence itself that licenses taking absence of evidence as evidence of absence. The license would have to come from elsewhere.

This perhaps is connected with Sagan's own diagnosis of the problem to which the Absence Maxim is proposed as a solution -- what Sagan calls 'impatience with ambiguity'. 'Ambiguity' is a term Sagan elsewhere uses to describe the situation of not having an answer to a question. Sagan does not, as far as I am aware, explain his use of the word 'impatience' in this precise context, and Sagan is usually quite sloppy when it comes to supporting his claims about critical thinking, but I think one can argue that the term is genuinely appropriate in this context: in cases of cursory search, it is absurd to hold that not having a clear preponderance of evidence for one side is a reason to conclude it false, because you should exercise the patience required to make your investigation reasonably thorough before drawing such a conclusion rather than hurrying to draw a conclusion. There will be times in many kinds of investigation in which it will be unclear or debatable what the evidence actually shows; this is not on its own a good reason to cut short the investigation. Thus the Absence Maxim could perhaps be regarded, and sometimes seems to be used, as a non-stopping rule for inquiry -- we should not stop inquiry (even temporarily) on nothing more than the ground that we have found no evidence yet. This raises the question of how to assess the thoroughness of an inquiry. It also raises the question of the exact relation between evidence and warrant to draw a conclusion.

(3) In general, if one models inquiry as a search, it would seem to make very little sense to deny the Absence Maxim for search in general; one has to justify a negation-as-failure rule, by establishing that one's search would have picked up the evidence for something if it were there. Thus, someone who claimed that there was no extraterrestrial life because they didn't see any evidence of it when they glanced at the sky has not done the kind of search appropriate to the conclusion they are drawing. And this seems to hold even if we say that glancing at the sky would be at least a minor part of a genuinely appropriate search.

Searches are organized for different ends and can be very different in character; you cannot assume that every search is a negation-as-failure kind of search. Thus if one models inquiry as a search, there seems some reason to take the Absence Maxim to be true as a default -- one has to set up a search of the right sort to get a case in which one can conclude No X from No Evidence of X. (And note that if you relativize Bayesianism to a given kind of inquiry -- so your probability measures are always only relative to a given inquiry -- then this at least qualifies any rejection of the Absence Maxim on Bayesian grounds.) In general, if one treats the nature of the inquiry in which the evidence is being used as important to the evaluation of evidence, this seems to favor something like the Absence Maxim.

I think there does seem to be a divide in approaches to evidence, one that has not been adequately examined -- namely, between inquiry-focused and general-measure-focused accounts of evidence. There is a lot of variation within the two approaches, but the former tends to see evidence as something constructed or formed in light of an inquiry that is limited by particular ends, tends to be favorable to the idea that what counts as evidence in one field of inquiry may not always do so in another, and tends to be pluralist about how evidence supports claims (i.e., it tends not to assume that all evidential support is of the same kind), whereas the the latter tends to reject all of these things. These differences can obviously lead to some significant oppositions in how evidence is evaluated.