Undivided Looking

I.  The Setup

On my post Black Swans, I received the following question from St. "naclhv", who is also a physicist... and a Christian... and a blogger who has discussed Bayesian arguments that the Resurrection of Jesus is highly probable!  So that's a fair amount of commonality... and yet there are also some differences, as we shall see!  I had said:

First of all, I should say you should be VERY SUSPICIOUS of any person who starts their argument by making concessions that huge to the other side. Factors of $10^{297}$ are ridiculous numbers that should never be thrown around in almost any real life situations, and if he concedes something that ridiculous to his opponent, he ought to be guaranteed to lose, plain and simple.  He's like a stage magician who makes a big show of how he's blindfolded and his hands are tied behind his back and so on.  You can be very sure there's a trick somewhere, and that all that patter is there to distract you from the way he actually does the trick.

(The other guy, St. Calum Miller, is also making a fallacy, when he quotes a likelihood factor of $10^{43}$ for the Resurrection; this number incorrectly assumes that the evidence from each apostle's testimony counts independently.  The odds of a group conspiracy to lie are certainly bigger than $10^{-43}$, which is an astronomically tiny number.  No real historical event is ever that certain.  That being said, he's right that the evidence for the Resurrection is extremely strong, as far as historical evidence goes!  It's just that nothing in life is really that certain.)

naclhv responded:

Hey Aron,

Long time lurker here. I love your site and the work you do. I would have stayed lurking longer, but I decided to comment because I happen to be writing my own argument for the resurrection over on my blog (http://www.naclhv.com/2016/03/bayesian-evaluation-for-likelihood-of.html).

Specifically, I'm also getting likelihood ratios around $10^{43}$ from my own calculations, and I thought they were quite reasonable - very conservative, even. So I thought that I'd run that value by you again, as someone whose opinion I highly value.

[some parallels to physics and history which I will quote in a later section...]

So, I'd love to get your feedback on this way of thinking about probabilities. It forms an important part of my argument for the resurrection, and I'm always looking to refine my ways of thinking.

Thanks in advance for your reply, and thanks again for the work you do here!

You're welcome!

So I read his blog series, which turns out to be quite long, and still continuing.  (This response will also be quite long.)  I find it hard to read long blog series without an outline of where I'm supposed to be in the argument, so I've broken it into some major sections so you can decide for yourself how much you want to read.  Fortunately much of what I want to talk about is in the first four posts:

1. The Main Argument
1 2 3 4

2 Considering possible objections
5 6 7 8 9 10 11 12

3. more examples for calibrating based on testimony
13 14 15 16

4. comparison to other claimed resurrection events
17 18 19 20 21 22 23 + more to come

As requested, I will now provide some friendly fire, against my own side of the argument.  But there's plenty of good stuff in there which I won't be addressing.

II.  Is an individual testimony worth 8 orders of magnitude?

First though, a commendation.  One of the major strengths of this series is that, instead of simply guessing how much evidence a single "seemingly earnest, sincere, personal testimony" is worth, he actually tries to explicitly estimate it using a variety of real-life examples (some of them are thought experiments, while others are taken from his own life, or the news, or gambling situations, and other such situations).

(If you want to decide for yourself how you'd evaluate these decisions, without being tainted by his own suggestions, you should read his first post before proceeding.)

The second post is an interlude in which, for no particularly good reason, he spots the skeptic an enormously tiny prior probability for the Resurrection, namely 22 orders of magnitude: $10^{-22}$.  This is, of course, just showmanship—the exact same thing I chewed out Dr. Robert Cavin for at the very top of this post, albeit more modestly—because the goal is to show that the evidence for the Resurrection is powerful enough to overcome even this handicap.  Well I don't think it is, as we shall see below.  If tomorrow I learned a new fact that was $10^{22}$ more likely to occur if Christianity were false, then if it were true, I'm pretty sure I would deconvert.  I think it's not possible for controversial historical judgements to be that powerful... I intend to explain why below.

In the 3rd post he writes:

Let's use my personal answers, given below, as an example for how to do these calculations. These are my gut answers to the questions, before doing an actual probability calculations. Remember, these are the events that I'm willing to give even odds (50/50 chance) on, based solely on an earnest, personal testimony. It does not mean that I'm willing to believe 100%, and it does not mean that I'd stop looking for more evidence. It only points to how much I'm willing to adjust my beliefs based on someone saying "yes, I know it's unlikely, but it really happened".

For the shared birthday question, I would easily believe that my friend shared a birthday with me. I would also not have any real problem believing that our mothers also shared birthdays. At three people - myself, mother, and father - I would start becoming skeptical, but would probably give my friend the benefit of doubt. Starting with four shared birthdays in the family, I would start leaning more heavily towards skepticism.

On winning the lottery, I would not really doubt that my friend won the lottery. I would start doubting if he says that he won two consecutive lotteries.

On getting a royal flush, I think I could almost believe that my friend got two such hands in a very lucky night at the table. I feel like three would be entering the realm of the fantastical, and I would doubt my friend at around this number.

On pocket aces, I would be willing to believe that my friend had up to four or five pocket aces in a lucky night of Hold'em.

On the multiple births, I would not have any real problems believing that someone was a part of quadruplets. A claim to be in a quintuplet would start to cause a little bit of doubt to me, and a claim of sextuplets would need additional evidence.

On being struck by lightning, I actually had someone around me claim that this had recently happened to her. I had no problem believing it. Even if she had claimed two such accidents I don't think I would have really doubted her. If she had claimed three, I would start to be skeptical.

Now, calculating the numerical probability values for all these things is pretty straightforward:

[He goes on to calculate and gets numbers approximately equal to $10^{-8}$]

(In the fourth post, he calculates the testimony of the disciples as being worth a whopping 54 orders of magnitude, but I will hold off on criticizing this number until later.)

There is room to criticize some of the specific examples here.  Maybe I'm just cynical, but I don't think I would believe an acquaintance who claimed to have gotten two royal flushes in the same sitting of poker!

And I also don't think he's right to say that, if someone were to lie on LinkedIn about having a Ph.D. from Harvard, "there is not much concrete negative consequences for lying, while the incentive of getting a job or a business contact can be quite appealing".  There's little point in lying on LinkedIn unless you plan to sustain the lie for your next employer.  But doing that is very high risk, since it's an easily checked fact, and getting caught would result in you getting fired and maybe blacklisted.

But this is quibbling around the edges with the exact numbers.  I think there's a really important point here, namely that sometimes human testimony can really be surprisingly powerful in its effects.

To make my own example, if somebody on a college campus told me, in a nonjocular way, that they'd just seen a building that was on fire, I would think they were probably telling me the truth, even if I was indoors and couldn't check to see if there was smoke.  Even if they looked drunk or disreputable, so long as I had no specific reason to think they were lying, I would certainly entertain the possibility that they were telling the truth.  But, the odds that any given building is on fire at any moment is very small.  If we suppose that a campus has at most one visible building fire (on average) every few years, and that the fire lasts for an hour before being contained, that's a prior odds of at least 1:25,000, brought up to around parity by a not-particularly reliable seeming source.  One could bump the prior odds still lower by adding on some extra details (e.g. somebody jumped out of a window into one of those nets that looks like a trampoline), so long as the extras didn't seem too implausible to be believed.  So I agree that testimony can do a lot!

But I don't think I would interpret this fact in exactly the same way naclhv does.  Suppose it were really true that, in general, "seemingly earnest, sincere, personal testimony" is false only 1 in $10^{8}$ times.  We can check this by asking how many times in my life have I been lied to?

Now except for pathological liars, people seldom lie about inconsequential facts that they have no emotional stake in; they may lie about trivial matters that make them look bad, but not when you simply ask them the time of day.  Let's instead ask how often people lie about matters of emotional significance.  Things that meet this threshold probably don't come up more than about 10 times a day.  Multiply by about 300 days in a year, and 30 years of life, that's probably about 100,000 situations in my life when somebody has been tempted to lie to me.  If the odds of them lying to me were really $10^{-8}$, then that means I might expect to live to be a thousand times my age before somebody would lie to me once. 

Maybe that's is a little unfair because naclhv does specify that the testimony must be "seemingly earnest, sincere, personal testimony", whereas a lot of lies are insincere, easily detectable, or the person backs down immediately when confronted, etc.  But even that sort of really serious lie, surely has happened several times to any of us!  (And there are fewer opportunities for people to make them, too.)  So I think the point stands that the general honesty of human beings ain't $10^{-8}$, or anywhere close to it.

So this raises an apparent conflict with the examples naclhv provides, some of which seem fairly reasonable.  I think the resolution of this paradox requires noticing another important principle, which can be illustrated as follows:

Suppose someone tells you that their license plate number is 4ZIW623.  Discounting the possibilities of a vanity plate, them not owning any vehicles etc. the prior odds of this are $10^{-4} \times 26^{-3} = 5.7 \times 10^{-9}$.  But more likely than not, they are telling the truth.  Why?  It is emphatically NOT because the odds of them lying about their license plate number are that low.  Instead, it is for this reason: even if they chose to lie, they would have no particular reason to pick that particular plate.  If they randomly make up a license plate, the odds of getting that specific one are also $5.7 \times 10^{-9}$, so those two large factors cancel out.  You're just left with your gut feeling about how likely a lie was (say 1 in 100).  That's why you should be more suspicious if they say their plate was (e.g.) 1DVL666.  The odds of getting that plate by chance are the same (assuming your DMV doesn't throw it out for looking devilish), but the odds of somebody thinking it's funny to lie about having that plate are substantially larger because it's not randomly selected; it's special.

This has a number of implications for evaluating human honesty.

One is that weird things happen all the time, and we tend to talk about them because they are more interesting then all the non-weird things that happen to us.  So if somebody says they got a royal flush in poker, that's the particular weird thing that happened to them.  If it hadn't happened, and instead they'd had an affair with a Soviet spy, they'd talk about that instead.  1-in-a-million things happen to a lot more often than 1-in-a-million people, because every day we do a thousand different things where an interesting thing might happen.

So, supposing it's really true that a typical piece of testimony is worth 8 orders of magnitude, I'm guessing about 6 of those orders of magnitude are due to the license plate effect, while only about 2 of them are due to people being reluctant to lie.  At least 1% of the things you hear are lies, but the 99% that is true is nonrandomly selected from the most interesting things that have happened to a person, so even the stories whose prior odds are 1 in 10^{-8} are still true most of the time.  But you shouldn't believe that even a plausible ordinary fact some schmoe tells you is 99.999999% likely to be true, as you would if you naively slapped 8 orders of magnitude on a 1:1 odds proposition.

This means, that if somebody claims to have gotten two royal flushes in one sitting, that's a lot more improbable than what you'd expect from simply squaring one royal flush.  Because getting one royal flush is just one of a gazillion different noteworthy things that might happen to a person, but getting two in one day is relevantly special, like the numbers matching on a license plate.  A liar can add on an extra royal flush with barely more trouble than it took to lie the first one, but a truth-teller had to be just that lucky.

In other words, if I'm right about the 8 = 6 + 2 split, you can only discount that 6 once.  Any additional improbability of the same sort, is on your own head.

So, a sufficiently implausible story is indeed more likely to be a lie than the truth.  But, the implausibility has to arise from some inherently improbable aspect of the story, which would be more likely to be invented by a liar than it is to really happen.  Merely adding additional details, more information ("and it turned out he was really named Aleksey Smirnov and was dropping off the secrets to a man who drove up in a green car..."), lowers the prior probability, but it doesn't matter to whether you should believe them because of the license plate effect.  (Of course the details do matter if they seem to involve corroborating or suspicious aspects, but the mere presence of lots of detail isn't the crucial thing.)  So this is a magical aspect of testimony, that it can cancel out any amount of low prior probability so long as it's merely due to there being large amount of detail, instead of something intrinsically unlikely happening.

(Of course, with a sufficiently large amount of detail, the odds are good that the person would make at least one mistake of perception recall.  But I am talking about evaluating the odds that the testimony is substantially true, not the odds that it is absolutely inerrant.  Minor mistakes and discrepancies are not to the point here.)

III.  What happens when we stack up multiple testimonies?

This also shows the wisdom of the biblical rule that a person should only be found guilty of a crime on the testimony of at least 2 witnesses.  (Still more or less true in Scots law, although the rule has been adapted to modernity by saying that the witnesses need not be human beings, one of them could be a DNA test or something.)  1 witness can just make up whatever details, but if 2 witnesses agree on the same highly specific thing (the more specific, the better), the probability of all those details being false is infinitesimal unless the witnesses aren't independent.  (For example, if there was a conspiracy to perjure themselves).

Informally, it might seem like this means that 2 witnesses can be more than twice as good as one witness.  That's not really the way the math works though.  What's really happening technically from a Bayesian point of view, is that most of the first witness testimony was used up fighting against the low prior probability of the specific claim (see the "prosecutors fallacy"), leaving the second witness testimony free to provide lots of extra gravy on top!

But what if we keep on stacking on more and more witnesses?  Does each one of them produce an additional new factor of $10^{8}$?  No, no, no!  First of all, as I argued in the previous section, I think $10^{8}$ is already too high for evaluating a single witness.  The odds of getting a liar are at least 1 in 100, for the reasons I said above.  Secondly, conspiracies between multiple people do happen.  (As well as other forms of nonindependence, for example someone being influenced by another person's recollection.)

Suppose that, to the best of our ability to tell, based on the factual details of situation, it looks like the witnesses are all more-or-less independent.  Can we then multiply out all the numbers to get a tiny probability of them lying?  (Say, $10^{-54}$, as naclhv claims for the various disciples mentioned in 1 Cor 15.)

Absolutely not.  Because it is always possible you are wrong about the factual details of the situation, and the witnesses are not in fact independent.  How would we go about evaluating the probability of this?  Well, to do proper Bayesian reasoning, you have to think about all the possible scenarios, and assign each one of them a prior probability.  You aren't supposed to assign anything a 0 probability, unless it really is absolutely impossible, nor are you supposed to make it really really tiny without good reason.  So, the probability that the witnesses are not independent should always be assigned some not-gigantically-tiny probability.

Now, consider 2 rival scenarios, one in which $N$ witnesses are e.g. independent and lying, and the other where there is a gigantic conspiracy to lie.  Is it not clear, that, as $N$ gets bigger and bigger, the probability of the second scenario will always exceed the probability of the first?  The plausibility of the independence scenario falls off exponentially with the number of witnesses.  While the plausibility of the conspiracy always remains at a reasonably small (but not too small) tiny value.  Since larger conspiracies are harder to hold together than smaller ones, a big conspiracy is going to be somewhat—perhaps even rather—less likely than a small one, but at least it doesn't fall off at a steep exponential slope, as a function of $N$.

One can generalize this argument further.  Any time you've successfully argued that some hypothesis which uses independence has a likelihood of $10^{-54}$, this pretty much guarantees that any hypothesis which does not assume independence is going to do better.  Unless you think the argument for their independence is itself a 54-orders of magnitude slam dunk, but that just pushes the question back to how one could be so sure of that question.

It's absolutely fine, as a rhetorical technique, to try to show that a viewpoint is implausible by showing that all of the most obvious ways for it to be true would involve the conjunction of several improbable events occurring.  But if one actually multiplies out the numbers, one should not take the final answer too seriously—because the most likely way for you to be wrong, is always going to be that you were in error to multiply out those large numbers in the first place, due to some breakdown of your model (including, but not limited to, failures of independence).

IV.  Why we should not be fantastically certain about almost anything

Here are a couple highly relevant blog posts on the subject, by an expert in reasoning I highly respect, who blogs by the pseudonyms Scott Alexander / Yvain (unfortunately not yet a Christian).  The first is about not taking arguments completely seriously when they lead to hugely confident predictions:

Confidence Levels Inside and Outside an Argument

The second one is about a super-Artificial-Intelligence (AI) taking over the world in the near future.  I don't take this hypothesis anywhere near as seriously as the community of Less Wrong rationalists does, but I have to agree with him that it's way more likely to matter than $10^{-67}$.  But you can take this as a general parable about a broader issue:

On Overconfidence

So, when you are evaluating the odds of e.g. the disciples claiming to have seen Jesus risen from the dead, the scenario to worry about is always going to be the one where the disciples are not independent, possibly for some reason that didn't fully make it into the historical record.  So when naclhv says that:

Incidentally, if you thought that I forgot to adjust my calculations for the fact that the testimonies are not independent, this is why - the three named witnesses in my argument ARE largely independent; they come from very different backgrounds and met the risen Christ under different circumstances. Especially in Paul's case, if anything you'd expect his testimony to be anti-correlated with Peter's. For the other witnesses where dependency is expected, I explicitly called it out and severely discounted the Bayes' factor values in the calculation.

for the reasons stated above it's hard to imagine that any three witnesses could ever be "largely independent" for purposes of multiplying many tiny probabilities.  Because the "error" due to them maybe not being independent is always going to swamp the situation where they are.

They may still be "largely independent" in the sense that postulating a common conspiracy requires making some improbable background assumption.  But, in that case you only pay the price of that background assumption (assuming that is more probable than multiplying out all the numbers on the assumption of independence).

V.  A similar issue with the McGrews

naclhv isn't the only smart person to make this mistake.  In an otherwise very fine article on the evidence for the Resurrection, Sts. Tim and Lydia McGrew claim a Bayes factor of around $10^{44}$ for the Resurrection, coming largely from the assumption that the testimony of the Twelve Disciples should be independent of each other (together with smaller additional boosts from the women, St. James, & St. Paul).

They then consider the possibility that the disciples were not independent, explaining that:

But when probabilistic independence of testimonial evidence fails, it need not fail in the way sketched above.  Probabilistic relevance can be either positive or negative... [some math follows]

This general statement about probability theory is correct.  But it is not really relevant, once you start claiming that something is really, really implausible.  Suppose that you aren't sure whether the failure of independence is going to be in a positive positive or negative.  In fact it depends on your background assumptions.  (And in a good Bayesian calculation, you should never really allow yourself to be 100% certain of anything.)

Suppose, just for the sake of argument, we granted to them a 99% chance to Scenario X, where the disciples' testimony would be negatively correlated (or else independent), and only a 1% chance to Scenario Y, where it is positively correlated.  Well, X gets killed by a huge factor of (according to them) $> 10^{44}$, while the latter gets beaten down by a much smaller factor (since the disciples testimony is now positively correlated).  So Y is always going to win!  (Even if the final result for Y is damped by the 1% factor, that's nothing compared to $10^{-44}$!)

They go on to articulate a particular reason to believe that some of the disciples' testimonies might be negatively correlated instead of positively correlated:

If A dies (especially in some unpleasant way) for his testimony to the risen Christ and B hears about it – and there is no serious doubt that the apostles knew when one of their number was put to death – does this make B more likely to stand firm until death in his own testimony? It seems to us that the opposite is true, that knowing of such a death is plausibly and under ordinary circumstances negatively relevant to B’s willingness to remain steadfast. B may well be frightened by the fate of A and drop his claims. In this case, treating A’s and B’s deaths for their testimony – their “martyrdoms” in the original sense of the term “martyr” as “witness” – as probabilistically independent actually understates the case for R.

This correctly identifies a possible mechanism, by which, given certain background assumptions, one disciple's false testimony might make another's (continued) false testimony less likely.

Personally I don't think that this is a more important effect than the sort of obvious social fact that people tend to imitate their friends' behaviors even when those behaviors are self-destructive.  (Consider how gang members react to the death of a gang leader.)

But it doesn't really matter much whether the failure of independence is more likely to be positive or negative.  So long as somebody can articulate any scenario in which the disciple's testimony was positively correlated, that is the scenario to worry about.  (So long as it doesn't also involve implausibilities worth many orders of magnitude, but it's hard to get there without multiplying a bunch of small numbers, and the whole point of these scenarios is that they try to avoid these things...)

Hence, the McGrews analysis provides an overestimate of how likely the Resurrection is.  That doesn't mean there aren't some strong historical arguments in their paper.  But the mathematical statements are hyperbolic and need to be discounted.

VI.  Are alternatives already factored in?

In a later post, naclhv fights against the possibility of alternative analyses here.  After mentioning some specific whacko conspiracy / delusion theories of the usual sort that people bring out to explain the Resurrection—and quite correctly saying that they not are well supported by any of the data that we actually have—he goes on thus:

First, note how weak this argument is, even if we were to grant it everything that it asked for. Remember, the odds for the resurrection are currently at 1e32, so the odds against it are therefore at 1e-32. Now, we'll allow for each independent objection to count as having the full weight of these odds. Never mind that many of these objections contradict one another and therefore reduce the probabilities of the other objections (increasing the probability for 'insanity' decreases the probability for 'conspiracy', because a conspiracy is less likely to succeed with insane people in it). We'll just ignore that. Never mind also that these complex speculations are naturally less likely because of their complexity. We'll also ignore that as well. So, if we can think of a hundred such objections, each of which carries the full weight of the 1e-32 odds for 'no resurrection', the final odds for the resurrection would drop all the way down to... 1e30

Let me first extract a correct and important point from this paragraph.  One doesn't really get a lot of mileage from simply coming up with large number of fantastically improbable anti-Resurrection scenarios.  For example, the Swoon Theory, the Identical Twin Theory, the Hallucinatory Drugs Theory etc.  For if it is true that each theory contains some individually highly unlikely coincidence (even a 1-in-a-million event) then simply coming up with a hundred or so different theories doesn't get you out of the hole.

But, the skeptic does get some mileage out of suggesting scenarios in which independence of the disciples breaks down, for the reasons explained in the previous section.  naclhv goes on to argue:

But more importantly, this kind of objection is simply, fundamentally wrong: it would not fly in any other investigation into a personal testimony, because it completely ignores the rules about how we evaluate evidence in a Bayesian framework.

Imagine, for instance, that your friend claims to have been struck by lightning. You've taken stock of this claim and have decided to assign it a Bayes' factor of 1e8. But then you say, "well, you may be just a little crazy. And you might have had a nightmare about a thunderstorm last night. Then you might have gone to a hypnotist and who had you recall your dream, which you're now confusing with reality. Or maybe it was the hypnotist who planted the suggestion in your mind first and that caused your nightmare. Really, it might have been any of these things - and isn't it more likely that at least one of these possibilities is true, rather than for you to have been actually struck by lightning?"

Should you or your friend then discount the previously assigned Bayes' factor in light of these new possibilities? Absolutely not. The thing to note here is that the Bayes' factor ALREADY includes all of the ways that this claim may be wrong. It is the numerical estimation of the weight of evidence for a human testimony, and as such already inherently includes the possibility that the evidence may be misleading.

Having established its value, it is simply incorrect to further modify it with no evidence, based on enumerating possibilities that were already included in its evaluation. Your friend's proper reply to your wild speculation would be to say, "what makes you think that I had visited a hypnotist or had a nightmare? Of course, anyone might be wrong about anything in any number of ways - but don't you already know how much you trust me? How does a list of ways that I might be wrong, with no evidence behind any of it, make you trust me less?"

That is quite true and correct for evaluating a single witness, if we have already calibrated the probability of error using everyday examples, as naclhv has attempted to do.

But it does not apply to hypothesis in which independence of multiple eyewitnesses breaks down, because the effects of those scenarios have not already been taken into account.

VII.  On tiny probabilities in physics

You mention that numbers like $10^{43}$ or $10^{297}$ are ridiculously large and should not be taken seriously, especially in historical settings. I would, in general, agree with you - but there are exceptions to this rule in some kinds of math, and probabilities is one area where such numbers are not uncommon. Here's how I'm thinking about this:

Let me give some examples from probabilities inherent in everyday objects. The probability of shuffling a deck of cards to a specific order is about $10^{68}$. The probability of recreating a game of chess through random play is about $10^{120}$.

Even in physics, $10^{43}$ would be a ridiculously large number if we were talking about something like time (is that in seconds or years? It doesn't matter - it's basically "forever"). But in the branch of physics that deals with probabilities - that is, in statistical mechanics, $10^{43}$ is nothing.

For example, the standard molar entropy of water vapor is 188.8 J/K/mol. So the number of microstates for a mole of water vapor at standard conditions is $e^{(188.8/k_{boltzmann})}$ - that is, about $10^{(10^{25})}$. Lest anyone think that this is so large only because we're talking about one mole of something, even if we take the moleth root of this number we still get about $10^{10}$ - so, even just five molecules of water vapor will have something like $10^{50}$ microstates.

The trouble with these examples is that they are all conditional statements of the form:

• If model M is correct (where independence holds) the probability of event E is tiny.

where the model M is a truly random shuffle, or the statistical mechanics of water, or whatever.  But that does not mean that the probability of an actual shuffle to result in a given configuration is that low.  The cards might be being "shuffled" by a card sharp like Scarne!

Similarly if all the air molecules go to one corner of the room, that would mean there's some natural (or supernatural) effect we didn't take into account.  It would not mean that a $10^{-(10^{25})}$ event just happened.

In other words, the model M could always be false.

Also, you mentioned that probability values like 98% are actually not at all extreme. I also think that as well. But the five sigma probability of about $10^{-6}$ is also not all that extreme - it corresponds to something that we're barely certain enough to publish on, at the cutting edge of science.

That's what we do in particle physics, anyway.  But in the soft sciences, they publish at 2 sigma which is why you can't trust anything you read in science news about people.  :-)

However, the 5 sigma rule $= 3.5 \times 10^{-6}$ doesn't actually mean that the odds of being wrong are less than one in a million.  The reason why particle physicists adopted that rule is that, when they used 3 or 4 sigma, they kept getting false alarms!  There seems to have been a recent example of this at the LHC.  This makes it clear that it's an overreaction to guard against biases that weren't taken into account.

One possible source of bias is the Look Elsewhere Effect, where there are a large number of possible theories that you could have checked for, and you just notice the thing that happens to look anomalous.  In Bayesian terms, this is closely related to the fact that theories which predict specific new particles and forces have low prior probabilities.  Finally, there's good old systematic error, the bane of experimentalists everywhere.

So really the 5 sigma rule is just a kludge, which exists precisely because things are never quite as sure as they appear to be, so you need to up the standards a little.

Several independent verification at the $10^{-6}$ level would easily bring the overall probability to something like $10^{-43}$, and any well-established scientific laws would easily break $10^{-100}$, by a large margin.

Assuming complete independence, yes.  But systematic error is not independent, nor is failure to properly consider alternative explanations, nor group-think bias, nor grand scientific conspiracies to mislead the public, nor malicious spirits playing jokes on us, etc.

So, even in history, I can easily imagine a statement like "The Roman Empire existed" having an odds of $10^{300}$ for being true. Basically, my rule of thumb is that probabilities or odds are not "too large" unless their logs are "too large". This makes sense, given the multiplicative nature of probability.

Same as above.

VIII.  Back to Jesus and the Resurrection

So where does this leave probability arguments for the Resurrection?  I made my own attempt to do a probability calculation in these posts:

Let us Calculate
Christianity is True

For the moment let's ignore the philosophical stuff about the argument from evil and fine-tuning, which maybe could also used to be ramped down a bit, and let's discuss the historical stuff.

Well, I still think that all of the basic component arguments here are good.  That is, it's still true that there's good reasons to believe each of the following is true:

a) Jesus was a very special person, apart from the unusual circumstances after his death

b) a few days later his tomb was empty

c) many of his disciples claimed to have seen him alive, including both women (the first eyewitnesses), the full group of 11 remaining apostles, St. James the brother of Jesus, and others.  [Consider "as read" the standard arguments about the testimony of women not being highly regarded among 1st century Jews, and at least some key witnesses being martyred for their faith.]

d) that some highly unusual vision/phenomenon—according to Acts it was noticeable to others and caused him temporary blindness, but even if we consider this to be an exaggeration, it seems likely to have been at least an epileptic fit of some kind—caused St. Paul, an enemy of Christianity, to convert and become a zealous missionary (and eventually get executed himself).

I originally said that (a), taken by itself, roughly cancels out a factor which is basically the Look Elsewhere Effect (discussed in section VII).

I also said that (b,c), taken by themselves, amounts to about 8 orders of magnitude (from many witnesses) and I'm prepared to stand by that given the weirdness of the situation.  Bear in mind that since tens of billions of people have died in historical times, a mere $10^{-8}$-level coincidence following somebody's death should still have happened at least a hundred times in history.  For the kinds of skeptical reasons I stated above, it would be hard to get this much above $10^{11}$ by itself since then we run out of the ability to check how many potential parallels there are.

Finally, (d) taken by itself, is at least a 1-in-a-million event and I stand by that.  I'm pretty sure there are not 40,000 non-Christians alive today who have had similarly dramatic conversion visions leading them to become zealous for a religion they previously disliked.  (It would be circular to count the Christians here, since if we're right God still does dramatic things to convert some people.)  Maybe we need to shave off a factor of 10, because of the existence of multiple possible persecutors in early Christianity whose conversions would have been equally dramatic (e.g. Caiaphas).

Now, under some fairly reasonable background assumptions, if we trust the New Testament texts even a little bit, some of these assumptions seem at least partially independent of the others.  (For example, even very skeptical scholars agree we have at least some information about Jesus' teachings prior to his death.  And that Paul was originally a persecutor of Christians—and therefore not likely to have welcomed his experience—we have from his undisputed letters, as a testimony against his own current interests.)

But, clearly the right approach for a skeptical attack, the only one that has a hope of success (other than an almost complete skepticism towards the texts which I really don't think is justified), will be to attack the independence of these events.  And there are some ways of doing this that probably do shave off several orders of magnitude.  I just don't feel like they are strong enough to explain all of the data.

For example, it probably IS true that if an unimportant rabbi seemingly rose from the dead due to a coincidence, that people would make up a bunch of stories about him and maybe put some words in his mouth.  But I don't think such an invented composite would end up being plausibly the most insightful and challenging moral thinker the world has ever known.  (And I don't think this is that subjective of a criterion.  The vast majority of people wouldn't pass the "laugh test" for that position.)  Nor would I expect multiple early detailed texts along the lines of the Gospels.

Going in the other direction, if a charismatic religious leader made grandiose claims about his own identity, I quite agree that it makes it more likely for his followers to report grand miracles after his death.  But I wouldn't expect it to involve quite so many coincidences as we find in the New Testament, I wouldn't expect such a large base of eyewitnesses, and I wouldn't expect the whole thing to be so well documented so early.  (Whereas legends that develop over centuries, that can happen to anybody.)

Finally, crankish people converting to a false religion is commonplace, but it's more surprising when one of your biggest persecutors has a vision of Jesus and goes blind until someone comes to baptize him, and it's also a bit surprising when he then goes around doing miracles, all of this described in a text (Acts) which to all appearances looks like a careful historiography, in parts styled very like a personal memoir by a close companion.   (Of course St. Paul's conversion is also mentioned in his own letters, I mean the ones that even anti-Christian scholars think were really written by him.)  You can, of course, say he was a sincere fanatic who (overcome by guilt for his persecution) confabulated multiple miracles, but that still leaves him more or less separate from the others.  To really undercut the independence from (b, c) you have to say he was a plant, or that he was a fraud who made up most of the other disciples' testimonies, but any of these tactics is an uphill battle for various reasons.

So, if you disbelieve the New Testament accounts of the Resurrection you can and should deny the independence of these pieces of evidence.  It's just, you have to pay a price for doing so.  I still think the most parsimonious explanation would be that a large group of people deliberately and intentionally conspired to make up the whole thing.  It seems more likely than the other naturalistic explanations, it's just not all that likely.

But because naclhv invited me to critique his argument, I'm going to be merciless and observe that he oversteps again when he says this:

Let me reiterate and clarify that, because it's important. There is an utter lack of evidence for disbelieving the resurrection: literally every single record we have from the people who were actually connected to the event to any reasonable degree ALL portray the resurrection as something that actually happened.

If you believe in the resurrection, you have the unanimous support of all the people who were actually close to the event and would know for certain. If you disbelieve the resurrection, literally every piece of evidence - every single testimony of every single person who ever testified about the actual event - is against you.

He has forgotten an important class of witnesses against the Resurrection, namely the guards at the tomb.  St. Matthew's Gospel tells us quite frankly that:

While the women were on their way, some of the guards went into the city and reported to the chief priests everything that had happened.  When the chief priests had met with the elders and devised a plan, they gave the soldiers a large sum of money, telling them, “You are to say, ‘His disciples came during the night and stole him away while we were asleep.’ If this report gets to the governor, we will satisfy him and keep you out of trouble.”   So the soldiers took the money and did as they were instructed. And this story has been widely circulated among the Jews to this very day.

Of course we owe this account to a Christian, but it is hard to imagine anyone would write these words unless either (1) the guards really did report that somebody had stolen the body, or at least (2) some of the Jews claimed that the guards had said this.  Now people do not usually make up, entirely out of whole cloth, arguments against their own position to respond to.  Maybe they unfairly caricature them as strawmen, but usually they are responding to real people.  So it seems historically very probable that there was in fact some kind of anti-Resurrection testimony to this effect.

It is a separate question whether this anti-Resurrection testimony, as we have it, is at all plausible.  It does nicely undercut the independence of (b) and (c) by postulating that the nefarious disciples conspired to produce both effects, even if their motivations at this stage would be obscure.  But, we can expect that the guards would have been severely punished for sleeping on duty, especially if all of them slept at once.  (This would be true for a Jewish guard, but even more true for a Roman one where the punishment would be execution.  Since Pilate's words in the Gospel were "you have a guard": it is unclear whether he was providing a guard or observing they already had one.)  And, if there was in fact a heavy stone and a seal, it would have been quite challenging to move it without wakening anyone.  And, if the guards were really asleep, how could they possibly know who had stolen the body?

Their testimony may even ultimately favor Christianity, since it's existence helps confirm that there was a guard, which makes the empty tomb a lot more impressive.  But, it is false to say that no one was claiming the Resurrection hadn't happened.  The guard—and apparently the Jewish leaders that allegedly bribed them—were putting forth a different story.  But for some reason, these days even the skeptics prefer to tell other tales.

So where does this leave us?  I'm reluctant to slap a number on this now, because earlier I concluded that, if you're really sure something is true, inevitably the best possible skeptical hypothesis is always going to be the thing you didn't think of, something that undermines all of your assumptions.  This means, the more and more sure we get, the harder it is to even calculate just how sure we should be.  But, we should not be too sure.

Leaving aside truly awful skeptical scenarios, like we're all in brains in the Matrix being toyed with, surely we can be pretty darn sure that e.g. Julius Caesar was assassinated.  As I have argued before, the evidence for Christ's Resurrection is almost as strong.  But, very tentatively, it seems reasonable to maybe put a cap on how sure we can be of any particular historical event, maybe 99.99% tops for the final answer, to something we've carefully investigated that seems to require an unlikely "conspiracy" to explain away.  Unless it's something really basic like "The Roman Empire existed", where we should be able to go a bit further.  (Part of me feels a bit dirty assigning some historical conspiracy theories a probability of more than 1 in a million, and maybe that's correct, I'm really not sure where the threshold should be.)

This is just a kludge, until somebody figures out a way of assigning a number to "failures of independence in ways that you haven't even thought of yet".  But, this is good enough for now.  It seems to me one can still be highly confident, on the basis of historical data, that Jesus rose from the dead.  Just not quite as confident as naclhv and the McGrews claim you can be.

(Of course, a complete analysis would have to include all the rest of the evidence from philosophy, experience, etc.  aside from the immediate historical data for the Resurrection.)

IX.  Epilogue

Some people might wonder why I'm spending time criticizing an argument for my own religion, saying that it is too strong.  Most people spend their time arguing against things they don't believe in.

Well, I'm not most people.  I'm hoping to do something a little more unusual, which is trying to follow the truth wherever it leads.  Superficially, it is rhetorically effective to play up the strengths of one's own argument, and the weaknesses of the other side.  Unfortunately, this can lead to a tendency towards dishonesty, ignoring the flaws on one's chosen side.

So I have a different evil plan, which is to evaluate arguments in a fair and unbiased way the way a rational person would.  You see, if I can successfully pretend to be doing that, then people on the other side will say to themselves,

"Here's this reasonable looking person, who doesn't seem biased, crazy, or stupid, and he knows about science, and yet he still thinks it's historically plausible that some dude was God's Son, and came back to life again.  Maybe there's something to it, and I should take another look."

So, there are advantages to pretending to be reasonable.  But I find that the easiest way to pretend to be reasonable, is to actually be reasonable.  And—joking aside—my first priority is to the Truth.  If Christianity is right, Jesus is the Truth, so loving Truth and loving Jesus works out to the same thing in the ultimate analysis.  But, if that weren't the case, I would want to know it, rather than living out my life based on a lie.

Other Christians might say, well what about the certainty which comes through the testimony of the Holy Spirit?  Who cares about probability theory and this historical jibber-jabber?  I kind of doubt whether anyone like that has read this far, but if you have, here's my response:  Obviously I'm not going to tell the Spirit not to bear witness to the truth in people's hearts.  And while much of the time he leaves us to our own devices, sometimes it does seems like he's bearing witness to my heart.  But, although I've had some fairly dramatic spiritual experiences, none of them are so strongly powerful that there's absolutely no chance I could be wrong about their cause.   Which is not unexpected, given that "we live by faith, not by sight" (2 Cor 5:7).

So, they also can't make me perfectly certain as a Bayesian reasoner.  But Bayes' Theorem isn't how people actually think internally.  It's just a somewhat useful model of what a hypothetical Spock-like rational entity would do.

When it comes to emotional certainty, I honestly don't think there's that big of a difference between, a calculation that says you should be 99.5% sure, and one that says you should be 99.999999999999999999% sure.  The heart doesn't really resolve that kind of difference.  Whether or not you trust in Jesus isn't really a matter of having an enormous probability, although you shouldn't do it if you don't think it's true.  It's a matter of making a decision to trust.

Once you've decided to trust, additional percentage points maybe help you sleep at night but I don't think they are all that spiritually valuable one way or another.  Emotional certainty can be spiritually valuable, if it's built up by trusting God in difficult circumstances.  As we all know, it doesn't come automatically from simply being intellectually persuaded.  That's where faith comes in.

To use a classic sermon illustration: what shows you have faith that a plane will arrive at its destination safely?  The answer is if you're willing to get on it.  One person may be trembling in fear, another may be cocksure, but whether or not you get on the plane is a yes/no question, not a continuous probability value judgement.  Maybe the first person gets on and the second doesn't.  So, you can even be a Christian even if you only think it only has a 70% chance of being true, as long as you are willing to get on the plane.  Those who do get on board usually become more sure, while those who don't often become less sure.  Which of these effects is primarily due to bias, I guess depends on who is right!

So, there are credences (i.e. probability assignments), there is the feeling of emotional confidence, and then there is trust, and none of these are exactly the same as each other, even though sometimes they are related.  What we are entitled to is just enough to get by on: "Give us this day our daily bread..."