The Tyranny of Assumptions in the “Jesus Tomb” Calculation

Well, I thought I was done writing on this for a while, but I thought I would post once more about the “math” behind the “Jesus Tomb” show.  I’m prompted by an article in last Friday’s Wall Street Journal (March 9, 2007) titled “Odds of ‘Lost Tomb’ Being Jesus’ Family Rest on Assumptions” by Carl Bialik (the WSJ’s “Numbers Guy”).  [Click here to read — I do not think subscription is required.]

I appreciate that the WSJ article brings out some things about mathematical work that often aren’t mentioned — chief among them: how dependent the mathematician is on the assumptions he makes or, to a great extent in this case, those given to him.  And I appreciate, as well, that Professor Andrey Feuerverger’s work is being seen in a better light (which means that he, himself, is hopefully being seen in a better light) than that cast by the horrible presentation of his work in the documentary.

For example, one of the key assumptions in the calculation is that the “Mariamene e Mara” tomb is Mary Magdalene.  Without that key assumption, according to Professor Feuerverger as quoted in the article, the mathematical results would be “statistically not significant.”  The assumption that “Yose” is so rare nickname for Joseph that it should be treated distinctly from the more common “Yosef” is also a key presupposition.

This is one of the places in which I thought the “docudrama” went south, and did so in a fairly contrived way on the backs of the filmmakers’ biases.  (Please note: I am not pretending to be unbiased myself!)  I recall the narrating voice of Mr. Jacobovici pointing out the crucial role identification of one of the ossuaries with Mary Magdalene played in the calculation.  Then, with an air of “Da Vinci Code” suspense, we see laid out the “case” for this identification.  Sadly, at this point in which honest scholarship should have played a key role, it was ignored and left to watch from offstage as Jacobovici, Tabor and company perform their “magic” and make the identification of “Mariamene e Mara” with Mary Magdalene seem an “air tight” conclusion.

Thankfully, many more responsible scholars have stepped forward to fill the gap, and anyone who has kept up with the matter has been able to learn what is not told on the TV program: the identification of “Mariamene e Mara” with Mary Magdalene is far from air tight.  In fact, even the translation of “Mariamene e Mara” has been debated, with a number of possibilities seeming as probable as “Mary the Master,” if not more probable — regardless of how confident Dr. Tabor sounds when he expresses his opinion.

But I’ve gotten a little off track.  In the article, Ivo Dinov, assistant professor of statistics at UCLA says, “I wouldn’t be comfortable coming up with a number like this, because the general audience will not understand that it is very, very subjective.”  And that is exactly right.  When I was working as an actuary (a mathematician for an insurance company), the audience’s interpretation of my conclusions always had to be foremost in my mind.  Often this lead to frustration between my department and, say, the marketing department, when they would want to use the numbers to say one thing, but we would have to rein them in because it was not an accurate use of the numbers.  Or, the numbers we would publish had to be explicitly qualified by an accompanying statement of the key assumptions behind them so that those assumptions could be considered by others for their reasonableness.  If others are going to be making decisions based on your numbers, then you want them to have all the info that they need to make a responsible decision.  Anyone deciding on the “truth” of Mr. Jacobovici’s proposal about this tomb based on “1-in-600” odds without understanding the elaborate set of assumptions made to get to that figure has made a horribly ill-informed decision.

In the WSJ article, Professor Feuerverger seems to recognize the deficiency in how his work was presented on the show when he says, “There is a mismatch between how the media works and how academia works.  Obviously it would have been a whole lot better if I had completed the paper [before the documentary aired].”  The “paper” in question would be his documentation of how he arrived at his result and the nature of his assumptions, which would have been reviewed and critiqued by his mathematical peers and relevant experts.  That’s the way that real science works, though none of it was on display on the Discovery Channel that evening.

I really do hope that Prof. Feuerverger is given a break by those who critique his calculation.  He seems (from what I’ve read) to be a decent fellow whose assistance to some filmmakers is earning him more notoriety than he would ever wish for.  As he says towards the article’s end: “When doing the calculation, I was naively unaware of the extent to which the filmmakers might be depending on the ultimate result of it.”  I hate to speak for him, but I would think that he probably regards this as his one mistake in this matter.

I would, however, offer that he has made one other.  One of the warnings given to us as actuaries by the Society of Actuaries was to beware the temptation to work with assumed confidence and competence outside your usual bounds of experience.  It seems to me that an error related to this warning has come into play here.

The Wall Street Journal article briefly references another critique of the calculation on the website of Scientific American, which I suspect is the article here.  As reported in the SA article, emphasis mine:

“I did permit the number one in 600 to be used in the film-I’m prepared to stand behind that but on the understanding that these numbers were calculated based on assumptions that I was asked to use,” says Feuerverger. “These assumptions don’t seem unreasonable to me, but I have to remember that I’m not a biblical scholar.”

Essentially, the calculation is incredibly assumption dependent.  That is the nature of the beast in this sort of work.  Prof. Feuerverger was asked to use certain assumptions, and he checked them against his own sense of reasonableness, yet he admits that he lacks the expertise to accurately decide how reasonable they truly are.  In most truly scientific publications he could communicate that uncertainty, but it doesn’t make for “compelling television,” which — as Mr. Jacobovici said in the hour long “discussion” that aired after the show — was the filmmaker’s goal.  And how certain should we be about the assumptions behind the “1-in-600” calculation?  As Scientific American states:

…the calculations made by Feuerverger and others rest on premises that must be decided by historians and archaeologists, who are still far from agreement on even the basics of the Talpiot tomb.

However sincerely or innocently done, the filmmakers’ misuse of Prof. Feuerverger’s calculation — using it to add an unearned sense of solidity to what is, by its very nature, a house of cards — is regrettable.  And the anxiety that the professor admits he has gone through since the show aired is also regrettable.  But he says in the WSJ article that the distress will make him a stronger human being and a stronger statistician, and I hope this is the case.  And, I hope those who look into how mathematics was used in this film come away more educated about the role mathematics plays in such things.  The presence of mathematical results brings an air of authority to one’s conclusions, but hopefully we will learn that it should not be an authority that goes unquestioned or unchallenged.

Jesus Tomb Commentary – Original Draft

Howdy, all!  I hope to leave this topic, but thought I would add this last post.  I wrote an interent commentary for possible use by the office in Charlotte, but it was way too long for that format.  Hopefully the revised, shorter version I submitted will work.  But in the meantime, I thought I would post my original draft here.  In some ways I like the shorter version better — it is tighter and more succinct (read: less hastily written, perhaps?).  If it ends up being publishable, you can compare for yourself when it comes out!  Until then, though, here is the longer, rougher draft (although, I will admit, adorned with a few edits that my editorial team (i.e., moi) found too tempting not to add).
~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
Edit, 5:25pm: I saw today on the Tomorrow’s World website that the “tighter” commentary is now up.  You can read it here.  (And if you didn’t read my instructions about how to subscribe to regular commentary e-mails, click here.)  I do think I like the final version of the essay better in a number of ways (if not all), and I often find that restrictions such as word length caps really do help make for better writing.  Anyway, the original draft is still given below.
~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~

The Tomb of Jesus? 

Have you seen “The Tomb of Jesus,” a Discovery Channel presentation brought to us by filmmakers James Cameron (of Titanic fame) and Simcha Jacobovici?  If not, it is only a matter of time before you hear about it – such is the way with society’s current fascination with the “Da Vinci Code” spirit of “creative history.”

Mr. Jacobovici and his team point to a tomb discovered in 1980 and thought to be uninteresting archaeologically at the time, and claim it is the family tomb of Jesus of Nazareth.  They note, for instance, that it contained ossuaries (coffins, or “bone boxes”) inscribed with the name “Jesus son of Joseph” as well as some other names associated with Jesus’ family (e.g., “Mary” and “Jose”) – though, including, as well, names that are not associated in any other resource or history with His family (e.g., “Matthew,” claimed to be a previously unmentioned brother, and “Judas,” who is supposed to be the son of this Jesus).  What are we to make of their claim?

Well, for starters, many researchers and experts are publicly pointing out that – while good for drama and publicity – the show’s assemblage and interpretation of the facts represent bad science and pseudo-archaeology.  Even the show’s probability calculation on the names found in the tomb is highly questionable in the way in which it is applied (and I noticed this, myself, having once been a practicing mathematician whose “bread and butter” involved performing such calculations).

Loose science and slipshod “archaeology” aside, the fact is that the arguments that have demonstrated the truth of Christianity to anyone not blinded by their own pre-formed assumptions are still as strong as ever, and suffer not even a dent from this television stunt.

Imagine the scene, if you will…

A hush spreads across the court of King Agrippa as the Christian apostle Paul begins to conclude his defense, discussing the crucifixion and resurrection of Jesus Christ: “…this thing was not done in a corner.  King Agrippa, do you believe the prophets?  I know that you do believe.”

Then Agrippa says to Paul, “Very passionately delivered, Paul, but your tale is a fantasy.  The tomb of this man you say rose from the dead is there, this day, outside Jerusalem!  His bones lie along side those of his mother and the rest of his family.  Why you persist in this horrible and perverse lie to your own destruction is beyond me!  Festus is right: You are truly mad…”

However, this is NOT how it happened!  You can read how Luke, the first century church historian, recorded the actual exchange in Acts 26:22-32.  Even in the white hot heat of terrible first century persecution of the new religion, when the Jewish leaders of the day were desperate to end this “vile” and troubling sect that kept talking “nonsense” about their leader’s resurrection, no one was able to produce a body to refute the claim – let alone an entire family tomb.  No one.  No matter how passionately they must have searched for it, no matter how viciously they must have interrogated the many Christians they arrested: there was simply no body to be found to demonstrate the “lie.”

If the Jewish leaders of the time could not, with all the power and means at their disposal, discover the location of a family tomb that was located in the very area from which the heresy was spreading, then surely they are amongst the most incompetent individuals in the history of mankind.  And if the original followers of Jesus – who suffered torments unimaginable – voluntarily endured such horrific treatment and violent deaths simply because they were unwilling to admit that their profession of a risen Christ was a lie they knew to be false, then surely they are amongst the most insane individuals in the history of mankind.

Believing that scenario requires an incredible amount of faith – more faith than a bad-science-but-ripping-yarn television program deserves.  And more faith than believing in the other scenario that has been continually verified by the accumulated facts of almost 2,000 years of human history and experience: He is risen.  And He lives today.

The apostle Paul and his fellow workers in the gospel would have gone down in history as laughing stocks if Jesus’ family tomb existed and contained His bones, not as a team of men who “turned the world upside down” (Acts 17:6).  And yet, they – with their seemingly unbelievable and assumably easily disprovable “resurrection” story – did, indeed, turn the world upside down.  This fact, history does not question.

While talented filmmakers and imaginative writers, these men have given us nothing more than an interesting tale – a pinch of fact surrounded by a good fistful of storytelling. Whoever this “Yeshua” is whose tomb has been found (and many were named so in Jerusalem at the time), one day in the general resurrection he might have a laugh hearing about how his “bone box” was paraded as the “silver bullet” that shot down Christianity.  But this fellow will know then, as we can know now – the tomb was, and still is, empty.

(If you are interested in really proving the truth of the Bible, please consider requesting our free booklet The Bible: Fact or Fiction?  Don’t take anyone’s word for it – prove it for yourself!)

Whew! Finished a “Jesus Tomb” Commentary and realized…

I just finished submitting a commentary to Charlotte about the “Jesus Tomb” television program, so I have been a little tied up for a while this morning.  (If they already have a better one then hopefully they will skip mine, though I might post it here for all of you, if so.  I tried to say something different than I have in previous posts — here and here.)  If you have submitted comments to show on this blog, I apologize that you are experienceing a delay and I hope to get to them later today, as I suspect that they are building up.

However, having just written a commentary, it struck me that many of you out there may not know that you can subscribe to free e-mail commentaries from the ministers of the Living Church of God and those behind the Tomorrow’s World program.  The webpage to read for the current commentary is always http://www.tomorrowsworld.org/cgi-bin/tw/commentary/tw-comm.cgi?action=view_comm.  There is link there to a library of old commentaries, as well.

HOWEVER, what you may not know is that you can request that new commentary links be e-mailed to you automatically without having to check the website every day to see if there is a new one.  To do this, go to the main Tomorrow’s World website, itself (www.tomorrowsworld.org) and at the bottom left corner of the screen you will see a small box for entering your e-mail address and a button marked “Submit.”  This will add you to our electronic subscribers, who not only receive our frequent commentaries, but who also receive other updates, as well — chiefly our very popular “News and Prophecy” e-mails.

If you haven’t subscribed, I highly recommend it!  We do not give your e-mail address to others, and the service is completely free.  Christ said, “freely you have received, freely give” and we believe in that command.  Our work is supported by the tithes and offerings of those in the Living Church of God and others who voluntarily join hands with us because they believe in what we are doing — so we are able to offer all of our information as a public service and ask for no compensation or donations (or “love gifts” — I hate that term!).  In fact, think of this post as a Public Service Announcement. 🙂

Have a great day — and, again, later today I plan to get to those comments that I suspect are building up!

The “Tomb” of Jesus — A Few More Comments

Howdy, again!  I figure some of you (having read my February 27 post about the subject) have been expecting me to write again about this with more to say.  Unfortunately, I did not give you a time element in that last post!

I really wanted to see the Discovery Channel show, “The Tomb of Jesus,” that Simcha Jacobovici and James Cameron have given us before I wrote anything too specific.  Well, earlier tonight I (and the rest of you who might have seen it) got to do just that!  And I was pleased that after the two hour “popcorn” treatment was over, the Discovery Channel had Ted Koppel host a panel of experts discussing (& disputing) the issue for an hour or so with Mr. Jacobovici and Dr. James Tabor, his partner in this matter.  Actually, the length of the main program, itself, could have been cut by an hour (or more) while still giving it time to present the important facts and “analysis” (the quote marks there are very intentional) — which would have given the Discovery Channel more time to allot to the panel discussion, which was much more beneficial, and could have been even more so.

Augh!  I have much more on my mind about this show and its issue, but I absolutely MUST get to bed!  Suffice it, for now, for me to make a few comments (else I may get to bed, but I won’t be able to sleep!):

•  The real “evidence” presented on the program is less than overwhelming, especially when stripped of the rather thick coat of gloss and varnish laid upon the actual facts at hand.

That the program’s creators have an agenda (and one that goes beyond, “Let’s just get people talking!”) should have been rather clear for those who have eyes to see.  For example, Dr. Tabor was true to form when he said at the very beginning of the show that the “historical” approach is to assume that Jesus’ body was buried and should theoretically be with us today.  Christians claim that the transformation and resurrection of Jesus’ body was a historical event — how can you claim that the “historical” approach is to assume from the start that a potentially historical event did not happen?  (I should give him credit, though, in that it is my understanding that by associating himself with this production he shows a willingness to change his previously stated position on some matters.  But trading one falsehood for another can’t earn one too many points…)

•  The mathematics used to calculate the “probability” touted on the program is fundamentally flawed as a measure of how confident we should be in Jacobovici and gang’s conclusions.  I don’t mean to diminish the reputation of the statistics professor who did the calculations for the Mr. Simcha, but anyone (like me) who has had to do real calculations like that in real situations so that you can advise someone to make real decisions recognizes that no hypothesis was being tested in those calculations.

If you want to test the credibility or confidence level of a hypothesis, fine — do that, and then tell me how confident that I can be in that hypothesis.  There was no hypothesis tested with those calculations.    All the talk about the “probability of the cluster” is impressive sounding, but it isn’t the analysis that is really needed.

Again, I do not mean to besmirch the reputation of Professor Feuerverger — he makes his own choices, and I do not know exactly what he was asked to do.  But asking someone, “What is the probability of this cluster of names?” is simply NOT the same as asking, “How statistically confident should we be that these names represent the family members of Jesus of Nazareth?”  Not only is the first question a bit vacuous (which would explain why not much more than a single “high school” level rule of probability is used to calculate it), it would likely be answered by most using a completely different (an not applicable) set of beginning assumptions.

(I could go on and on and on this point, and if I feel like rambling some time in the next few days, I just might.  I hate it when mathematics is abused and MISused to lend an air of authority to something that does not deserve that authority.  If you like the sort of “math” that you saw on “The Tomb of Jesus” tonight, you might as well go out and buy a Michael Drosnin “Bible Codes” book — the “mathematics” is just as “sound” in those things (I refuse to dignify them by calling them books).  It might help sell books and boost ratings, but it just ain’t responsible mathematics!  OK, Smith: surpress the urge to rant, surpress the urge to rant…  I think I need to find a paper bag to breathe into…)

•  There is currently a great deal of discussion about the assumptions concerning the names, as well.  In fact, many have quite authoritatively argued that the supposed “Mary Magdalene” inscription should not be interpreted to be Mary Magdalene.  In fact, it has been powerfully argued that the way in which the name is inscribed on that ossuary actually provides evidence against the identification of the woman with Mary Magdalene.

•  On some of these matters, you might consider taking a look at the blog of Ben Witherington.  I read his article in the Wall Street Journal on March 2, “Tomb of the (Still) Unknown Ancients” and found it well done.  (You can read here, though it may require a subscription, I’m not sure.  If the link works for you, I do recommend the article to those who might be interested.)  Though written before the show aired, he had seen an advance preview and has actually worked with both Simcha Jacobovici and James Tabor, before, on other matters in which they are in agreement.  His conclusion?  “Make no bones about it — they have not found Jesus’ tomb.”

I mention his blog because he has posted insightful news and comments about many additional, convincing critiques of the conclusions shoved forward by the team of Jacobovici, Tabor, and Cameron.  His February 26 post, “The Jesus Tomb? ‘Titanic’ Talpiot Tomb Theory Sunk from the Start” is a good start.  Two more later posts (February 28, “Problems Multiply for Jesus Tomb Theory,” & March 1, “The Smoking Gun—Tenth Talpiot Ossuary Proved To Be Blank“) contain more details and dissecting, which will be more than some will feel the need to read, but which I very much enjoyed reading after the show.

And that’s just got to be it, for now!  In summary, let me just say that the arguments for the truth of the Christian claim that Jesus was, in fact, raised from the dead are just as true now as they have ever been.  For 2,000 years that claim has been attacked by hardier foes than we see today, and for 2,000 years it has survived because it is a matter of history, not fantasy.  The resurrection is not some “cunningly devised fable” (2 Peter 1:16) — a description which just so happens to fit tonight’s Discovery Channel telecast of “The Tomb of Jesus” quite nicely.  The apostle Peter prophesied that scoffers would come in the last days (2 Peter 3:3)…  Don’t let the fact that some of them have television cameras and slick websites bother you any more than it should!  Let the fairy tales crumble in the waste can, the truth will outlast them all.

Have a great night — or morning — or whatever part of the day you find yourself in!

The “Tomb” of Jesus – Early Comments…

Well, I find myself in a pickle…  I would love to spend some time writing on this supposed “tomb of Jesus” that has been “discovered” but I barely have any time to write this morning as it is, and I’ve already got one foot out the door (well, perhaps not one whole foot — maybe about two or three toes).

So just consider this a place holder — I do plan on writing about this later, so check back!  Until then, let me give a couple of very quick observations:

Concerning his proposition that the bodies found are the family of Jesus — including Jesus, Himself — James Cameron made the comment on CNN that “people of deep faith that sort of reject this out of hand are on safe ground, because we can never prove it.”  Hearing this, I cannot help but add that there will be others of “deep faith” who accept it uncritically.  The facts — over the years — have routinely supported the key elements of the Christian faith, including the death and resurrection of Jesus Christ, but those facts tend to be rejected because accepting them requires one to accept the truth of Jesus’ ministry and teachings and the necessity of changing your life accordingly.  Such people are being hypocritical when they ignore the full weight of evidence in favor of the “facts” they like; in fact, in that sense, I respect those who admit their bias up front and without reservation a great deal more.  And all have their biases to be admitted — such is the nature of humanity (yes: me, too!).

The other early observation is that Simcha Jacobovici has certainly proven that he is not someone to be praised uncritically.  (Of course, who is?)  I mentioned him and his show, The Naked Archaeologist, in a blog entry way back on November 10, 2006, at which time I tried to include a few qualifications to indicate that — as much as I like and appreciate his show — I do not always agree with his conclusions, which are hit-and-miss as are almost all conclusions in such work, it seems.  And, after watching some additional Naked Archaeologist episodes, I found some things I disagree with concerning his methodology at times, as well, as opposed to simply his conclusions.  For example, after his episode which gave an air of acceptance and praise to the fellow who — as was ultimately proven, I recall — forged the infamous “James ossuary,” did he film a follow up a episode giving a more accurate picture?  Perhaps he has and it has not yet aired.  Also, his tendency to accept conjecture as fact without qualification can be a bit annoying — but then, he is likely in good company in his field (please forgive me, archaeologists who do not do this!), differing only in that he pulls his conjectures out of a different bag.

I would probably still watch his show on occasion, and it is still refreshing to see an archaeologist who takes the Bible (that is…) part of the Bible seriously (that is…) mostly seriously.

Hmmm… Did I say “quick observations”?  I really need to follow my toes out the door — but I will write on this again when I have more time!  Check in later…