Here is the raw data comparing the report cards of the two candidates, and calculating the observed credibility of all their statements that received Politifact ratings:
In today's analysis, I'll do the following:
- Estimate the uncertainty in each individual candidate's statement credibility rating.
- Estimate the uncertainty in the comparison of the two candidates' credibility ratings.
- Compare each candidate to their running mate.
Let's get on with it, because I have to (a) finish an #AskAGuru answer, (b) finish a Guru magazine article submission, and (c) go to a funeral (no fieldwork today).
Starting with the comparison of Biden and Ryan:
- We can be 95% confident that the credibility of Biden's statements is between ~0.34 and ~0.58. He's at worst making mostly false statements and at best making barely better than half true statements.
- We can be 95% confident that the credibility of Ryan's statements is between 0.17 and 0.56. He's at worst bordering on zero credibility and at best making barely better than half true statements.
- Indeed, the confidence intervals for the two VP candidates are wide, and overlap. Although the maximum likelihood estimate of Biden's credibility is ~1.25 times that of Ryan's, we can only be 95% confident that Biden's credibility is between ~75% to more than 275% of Ryan's.
- The chances that Biden's credibility is greater than Ryan's, however, are nearly 3.5 times greater than the chances that his credibility is less than or equal to Ryan's credibility (~78% probability versus ~22% probability).
So guessing the Biden will make more credible statements than Ryan during the debates is at least better than a coin toss, but it's by no means certain. That's the lesson I'm trying to give with this whole credibility simulation exercise.
Now for a comparison of the VP candidates to their running mates:
- Obama's observed credibility is 117% that of Biden.
- We can be 95% confident that Obama's credibility is between ~93% and ~158% that of Biden.
- Obama is nearly 8 times as likely to be making more credible statements than Biden as he is to be equally or less credible than Biden (~89% probability versus ~11% probability).
So we can't be quite certain that Obama is more credible than his running mate, but such a guess has far better than a coin toss's chance to be true.
Now for Ryan versus his running mate.
- Ryan's observed credibility rating is nearly equal to his running mate's.
- We can be 95% confident that Ryan's credibility is between ~49% and ~172% that of Romney.
- Ryan is just over 1.5 times as likely to be more credible than Romney as he is to be equally or less credible than Romney.
So we are even more uncertain about whether Ryan is more credible than Romney because guessing that to be so has little better than a coin toss's chance of being correct.
Based on these analyses, I make the following predictions:
- Biden will be somewhat more truthful than Ryan during the first VP debate.
- In his first debate, Biden will be somewhat less truthful than Obama was in his first debate.
- In his first debate, Ryan will be about at truthful as Romney was in his first debate.
Next time, I will aggregate all four candidates' report cards by party and estimate the certainty with which we can make statements like "Obama/Biden makes more truthful statements than Romney/Ryan". And once Politifact publishes the definitive list of their ratings of the VP candidates' 1st debate statements, I'll analyze how well our predictions played out.