I opened up the Politifact.com Truth-O-Meter report cards for the Republican and Democratic National Committees' official statements and did what I've been doing a lot lately: calculate their respective truthfulness rating using my scoring method. Here's the data:

Look familiar? Here, let me show you the data comparing Romney to Obama (as of today at 8:44 am EST).

Note that Obama's truthfulness has dropped a percentage point from where he was when I first calculated his truthfulness a few days ago. I used my truthfulness simulator to test the hypothesis that Obama has lost credibility. The odds aren't much better than one-to-one that he has lost credibility. The 95% confidence interval for ratio of Obama's truthfulness before to after is between ~0.92 and ~1.11. The probability that Obama lost credibility ~56%. There is even less evidence that Romney's credibility has changed over the last week. The 95% confidence interval for the ratio of Romney's truthfulness before to after is between ~0.73 and ~1.37. The probability that Romney gained credibility is ~51% (basically the toss of a very slightly unfair coin).

If you think the analysis in the previous paragraph is kind of silly, you're kind of right. But I'm just demonstrating another use of my simulator: you can compare an individual report card to that individual's report card at another point in time. Eventually, I will extend the method to allow for the analysis of report cards over several points in time. Actually, I'll use different methods all together. But the point is that you can statistically analysis truthfulness over time, not just over individuals.

But let's get back to the main purpose of this post, which is to compare the presidential candidates' truthfulness to that of their respective parties' national committees. Romney's truthfulness is 0.35. So is the RNC's. Obama's truthfulness is 0.54. The DNC's is just two percentage points higher. Can we draw any distinctions between the truthfulness of the candidates and the truthfulness of their parties' committees? Let's use the simulator to find out. Along the way, we'll also compare the DNC to the RNC.

Is Obama just as truthful as the DNC? Conversely, is the DNC more truthful than Obama? The 95% confidence interval for the ratio of the the DNC's truthfulness to that of Obama is between ~0.82 and ~1.26. The probability that the DNC is more truthful than Obama is ~63%; better than a coin toss, but not good enough to be almost certain. Then likelihood that the DNC is more truthful is only about 1.7 times as high as the likelihood that it is less truthful. We can't say with much certainty that the DNC is more truthful than Obama, but can we say that the two are equally truthful? Well, remember what I said in a previous post: absence of much evidence isn't evidence of absence. Still, we can say that we're hesitant to reject the null hypothesis that the two are equally truthful. 

Can we tell any difference between the truthfulness of Romney and the truthfulness of the RNC? The 95% confidence interval of the ratio of the RNC's truthfulness to that of Romney is between ~0.43 and ~1.56. The probability that the RNC is less than Romney is ~52%; it is basically a (very slightly unfair) toss-up. We should be hesitant to reject the null hypothesis that Romney and the RNC are equally truthful, but we shouldn't leap to the conclusion that the two are equally truthful.

How does the RNC compare to the DNC? The observed truthfulness of the DNC is ~1.6 times higher than that of the RNC. Can we state with statistical confidence that the DNC makes more truthful statements than the RNC (based on their Politifact report cards)? The 95% confidence interval of the ratio of the DNC's truthfulness to that of the RNC is between ~1.03 (DNC very slightly more truthful than RNC) and ~3.5 (DNC 3.5 times more truthful as RNC). The probability that the DNC is more truthful than the RNC is ~98%. Much as we could for the comparison of Obama to Romney, we can guess that the DNC is more truthful than the RNC with a high degree of confidence. 

In a future post, I will address the concerns than many have raised that Politifact.com data is liberally biased, and incomplete. For now, however, just take this as one piece of evidence among many regarding the comparative truthfulness of the RNC and DNC.

Lastly, I stress that comparing the truthfulness of the RNC to the DNC is not the same comparing the aggregate truthfulness of politicians in each party. To do that, I would have to sample the truthfulness of each individual from the appropriate multinomial distribution, average the sampled truthfulness ratings by party, compare the party average truthfulness ratings, and repeat several thousand times to produce a probability distribution, from which I would derive confidence intervals and test hypotheses. Of course, this analysis is also distinct from a comparative analysis of the truthfulness of the Republican and Democratic National Conventions. There, I would have to sample the truthfulness of each individual speaker's statements during the two conventions, average the sampled truthfulness ratings by party, compare the party average truthfulness ratings, and repeat several thousand times, et cetera.

Anyway, Politifact, I'd love some feedback. Here's to continuing my open and transparent pitch of this method, and my commentary on it. I think it would add a lot of value to your Pulitzer prize winning enterprise, which I've found invaluable over the years.