Are Blogs Evidence? What Is?

October 18th, 2012 at 8:38 pm

Of the six studies that Gov Romney cites to defend his mathematically-challenged tax plan, a few were blogs and were thus often dismissed by critics as being de facto inadequate evidence simply by dint of being blog posts.   As an active, evidence-based blogger, I can only say…quelle horreur!

Actually, I think it’s entirely fair to heavily discount blogs as evidence.  Obviously, quality varies widely and there are no gate keepers on the net, and nothing approaching rigorous peer review.  However, there’s room for nuance.

First, some blogs, like the ones at CBPP and EPI (not to leave anyone out, but those are places I’ve worked so I know how they operate), generally report shortened, reader-friendlier versions of their studies, so judging those blogs as evidence depends on your view of their studies.

Which gets to my larger point—I wouldn’t generally trust blogs as evidence but neither would I trust every think tank report.  The most reliable basis for trustworthy evidence in social science is peer review, a process by which experts of the established knowledge on the topic, along with rigorous application of the rules of statistical evidence, evaluate the claims in the study before it can be published.

This process too, of course, is far from fool proof.  There are statistical outliers, but more problematic, embedded prejudices in social science—assumptions and even “fads” that undermine the peer review process.  For years, economic journals printed studies supporting self-regulation by rational actors that gave succor to Greenspanian laissez faire regulatory practices which in turn helped spawn the Great Recession.

Now, slowly, those articles are being replaced by new takes on an older set of ideas once espoused by Kindleberger and Minsky about inherent instabilities in the economic system.

But back to blogs, think tanks, and what constitutes evidence.  The thing I’ve always found useful about the work of the think tanks I trust is that they tend to take a pretty-much “just the facts ma’am” approach in their reporting.  CBPP and EPI reports, for example—again, places I cite because I know first hand how they make the sausage—tend to be based on data series from established data sources with little manipulation.  They offer interpretation, but the facts and data processes are right out there and derived from trustworthy source material.

In terms of using blogs for evidence, you’ve got to know with whom you’re dealing.  I can most reliably speak for myself and much of what I do in this space is report evidence from sources I judge to be reliable in ways that are intended to go down more easily.  When I present my own statistical evidence, I’m trying to follow the standards noted above re EPI/CBPP—plotting/interpreting reliable data.

But blogs are by nature quick, impressionistic takes on issues.  They can point you in the direction of deeper dives, but they will very rarely, I think, provide adequate evidence for policy decisions.

Case in point: Romney’s tax plan.  It would be ludicrous to trust a blog or an oped that threw some rough numbers together over the Tax Policy Center’s results, as the latter are based on a well-established tax model built by non-partisan experts that has a long track record of generating reliable facts about tax policy.  Certainly the blogs can play a very useful function in raising questions about the deeper studies, and even send the authors of the study back to the model for tweaks suggested by the bloggers.

But it’s actually pretty shocking to me—though I know I should develop a thicker skin—that the Romney campaign is now lurching around, daily trotting out new “studies” and “solutions” to their math problem–“we’ll broaden the base (but can’t say how)”…”we’ll cap deductions at $17,000”…”whoops…that doesn’t work…we’ll cap them at $25,000.”

[See here for TPC evidence that these caps also fail to raise enough revenue to offset the $5 trillion, 10-year cost of the tax cuts.  Since most deductions and credits accrue to higher income households, the caps do, however, raise revenue in a progressive manner.]

The Romney team clearly threw out their tax plan—20% cuts across the board—without any of the requisite spade work to see if it actually made sense.  And now that real studies are challenging it, they’re veering from “just trust us” to setting the evidentiary bar down so low that anything with numbers on it can clear it.

Not surprising, given the beating the facts have taken in this election cycle, but not so good for democracy, not to mention the truth.

Print Friendly, PDF & Email

9 comments in reply to "Are Blogs Evidence? What Is?"

  1. mitakeet says:

    However, none of this matters if the populous is so credulous that they can’t see the difference between well respected non-partisan research and highly partisan blogs. I am (or perhaps was) a scientist and have a decade or so of experience reading primary literature and know that even there people make mistakes, exaggerate results and have bias. Because I know this and have extensive experience I can tell which bits of primary literature deserve my close attention and which do not. However, ‘ordinary’ people don’t read primary literature (ever), let alone have the skills to tell high from low quality, so there exists a group of people who read primary literature and then translate so interested lay people can get the gist. Sadly, in a rather substantial fraction of the time (10-20% in my experience) the digested report misrepresents the underlying literature, sometimes comically. Since (most) science is without political agenda, yet is still rife with these problems, I can in no way think that stuff like economics, job reports, etc. could ever be comprehended at the primary (peer reviewed) literature level by lay people. Given that most people don’t even have a strong lay interest in the primary data, I find it very difficult to believe that any undecided voter will give a damn about what is ‘real’ and what is blather made up by political hacks.

    I wish we had an electorate that cared enough to do some research, but if we did we wouldn’t be in our current idiotic situation.

  2. wkj says:

    My income is less than $250K/yr.

    It is mostly interest & dividends.

    Romney says he would reduce my tax rate on that income to 0%.

    I should be tempted to vote for him, but I don’t believe a word he says.

  3. Adi Venkata says:

    The issue with US taxation is two fold (1) Complex (2) Creates classes of income

    I suggest the following:

    1. Deductions limited at 25% of gross income or $40,000 – whichever is lower
    2. There is no difference between any source of income – salary, dividends, capital gains
    3. There are 5 rates: 7.5%, 15%, 20%, 25% and 30%
    4. The slabs are at $15,000; $50,000; $75,000; $150,000 and $300,000
    5. Estate tax has 2 tabs: $2M – $10M: 10% and $10M+: 15%
    6. SS Tax limit is raised from $107K to $300K
    7. Non-banking financial transactions are taxed at 0.2% per transaction (shares, swaps, options etc)

    My estimate is that remove the classes of taxes; bringing more equality to salary; increasing overall revenue and reducing deficit. i want to ensure that all increase of revenue from this change in income tax code goes towards deficit closure rather than increased govt spending.

  4. Rima Regas says:

    Romney, and Bush before him to a lesser extent, brings a high degree of intellectual dishonesty that’s just barely above acceptable levels. This is how he ran his business and it’s how he runs his campaign. It’s no wonder that he would choose to rely on opinion and call it fact.

    Where his numbers and Paul Ryan’s come from should be reported on far more widely than they have been in the context of what’s acceptable and what is not, when coming up with a policy.

    You’ve mentioned this, Ezra Klein has mentioned it. Paul K has been calling Ryan a fraud for some time now. Someone else, a reporter, needs to tell the story of where these studies come from and how they deviate from serious number crunching at an institution with repute.

    So far, they’ve come up with alternative science to refute Roe v. Wade. They’ve invented an alternative science to refute global warming. Now, they’re inventing new economics.

    Slowly but surely, they’re managing to delegitimize our institutions.

    This is bad.

  5. Sharks says:

    The truth is they never planned on paying for the tax cut. They never had a plan to pay for it because they never will pay for it. When have republicans ever paid for anything.

    This is the party of tax cuts pay for themselves. Tht is their belief. Bush was told by the CBO that his tax cuts would do everything the opposite of what he was telling the American people. He knew. Romney knows.

    Tax cuts pay for themselves.

    The only reason they can’t use that line anymore is because Bush proved their entire fiscal policy to be nothing but a fraud.

    The fraud continues with Romney.

    Economists should stop chasing around these numbers Romney throws out and look at the history of the Republican party.

    They don’t pay for anything.

  6. Bob Dole says:

    A blog asking if a blog is evidence??? So this blog is evidence of what…hilarious…

  7. Rich Anderson says:

    Our electorate barely cares about who’s running for president, no where near the amount of people who could vote do. There’s no way they’ll care enough to separate good information from bad when it comes from similar sources.