Of Studies And Numbers And Questions

I make a decent living fixing very complicated broken things.  Much of my life is dominated by trends and reports; statistics.

So when someone says to me:

Cardiac arrests in marathon and half-marathon runners have become more common during the past decade – a fact that has generated more terrifying accounts about fit athletes keeling over in the midst of competition.

I always, ALWAYS, question the data, the gathering and the incentives.

This is true in my professional life where the reports describe very complicated broken things or when I stumble on a report that says:

…many researchers have reached a conclusion that turns conventional wisdom on its head: Americans enjoy less economic mobility than their peers in Canada and much of Western Europe.

When a “report” doesn’t ring true, you have to investigate.  For example, the report about the alarming increase in heart attacks among marathon runners?

The increase in cardiac arrests, it turns out, has been accompanied by a dramatic increase in the number of participants…

Honest to God.  “They” reported that heart attacks were on the rise!  Regulate marathons!  Then “they” did a study to identify the reason why more runner were experiencing heart attacks.  And it was because there were more runners.

When someone tells you that America doesn’t have the world’s best medical care; don’t believe them.

When someone says that America doesn’t have the greatest ability to lift you out of poverty; don’t believe them.

Dig deep.  Find the data.  And discover for yourself that truth.

11 responses to “Of Studies And Numbers And Questions

  1. Yes, that’s one of the first lessons in social science or any use of statistical analysis – look at the methodology, look for anomalies. The best one I ran into was a dramatic increase in the death rate of East German males under 65 after unification. Original articles thought perhaps the stress of shifting to capitalism, the loss of their culture and status and a whole bunch of other factors played a role. Digging it turns out that the increase was due primarily to the fact that East German men were having fun driving big West German BMWs and Mercedes on their old highways built for tiny Trabis. The death rate from auto accidents rose so dramatically as to alter the stats completely!

    • Yes, that’s one of the first lessons in social science or any use of statistical analysis – look at the methodology, look for anomalies.

      I agree. When you dramatic increases in marathon deaths or in East German male mortality numbers going down, you have to look harder.

  2. So are you telling me that last years report showing a million percent increase in the number of hand injuries related to the use of iPads compared to the base line data collected in 1981 may be flawed?

    • So are you telling me that last years report showing a million percent increase in the number of hand injuries related to the use of iPads compared to the base line data collected in 1981 may be flawed?

      Indeed. It’s important that the thing being measured is relevant to the thing being concluded.

  3. “When someone tells you that America doesn’t have the world’s best medical care; don’t believe them. When someone says that America doesn’t have the greatest ability to lift you out of poverty; don’t believe them. Dig deep. Find the data. And discover for yourself that truth.”

    This is rather a strange conclusion to a post that’s ostensibly about chasing down the data.

    We have seen that study after study shows that we don’t have the world’s best medical care. Links available at this comment: “This one puts us in 37th (this criticism says it should put us at 15). This one gave us a failing grade, noting that of the couple dozen countries surveyed, we were worst in infant mortality and worst in life expectancy once people reached age 60. On this one, we ranked 7th, 4 straight years. Woot! Now the bad news: there were only 7 countries in the study.”

    When we were discussing it then, I linked to study after study, and you said, “well, yeah, but you could do other stuff to the data that would tell me we’re awesome,” but didn’t have any substantiation.

    Yes, we have some great things here, but overall, Americans are poorly served by the health care institutions we have in place, compared to other well-off countries.

    And it’s the same for social mobility. Here’s a line from a 2010 article in The Economist, hardly a subversive left-wing anti-capitalist rag, examining the causes & extent of our reduced mobility: “Parental income is a better predictor of a child’s future in America than in much of Europe, implying that social mobility is less powerful. … The children of the rich and poor, meanwhile, are less mobile than the middle class’s. More than 40% of those Americans born in the bottom quintile remain stuck there as adults.”

    Your views on these two issues are resolute and unshakable indeed, but I don’t see what they have to do with chasing down the data.

    • We have seen that study after study shows that we don’t have the world’s best medical care.

      Yeah, we’ve been ’round and ’round on this. I continue to be amazed that you won’t accept the fact that the data as reported isn’t reporting about things like health care. The studies that Liberals throw out there that they THINK measures things like health care are really measuring things like the impact of auto accidents and gang land murders.

      For example, it’s said that the United States has a lower life expectancy than other nations. But those other nations are not exposed to the same life ending [non-medical care delivery related] things like motorcycle accidents. Or hand-gun deaths. In the same manner, studies that Liberals use to show to demonstrate health care quality centers on infant mortality. Again, these studies don’t take into account things like incarcerated dads. Single mothers. Ethnicity of mom. Birth weight of baby.

      In both cases, when factors are normalized, the United States ranks at or near the top.

      The point of this post is that when people report higher number of marathon runners are dying from heart attacks, it doesn’t mean that American’s marathons are more deadly than other marathons. It just means that we have other things going on in the population.

      And it’s the same for social mobility.

      Yeah, that’s another one too.

      I recently quoted data out of The Oxford Handbook of Economic Inequality. I am interested to understand why this data is showing what it’s showing. I suspect that it has to do with several things, but I feel that our declining marriage rate and the number of immigrants are leading reasons. For example, I think that immigrants come to America poorer on average than natural born citizens.

      Your views on these two issues are resolute and unshakable indeed, but I don’t see what they have to do with chasing down the data.

      It would be fascinating to be able to sit and have a beer one day. I absolutely cannot get past the fact that we can’t agree on the data. For example, I don’t know why you insist on counting high powered assault rifle deaths while deer hunting as a measure of medical care in the US. And you must feel equally perplexed as to why I don’t. 😉

      • are really measuring things like the impact of auto accidents and gang land murders

        You throw out hypothetical possible reasons as to why The US Is Different, and our health outcomes aren’t really actually our health outcomes when you account for rednecks and thugs killing people, but there isn’t any empirical data to support that point of view. It’s just a bunch of stuff thrown at the wall.

        In that previous thread, you mentioned obesity; but obesity accounts for only about $25 billion per year in spending. So the numbers don’t support the hypothetical, theoretically plausible point that obesity means that “USA #1!”

        In this post, you mention high powered assault rifle deaths while deer hunting, but “According to National Safety Council statistics, far more people per 100,000 participants are injured while bicycling or playing baseball than while hunting. And the Council’s most recent statistics show that while roughly 100 people die nationwide in hunting accidents each year, more than 1,500 die in swimming-related incidents.” So the numbers don’t support the hypothetical, theoretically plausible point that hunting fatalities mean that “USA #1!”

        these studies don’t take into account things like … Ethnicity of mom.

        Um… how is it possible to “normalize” for that? Presumably you’re arguing that a black mother’s health outcome is less meaningful a measure than a white mother’s death– perhaps it’s around three fifths?

        As to the gini coefficient & inequality: (1) you don’t compare the post-marriage US gini to that of other comparable nations, you just argue it should be smaller w/n the US with higher marriage; (2) correlation != causation; (3) you go into the workforce with the population you have, not the population you wish you had; (4) you don’t look at trends on marriage & inequality in recent times to see whether this holds up over time, & whether it might explain 1% or 90% of worsening inequality; (5) you don’t compare the US’s inequality & marriage rates over time with those of other comparable nations; (6) I’m tired of typing.

        (Incidentally, I’m in agreement that marriage is a Good Thing. But, as with, say, affirmative action, while I support it in theory, I don’t know enough about specifics of implementation to know whether/how we can effectively nudge things that way with policy).

        So it’s the same thing as bringing up hunting & obesity in health care stats. It’s a possibly maybe theoretically relevant reason to ignore what every study says, but does it hold up? You don’t know.

        It’s the opposite of chasing down the data. It’s inventing hypothetical reasons as to why you might ignore the data.

        • You throw out hypothetical possible reasons as to why The US Is Different, and our health outcomes aren’t really actually our health outcomes when you account for rednecks and thugs killing people, but there isn’t any empirical data to support that point of view. It’s just a bunch of stuff thrown at the wall.

          And you refuse to look into reasons that might explain the US low ranking. I’m sure this is due to the fact that it fits your narrative, but the facts don’t support that line of thinking.

          In this post, you mention high powered assault rifle deaths while deer hunting, but “According to National Safety Council statistics, far more people per 100,000 participants are injured while bicycling or playing baseball than while hunting. And the Council’s most recent statistics show that while roughly 100 people die nationwide in hunting accidents each year, more than 1,500 die in swimming-related incidents.” So the numbers don’t support the hypothetical, theoretically plausible point that hunting fatalities mean that “USA #1!”

          Fair enough. I trust that you’ll be able to demonstrate the data that shows mortality rates for the USA as compared to other nations as it relates to hunting, swimming, bicycling and baseball. Which, by the way, is my point. The United States experiences more deaths related to causes not related to medical care delivery than other nations.

          You, however, ignore this and don’t even attempt to account for it.

          Do you really think that traffic fatalities is an accurate gauge on our medical system?

          Um… how is it possible to “normalize” for that?

          Um…Work harder.

          Presumably you’re arguing that a black mother’s health outcome is less meaningful a measure than a white mother’s death

          I’m gonna assume that you posted this at a very late night/early morning hour and didn’t mean to imply what you just said. Because not to assume that would mean that you are just an ignorant bastard, and I don’t think that you are either ignorant or a bastard.

          perhaps it’s around three fifths?

          Keep bangin’ that racism drum brother. However, again, I’m willing to overlook the implied insult due to the late hour and the probability that you had too much to drink.

          As to the gini coefficient & inequality: (1) you don’t compare the post-marriage US gini to that of other comparable nations, you just argue it should be smaller w/n the US with higher marriage;

          And you refuse to look further into the data. You are making the argument that more marathon runners dying in America is because marathons have gotten more dangerous.

          Dig, man, dig!

          • I meant to be highlighting what I perceived to be the absurdity of the claim that we can “control” for ethnicity of mother. I certainly didn’t mean to be charging you with believing that the Constitution was correct to treat blacks’ lives as worth 3/5 of whites’ for some purposes. So, this time without any obnoxious absurdity-highlighting: why, conceptually, and how, statistically, would we control for that?

            I linked to one study that examined quality, access, efficiency, equity, quality of life, and put us 7/7, and another that examined health level, responsiveness, equity, health distribution, and responsiveness distribution and put us 37th. In response you linked to one guy’s blog post from five years ago looking at data ending in 1999, arguing that as long as we don’t worry about how many more people die violent deaths, our life expectancy is in line with that of other wealthy countries. That’s doesn’t get us out from under the findings of all those studies. That’s not following the data.

            If indeed “[t]he United States experiences more deaths related to causes not related to medical care delivery than other nations”, that strikes me as a rather important public health issue. Should we stop subsidizing auto-based designs, and/or permit higher-density residential construction?

            At the end of the day, I’m not a health care professional, and I’m not a statistician. I can ask questions, and hopefully catch errors. But I’m not going to, in a blog post, come up with a hypothetical that debunks the views of experts on these issues. All the comprehensive studies that I’ve seen find that we have mediocre health care, and worse social mobility than most European countries (not as bad as the UK, though, I believe).

          • I certainly didn’t mean to be charging you with believing that the Constitution was correct to treat blacks’ lives as worth 3/5 of whites’ for some purposes.

            Whew.

            It’s strange but over time it feels that I’ve come to know the regulars around each of our blogs. I had a serious degree of distress that ya’ll thought I might really feel that way or hold those opinions. If i reacted a little defensibly, or a lot, I wanna apologize. I tried to wait as long as I could to let some of those feelings dissipate [never blog mad].

            Anyway, glad to know you were only making a point and not making the accusation!

          • So, this time without any obnoxious absurdity-highlighting: why, conceptually, and how, statistically, would we control for that?

            Certain populations and sub-groups have different outcomes. Latin American women have lower mortality numbers [better numbers] while African American women have higher blood pressure. Single mothers do worse than married mothers. When you compare a homogenous population like Japan or Norway and compare it to America, you aren’t measuring the same thing.

            I linked to one study that examined quality, access, efficiency, equity, quality of life, and put us 7/7, and another that examined health level, responsiveness, equity, health distribution, and responsiveness distribution and put us 37th.

            Most of the data is centered on the WHO rankings. And the WSJ piece talks to that specifically. It note multiple problems with the WHO ranking especially citing the drop from 15 to 37 if we remove the expenditure piece. But even the 15th ranking is due to the criteria established by WHO. Criteria that many folks have issues with.

            you linked to one guy’s blog post from five years ago looking at data ending in 1999, arguing that as long as we don’t worry about how many more people die violent deaths, our life expectancy is in line with that of other wealthy countries.

            The blog links to a broken link. I’ll try to find the Iowa study.

            But the point is that studies that use life expectancy don’t account for non-medical care causes of death. There is little reason to measure fatalities due to unmanned drones on the War on Christmas when measuring health care.

            Should we stop subsidizing auto-based designs, and/or permit higher-density residential construction?

            That is a better response than blaming health care delivery.

Leave a Reply to Pino Cancel reply