Management by Baseball

I'm in the middle of a three-part post, and while as a reader I really dislike it when someone goes off on an aside in the middle of a series, I'm about to do that. I'll get back to the series; I just had to deal with this archetypal foolishness.

BRAIN-CELL KILLERS VS. WASTE OF ELECTRONS

ESPN ran a table in yesterday's MLB index page, which I include below. The Disney sports channel is trying to be in Frontierland (opening up the savage statistical desert and creating a settlement of civilized modern interpretation), and careening into Fantasyland with this particular effort, something they call "The Juicebox".

The idea is (I think. The table has no analysis presented, just what you see below. This is based on my induction) to use numbers to help fans figure out the on-field effects of the new MLB steroid/supplement-testing policy.

It's a clinic in how to strike out.

The Juicebox

Strike 1: The table appears without any accompanying narrative beyond that little bit of text. No analysis, barely even enough context-setting to allow a consumer to judge what the numbers are supposed to tell you. Like a Disneyland character dressed in a Pluto suit, it has nothing to say for itself

Strike 2: They don't bother to tell you whether these numbers "per game" are "per team, per game" or "per game (both teams)". It's per team per game, by the way.

Strike 3: In an effort to impress the reader with their extra-beefy numeric ability, they include not just one, but two insignificant digits, as though if they printed "1.1" readers would think they were girly-men, but if they printed "1.0821074", readers will be impressed with their smack-umen. Look, the difference between 1.082 and 1.071, is ONE homer every 90 games, or generously, two per season per team. Not significant.

Strike 4: As though in a bad dream from Mr. Baseball, they continue to whiff by failing to deliver any conclusions. I think they did that because when the Disney-lads started this Juicebox feature, they didn't have any idea what the statistics might tell them when they "designed" this feature. Do they think it shows steroid use is down? Up? That steroids don't have any effect on statistical accomplishment? That steroids do, but that they increase the capabilities of pitchers and batters equally? Like the last 36 years of Federal drug policy through every President since Nixon, the Disney-lads don't know any of the science, they don't care about any of the science, they just want to waste oxygen by vigorously waving their arms about it.

Strike 5: They didn't normalize the data. So they're comparing the rate of numbers produced in the first three months of one season with the rate produced over entire previous seasons. Given seasonal variations in weather and production, this is a weakness (not fatal, necessarily, but why mix when you could match?).

BEYOND BASEBALL

There are some good lessons laid out here, and not all of them at the expense of ESPN.

In their defense, sports coverage is early in its transition from the old Bitgod (Back In The Good Old Days) stats model to the more informed Sabermetric model. As with any paradigm shift, there will be some adopters who shift to the new model without really understanding it, just knowing that they need to to "keep up". These early efforts in organizations wedded, well, welded, to the old model are bound to have rough spots. But it's better to shut up than to fill the universe with more verbal plaque.

And sometimes you develop an idea for a study that confirms the null hypothesis (that what you thought might indicate a trend or result instead indicates nothing significant). If you're never hitting that wall, you're probably not experimenting hard enough. Don't be worried about presenting the results of a study that did this, but explain what you think it means. Too often in our sabermetric community, people invest a lot of effort in analysing data and present their results as though they had meaning, when in reality, it was a large effort that proved almost nothing. It's okay that it proves little or nothing; one just needs to explain that.

* - When you present data on any topic, don't add a bunch of insignficant digits. 1.082 in the context of a 83-game series of events (the average # of games played this year so far) really should be 1.1 or perhaps 1.08 if you want to get Beyond The Valley of the Super-Fine. One divided by 83 is 0.012, so the last digit doesn't inform, just uses up ink or electrons.

* - Have an idea of what you're trying to indicate with data before you start running numbers. If ESPN had thought through the basis of the Juicebox, they'd know at this point that it looks like MLB's steroid/supplements policy hasn't had a sigficiant effect of statistical output so far, and they could have mentioned that.

* - Try to normalise data. Sneaky statisticians can prove almost anything by carving out samples, picking out some four month period that "proves" the economy is booming or tanking, depending on what they need. Weak statisticians can just choose odd periods unintentionally. Try to make sure your numbers don't have wierd environmental artifacts in them, like comparing Spring+early Summer with numbers from an entire year.

* - Contextualize conclusions. Always present findings in a few different ways so you can reach people with different skill sets. For example, a short narrative here could have explained that home run output was up a tiny bit so far this season, and that perhaps the new steroid policy wasn't surpressing that expression of batting power.

There are always going to be imperfections in data presentation, we can all live with that. But try not to chain together every single one of these problems the way the Fantasyland characters at ESPN did.