Management by Baseball

Most large organizations (about 80%) make no meaningful attempt to do performance evaluation. About 15% strive for meaningful results and squeak out some utiltarian insight. Only 5% actually master the concepts required to figure out how to figure out what performance is and should be and can keep it useful through evolving events and environments.

The single most common stumbling block to effect performance evaluation is that would-be measurers forget about the context. Context isn't simple. We don't live in a good-or-bad, black and white world where the good guys 'n gals are easy to pick out from the bad ones. Like in a film noir, people are all human, all have foibles and moral weaknesses -- sure, some just stink, but even the best have weak spots. Protagonists in a Noir film may be people the writer wants us to root for, like Joan Crawford in Sudden Fear, but writer Leonore Coffee makes to understand she is capable of evil, too. The classic hero of the genre, Philip Marlowe, knows he lives in the real, ugly world and is willing to steal needed evidence or let the innocent suffer to achieve his goals.

In the noir view, the difference between protagonist and antagonist is a matter of degree. The lines can fuzz at times, and the relative good/badness of an individual depends on the exigencies of the environment and the moment. Nowhere are measures, the good ones, anyway, more clarifying on this point than they are in baseball.

MALCOLM NORMALIZES THE RANGERS
Because in baseball, researchers "normalize" performance. Normalize, in this context means, removing from an individual's performance the specific factors that boost or supress average performance. Take a 5'5" woman, and measure her standing height on the ground, upright in a one-foot hole and perched on a nine-inch high pitching mound. How tall is she?

Right.

But to the average manager doing performance evaluation for a big organization, she is going to be measured as 5'5", 4'5" (petite) and 6'2" (call the WNBA), because he hasn't normalized, evened out the playing ground. Sales people are most frequently afflicted with this inability. Where compensation depends on commissions, varying quality of assignments and territory demographics and pre-existing customers shape performance, but usually not performance measures.

Sometimes the context differences are subtle. Sometimes they change the way you view the whole shebang.

Take the 2004 Texas Rangers. Take a look at their roster's core performance measures from last season by clicking here. It's a bunch of heavy hitters and a pitching staff made up of a couple of league average ERAs and then the deluge of 5-something ERA tater tots.

That's what it looks like to conventional wisdom. Just about all the observers, including the press, fans and cognoscenti agree what Texas needs (besides a film noir remake of The Alamo), is pitching. When people analyze the Rangers' performance, the conventional wisdom is it's their pitching that has limited their achievement.

It's not true. Thanks to Don Malcolm, who not only messes with Texas, but with conventional views of the Rangers' performances, it's now obvious their pitching was fine, it's their hitting that needed improving.

Malcolm, one of the most interesting (and persistently contrarian) baseball minds, is issuing his team by team previews for the 2005 season, and recently did one for the Texas Rangers. His numbers for the team over the last few years normalized for their offensive-boosting home park will surprise many people.

Normalized to 1.00 (where 1.00 is an average performance for a team playing in a composite league-average home park), the Rangers hitting last year was 0.95...5% below league average, while the team's ability to prevent runs was 1.13, 13% better than league average. Conventional wisdom, a mere 180 degrees from its belief in reality, stood on its head.

The masking factor was their home park, which everyone knows is an offense engine, that boosted hitting 11%. And even though everyone knows this as a given, they still overlook it when they analyze what Texas needs to go the next step. GM John Hart knows, actually. His biggest move of the off-season was to sign solid-hitting and -fielding outfielder Richard "Guadalupe" Hidalgo, and not to spend the bulk of what dollars he had in his budget to acquire more pitching.

BEYOND BASEBALL
I discussed the common failure of performance measurement in sales departments (where you'd think it'd be easiest, given that results are already measured, as they are in baseball, in hard numbers, usually dollars or units-sold). I had an interesting consult several years ago with a big multi-state law firm that was trying to figure out a way to partially equalize compensation based on performance.

I headed them in a direction they ultimately couldn't bring themselves to commit to as a compensation system foundation, but it did re-arrange their thinking on performance evaluation.

I suggested they do regression analysis on what factors were knowable in advance on a piece of legal work. What were the merits of the work, and what was the expected, realistic, chance of coming away with a settlement or victory if it went to trial? If it went to trial, what effects on success would the judge (most frequently out of the control of the talent) have? There were a handful more factors, but you get the idea.

To some degree, they were already doing this kind of analysis, but not on paper, and only in deciding whether to take a piece of work, how flexible to be in pre-trial negotiation, and how much and which resources to assign to an individual piece of work. That is, senior partners knew roughly how to assess everything I suggested they use to measure outcomes relative to what the talent actually achieved, they just weren't, ultimately, willing to codify it or use it after the fact. I don't think in this client's case, they were afraid of accountability, although that's the most frequent reason for slipping context-sensitive performance analysis. Most often when decisionmakers hate accountability it's because the system that got them to a level where they get to make important decisions is one where they haven't had to be accountable -- add that in, and all of a sudden, you're changing their job and putting them aat risk of being outed for their non-successes. And a fair number of the decisionmakers have actually worked the system to slipstream things like the Texas Rangers' park factors, appearing to do better because they've had easier assignments or appearing to look taller because they're standing on a pitcher's mound instead of a hole.

Malcolm's work will cast light on many subjects -- this was just one team preview. Take a look at his work -- his contrarian and tart outlook should give you other management ideas you might apply in your own performance evaluation efforts.