Management by Baseball

My compañero Dave "Doctor K" Kurlan (author of the insightful/useful baseball-themed sales "how-to" book Baseline Selling), has written a cool entry on his blog, Baseball and Sales Management by the Numbers about how sales stats too often are like ERA is in baseball -- providing an incomplete or even highly inaccurate indication of the effectiveness of individual sales staffers.

I believe one of the least consistent, and most misleading statistics in Baseball is ERA or Earned Run Average; the number of earned runs a pitcher allows per nine innings pitched. {SNIP}

For example, during his final years in Boston, Roger Clemens was virtually unhittable for 5-6 innings each time he pitched but gave up a lot of his runs in the 7th and 8th inning. With a stronger, more reliable bullpen, Clemens would have been out of the game after six so his ERA for those years was higher by perhaps a run or two because of the lack of a bullpen.

{SNIP}Sparky Anderson, manager of the Cincinnati's Big Red Machine in the seventies, and later, the Detroit Tigers, was known for his quick hook. Grady Little, famous for leaving Pedro Martinez in for one batter too many in the 2003 American League Championship Series, had a tendency to leave pitchers in longer than he should. I'm guessing (without statistical backup) that pitchers under Anderson had a lower ERA than pitchers under Little.

Turning to sales, there are a number of statistics that are equally difficult to equate with performance, the most obvious being revenue. Many salespeople, considered top producers by their companies, top the charts for revenue but don't perform in such a manner as to justify the attention, rewards or commissions that they earn. They may have inherited their accounts, built them up over decades, or have the best territory. For many of them, if you took those accounts away and directed them to go out and sell something, many of them would fall flat on their faces.

Great point Dr. K. I can't add anything important to what's in his full post -- go read it in full. It transcends sales stats, by the way -- Kurlan's description of metrics without context (ERA among them), applies to any performance measurement model. My own essay on the limitations of metrics such as ERA appeared in two parts: an October '03 essay here and another reference here (warning: the graphics they linked to are no longer on the web -- you can see most of them by right-click copying the link references and then going through the Internet Wayback Archive at http://web.archive.org, pasting those copied addresses into Wayback's search box.

I'd like to elaborate a little on his Clemens assertion. He was working from memory. I get to work from Sean Forman's magnificent Play Index, a breathtakingly useful research tool added to his (already best single www research site in any discipline) Baseball-Reference.Com. My memory was a little different, and we were both right -- a good example of what happens when two numerate people look at the same data.

Clemens' last year in Boston was 1996. Was he unhittable for 5 or 6 innings but not after?

By Inning, 1996

 Split         G   PA   AB   R   H  2B 3B HR  BB  SO   BA   OBP   SLG   OPS  BAbip tOPS+ 
+------------+---+----+----+---+---+--+--+--+---+---+-----+-----+-----+-----+-----+-----+
 Innings 1-3   34  425  378  40  86 19  2  8  42 107  .228  .307  .352  .659  .294    95 
 Innings 4-6   34  418  370  39  84 14  2  7  42 100  .227  .307  .332  .639  .289    90 
 Innings 7-9   27  189  163  20  46  6  3  4  22  50  .282  .364  .429  .793  .378   135 

Wow. How's that for a smart fan's memory. Use the right-hand column, tOPS+, as an indicator...this metric normalizes to 100 Clemens' 1996 composite performance, with a 95 for example, meaning 5% better than his composite and 135 meaning 35% less effective at stopping offense. He was a different pitcher in innings 6-9, not terrible certainly (look at his BA, OBP and SLG), but not good either. As Dr. K notes, his manager Kevin Kennedy should have had him on a short leash after 6 innings...at least if you look at this chart.

But let me give you another table for Clemens' 1996.

By Inning, 1996

 Split         G   PA   AB   R   H  2B 3B HR  BB  SO   BA   OBP   SLG   OPS  BAbip tOPS+ 
+------------+---+----+----+---+---+--+--+--+---+---+-----+-----+-----+-----+-----+-----+
 1st inning    34  134  116   5  21  5  0  0  18  35  .181  .291  .224  .515  .259    54 
 2nd inning    34  148  138  18  35  6  2  6   8  39  .254  .299  .457  .756  .312   122 
 3rd inning    34  143  124  17  30  8  0  2  16  33  .242  .329  .355  .684  .308   103 
 4th inning    34  160  138  23  39  3  0  4  20  38  .283  .375  .391  .766  .361   127 
 5th inning    33  137  121  13  27  7  2  1  13  28  .223  .294  .339  .633  .277    87 
 6th inning    32  121  111   3  18  4  0  2   9  34  .162  .231  .252  .483  .213    43 
 7th inning    27  103   87   8  19  2  0  2  13  28  .218  .317  .310  .627  .293    87 
 8th inning    16   63   54  10  19  2  2  2   8  16  .352  .429  .574 1.003  .459   196 
 9th inning     5   23   22   2   8  2  1  0   1   6  .364  .391  .545  .936  .500   176 

On closer examination of tOPS+, you'll notice he actually was pretty good in the 7th, 13% better than his composite "norm". Compared to the lights-out extra-crunchewy 6th-inning performance though, it's a shear off that then tanks in the 8th and 9th innings. I think Dr. K might have blended together 1995 with 1996. Look at Clemens' numbers for the truncated-to-144-games season.

By Inning, 1995

 Split         G   PA   AB   R   H  2B 3B HR  BB  SO   BA   OBP   SLG   OPS  BAbip tOPS+ 
+------------+---+----+----+---+---+--+--+--+---+---+-----+-----+-----+-----+-----+-----+
 Innings 1-3   23  300  265  40  69 11  2  5  25  72  .260  .340  .374  .714  .337    93 
 Innings 4-6   22  255  222  24  55  9  0  9  25  48  .248  .336  .410  .746  .277   101 
 Innings 7-9   12   68   57   5  17  4  0  1  10  12  .298  .412  .421  .833  .364   126 

 1st inning    23   94   84   9  17  3  1  1   7  23  .202  .287  .298  .585  .267    59 
 2nd inning    23   97   85  14  20  2  0  2   8  20  .235  .320  .329  .649  .281    76 
 3rd inning    22  109   96  17  32  6  1  2  10  29  .333  .404  .479  .883  .455   139 
 4th inning    22   92   81   6  17  2  0  2   9  20  .210  .304  .309  .613  .254    66 
 5th inning    22   98   83  15  25  3  0  7   9  13  .301  .385  .590  .975  .281   161 
 6th inning    18   65   58   3  13  4  0  0   7  15  .224  .308  .293  .601  .302    64 
 7th inning    12   47   39   4  10  3  0  1   7   8  .256  .383  .410  .793  .300   115 
 8th inning     6   21   18   1   7  1  0  0   3   4  .389  .476  .444  .920  .500   150 

In that previous season, The Rocket really was pretty poor in 7th and 8th innings, and his red glare burned out pretty markedly -- so much so that he couldn't (or perhaps just didn't) appear in a single one of his 23 games' 9th innings. In 1997, pitching for Toronto, his pattern returned to his 1994 and previous pattern -- pretty consistently great across innings.

Consistency is a necessary ingredient for most organizations' success. But most organizations overstate the value of highly context-sensitive measures without taking context into consideration won't achieve consistency & will only be able to succeed with lot of luck. Dr. K knows that, and you should listen to him.