Umpire controversies and the fallibility of Pitch F(x)

At Baseball Prospectus yesterday, Colin Wyers offered an unusual take on the Pavano/Berkman umpiring controversy.  I’ll attempt to summarize his piece, but if you have a BP subscription you should go on over and read the whole piece.  It’s rather well-done.  Wyers first notes that there are a plenitude of factors conspiring against the viewer to prevent them from calling a pitch a ball or a strike.  Part of the problem is that the camera is not positioned directly behind the pitcher, as it would have us believe.  The camera is positioned in the outfield at an offset and zoomed in (which really says something about the quality of the camera, no?).  This also affects our depth perception and our ability to perceive the distance between the pitcher and the catcher and the break of the pitches, no small thing.  The result, Wyers argues, is that the umpire has a leg up on the viewers at home:

The act of recording a pitch on video significantly changes how it looks compared to how it really is. This is why if you’re watching on TV, and the umpire calls a close pitch in a way you disagree with, it is far more likely that the umpire is right than you.

(And of course, the exact offset and the amount of magnification changes from park to park and sometimes batter to batter. Some camera setups are going to make it look like more pitches are inside than they really are, others may make it look like pitches are lower than they are, and so on.)

And then there’s the question of Pitch F(x).  The reliability of Pitch F(x) data is predicated on proper calibration, which is why the operators calibrate it before every game.  Yet the presence of some 42,000 people in the stadium that night would cause the stadium to actually move, taking the cameras with it.  This heightens the margin of error:

PITCHf/x reported the pitch at .67 feet away from the center of home plate as it crossed the front of the plate; according to Mike’s corrections, it was probably .72 feet away, with a margin of error of .06 feet (accounting for the random error in pitch location measurement, plus the estimated error in the correction.) The edge of the zone (in other words, the edge of the plate) extends to .71 feet from the center of the plate in either direction. Now, PITCHf/x is giving us the center of the ball—if any part of the ball catches the plate, it’s a strike. So the effective zone extends to roughly .83 feet from the center of the plate.

So we think that pitch was probably a strike, given what we’ve seen with PITCHf/x—but we’re not entirely certain. (Remember, standard error means a 68 percent chance outcomes occur within the MOE.) It is, essentially, a borderline pitch.

Not only that, but there is some difficulty in judging the vertical zone.  Every baseball observer knows that the strike zone varies from batter to batter, and so Pitch F(x) operators will set the top and bottom of the zone before every at-bat.  This adds margin of error to Pitch F(x) data:

So when using the F/X data to judge whether or not a pitch is in the zone, you need to account not only for measurement error in the pitched ball, but the operator’s estimate of the strike zone as well.

This is why it is expressly unhelpful to go to your favorite PITCHf/x website, pull up a scatterplot from a single game, and use it as evidence the umpire did a bad job. The responsible thing to do (and this is what MLB does when using PITCHf/x to grade umpires) is to correct for these calibration errors and to look at a larger sample of data.

This is also one reason it’s infeasible right now to use PITCHf/x to call balls and strikes in a live game. (There are others—timeliness is one, of course. Another is operator error—game scorers are just as human as umpires, and they sometimes make mistakes in associating the PITCHf/x data with the right pitch in the game, for instance.)

Over at Inside the Book, MGL offers a response to Wyers.  His argument is essentially that astute baseball observers have developed advanced pattern recognition that enables us them to judge with relative accuracy whether a ball was a strike or not.  In classic MGL form, he writes:

I say poppycock!  I claim I can call most pitches almost as well as the pitch f/x graphics you see on TV.  How can I do that even with all those camera problems that Colin talks about?  Well, when you watch thousand of games and you get feedback from umpires, batters, pitchers, AND, most importantly, the “pitchtrax” graphics on TV over the last 5 or 10 years, you somehow mentally can make all the necessary adjustments, the same way that a batter can figure out whether a pitch is going to be a ball or strike (Jeff Francouer and Pablo Sandoval excepted of course) in less than 1/2 a second.  In other words, for every pitch you see, you have seen that same pitch in the same visual location hundreds of times, and you have also seen what the umpire calls it, the reaction of the players, and many times, the exact location according to the TV strike zone graphic.  You can reach into your memory bank, and call the pitch pretty much as well as the average umpire, the average player, the pitchtrax graphic, etc.

Of course, there’s no real way to judge whether MGL is right unless he actually got behind the plate and tried to call balls and strikes as an umpire.  Wherever you come down on the issue, it’s safe to say that the issue is more complex than: “the umpire is an idiot because Gameday/Brooks says so”.  The umpire probably is an idiot though.