Over the weekend, there were two pieces on Boston.com that caught my eye.
Both were on the ever controversial subject of Ultimate Zone Rating, or UZR for short. One was very well reasoned and objective. One was, well, less so.
The first was from old friend Pete Abraham and it was, as Moshe said on Twitter, it was fair, reasoned, and balanced.
Abe’s article discussed the ups and downs of UZR, and highlighted something that few, especially the latter writer (Dan Shaughnessy), seem to realize:
“UZR? That stuff seems like a lot of mumbo-jumbo to me,’’ Drew said. “Do those guys even watch the games?’’
Every play of every game, several times over, in fact. The data used to compile UZR and other advanced statistics comes from Baseball Info Solutions, a company in Coplay, Pa. Owner John Dewan hires only former college or professional players and trains them to chart games via video and record the specific characteristics of every batted ball.
I’m going to repeat that: UZR is compiled by former players who watch every play of every game many times over.
Shaughnessy’s article makes one long for Fire Joe Morgan. River Ave. Blues reader Robten did a mini-FJM job:
“What if you have a bad hamstring and you can’t get to a ball up the line?”
Then your range will be diminished, which will show up in the stats.
“We’ve got too many stats in this game today.”
How about we just get rid of all stats. The only record of the game will be the eyes of Tom Giordano. Since he’ll have to attend every game, the season will now last three years. Of course, the smart guy that he is, I’m sure that he’ll develop some sort of short-hand way of keeping track of things. For instance, some numerical notation for keeping track of things like the times someone gets on base or doesn’t make a play, but like I said, having to develop a numerical system to keep track of games will be worth it just to get rid of all those damn stats!
While baseball is played on green grass in fresh-air stadiums, an army of geeks will be holed up in their basements, under a naked light bulb, crunching numbers and finding new equations to measure something that simply can’t be quantified.
Like the amount of times that a sportswriter denounces stats as a threat to the game of baseball? He’s right. Quantifying the arrogant ignorance of most sportswriters today is like dividing by zero; one cannot simply calculate infinity.
I don’t trust the defensive numbers.
Seriously. I mean, what do they have to be defensive about? JUST WHAT ARE YOU HIDING UZR?!?!? Drug dealing? Gun running? Death Panels? Oh god…the call is coming from inside the house! RUN FOR YOUR LIVES! IT’S THE DEFENSIVE NUMBERS!
Instead of going through Shaughnessy’s article piece by piece, I’ll simply say that this type of article is everything that is preventing advanced metrics from gaining leverage in the mainstream baseball world. Closed minded ignorance–make no mistake, that’s exactly what it is–helps no one. Hopefully, a lot more people read Petey Abe’s article over the weekend. Articles like his will help the casual fan understand something that isn’t so scary after all. I applaud Peter Abraham for stepping out of his comfort zone a bit and attempted to grasp something new; let’s hope it catches on.
If you enjoyed this article, please consider sharing it!
6 Responses to Two Differing Perspectives on UZR
Leave a Reply
TIQ IQ
This was also a great quote:
“Lichtman believes UZR is 90 percent accurate, but in the estimation of Dewan and Pinto, defense is only being evaluated at 60 percent efficiency, far lower than offense and pitching.”
Proponents of UZR also need to understand this point. The numbers of a certain player may be completely inaccurate. Please, at this point in time, do not use UZR to make definitive statements about a player’s value, because there’s a good chance that number is very wrong. Unfortunately, WAR incorporates these very inefficient and sometimes inaccurate defensive statistics in their valuations of players, and more unfortunately, people use WAR as the go-to statistic for determining the entirety of any given player’s value. Don’t do that. And don’t call people who question this statistics’ validity ignorant.
There’s very good reason for people to not trust today’s defensive statistics. They’re simply not trustworthy, and the leaders in the statistical market are the ones who are saying this. Let’s try not to be close-minded on the other side of the spectrum either.
As a quoted source in this article, I think I can speak with (little to no) authority to say that the problem is that writers like Shaughnessy are not critiquing the limits of UZR from a rational and well-thought perspective. Rather, they are simply railing against any and all statistics which they find to be “strange” and “different” and “ARod-ing baseball.” Even those who make use of UZR know, for instance, that any one season is imperfect in that it is essentially a small sample size and that it takes several years to get a sense of a player’s fielding ability. In other words, some of the complaints about UZR are essentially a straw-man argument. No one points to any one stat and says, “This tells me everything I need to know.” The issue is trying to keep track and measure something that, to this point, has only been marginally addressed in a limited way (errors, Sportscenter highlights, etc.).
Now, if there are limits to defensive metrics because they are new and in the process of development and if you have a patient, rational and thorough critique of them, that’s one thing. But, if you’re just spouting off about how there are “too many stats” which don’t capture the “heart” of the game–like, I don’t know, getting on base and scoring and preventing runs–that’s something else. That “Je ne sais quoi” is what Shaughnessy et. al are full of.
I tend to agree with much in both of the comments above. In tha Abraham article, John Dewan and David Pinto, who have,created metrics designed to measure defensive ability, state that in their opinion, current methodologies are operating at approximately 60% efficiency. This is from individuals who have a vested personal and financial interest in promoting the validity of these measures. As Andrew correctly states, this concededly inefficient metric is one of the components for WAR which purportedly measures a player’s entire value. To then blindly quote someone’s WAR value in comparison to another player’s as an absolute measure of their relative abilities is a logically fallacious appeal to an authority which doesn’t exist.Further, if these metrics- UZR, plus/minus,etc- purport to measure the same thing- defensive range and sure handedness- why do they often disagree in terms of the relative abilities of various players?.As people well grounded in the subject, from Bill James to Theo Epstein , who is quoted in both articles incorporated in the original post, understand, even the most finely calibrated statistical analysis must be augmented by the eyeball. Do they match? If not, why not? Where is the disconnect? Obviously the newer metrics, in attempting to quantitatively evaluate range, are a considerable improvement over gross fielding percentage and the like. However,few with any sophistication about baseball, ever took fielding percentage, standing by itself, as a true measure of a player’s defensive abilities. Most understood for example, that a Steve Garvey who positioned himself in close proximity to first base was not nearly as good a fielder as Keith Hernandez who had considerably more range but consistently came in with a lower fielding percentage. Obviously, Shaughnessy’s complete disdain for the newer defensive metrics is ignorant. However, a complete obedience to measurements which even their designers admit are far from perfect and sometimes produce anomalous or questionable results – Texeira 2009 UZR for example-without augmentation from experienced and trained people who are at the games is equally incomplete
A successful incorporation of both the numbers and visual observation will produce the most complete and accurate assessments.
If it depends on a human’s observations, it’s not very likely to be completely free of subjectivity. From what I’ve seen, it’s not very accurate at all.
On the other hand, the attacks from the articles have very little to do with the usefulness of UZR or other defensive metrics. The word ‘luddite’ always springs to mind whenever I read anything like the comments quoted in the article.
i agree with your premise essentially- but there is the issue of where does it end? stats are great up to a point. yet at another point it becomes extraneous and enough is enough. stats for analisys has a place. but we need to keep it from growing out of hand
ps: your point about the advanced metrics not being accepted universally has something to do with fans and commentators/writers not really understanding them, it might well be a generational thing, or an education thing