Creating A Prospect Rating System

In the past, I’ve toyed with the idea of a prospect rating system with a core structure borrowed from Hockey’s Future’s rating system, with some cosmetics borrowed from Project Prospect. I like the rating system because instead of just giving a prospect an overall rating (Sickels’ grading or Kevin Goldstein’s stars), it attempts to answer the two big questions about minor league players: how good can they be,  and what are their chances of getting there? It does so by assigning two grades to the prospect. You can see an explanation of the system at the link above.

I’ve designed a system similar, but in my opinion better defined, to the HF systems. I assign two grades to each player – a “Talent Rating” from 1 to 10 and a “Risk Rating” from A to F. They are separately ordinal variables, but do not interact. That means that you can say that a 7.0C > 7.0D, and that a 7.0C > 6.0C, but you can’t say that an 8.0c > 7.0B.

Talent rating represents what I believe the prospect can play like in their prime years if 75% of things go right for them. I hate the term ceiling, because miracles can happen all the time in baseball. Tim Wakefield was drafted as a 1st baseman when no one in the world thought his ceiling was actually a 200-win pitcher. Colin Curtis could wake up tomorrow, have eye surgery, and somehow start hitting 30 home runs. Virtually any player could become a Hall of Famer if absolutely everything goes right, but that doesn’t help us in evaluating prospects. Instead we look at the case of what happens when most things go right. Prime years is defined as the average of what you would expect a player of that rating to play like in the top third of their career single seasons. The following tables explains the ratings:

Risk rating represents the relative probability that the player will have prime years in their career equal to their talent rating. Prospects are inherently uncertain, and not even the safest prospect is a sure bet in the major leagues. However, some prospects are more safe than others, and that’s what risk rating is trying to get at. By no means is an ‘A’ rating a sure bet. The following table explains the risk rating system:

Risk rating is actually a qualitative assessment of three different types of risk. I call these types of risk Distance Risk, Injury Risk, and Talent Risk. They aren’t completely without overlap, but I try my best to define them precisely. The following table explains the different types of risk:

The definition of injury risk should be pretty self evident. Talent risk generally deals with what I call “scouting report problems” – Slade Heathcott may develop into a poor man’s Kenny Lofton, or he could never get his strikeouts under control and fail to develop as a hitter. Distance overlaps a bit with the others. It refers to the general principle that the more unproven a prospect, the more uncertain the future is for them. I think that its important to consider when assessing the risk of any prospect, even if in a perfect world the overlap would render it unnecessary.

Put it all together and you’ve got prospect tables like this:

Which I think look pretty darn cool, if I don’t say so myself. Tomorrow, I will be releasing an updated prospect list just like this one, but with all 30 prospects rated.

I really want to reiterate that all these ratings are just clearing defining what my opinion of a prospect is. Its qualitative and subjective. However, I think it is a much better way of ranking these guys. Sean has some ideas to make some of the risk ratings a little more objective, but that’s an improvement for another day.

What do you all think? This is the first draft of the system. I’m open to criticism. If it all works out, I’d hope to develop some kind of dynamic spreadsheet with every player on every Yankee minor league team rated.

Follow Me On Twitter