Monday, October 03, 2005

Analytics vs. Intuition

As a fairly analytical person myself, I found Michael Lewis' book "Moneyball" to be a revelation. Lewis chronicles the Oakland Athletics' use of baseball statistics in forming their team and making day-to-day decisions on the field. His underlying thesis is that General Manager Billy Beane's embrace of a statistical approach to baseball has allowed the Althletics to compete year in and year out with teams that have much greater resources at their disposal. Along the way, Lewis discusses the tension that was created between old-time baseball people who primarily used intuitive and emotional approaches to decision-making (e.g., identifying five tool players, going with gut instincts, staying with the hot bat, etc.) and the new-breed of statistical geeks (e.g., OBP, previous head-to-head matchups, ignoring pitchers' save numbers and won-loss records in favor of different performance measures, etc.).

I suspect that similar approaches (and tensions) are applicable in the US Chess League as well. In a previous post I discussed how analytics might play a role in team formation; but, I think over time, the search for an extra edge will lead to additional applications. For example, in determining the line-up each week previous head-to-head matchups might be considered. The historical success of your player against the typical opening repetoire of the expected opponent might also be factor. And with the existence and accessibility of large scale chess game databases, these types of analyses are reasonably easy to perform.

Today, I want to take a look at another potentially useful measure. Five weeks into the season, we can begin to look at the performance ratings of the various players. This measure can be thought of as the chess equivalent of a batter's OBP or a pitcher's ERA. By comparing a player's performance rating to their actual rating, we can assess how well each player is performing for the team relative to the expectation implied in their official USCL rating. Since the USCL imposes a rating cap each week, the overall success of a team is largely driven by the individual players' ability to perform above rating expectations.

So let's take a look at the relative performance ratings of the Boston Blitz through week 5. The format of the data is as follows -- Player Name (# of games played) - USCL Rating, Performance Rating, Performance Rating-USCL Rating. I have sorted the players by relative over/under-performance:

IM Perelshteyn (2) - 2576, 2953, +377
NM Riordan (2) - 2272, 2400, +128
IM Friedel (4) - 2447, 2520, +73
--------------------------------
GM Christiansen (3) - 2596, 2562, -34
Krasik (5) - 2123, 1979, -144
FM MacIntyre (2) - 2316, 2137, -179
FM Kelleher (2) - 2383, 1992, -392

Of course the sample sizes are fairly small, so you need to be careful about reading too much into these numbers (especially for those players who have played only two games). Nevertheless, they do reflect the actual performance of the players under USCL playing conditions.

The first thing to notice is that the team as a whole has just slightly underperformed by -33/game. As such it is not surprising that the Blitz' record through week 5 is right at .500 (2.5-2.5). This data can also be used to assess the validity of statements like "player x is on the team because he/she is actually stronger than their published rating." These claims of 'under-ratedness' can now be evaluated in the light of day.

The last use of performance rating information which I would like to discuss today is its application to the week-to-week team selection process. It definitely should not be used as the only input (that would be equivalent to riding the hot batter regardless of other mitigating factors), but I think it can be a useful source of information in selecting the line-up with the best chance of winning each week.

While not the best example to illustrate this point, let's take a look at the line-ups for this week's match against the Miami Sharks:

IM Josh Friedel (2477) vs. GM Julio Becerra (2622)
FM William Kelleher (2383) vs. FM Marcel Martinez (2469)
FM Paul MacIntyre (2316) vs. NM Miguel Espino (2272)
NM Charles Riordan (2272) vs. FM Javier Torres (2231)

Notice that the Blitz' line-up includes two over-performing players and two under-performing ones. This in itself is not remarkable; the Blitz don't really have the option of fielding all over-performing players at this point in time. However, I don't particularly like this line-up because one of the over-performing players (Friedel) is facing a major rating deficit on Board 1. As such, there is a reasonable chance that his relatively strong play won't result in any points for the team. In the middle of the line-up, Boston's under-performing players face a small rating deficit on Board 2 and a minor advantage on Board 3. Only on Board 4 does the team have both a rating advantage and an over-performing player. Of course, anything can happen in the games when they are played on Wednesday, but my pre-match assessment is that a 2-2 draw is the best the Blitz can reasonably expect from this line-up (Friedel 0, Kelleher 0 or 1/2, MacIntyre 1/2, Riordan 1).

Let's look at a couple of other options the Blitz might have considered (and, for all I know maybe they did -- e.g., IM Foygel may still be unavailable due to family issues and IM Perelshteyn may not have been able to play for some other reason). The first option would be to replace IM Friedel with IM Perelshteyn on Board 1. This doesn't change the mix of over and under-performing players, but does put a player on Board 1 with a much smaller rating deficit vs. his opponent. In addition, Eugene has already demonstrated an ability to successfully take on GMs in this league.

The second option is to go with the lineup -- Foygel, Friedel, MacIntyre, Riordan. Certainly Igor would have a hard time on Board 1, but in compensation the Blitz would have had rating advantages on the remaining three boards (two of the three with over-performing players). Had this option been feasible (and as mentioned earlier, it may very well have not been), it would have been my first choice.

For now, we'll wait for Wednesday and see what happens. As for BCC Weblog coverage of this week's match, I may do something a bit lighter on Wednesday night or just a post-match wrap-up. We'll see how much energy I have left after watching another round of San Luis.

No comments: