Truechess.com Compares the Champions: Who was the greatest chess player of all time?
Return to Compare the Champions Main Page
or Return to Truechess.com home
The Project
I highly recommend "Computer Analysis of World Chess Champions" by Matej Guid and Ivan Bratko which can be found
here. Many of the ideas discussed by
Guid and Bratko are applicable to this project.
(I had already started the programming for this project when I read the Guid-Bratko paper. My real inspiration
was Jeff Sonas's ChessMetrics site.)
For 15 months (from February 2007 through May 2008) I used 12 computing threads (on three Intel quad-core Q6600 computers running at 3.0 GHz)
to computer-analyze a total of
617,446 positions from 18,785 games involving the world's greatest players from Morphy to Anand.
(As a comparison, Guid and Bratko analyzed 37,000 positions from 1397 games and called this "a relatively large sample.")
Guid and Bratko analyzed only the games played in the World Championships while this project tried to analyze entire playing careers
-- for example, 69,084 positions from 2318 games were analyzed for just one player (Smyslov).
The first 8 moves in each game were ignored, but each subsequent position was analyzed for a full six minutes
(the average search was 17.4 iterations) by a modified version of Bob Hyatt's Crafty program (version 20.14) to
determine the a score for the best move available. A second search, of the same depth as before, assigns a score
to the move played in the game.
(The difference between the move made and the best move in the position is the raw error score.)
Finally, a third search calculates the complexity score for the position. (How complexity
for this project is calculated is explained below.)
Each move with a "raw error" of at least 1.25 pawn was subsequently examined (with no time limit)
with Rybka (version 2.3.2a running by itself on a quad-core Q6600 computer), the strongest program available at the time,
and analyzed until it was determined whether the move exceeded 0.75 pawns, the (arbitrary) blunder-threshold for this project.
The result of this analysis is found in the BLUNDERS PER 1000 MOVES column.
(Some blunder-candidates were examined for as long as 12 hours before a definitive classification was made.)
A complexity table was constructed to hold the
average raw error score, given a specific complexity of position, that the grandmasters' achieved, based on all 18,785 games.
Each player's non-blunder moves were compared to this table to determine the value found in the
PERCENT BETTER THAN THE AVERAGE GRANDMASTER MOVE column.
Larger values indicate better moves than average.
Blunders
A move is a blunder when: (A) the "best" move is rated at least 0.75 pawns better by Rybka [2.3.2a] and at least 1.25 pawns better by Crafty [20.14].
(B) the player is not making a desperate move because he is "lost" [i.e., the position is no worse than -2.00 pawns]; and
(C) the opponent has at least a reasonable chance of saving the game
[i.e., the opponent's subsequent disadvantage is less than 1.50 pawns].
(I analyzed with Rybka until I was sure the opponent actually had reasonable practical chances to at least draw.).
Average Raw Error per move
The Average Error per move is the average difference, in centi-pawns, between the "best" move in the position and the actual move played, excluding blunders
(as defined above). A score of 7.69 means that the player's average move was 7.69 centi-pawns worse than the best move -- this is the difference between
a computer score of +1.32 (for the best move) and +1.24 (rounded off to the nearest centi-pawn).
These scores were calculated by Crafty [20.14] at 6 minutes per move (see notes). Because Crafty, not Rybka, produced the "Average Error per move" and "Complexity" data,
these data should be taken with a grain of salt. For example, Karpov and Kramnik have somewhat similar styles (as measured by Crafty by their Complexity scores)
and very similar Average Error scores, so I would say that Karpov and Kramnik were not clearly differentiated in these two categories.
On the other hand, comparing the numbers between Capablanca and Karpov during their late 20's shows Capablanca with a more complex style and a lower average error,
so I would say that Capablanca is clearly differentiated as better during that period.
Percentage of Points Won
Percentage of Points Won is calculated, for all the games analyzed, in the normal way
(based on 1-point for victory, ½-point for a draw, and zero for a loss).
Complexity Details
I spent a lot of time devising the computer algorithm (and inserting it into Crafty) to calculate the Complexity score. This score is
meant to be a measure of the volatility or wildness or complexity inherent in a position.
It works this way: (1) the score (call it BEST_SCORE)
for the best move (call it BEST_MOVE) is identified
and the iteration (call it ITERATION) in which it was so identified is remembered;
(2) a new score (call it NEW_SCORE) during a search of
depth ITERATION-1 is done for BEST_MOVE;
(3) the difference between BEST_SCORE and NEW_SCORE
is the raw Complexity score (call it RAW_COMPLEXITY);
(4) the Complexity score you see displayed is RAW_COMPLEXITY * ITERATION / 10.
I was surprised and delighted that these Complexity scores actually seem indicative of a player's style.
For example, Steinitz gets into positions that are much more wild and out-of-control (on average) than those of any other player.
And Kasparov scores consistently higher (usually by a wide margin) than any other modern player, although Fischer, Tal, Botvinnik, Alekhine, and Euwe(!)
also had periods of relatively high complexity.
See the FAQ for an example of raw complexity.
Selection of Data
As already mentioned, the first eight moves were not computer-analyzed. I later decided, because modern players often play far more than the first
eight moves from memory, to begin the data analysis with move sixteen.
In addition, I eventually decided that computer-analysis of the endgame
still leaves much to be desired, so moves beyond the 40th are not included.
Other Notes
Since I wanted to avoid trying to determine what was a fashionable opening variation and what was a blunder, I did not analyze the first eight moves of a game.
(As it was, I did override the computer evaluation of two or three opening gambits (sacrifices) that occurred after move eight.)
Also, because I have found that correctly analyzing an endgame position can take days or weeks,
the data does not include moves greater than the 40th.
The Crafty search for the best move was limited to six minutes on a single core of an Intel Q6600 quad-core chip running at 3.0 GHz.
Eighty-six percent of the searches reached the 16th iteration;
95.5% of the searches reached the 15th iteration. (Guid-Bratko did a fixed search of 12 iterations.)
On average, "best" move was identified during the 9th iteration.
(Twenty-nine percent of the time the best move was not identified until iteration thirteen or beyond.)
Analysis with Rybka used all four cores of a 3.0 GHz Q6600 computer and was run until the result was clearly established.
Some Rybka searches were as long as 12 hours (and 40 iterations) for a single position, but the normal search was a few minutes and about 22 iterations.
One of my next projects is to categorize, with Rybka, those moves which Crafty found in the range of 0.75 to 1.25 worse than the "best" move.
This will obviously raise everybody's blunder rates by a slight amount.
The goal was to be as correct as possible, not to adhere slavishly to the computer's evaulation.
For example, I overrode the computers' decision and classified Fischer's 29...Bxh2 in the 1st game of the 1972 Fischer-Spassky match as a blunder --
true, the resulting position is not completely lost, but 29...Bxh2 turned an easy draw into a fight for survival (a fight that Fischer lost).
I prefer to look at the data with draws excluded. (Although the Summary Rankings are presented both ways.) I think it is misleading that certain players
should have their blunder rates dramatically lowered because of a fondness for the "grandmaster" draw.
Return to Compare the Champions Main Page
or Return to Truechess.com home