The second half of the Computer Goat Rally,
courtesy of Kamakshi Tandon
*
Steve,
Sounds like you’re still scarred by the experience of trying to figure out who the greatest of the last 40 years was. At least you were choosing between the right names. :) I remember the same flak when running a greatest shots series for the website that combined the men and women. What’s a more interesting question, which player you’d place the most money on to win or the player you’d pay most money to watch?
As you said, it’s funny we were talking about adding bonus points to rankings the week before last and now here’s a system that is nothing but bonus points. Trying to figure out a system for calculating who’s the best right now is complicated enough, so you can imagine how hard it gets when you’re trying to calculate the best of all time—different era, different equipment, different levels of play.
Because there are so many factors, any discussion that goes on long enough will eventually get to a point where someone will say that we need to agree on criteria. How do we determine greatest? Slam wins? Dominance? Absolute level of play? We all know the drill. This is usually the point when it’s time to finish off your drink and call it a night, because the fun part of the discussion is over and it’s going to be all metrics and formulas from here.
If you’re sufficiently hardcore and carry on, statistics are the easiest thing to reach for because they’re the most objective. Once you have the criteria, there’s no argument about how to apply it—you just crunch the numbers and get the results. Then the debate restarts, only this time over the list that’s been produced.
Like most people, I raised an eyebrow at the latest attempt, by a science professor at Northwestern. It puts Connors at the top, followed by Lendl, McEnroe, Vilas, Agassi and Edberg, none of whom usually make the top three of the traditional GOAT debate, which centers around Federer, Sampras and Laver. But I also feel a little bad about beating the guy up about it, because he was just applying a statistical concept to tennis and seeing what happened. I’m not sure he was signing up for having it kicked around and discussed by tennis people and media articles on its practical merits.
Like you said, what comes out depends on what goes in. One problem is that I can’t tell exactly what’s going in here, despite ploughing through the study twice (the fact that I kind of skipped the math might be the problem). Did you know that a draw is a “binary rooted tree?” Me neither. As far as I can tell, you get points for every player you beat based on how many players the other player has beaten. So for a single tournament, you’d get x points for a first-round win, then a little more for a second because you’re beating a player who’s beaten another player. Then in the next round you get even more because your opponent will have beaten two players to get there. The useful thing is that you also end up with more when you win bigger tournaments because each extra round you beat a more valuable player, so that seventh round at a Slam ends up making you double what you’d get for winning a six-round even like a Masters.
And then you go on to the next event, where if you beat the winner of the last tournament you get a lot more points than if you beat the guy who lost in the third round. It’s quite an interesting concept, and it certainly gets us out of our usual mindset of Slam wins. But I’m not sure if it counts all your wins when measuring your value as a win—if it does, it’s a major flaw because wins over a player would be most valuable when he was at the end of his career, but he wouldn’t be playing at the level he was when he won all the matches that built his quality score. Why should Soderling’s win over Nadal at the French be worth less than when some kid beats the ten-time French Open champ at 30 years old? Or maybe it doesn’t do that. I couldn’t parse the equations well enough to figure out exactly that’s going on.
In terms of the criteria I gave last time for rankings systems, what this means is that this one doesn’t have the level playing field, and only has the quality control aspect in terms of opponent, not level of tournament (I get the argument that it does this internally by looking at the quality of the field instead of the prestige of the tournament, but that’s not the only thing that counts—there’s also whether players try to play their best there, how good a test the conditions are, etc.) The other thing is that losses don’t count at all. Connors certainly won a lot, but he lost a lot of crucial matches too.
In fact, when you read our fellow NY restaurant frequenter Tom Perrotta’s article about it in the Wall Street Journal, there’s a strange mention about a player’s quality score being determined by “matches, especially victories, against quality opponents.” Wait—especially victories? So just playing a match to a quality opponent counts for something, even if you lose?
Also included is a list of the players’ "quality wins," which isn’t in the study itself. I have no idea what it takes to get a quality win, but Connors has 178 and Nadal has just 21. Now it’s acknowledged that Nadal is underranked because he’s still relatively early in his career, but as our fellow tournament restaurant frequenter Matt Cronin noted in his column on tennisreporters.net about the system, “Nadal … owns a combined 30 wins over Federer and Novak Djokovic, and has beaten Slam winners Agassi, Lleyton Hewitt and Roddick.”
Another thing, which the comments on your post talked about—there’s no accounting for surface, which obviously has a big impact on the significance of a win against a top player.
Because Connors came out on top, followed by Lendl, it’s tempting to think that the system gives an edge to grinders who play and win a lot of matches. But McEnroe and Agassi are high on the list too, and certainly they can’t be considered heavy mileage players. As Matt suggests, the results suggest its weighed too heavily in favor of longevity. I mean, Federer’s been playing for a long time now, and he’s only seventh on the list. Sampras is way below Agassi. And goodness knows how Nastase managed to top Borg. In a way, I think what this system measures best is the sustained ability to go deep into tournaments. That’s an achievement, certainly, but we tend to salute the players who do that, not bow to them.
The most interesting this about the study is probably the early commentary on the data, where it’s mentioned that there were 75 tournaments a year in the early Open Era, and that the number of players playing ATP events has shrunk from about 400 to 300.
There’s also a comment about the predictive value of the system—apparently the player who ended on top in any given year often ended the following year at the top of the ATP rankings. In way, this supports what Robert Waltz, who does a lot of stats for Daily Tennis, has long said about bonus points being a good predictor of who’s about to rise or fall in the normal rankings.
Anyway, to return to what I said at the beginning, the question of who’s the greatest ever is too complicated for anyone to ever design a system that’ll come close to capturing the essence of the way we debate it. There’s too many criteria, and they’re too subjective. Is a player dominant because he’s so great, or because it’s a "weak era," as the phrase goes? Do your wins count when your nemesis is injured? When we judge wins and losses, we can take into account very precise things you can’t set up a system to capture—that was a huge win because he beat so-and-so when he was playing great at home on his best surface. Or, he was hurt, or tired, or just not playing well. Or, yeah, but the court was really fast, or really slow. It was closer than the score. He had a match point. And perhaps most subjectively of all, we look at their legend, their legacies, their impact. Maybe it shouldn’t matter. But maybe it should.
We always talk about the difficulties in comparing across eras and different fields and technologies, but the other thing is that players themselves are hard to compare because they’re so different. Who was better, Lleyton Hewitt or Marat Safin? Same era, comparable records, and yes, a 7-7 head-to-head. Some people would say Safin because Safin on his best day was a better player. But Hewitt’s normal level was probably better than Safin’s.
But just thinking about these things gives you a better insight into each of them, and into your own values. Comparing the greats helps to see what made them so great, and all these different ways of doing it, statistical and not, illuminates another aspect of the picture and gives it more dimension. When it comes to the greatest ever debate, it’s the journey, not the destination, that counts.
Kamakshi