A look at ratings #2 – the ELO system

In a previous post, I took a look at some of the statistical correlations between a player’s scoring averages and their Scrabble® rating.  Over this past year, I’ve seen my own personal rating jump by over 400 points (in tournament play) and about 200 points at club play.  Seems like everyone likes to talk about ratings – either theirs going up or down.

In this post, I’m taking a closer look at the rating system.  For those of you who don’t know, here in North America, the National Scrabble® Association (NSA) uses the ELO system originally developed and still used for chess.  I won’t go into too much detail, but this system is based on estimating a player’s "true skill" by using statistical measures of interpreting wins & losses.

The ELO rating system
Basically, each player has a rating.  This rating is then compared with your opponent’s rating and an "expected" win % is determined based on how far apart you and your opponent’s ratings are.  After each game is played, you calculate the difference between the expected % of wins and our performance.  If two players are equal in skill, then in theory, they should each win 50% of the time against one another.  A higher rating would thus suggest higher skill and higher likelihood of winning any particular game against lower rated players.  The same works when playing against higher rated players.  The theory is that over time, your true skill should be reflected in your rating because your "good and bad" playing should even out over time and your true rating should emerge.

For a more detailed explanation of the ELO system, please view the Wikipedia entry titled "Elo rating system".

Complaints about the rating system(s) in Scrabble
In the recent issue of Scrabble News (#218 I believe), there was a note about the formation of a ratings committee.  This group is going to suggest an alternative ratings formula and come-up with something for use by 2009.  I’m not sure why people feel a need to change the current system, but I’ve heard the following complaints:

  • Ratings have been decreasing/"deflating" (read below for some more info about this)
  • Point "spreads" should be acknowledged – for example a player winning by 500 points should be recognized
  • Similar to the point above, "all wins aren’t equal"
  • It’s too easier to "lose" rating points but harder to gain them

Personally, I’m not sure what all the complaining and clamoring is all about.  No system is perfect.  In science, it’s what we call "measurement error" in that every measurement we take has some sort of error involved.  For things like measuring the height of something, we can be pretty good, but with things like measuring someone’s intelligence, it can be hit or miss.

Ratings deflation in Scrabble®
In the case of people complaining about ratings "going down", the Wikipedia entry has a very interesting example that describes this phenomenon of ratings decreasing even though a player’s skill level remains the same (it’s about halfway through the section titled "ratings inflation and deflation").  Because of the way the system is set-up, if only a few players improve their skill while everyone else stays the same, what happens is that the player who got better gets a big jump in ratings while the others who stayed the same decrease in ratings – thus the "deflation".

I believe this is what is happening in Scrabble today.  Players who used to be in the 1500-1600 range have had their ratings drop a good 200-300 points in the past few years.  What I’ve seen as a new player is that more players are getting better and at a quicker rate.  We can see this in some of the younger players getting involved, but mostly because of the better study tools available.  Computer simulation software allows players to analyze games and become better.  We also see this in the decreasing number of (active) players in the 1900s and 2000s.  What we don’t see, however, is a change in the relative rankings of players.  The top players are still at the top in roughly the same order as they were before.  It’s those players who haven’t improved much over the past few years who have seen their ratings plummet.  They aren’t any worse than they were before.  It’s just that everyone else is getting better and so gaining and/or maintaining a rating is much more difficult.

In my example, based on my play at my local club, I’ve felt that I was about a 1150-1250 player and yet my tournament/NSA rating was about 800 for over a year.  Once I figured out a few things about playing in tournaments, particularly about playing against lower rated players, I’ve seen my rating jump 400+ points to about 1205 (estimated).  I’ve jumped up so much because my true level of play is higher than 800.  Now that I’m at the 1200 level, I don’t expect to see much gains in my rating until I start improving again.  Those people who I beat would see their rating go down, not because they played poorly, but because they played someone who was rated lower than them.  Similarly, I would expect it very difficult to keep my rating if I start to play against opponents who are getting better.  We’re all on this treadmill of needing to improve or else see our ratings go down.

What people also complain about is the fact that unlike in chess, Scrabble® involves quite a bit of randomness and luck.  Sometimes you get some crappy tiles and it’s almost impossible to win.  Unfortunately, there’s really not much to do.  In the card game Bridge, the solution was to create "duplicate bridge", where every team plays the exact same hands/cards.  The theory is that if everyone plays the same cards/hand then those with better skill should do better.  It’s like having everyone take a standardized test and getting a score.  Unfortunately, I don’t think this is likely to happen in Scrabble.

Final thoughts
So after a brief look at the ELO system, I’m not sure if any new system that may (or may not be introduced) will fair any better.  Personally, I think the secret is not to worry about the rating and focus on just becoming a better player.  The ratings will take care of themselves.

Powered by Qumana

11 Responses to A look at ratings #2 – the ELO system

  1. Liz says:

    Thanks for your thoughts! This was helpful to a newbie learning about the elo rating system. 🙂

  2. Larry Beaird says:

    I’m trying to set up a rating system for a new Scrabble club where all (or almost all) the players have NO rating. I’d like to use scores in addition to wins and losses to figure the ratings.

    I created an Excel spreadsheet that keeps track of points for, points against, and wins and losses.

    What do you suggest for this new club?

    Thanks a bunch!!

  3. mary hobbs says:

    My rating is 1442 on my laptop, and 1378 on my home computer? Same games, same facebook account. what is the explanation for that? thanks.

  4. Marie Black Service says:

    I want to know what the E stands for and the L and the O.

    • PM says:

      It is the creator’s last name “The Elo rating system is a method for calculating the relative skill levels of players in two-player games such as chess. It is named after its creator Arpad Elo, a Hungarian-born American physics professor”

  5. Jim says:

    I have played 6 games of Scrabble…..I have lost 3 and won 3. The 3 loses show, but that is all. It only shows I have played those 3 games, not the other 3 I won????

  6. matt says:

    I have a rating of 1828 on Facebook Scrabble. What does this mean or how does it compare with other ratings? Thanks

  7. Paul says:

    I like the idea of duplicate scrabble. It would be fairly easy to set up using a database and could be an option in the online versions. Another online option would be to calculate the best possible score and work out the percentage that players are away from that.

  8. i think the point difference between the winner and loser of a game should be included in there score rating as extra point like if the winner won by a margin of 49 pts the winner should also get those points etc

  9. Saly B says:

    Adding in increases in points score for bingos or high scores is partly rewarding skill but more reflects the luck of the tiles you get.

  10. Clinton Dawkins says:

    Big wins aren’t frequent anyway. People lose big too.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: