The Perils of Five-Star Rating Systems

There are many ways of rating “things”. Michelin will give your restaurant zero to (if you’re very lucky) three stars. All sorts of things are scored on percentage scales. On hot ot not, and elsewhere in life, you’re judged on how well you match up to being a “perfect ten”. But probably the majority of rating systems operate on a scale of from one to five stars (or… insert non-starry symbol of choice here. Or even the numbers one to five). There are very good reasons for this. A major one, I guess, is the nature of the human brain: short-term memory generally has seven plus or minus two slots for us to store information in, so if you go much over five then you risk having more discrete levels than you can actually comprehend at the same time. Also, five fits a standard scale centred around an average value, with values for good and ungood, plus extreme values for very good and very ungood. So, five is cool.

But after a few months of using iTunes and rating my songs according to their five star system, I started to run into problems. I started to want half-stars in between the stars. Or at the very least, a sixth star. I had five star songs which were my favourites of all time, but I started wishing that I had an extra level to go to when a song was really one of my favourite favourites, one of those ten-or-so desert island songs. I also started getting confused somewhere between three and four stars, and between four and five, wondering all the time exactly what I meant by those ratings, and finding certain songs which I wished I could slot somewhere in between. I also found that one and two-star songs were very few and far between, and often I would delete a song from my library rather than ranking it as one star.

Recently I started using Adobe Lightroom and I ran into pretty much the same problem. Just what does “three stars” mean, what does any particular ranking mean? And then it dawned on me, where I had been going wrong. I’d been viewing three, being the central ranking, as “average”. Now, that’s fine if you’re reviewing films for your local newspaper (well, actually it’s not totally fine, but that’s a whole different discussion…), but for my tasks I needed a whole different approach. Rather than average, three should represent “pretty damn good, actually”. Even a lowly one should represent something which has some value for me, otherwise why am I bothering giving it valuable hard disk space? So today, I hard-coded my rating system. Here is what the numbers now mean to me:

  1. Not special, only really has value within its own category. For photos, these are the ones I want to keep in the set, to get a full picture, but which I would never look at outside the context of that set. For music, if I was listening to an entire album then I’d want to keep these tracks in, but I would never listen to them outside that context.
  2. OK, but again mainly valuable within its own category. If I wanted to give an extended over-view of a photo-set or a musician’s work, I would include these.
  3. Good. When I am going through photos in Lightroom trying to create a cut-down set which I can put on a website, I will look at the three-stars and upwards (unless I have a real paucity of good pictures). For day-to-day listening in iTunes, anything with three stars or more ought to be stuff that I really rather like, and would be suitable for random play.
  4. Very good. If a photo from within a category has four stars, I would consider using it elsewhere, e.g. if it’s a portrait then I will look at including it in a “best of” collection of portraits. In iTunes, these will all be favourites and I’d be quite happy to stick a CD-ROM of four-star tracks in the car stereo or stick on a random mix when friends come over (well, OK, some tracks wouldn’t be suitable for cars or friends, but that’s more down to the eclecticism of my music collection than anything else).
  5. Excellent. The best of the best. My masterpiece photos. The pieces of music I would be lost without. Contrary to my earlier iTunes experiences, five stars should actually be a hell of a lot rarer than one stars. While rating my photos, I’ve found that most of my photos are one or two stars, quite a few are three, very few are four and, in the couple of sets I’ve rated today, I haven’t yet found a five.

Anything which doesn’t even rank one miserable star should be deleted immediately.