An argument for “weighted” customer ratings
~ 24 January 2006 ~
So yesterday morning I’m on Amazon looking for a book on mobile devices, and I end up buying four books — three more than I planned on buying.
I blame it on their precisely persistent presentation (say that three times fast) of related books, strategically placed on product pages, shopping cart pages, and just about anywhere else on the site.
There is, however, one area in which Amazon — and many other commerce sites — could improve: Customer ratings on search pages. More precisely, these sites could improve the visual appearance of ratings (usually done with stars of some sort) when one product has been rated by a larger audience than that of a similar product with the exact same rating (or, same number of stars) by a much smaller audience. For the purposes of this article, I’ll refer to it as weighted customer ratings.
Here’s an example: Over the holiday break, I performed a search on “dvd player” while shopping at Amazon. The following results came up on the first page (click for large version):
Notably, two DVD players, one a Panasonic and the other a Philips, similarly priced, had the exact same customer rating — 4 stars:
However, click a step further and things are quite different:
Panasonic DVD Player (10 reviews)
Philips DVD Player (437 reviews)
So, here we have two DVD players, similar price, exact same rating, yet one reviewed by 10 customers while the other reviewed by 437. Which rating would you trust more? I find wisdom in crowds, especially large ones when buying online based on customer ratings. I’d naturally prefer the Philips over the Panasonic, if features and price were similar and no strong brand preference existed.
Thus, it begs the question, Should the two ratings be “weighted” visually somehow to indicate the fact that the ratings, while the same number of stars, are actually based on drastically different review counts? Hypothetically, a product with just one skewed rating of 5 stars could potentially see more clicks on search results pages than a similar product with 4.5 stars rated by hundreds of customers. Granted, this is resolved as newly added products age, but see the issues this may cause?
Perhaps what’s needed, then, is a slight alteration to the presentation of ratings. It’s not an easy issue to tackle, as any solution inevitably presents 1) additional interface clutter and 2) another hit to the database on product or results pages — for every product on the page, no less. But let’s explore a few options for weighting customer ratings visually.
The first is perhaps the easiest. Not the prettiest per se, but practical. Simply add a review count next to each rating.
A fair number of commerce sites already display numbers next to ratings on results pages. Seems fairly straightforward, but let’s look at other options.
How about using color to indicate one product has many ratings, while others have few? Something like red for rating hotness, maybe a cold color for little or none.
That seems to work in principle, but ignores color-blind users who may not be able to distinguish between the two.
Maybe size will do the trick — enlarging the rating icon (stars) for products with a larger number of reviews.
Hmm. Maybe not. Did someone say clutter?
The iStockphoto Flames
How about adding flames to the stars, much like iStockphoto does with popular photos (based on number of downloads)?
Well, works for iStockphoto. Not sure it works here.
The eBay Approach
I’ve got it — we’ll do an eBay approach of some sort. Like Power Seller or something, but instead “Power Rating” for products with more than X number of ratings (relative for each store).
Um, not only does that look a bit silly, but doesn’t make much sense either.
Well, I suppose it’s clear that the clear winner in this case is no winner. At least this served as an exercise in interface design for me, and I hope the same for you. Even despite the fact that Amazon has probably already done A-B testing on all this, so I’m probably exercising in vain…
Stock photography, type, and killer tees. Genuinely recommended by Authentic Boredom.
Authentic Boredom is the platitudinous web home of Cameron Moll, freelance new media designer, author, and speaker. More…
Full-time and freelance job opportunities. Post a job...
A selection of fine reading, available for a limited time only:
- Jobs home page reorg
- Coming soon: Mobile Web Design, the book
- Dyson ad: Text as more than just words
- Setting sail for Europe
- Review: Sumo Omni bean bag chair
- Dashboard widget for Authentic Jobs
- Limited-time offer: $99 listings
- Nine skills that separate good and great designers
- Fire sale
- Introducing AuthenticJobs.com
CSS Mastery: Advanced Web Standard Solutions A solid round-up of indispensable CSS design techniques by Andy Budd, Simon Collison, and Cameron Moll.
Mobile Web Design A guide to publishing web content beyond the desktop. Tips, methodology, and resources. Now available.
Letterpress Posters The unassuming beauty of a freshly letterpressed print.
That Wicked Worn Look. Techniques for that worn, aged, distressed look.
Mister Retro Machine Wash Filters Turn the dial to “Instaworn” with these filters.
Blinksale Dive in and enjoy shamelessly easy invoicing from Firewheel Design.
Basecamp My preferred web app for internal and client project collaboration.
HOW Conference Austin, June 24–27. Pentagram, Adobe, P&G, et al.
Web Design World Seattle, July 20–22. Practical sessions on web design.
Stimulate Salt Lake City, September 2009. Entrepreneurship and design conference.