Beware of the Ratings on Yelp

Image courtesy of mjtmail on Flickr.

If you haven’t heard of it before, yelp.com is an online rating website for businesses all over the the world where people like you and I can rate businesses as well as write up reviews (unpaid) and have them posted online for other people to see.  As a new user of yelp, I recently found out that the ratings shown for the businesses do not reflect all of the reviews submitted on those businesses.  When you use this site, you need to be aware of this, especially in order to avoid really negative business experiences like the ones I had.  I don’t want to see anyone else put into a bad position when dealing with people who aren’t ethical business people.  No one deserves to have their equipment damaged or to have a business give them a firm quote and then force them to pay more for what they had already agreed upon.

Here’s what I’m talking about.  When I recently reviewed a local business with whom I had had a very bad experience, I noticed that there was another patron who had also had a very bad experience.  But I noticed other reviewers with ok and great experiences.  I was pretty suspicious of the review with the 5 stars, but maybe it was legit.  After I submitted my one-star review I skimmed the business’ page again and noticed a little note at the very bottom of the reviews which said “filtered” reviews.  “Hmmm,” I asked myself, “what is this?”

So I clicked on it, and found tons of one star reviews and a couple of 5 star reviews that had been hidden from view!  What?!  I read through a good number of them and it wasn’t hard to realize that these one star reviews were all very legitimate.  They had specific and applicable details and were all different from each other in complaints and writing style.  And, having experienced the nightmare that was this business myself, I had no problem believing any one of them as true.  On the other hand, the five star reviews gave very little detail and were a bit too complimentary so they may have been fake, but it’s hard to say for sure.

I was a bit concerned about why all of these one-star reviews were being hidden from consumers, so I looked around for answers.  I clicked on the “about filtered reviews” link to see what I could find out.  It says that these reviews are not included in the company’s star rating.  “Why not?” I asked myself, “they are all legitimate reviews!”  I was starting to smell something a bit fishy, and it wasn’t the soup.

So I submitted a question to the company, and they responded quickly.  They said:

“Thanks for contacting us with your concerns.

We try to showcase the most helpful and reliable reviews among the millions that are submitted to the site. Unfortunately not all reviews make the cut, even some legitimate ones. However it’s worth noting that the system isn’t static. It does what it can with the information at hand, and regularly revisits each review as it learns more. As a result, filtered reviews can find their way back onto business profile pages and vice versa.”

After I received this, I felt placated.  “Ahh, ok, they’ve got a system, it’s all under control.”  But I then was prompted to go back to the site and see what happened with my recent reviews, and lo and behold, the two one-star reviews (both very legitimate and appropriate) I had written had both been filtered and placed in the “hidden” and uncounted section of the two businesses’ pages.  “What?!”  Ok, this can’t be right.  So I wrote back to the customer service rep who had responded to me and asked to speak to someone about their filtering system as it didn’t seem to be working properly.  I haven’t heard back.  So much for good customer service.

If their system works as they say, as more information comes in, the businesses’ ratings should be adjusted to reflect them.  That means that the one company that I reviewed which had 37 one-star reviews out of 54 reviews total should have a lower rating than 2.5 stars, and with so many one-star ratings, many more of them (if not all of them) should be placed on the business’ official ratings page and counted.

It sounds like their system may work to filter out fake 5-star reviews, but it seems to be doing so at the loss of keeping the valid one-star reviews.  In reality, it is more important to know about the one-star reviews than avoid some fake 5-star reviews.  I’d take another look at my filtering system if I was working at yelp, and redesign it so it better filters out only fake reviews and maintains the legitimate ones for consumers to use.  The entire point of the site is to (well for yelp’s owners to make money off of free consumer reviews…. but) provide honest reviews to consumers so that they can make good decisions about where to spend their money and where to get the services they need.  Knowing about bad businesses is a critical component of this service.

So I think the important lesson here is that if you choose to use yelp to help you decide whether or not to patronize a certain business, or to choose which business is the best fit for your needs, that you not only read the business’ main reviews but are also sure to check if they have any filtered reviews and to read those as well.  I certainly wish I had done that before I went to use the two businesses that I ended up giving one-star reviews.  It would have saved me a lot of time, money and distress in the end.

If you have any questions or comments please feel free to post below, I look forward to hearing what you think.