Blog

David Streitfeld of the New York Times has been looking hard at the issue of fake customer reviews.  A year ago, he called out freelancers offering to write positive reviews for a few bucks.  In January, he wrote up a service called VIP Deals that offered rebates in return for positive reviews.  And a couple days ago he published a piece on Todd Rutherford, the founder of gettingbookreviews.com, who sold 4,531 book reviews in 2010 and 2011 at $20-99 each, before backlash from Google and Amazon forced him to shut down the service.

No doubt, there is fakery out there.  The question is: how widespread is it?  4,531 seems like a lot of reviews, but it’s a small part of the billions of customer reviews available on the web.  The most recent NY Times piece says Bing Liu, a computer scientist at the University of Illinois, Chicago specializing in automated text analysis, “estimates that about one-third of all consumer reviews on the Internet are fake.”

Really?

One of the reasons there is so much suspicion of reviews is that so many of them are positive.  Liu has estimated that 60% of the reviews on Amazon are 5 stars and another 20% are 4 stars, but, he says, “almost no one wants to write five-star reviews, so many of them have to be created.”

Here’s an alternative explanation: there’s a lot more customer satisfaction out there than you might guess.  That’s a conclusion you might come to from reading customer answers to shopper questions gathered through TurnTo.  With the TurnTo system, there is almost no possibility for fakery.  When a shopper asks a question, the system chooses a group of people who actually bought the item (based on the transaction records of the store) and emails the question to them.  While the system allows in-line answers on the product detail page, >90% of the answers come in reply to this question email.  So unless there’s a big population of people buying products they don’t need for the purpose of providing artificially positive answers to shopper questions they may never even receive, these answers are legit.

And one of the most striking aspects of the answers provided by these real product owners is how effusively positive they often are.  For example, here are some customer answers to a shopper question about the height of the drip spout on an espresso machine at SeattleCoffeeGear.com.  The question doesn’t ask for any sort of overall evaluation of the machine – it’s just looking for a measurement.  Yet many of the respondents (including me) spontaneously volunteered our enthusiasm for the product.  (You can find this page here.)

Now I don’t want to be Pollyanna about the problem of fake reviews.  I suspect they are much more common on destination review sites like Yelp and Trip Advisor where anyone can submit than on ecommerce sites where the ability to verify purchase is an easy and effective way to police.  It’s also harder to believe the uniformly high ratings sometimes found on products which are judged on subjective personal taste, like books and food; personal tastes differ too much.  As Streifeld points out, even The Great Gatsby (which, first published in 1925, is presumably not attracting many fake reviews) has plenty of neutral and negative reviews (>300 reviews are 1, 2, or 3 stars out of 1,400 total at Amazon).

But while the battle goes on between the fakers and those trying to root them out, it’s possible that in many cases when the reviews are positive, customers might just be happy.

There was an interesting  piece in the New York Times over the weekend about the experience Urban Outfitters has had with personalization.  In brief: they tried altering the merchandise displayed on the site based on the gender of the visitor – it seemed like an obvious win to show women more women’s clothes and men more men’s clothes.  But it backfired.  Dmitri Siegel, then at Urban Outfitters and now at Patagonia, noted that shoppers “took offense at being subjected to gender-based marketing.”

And yet some aspects of personalization – in particular, recommendations – seem to perform well, and even better when aided by automation.  Last week I was speaking with an IR50 retailer who said that in a recent test their automated personalization engine outperformed their human personalizers by 3X!  (Recommendations proposed by the automated engine received 3X as many clicks as recommendations manually composed by the merchandising team…)

This contrast – between personalization that works and personalization that doesn’t – reminded me of the windshield wipers on our car.  OK, OK, bear with me here.  Up until the car we own now, the intermittent wiper always worked the obvious way: you turn it up, the delay between wipes gets shorter.  But on our current car, it’s different; the wiper setting determines the amount of rain that a sensor needs to detect before triggering a wipe.  That is, you are not setting the wipe frequency directly, you are setting it relative to the amount of rain that’s falling.  The idea is this: if you like a dry windshield, turn up the knob, and the system will keep your windshield dry for you by increasing the frequency of wipes as the rain gets heavier.  And if you prefer to minimize the distraction caused by wiping, turn down the knob, and the wiper will only trigger when it senses a very wet windshield.  In fact, you never even need to turn off the intermittent wiper, because on a sunny day it never triggers, even on the highest setting.

Like personalization, I suspect that reactions to this feature vary, and some folks probably like it.  But I’m not one of them.  I prefer to have direct control over the frequency of my wiper.  While the sensor is pretty good, it’s not perfect, and when I want a different frequency than what the sensor has chosen for me, it’s hard to get it right.  Further, changing the speed manually is not hard, so the intelligence built in to the system isn’t really saving me from some difficult burden.

You can see where this is going.  In an effort to “personalize”, the wiper system forces me to give up some control.  But the cost to me of the loss of control is greater than the value to me of the personalization.

I suspect that my experience with the wipers is similar to the experience of shoppers at Urban Outfitters.  By personalizing based on gender, the site removed a degree of control from the shopper.  After all, women shop for men’s clothes and (ahem) men shop for women’s.  So there’s a cost to the shopper when the personalization is off.  On the other hand, there’s very little burden for the shopper in controlling the system manually – ie clicking the tab that says Women’s when shopping for women’s clothes.

This doesn’t mean personalization is always a mistake.  There are some times when direct control is too hard and having a system help is very valuable.  One example: in modern fighter jets, direct control of the airfoils through wires and hydraulics – the way pilots used to control planes – is no longer possible.  So instead, the pilot tells a computer what she wants the plane to do, and the computer tells the control surfaces how to move to accomplish that.  Similarly, on an ecommerce site with many SKUs, smart, personalized recommendations can help me discover items that I would miss with less-smart recommendations.

The trick in getting the user experience right is to figure out when allowing direct user control is preferable and when automated personalization is the better choice.

Here’s what Citysearch CEO Jay Herratti told the New York Times last week regarding their recent trial of Facebook Connect:

In the four months the site [Citysearch] has been testing Facebook Connect, 94 percent of reviewers have published their reviews to Facebook, where an average of 40 people see them and 70 percent click back to Citysearch. That has translated into new members: daily registrations on Citysearch have tripled.

If you are an on-line merchant, don’t leave all the Facebook Connect fun to the publishers!  With tools like TurnTo, a Facebook Connect implementation is far easier and quicker than you might imagine.  And results like those from Citysearch show the benefits can be dramatic.