Monday, June 21, 2010

Fraudulent Ratings and Reviews

When I set out developing RateTea, I wanted to create a rating site whose accuracy could be trusted. Many rating websites are simply moneymaking schemes, designed to make a fun, interactive site, but with little regard to the accuracy of the information. While it would be nice for me to earn money through RateTea, I am not willing to compromise the integrity of the site in order to do so. I want to focus first and foremost on editorial integrity--ensuring that the material on the website, including both user-generated ratings and reviews, and the researched information on tea, is of the highest quality possible.

This attitude is central to all aspects of my life. I want to conduct myself with honor and integrity in everything that I do, and I will always sacrifice financial gain to do so, because I don't need to live an expensive lifestyle, and I have always had more than I need to survive comfortably.

Fraudulent Ratings:

RateTea has had a surprising problem with fraudulent ratings and reviews. The website clearly identifies both in the terms of use, and during the signup process, that it is prohibited for people to rate and review teas of a company they have an affiliation with. When signing up, people must select a pop-up menu that identifies whether or not they are affiliated with a tea company. People might object that no one reads terms of use, and while I agree with this, the message is communicated clearly in the signup process, and more importantly, it is common sense that it is dishonest and unethical to rate and review your company's own teas, especially when you lie about the fact that you are doing so.

There are several companies, all of whose teas are listed on the site, where there have been problems with such reviews. The problem is surprisingly transparent: a new user joins the site, usually with a hotmail or gmail email address, and creates some very high (usually 30/30) ratings of the company's teas all at once, never reviews any other company's teas, and never returns to the site or participates in the site in any other way.

Tracking Down the Offenders:

I have a bit of skill as a hacker and am very good at researching. With little effort, I once tracked down the identity of one of these reviewers, and found that he was a media consultant that had been hired by the tea company that the reviews had been written about. I deleted the ratings and politely contacted the person explaining our policy. I received no apology and indeed, no response at all.

How is there any honor and integrity here? If this person had admitted that he had written the reviews and apologized, there would have been integrity in that action; it would have been honorable and I would have forgiven the person's past actions: we are all human and make mistakes. But there is no integrity in failing to do so. Such inaction is shameful and sad.

How do other rating and review websites handle these problems?

Websites with a team of reviewers, such as Teaviews or Tea Review Blog generally don't have this problem because they know each of their reviewers, and the sites are not open to the public. Rate Beer, the site that RateTea is in many ways modeled after, handles things very similar to how RateTea does--their terms of use strictly prohibit rating beers when you are affiliated with the company.

Companies Reviewing their own teas on Steepster:

Steepster seems to have avoided the issue entirely...although their terms of use are much longer and more complex than RateTea's, my brief skimming of them shows no prohibition against a company rating and reviewing their own teas favorably. Indeed, extensive exploration of their site has shown me that a large number of tea companies rate and review their own teas. For many smaller companies, the overwhelming majority (and in some cases, ALL) of their tasting notes and ratings come from a company rating their own teas.

Do the ratings on Steepster have any integrity at all, if companies are allowed to rate their own teas? I know I am biased in answering this question because I have an interest in RateTea, and I have chosen to set it up the way I have, but I had thought about this issue long before Steepster had ratings. In fact, I had thought about this issue at length before I even knew Steepster existed. And many of the users on Steepster are honest (after all, I've participated there too, as cazort) and those are honest reviews just like all 278 of mine on RateTea.

Perhaps Steepster has a different purpose from RateTea. Is its purpose first and foremost to be a fun, interactive website about tea? If so, then it seems to be doing a very good job of this (quite frankly, a better job than RateTea, as judging by their higher level of participation). But this is not what I was aiming for. On RateTea's about us page, there is a sentence that reads: "We strive to offer the most accurate information on tea available on the web.".

This is what I care about. And honestly? I think we're already there. We screen out the fraudulent ratings. And I've put a tremendous amount of time into carefully researching the information on styles, regions, and brands of tea...and citing the sources of information when it is potentially controversial or when it pertains to critical topics like health or wellness.

So what do you think?

I have two questions for my readers...the first is a more practical one, the second is more existential, about what I should be doing with my time and if RateTea is really worth the while.

(1) Should I do more than just deleting fraudulent reviews? When there is a chronic problem, should I remove / unlist the tea companies that have had problems with fraudulent ratings? (Especially given that no legitimate users have ever reviewed any of these company's teas) Should I contact the company before taking such action (I'd be inclined to do so)? If the problem persists, should I bring out into the open that the company has been promoting its own products?

(2) Is this whole project a lost cause, or is it worthwhile? Do people care about the accuracy and integrity of a site? Or do they just want a site that's fun and looks good? Sometimes I fear that I'm never going to be able to make RateTea good unless it first takes off enough to pay for the skills that I don't have. I'm a lousy graphic designer and I don't always know what people want in a site. I do the best I can and sometimes it just doesn't seem good enough. What I am good at though, is researching, and maintaining the integrity and quality of information on a site. I just don't know sometimes if that's what people really value.

Thoughts? Thanks in advance.


  1. Alex - that you for attempting to maintain transparency and honesty in regards to tea ratings.
    Being the owner of Admari Tea - I would want only honest unbiased reviews of our product. I strive to put together the best tea list I can with an eye toward quality as well as value.
    You recently listed our tea on your site and for that I am very grateful.. I am pretty sure we have not had any ratings yet but we are a small company (at the moment:). But when we do get rated I would really hope they are from true, honest tea lovers. Plus I can't afford a media consultant ;)
    I think you should delete the companies with ONLY fraudulent reviews and yes, I would contact them. I don't know if I would bring it out in the open.. I think in the industry we all know who tries to maintain integrity in their product and truly cares about the leaf. Those who don't are somewhat apparent.
    Thanks again Alex and I really do think what you are doing is worthwhile.
    Adrienne Etkin/Admari Tea

  2. Thanks for your reply!

    I also think it is unfair for the companies, like yours, that refrain from promoting their own teas in this way.

    This gets at one of the toughest things that I struggle with about American society. Sometimes I get the distinct sense that our society and our economy punishes people who live / play by the rules of working towards the common good, while it too often lets the self-promoters and scammers slide.

    I think though that when people start working together and sharing information in a democratic way (such as through a decentralized network of blogs, word of mouth, etc.), it ultimately rewards the people who are doing honest work.

  3. I totally agree with you. I hate companies who try to boost themselves this way rather through actually producing a good product. Take Chicago Tea Garden. They never post fake reviews and yet are consistantly one of the most highly rated on review sites. This has been a problem with Steepster and one it seems the site builders aren't sure how to deal with.

  4. Wow, great read!

    Integrity is huge to me, and when potential customers see that in a tea company, it usually pays off.

    I have just a single review here for Utopian Tea on from @lissisblissx, and I'm happy with that one rating, without going and rating my own tea. However, on steepster I have put in my own rating on teas, but at least I make it obvious such as "As the head of Utopian Tea..I picked this tea because of yadda yadda..". This shows honest transparency.

    The good thing is, people are smarter than we give them credit for, and they can smell bullsh*t from a mile away. The companies giving fake, passionless self-promotion are the guys who are going to lose.

    I say give a warning email, if it continues, remove that individual tea being rated falsely, and if it continues again, remove the entire company.

  5. Thanks for the comments!

    I do think there's a complete difference between an anonymous reviewer, and someone like you openly identifying yourself as the owner. And I think it's good that you are sharing information about why you chose that tea.

    I was actually thinking of adding a feature to that would allow tea company owners to review their own teas and then separate those reviews (like, highlight them in a different color) and exclude them from site-wide rating calculations.

    This way people like Stephen Wise could share their perspectives honestly, and it would encourage all tea companies to participate in this way.

  6. To respond to Nicole, I am also impressed with Chicago Tea Garden, after trying only one of their teas (Large Leaf from Old Trees Pu-erh) parents are going to order some tea from them soon, and I look forward to trying them.

    Right now I'm still overwhelmed with the free samples I've been receiving, and it's summer so I'm drinking less tea and more herbals from my garden.

    I want to post more soon about the herbs I'm growing and making herbal teas out of.

  7. I think it would be kind to contact the companies with the fraudulent reviews, but I do think those reviews should be excised. After a while, you begin to know who has a palte similar to yours and are more inclined to take their advice. I am always worried about people who are consistently saying all teas sampled are good. Even the best of companies buys or poduces something icky sometimes. I worry that I will be too kind in my reviews if something is a free sample. It is good to be concerned because we should be striving for honesty and integrity and we need to hlp others to do the same, if we can. You're doing a good job!

  8. In response to your questions above I believe that when there is a chronic problem with fraudulent reviews you ought to delete the fraudulent ratings, contact the company in question and tell them if this pattern does not cease forthwith you will remove the company from your list, and then carry through. I also favor publishing a list of the companies you have removed from your list due to unethical behavior ie. fraudulent ratings.

    I suppose this make me "old school" in a time when balderdash is used to sell products but I refuse to budge from my ethics and don't think you should budge either.

    I am keenly aware that children do not as they are told to do, but they do as they see their role models do. If the adults of this generation do not take a stand against such practices what ethics will the next generation have?

  9. Wow, I had no idea that tea companies were that shallow. I would be interested in seeing the trends of the size of the companies doing this fraudulent self promotion. It really is awful to take advantage of their consumers, you know? When I am considering buying some new tea,I go around and ask my tea friends on social networks, and also search out ratings on tea rating sites, it makes me awfully saddened to think that some of these reviews that I seek out may or may not be real ones.

    As far as action,first delete the reviews. Then, contact the company personally out of respect, saying 'hey, i noticed so and so from your company has posted fake 30/30 reviews of your teas and they work for you...etc.". If there is no response, I would be weary of some of the reviews that come in, and if they look sketchy, investigate them. But I also find it unfair for other tea people to not be able to rate that specific companies teas or read some honest reviews on the tea just because the company was shady.

    Actually, this is quite a jam, because after typing that, I thought to myself that I would not want to be in contact or do business with a company that doesn't believe in integrity or their product, for that matter. I agree with timetheif about posting a list of offenders of this fraudulence, I'm sure i'm not the only one who would want to know what tea companies to stay weary of. I'd rather support a company that believes in their product and tea.

  10. Ouch.. I have to admit that I haven't really thought of all these issues but then...hanging head in shame...I don't tend to hit other sites to check out reviews. I probably should get around more though.

    One thing that I did want to point out is the fact that readers on TeaViews can post their own reviews on teas that we have already reviewed. I love it when they do because I love to see how other's tastes differ from mine

  11. I like that too, I think it's great when people's tastes differ. That's one reason I'm a bit skeptical of listings like the "highest rated teas"...I only listed those on because I was caving in to pressure from users.

    I think a much better feature is the compare users feature which helps you locate users who have similar tastes...unfortunately, there aren't enough users with overlapping reviews to use it fully. I have two other users that I can compare myself to...