Wednesday, May 16, 2012

an experiment in the amazon ratings system

Not long ago I accidentally ended up on some kind of semi-private forum for top reviewers at Amazon. It had never occurred to me how obsessed these guys are with increasing their rating. They have a whole forum that seems devoted to complaining the rating system is unfair and how to game the system to improve your rating. Yes, I know that is somewhat ironic in itself.
I really don't care much about my rating. I am not even sure I realized I had one until I landed there. It is not like it is hidden, so I have probably seen it, but just did not know or care what it is. Based on a few things I read on the reviewer forum, I thought I would try an experiment. It was speculated that negative reviews impact your rating, while positive reviews improve it less than negative reviews hurt it.

Not realizing how serious these guys take their reviewer ratings, I posted that I did not think much of the whole review system, given how obviously screwy the system is IME. Admittedly that is mostly in the book section and of late mostly in the self published book section where there is pretty obvious and rampant sock puppetry going on.

Shortly thereafter, I received two negative votes on each of a couple of hot sauce reviews I had written. I suspect that my comments on the forum may have had something to do with the negative votes on my hot sauce reviews. In any case, it gives me an opportunity to see just how screwy the system is. As suggested, I am going to delete the two negative reviews, and then repost the exact same reviews and see what it does to my ranking. I had given both the hot sauces the same exact review. This is what the reviews said.
I copied this from a post I wrote on my blog a while back. I only changed the price to XX.
This has become my preferred hot sauce. I had never heard of the brand before but saw it on the shelf at Walmart maybe a year ago and bought a small bottle to try. I really like it so it is now what I use mostly as hot sauce, several times a day usually. It comes in regular and extra hot. I have tried both and prefer the extra hot, although it is not that hot. It is about right for me. I go through a bottle or 2 a month.
Ingredients: Water, Chili Peppers, Vinegar, Salt, Spices, 0.1% sodium benzoate as a preservative.
As an added bonus, the 1 liter bottles are <$XX. Made in Mexico. it comes in a smaller bottle as well, but I get the 1 liter size. I have one at work and one at home.
Before deleting the reviews (5-14-12 and 9:30 pm CDT) my reviewer statistics were:
Customer Reviews: 101
Top Reviewer Ranking: 33,872
Helpful Votes: 102

The email I got confirming the hot sauce reviews the first time I put them in said this:
Reviews written: 95
Reviewer rank: 21,478                   
Helpful votes: 100  of  166 

Incredibly, after I deleted the reviews, a link appears to the product page so that I was able to immediately re-enter the same exact reviews. I will wait and see what happens. Supposedly the reviewer ratings are recalculated daily.
This morning I checked it again.
Customer Reviews: 102
Top Reviewer Ranking: 33,935
Helpful Votes: 103

I added a review in the meantime and I guess I picked up a helpful vote, but it did not seem to change my rating any. Both of the reviews were immediately reposted. They showed back up with the original date but with the negative votes missing. However, only one of the reviews shows up in the list of my reviews. The other one is still there, but you have to go to the product page to get to it. Very odd.

For some reason Amazon does not show the total votes cast on your reviews on screen, but does send them along with the notices it emails you when you post a review. So I went back thru the emails they sent me.

After the last review I wrote (after deleting and reposting the two hot sauce reviews but before posting the latest review), the email notification said this:
Reviews written: 102
Reviewer rank: 33,872
Helpful votes: 102 of 176

The email notification for the previous review said:
Reviews written: 101
Reviewer rank: 32,083
Helpful votes: 102 of 180

It does appear that it did remove the negative votes, but it did not change my rating very much. It is not like I really care anyway, and the whole episode starting with the inadvertent posting on the forum of self important reviewers has been pretty much a waste of time for me.
I went back to a few of the emails from Amazon. Not sure if they mean all that much.

4-17-12
Reviews written: 85
Reviewer rank: 24,773                   
Helpful votes: 86  of  140                 

2-20-12
Reviews written: 51
Reviewer rank: 25,985                   
Helpful votes: 52  of  79                 

10-3-11
Reviews written: 14
Reviewer rank: 8,452,461                   
Helpful votes: 9 of 23

Does any of this prove anything? If nothing else it does prove I was willing to waste time on something (the reviewer ratings) I consider all but useless.

ETA 5-16-12 7:15 PM.
Oddly, my ranking has gone up by >3000 places during the day. No new reviews, no additional good votes since this morning. But, both hot suace reveiws are now shown in my review list.
Customer Reviews: 102
Top Reviewer Ranking: 30,534
Helpful Votes: 103

ETA 5-17-12 shortly after lunch time I wrote another review and this is after I finished the review but before it went online. I wonder if when your rating changes it changes gradually.
Customer Reviews: 102
Top Reviewer Ranking: 29,111
Helpful Votes: 103

1 comment:

Anonymous said...

Big deal....do you care?

More time on the computer....