Questionable Ratings Plague Amazon 3D Printer Listings

Shopping for a 3D printer on Amazon? Watch out for twisted product ratings. 

Amazon is a very popular place for many 3D printer manufacturers to market their products these days. It’s a worldwide storefront that attracts a great deal of traffic. 

On the other hand, it can be challenging for a manufacturer to have their products float to the top of buyer search lists. Companies tweak their pricing levels, adjust the wording in their listings and select what they believe to be the most attractive images for their machines.

But there’s one aspect they cannot – or should not – control, and that’s the user ratings for their products. 

Ratings are supplied by the public by adding a rating to the listing, with a text description of their feedback. Often these ratings are critical for buyers, as they seek independent advice on a product. It’s a very handy feature everyone uses. 

For example, if you read multiple reviews on a product that pointed out flaws, you’d be less likely to buy. On the other hand, a series of positive reviews might make you more likely to buy. 

Unfortunately, the rating and review system can be gamed in some ways, even though Amazon does provide for “verified purchaser reviews”. There are three types of parties that can abuse the review system: 

1. Those that for some reason dislike the vendor or product and want to hurt it by adding negative reviews and ratings
2. Those what are enamored with the vendor, product or concept and want to boost it by adding a positive review or rating
3. The vendor itself who, directly or indirectly, adds positive reviews in an attempt to boost sales

These three factors can really mess up the ratings for some equipment on Amazon, but it’s hard to detect them. There are some tells, such as repeated identical text, simultaneous submission, nonsensical information or inaccurate data. But it’s very difficult to perform such an analysis by hand because some popular listings contain literally hundreds of reviews. 

I found an interesting service, ReviewMeta, which actually performs this analysis instantly for any given Amazon listing. While their analysis may not be completely accurate, it’s a very good start. Their algorithms involve checking that stuff I mentioned above and much more, far deeper than one could ever hope to do manually. 

Their process involves examining the original reviews and tossing out what they determine to be questionable. The result is a “fresh” rating based on supposedly “good” reviews. 

Sometimes the new rating is a lot less than the original. This would suggest someone or group is attempting to artificially boost the rating. 

Sometimes the new rating is more than the original! This suggests someone is trying to denigrate the product through suspicious negative reviews. 

Sometimes the new rating is the same as the old rating, but there have been a large number of questionable ratings, suggesting there is a bit of a battle between pro and con forces over the product. 

To see how this affects the ratings, I managed to do an analysis of some 50 3D printers currently for sale on Amazon.com. I selected the listings with the most reviews and a few notable others that I felt deserve examination. The results of the analysis were interesting. 

Of the 50 listings in total: 

• The average original rating was 3.8 (out of 5.0) and the average “new” rating was 3.6, slightly less. 
• 34% of the listings “passed” ReviewMeta’s test, meaning they did not have an unusual proportion of questionable ratings.
• 30% of the listings warranted a “Warning” from ReviewMeta, suggesting they had substantial questionable ratings.
• 36% of the listings were giving a “FAIL” rating from ReviewMeta, meaning the number of questionable ratings were so prolific as to significantly change the original rating
• The top ten rated 3D printers in my list did NOT change when using the ReviewMeta adjusted ratings; only one printer on the top ten slightly changed position within the list.

As for individual products, the results were essentially all over the map. Some were “overrated”, others “underrated” and many didn’t really change their rating at all. 

The most “underrated” products analyzed was a short list. Again, these are machines whose ratings, according to ReviewMeta, should be higher than they appear due to what they consider questionable ratings. It may be that these machines are suffering from negative reviews from external parties for some reason. The list:

Note that the difference from original to adjusted rating is fairly small and that’s why the top ten list didn’t change much. 

Here is the possibly controversial list of “overrated” units, again machines whose ratings, according to ReviewMeta, should be higher than they appear due to what they consider questionable ratings. It may be that these machines are accumulating suspicious positive reviews from unknown parties for some reason. The list:

This list is a bit longer and includes a variety of machines from different vendors. Some of these machines have actually been discontinued, but for some reason are still for sale by parties on Amazon. 

There is one particular machine I could not include on this list: the MakerBot Replicator 5th Gen. This machine’s adjustment by ReviewMeta actually had such a proportion of what it considered questionable reviews that it could not create a new adjusted rating. While MakerBot has long fixed technical issues encountered with this machine, it seems that it has attracted many positive and negative reviews. 

For vendors, my examination of ReviewMeta’s analysis showed that both 3D Systems and MakerBot seemed to have an unusually high number of what ReviewMeta considers questionable reviews. Notably, these two companies were for many years the most well known brands of desktop 3D printers and thus may have attracted larger public interest. Only Aleph Objects (LulzBot) had entirely “PASS” ratings. The remaining vendors had a mix of “PASS”, “FAIL” and “WARN” on their product listings from ReviewMeta. 

Do these machines deserve such treatment? Maybe, maybe not. It seems clear that in at least some cases there are reasons to doubt reviews, and that ratings have been incorrectly added. It’s also possible that ReviewMeta’s algorithms toss out reviews that are actually legitimate, but perhaps tripped up ReviewMeta’s system in some way. 

My advice: ReviewMeta’s service provides some additional insight into a product, but is certainly NOT the only information you should use. I strongly advise actually reading as many reviews as you can handle, as the text often includes points relevant to your specific situation.

I also recommend reading user forums for machines you’re considering buying, particularly if they are independently operated. There you’ll find actual users talking about the machine and you’ll quickly learn the real story of what it takes to operate one of them. 

Finally, if you do buy an item, please put an ACCURATE rating into Amazon that truly reflects your opinion and experience with the product. That will help everyone that follows.