I feel your pain.
When I try to evaluate something, I, too, go looking for the reviews and evaluations of others to help direct what details I should be looking at. I, too, find it difficult to find sites and collections of reviews I trust.
While I don’t think your subscription idea will work (I’ll explain why), I’ll review why I think we’re in the state we are, and what I do when looking for honest reviews myself.
Become a Patron of Ask Leo! and go ad-free!
Money matters
You certainly hit one on aspect off the top: money.
I can tell you from personal experience that running a site costs money — more than most people think. That money needs to come from somewhere.
So-called “affiliate sites” are sites which promote specific products and get a percentage of the sales price when people make a purchase. Many are clearly sales sites with promotional information about exactly one product. Unfortunately, others try to look like objective comparison sites, except that the “winner” of their comparison is the product for which they’ll make money.
Sometimes, such comparison sites aren’t affiliate-related at all, but are sites sponsored by the winning product.
Even when not biased by money, through, objective reviews are still hard to come by.
Opinions: everyone has one
It’s important to realize that reviews, even honest ones, are opinions. And when it comes to opinions, not everyone agrees.
Consider someone reviewing Windows 10. I hear from people who love it, and I hear from people who loathe it. Assuming either side put together a review site detailing their experiences, who would you believe? Who should you believe?
Each side believes the information they present to be completely objective and truthful — and yet they may completely contradict the other.1
Both are “honest”, but whether or not you’d consider them helpful is questionable.
Subscriptions aren’t a solution
Once again, based on my personal experience, the subscription model is hard; it’s much more difficult than most people think. While Ask Leo! has a number of patrons, for which I am eternally grateful, it’s still not enough to cover the expenses of the site.
On top of that, most people looking for a review of something aren’t looking for more than one. This means if they find the review they’re looking for, they’re done — they don’t need to subscribe to get more in the future.2
It’s awesome that you’d be willing to subscribe to such a service, but in my opinion, you’re in a small enough minority that I wouldn’t see building a site or business around the kind of technology recommendations you’re looking for as viable.
Honest reviews & what I do
There are plenty of “honest” review sites out there, though you must keep in mind that every review is a biased opinion of some sort.
Personally, I find the forum-based reviews most helpful. I can get a cross-section of opinions and weigh them each on relative merits, ranging from their willingness to substantiate their opinions with data to their ability to state their opinion clearly and concisely.
The problem is that it requires work. It requires more effort to get an understanding of what reviewers think is important, what their biases might be, and whether or not their opinion is worthy of merit. That’s all necessary prior to deciding whether or not to take a review seriously.
Most people aren’t willing to do that work. They would prefer a single source they can trust without taking additional effort.
I don’t know of such a source.
When I review
You may notice I don’t do many product reviews. For the few I do, I try to be as objective as I can be.
And yet, I also need pay the bills, so I absolutely use affiliate links or other forms of compensation when they’re available. My process boils down to this:
- Pick a product, decide to review it, and do so.
- See if they have an affiliate program.
I don’t decide whether something is worth reviewing based on my ability to make money with it.3
The question is this: those are just words. Do you believe them? Should you? I have detractors who absolutely don’t. More to the point, should you believe any site that claims objectivity?
And is it even wrong for a site to present honest reviews across only products that provide some form of compensation?
There’s no simple answer. It’s not a yes or no question. What’s needed is healthy skepticism, and ideally, confirmation from multiple sources before relying on anyone’s opinion, regardless of their motivation.
And, yes, that means more work.
For full disclosure, I’d encourage you to read my Product Reviews, Recommendations and Affiliate Links Disclosure page.
Do this
Subscribe to Confident Computing! Less frustration and more confidence, solutions, answers, and tips in your inbox every week.
I'll see you there!
Podcast audio
Footnotes & References
1: If this sounds dangerously close to politics, it’s because it is. Religion, too. Each side believes with complete certainty that they’re being objective and honest, and yet can be in complete disagreement with “the other side”.
2: This is actually true for my question/answer model as well. The vast majority of site visitors come, get an answer, and leave, never to return — until, perhaps, their next question.
3: Macrium Reflect is a good example — they have no affiliate program. I ended up adding EaseUS Todo to my recommendations as a less costly alternative for folks, and only after doing so did I find that they have an affiliate program.
That may be the best source of reviews, but still very subject to prejudices. For example, if you were to come to Ask Leo! to decide whether or not to upgrade to Windows 10, the majority of reader comments on the forum would convince you that Windows 10 is the worst OS ever. Personally, I find it does the job as well or better than any other OS I’ve ever used.
Then, there are organizations which target certain websites and products and attack them en mass. I know of some good websites which have a bad WOT (Web of Trust) reputation due to massive campaigns by some organizations. This can also work for product reviews.
One source of reviews that I find useful is the Amazon customer reviews for just about any product. Even though some of those can be planted by people promoting a product when a product has hundreds of reviews it is easy to look for patterns of problems that many people have encountered. I look at the 1 and 2-star reviews to find common problems. Usually, it is easy to spot very negative reviews by disgruntled people who didn’t even understand the product and disregard those. When a lot of 1-star reviews indicate that a product didn’t work or stopped working after a very short time, this is a good indication of product quality issues. Many Amazon reviewers include useful information that identifies legitimate problem areas.
At my age, skepticism has become an art-form. I purchase a lot of product from Amazon for the simple reason my small town has few resources. I take Amazon reviews with (cliche) a grain of salt and read between the lines. If I’m looking for a good laugh, I read the one and two star reviews. Yes, a lot of these folks have an ax to grind or have no common sense about…well…anything. I generally come to my own conclusions from past performance and experience. I return items on occasion but will leave a four or even a five star review. Just because a particular product did not work for me doesn’t mean it will not work for someone else. There is an art to being objective to something you disagree with or do not like…careful; what goes around comes around…Its ok to dislike anything, just say it nicely and in a respectful way…in a way your Mama would approve! : )
Consumer Reports comes the closest to an objective (but not free) review site for me. It’s not strong on software or very techy stuff but for general household products it gives useful information. As Leo mentioned though they can never embody your particular needs and wants with regard to any product.
They have user comments of their reviews and I find it very interesting that the users are invariably much more negative than the reviewers. I assume that folks that have problems are more likely to comment. But these negative comments are often related to problems that only show up after some period of use. So even these are useful to me when evaluating a product.
One of my issues with Consumer Reports (I do subscribe) is that the models they review are often out of production, or not generally available, at the time the review comes out, or the time I need whatever it is.
I agree. I used to subscribe to CR years ago, but I almost never made a purchase based on their reviews. I only buy major appliances every 12 – 15 years and cars about the same. I would read it every month and think it was good info, but I didn’t need anything they reviewed. Now that they have a website, I’d be willing to pay a couple of dollars for a one month, or even one week, subscription whenever I’m considering a major purchase, but other than that, it’s just interesting reading.
One site that I find helpful is thewirecutter.com. A bit like Consumer Reports, The Wirecutter is owned by the New York Times, which bought the site and its services a few years ago. The financial backing of the NY Times provides value in two ways. First, it means that the reviewers are not dependent on the product manufacturers to support them, so the reviews are much more objective. Second, since being a N Y Times company, they have been able to greatly broaden the range of products they review. When they were a private site, their reviews were limited to specific technologies, but they now review many kinds of technologies and many non-technology consumer products. They do provide links to where you can buy the products they recommend (usually at Amazon) and I assume these are affiliate links that provide revenue, but I believe this revenue is very small compared to what comes from the Times, so it would not influence their reviews.
I was glad to see that the Federal Trade Commission just fined this company for claiming objective reviews when they were pushing products based on advertising dollars.
[broken link removed]