Tool Reviews

[John Schreiber]

Both of these quotes came from another thread. Rather than hijack that one, I'll start fresh here.

I received a free 10" blade from the owner (he posts here from time to time). I was going to do a review for him here. I didn't because it didn't really work for me. . . .

Brian - My experience is very similar to yours...how I acquired it, how it performed, how it wore and came off the blade, and skipping the review because of all that.

That helps explain why so many of the reviews we see are positive. "If you can't say something nice about someone, don't say anything at all" may make sense in personal relations, but as far as reviews, it gives a distorted picture of the marketplace.

Why not give an honest review of each tool you receive? If you honestly say that a tool has problems, does that mean you won't be asked to review tools in the future?

I don't want to be controversial and I hope I'm not insinuating anything or insulting anyone. This may be the way things are done, but I was really surprised.

[Brian Kent]

John,

I bow to your wisdom since you have about 511 more posts than me, but I have seen a lot of negative reviews here. Do you mean like formal and official magazine - type reviews?

Brian

[John Schreiber]

. . . I have seen a lot of negative reviews here. Do you mean like formal and official magazine - type reviews? . . .

These were reviews where the manufacturer gave or loaned the tool for the purpose of reviewing. I often see blog and website postings where people do a thorough review of a tool. Occasionally they mention that they were loaned the tool for reviewing purposes.

I had assumed that if it wasn't mentioned, that they had simply bought the tool for their own use. I also figured that most people would point out a tool's downsides if it had them. It seems instead that people don't do a review if their experience is negative.

Do they do that because they want to be loaned more tools? Maybe I'm naive, but I was surprised.

[scott spencer]

Hi John - I found it hard to give a negative review to a product from someone as nice as Tom who participates here. Time and motivation are factors too....I'm busy like most of you, and pick and choose my free time accordingly. I tend to be more enthusiastic in general when I truly like a product. This one didn't catch my fancy for long as it didn't work out as well as I would have like, so I set it aside and moved on. Had I purchased the Final Cut system as a consumer, I'd likely be motivated by an expense that didn't pan out. My comments about the Final Cut system in general, have been honest when the subject is brought up, but I was just never motivated to originate a critical post about it. I have seen other less than glowing comments about the Final Cut on various websites, so I figure not everyone is holding back, and the good, bad, and the ugly always comes out eventually anyway.

I wouldn't necessarily draw any sweeping conclusions about the population based on Brian's and my reactions....it's just the way it unfolded for me in this case, and it was similar to Brian's situation.

In general, I tend to take reviews very lightly for a variety of reasons, some of which have been mentioned here. They all boil down to someone's opinion, and what's important to the reviewer may or may not be as important to any particular buyer.

[Brian Kent]

Well if that is the tit for tat, I would be happy to have Lee Valley and Lie Nielsen send me their whole line of tools for a comparison review. I'll let everyone know how excellent both of them are and we are all happy.

But seriously, I do see your point. It would be tempting to put loaned or donated tools in an unfairly good light.

[David DeCristoforo]

"Why not give an honest review..."

Most publications go with the "If you can't say something nice, don't say anything" position for one reason: advertising dollars. This can also hold true of web sites that are supported by advertising. But there is another force at work here. Post a negative review of a tool and 999 times out of 1,000, you are going to find yourself in the middle of a "flame war" with the devoted users of that particular tool. Try it and see....

[Ron Bontz]

Interesting post. Many points well taken. If I ever come up with a proto type of that shaper jig I'm working on I'll have to send it to Dave for a "private" review. I think Dave is right on. The loyalist vs the new comer, and the dollars vs sense. Reminds me of the Jointech vs Incra "discussions". Both good products.
Oddly enough I find myself currently having to decide between two good products. A JDS Accumiter with the optional hold down clamp and a Jessem Miter R Excel.
For me it comes down to what works best for me. The only dollars involved here are the ones floating out my shop door. I won't mention which one I'll keep. I don't want to start a war.

[Tom Henderson2]

This is a bit of a dirty little secret -- that many internet tool "reviewers" are not always disclosing that tools have been provided gratis, or other arrangements with the tool providers.

Some get quite defensive about it -- not sure why.

Some are also quite biased for or against certain vendors. That shouldn't surprise anybody.

-TH

-TH

[Larry Edgerton]

"Why not give an honest review..."

Most publications go with the "If you can't say something nice, don't say anything" position for one reason: advertising dollars. This can also hold true of web sites that are supported by advertising. But there is another force at work here. Post a negative review of a tool and 999 times out of 1,000, you are going to find yourself in the middle of a "flame war" with the devoted users of that particular tool. Try it and see....

Exactly. The best tools in the world are the ones I own.

John. When I buy a tool and don't like it one of two things happens. I give it away, or if it messed something up I smash it with a hammer! To improve the breed some must die.

I get some grief some times for negative reviews, but no one gives me free tools, so I will always tell you what I really think, as long as you do the same. Deal?

To tell you the truth, there are just a handfull of people on here whos opinion I pay any attention to for the very reasons that you bring up.

[Jeff Duncan]

There's one other aspect to consider with tool reviews also....is the reviewer qualified to do an un-biased review? I've read several reviews on this and other sites where the author was raving about the new (fill in the blank) he just bought. What qualifies this person to review the tool? What is their real world experience using this type of tool/machine/accessory? Has he/she worked with similar products by various other manufacturers for years, or is this the first one of the type he/she has bought?
Something to keep in mind next time you read a review.
JeffD

[Tom Walz]

I offered the Final Cut blades for testing. I posted the offer and sent them to anyone who wanted to do the test. The reviewer was free to say whatever they wished to say or not say.

We wanted to know what a random selection of real woodworkers thought. Obviously you can argue with our methodology here but we do find this sort of test helpful.

All the points raised above are valid but there is generally nothing very sinister about them. Some people just don’t like doing it. Some contact me privately and tell me what is wrong.

Some products are much bigger hits than others. In order to get some idea as to which products are going to be big hits we try to do real world testing.

We do this to find out how good a product is so we know how aggressively to promote it. Some people like the Final Cut and it is a reputable product so I had it put on our web site. It isn’t getting a lot of play so I am not advertising it. By comparison Tense wood watches are doing pretty well so we are promoting them. Whiteside, Southeast and Vortex tools are doing really well so we are spending a ton of money making them easier to find by model number.

As I said, we are selling Tense wood watches and we are looking for other wooden objects to sell. Some of what we are offered is great, some is good but doesn’t fit our concept and some is just poorly made. We try to be polite to everyone.


Tom

[William Falberg]

Tool reviews wouldn't even be necessary if the manufacturers were honest enough to disclose the pros and cons of their products. They know better than anyone where a tool's strengths and weaknesses lie. Only the manufacturer has access to customer feedback data. Whether or not they ever review that data is questionable in some cases. Whether they ACT on that data even more so. If they sell through distributors they'll be completely disconected and never know true consumer response because feedback ends in the distributors' round file. It's cheaper to refund money than change molds. Head-to-head comparisons are artificially contrived, banal, and unfocused attempts to rank tools based on artificially contrived criteria. Micro-manufacturers, start-up tool companies, and others serving the various niche markets thrive on customer feedback to evolve their tools' design. When a customer invests his money in a new tool it is because he's hoping that new technology will solve an existant problem that is unique to himself and maybe a few others. They're LOOKING for change: something new. A reviewer might not have any interest in change because he has no problems in that area. Such contrived circumstances wll reveal the reviewers' resistance to change as well as a general lack of understanding the new tools' advantages: they're looking at it through the distorted lense of conventionality. There's no such thing as an "objective" review. Only those with a "dog in that hunt" can render a meaningful evaluation of such new ideas. The best thing you can do for the start-up tooler is to complain about features you don't like and praise the parts you do like. None of this applies to the narcissistic tool giants who isolate their "Research and Design" departments from the consumer. Tool reviews are but a cheap (and lame) substitute for effective feedback looping that suits in the tool-giants' board room call "cost-effective" marketing. Is it any wonder you're not going for it?

[Eric DeSilva]

You seem very bitter about something. And, try as I might, your response doesn't make sense to me.

"Tool reviews wouldn't be necessary if the manufacturers were honest enough to disclose the pro's and con's of their products"? That doesn't compute. Things that one person views as a pro may be viewed as a con by another. That is the point of a free market economy--you get to vote on the features you like with your wallet. Even if they did disclose pro's and con's in somewhat of a reasonable fashion, how does that help? Take this example: MfrA says "on the pro side, we are 50% cheaper than the competition; on the con side ours is only X HP." MfrB says "on the con side, we are more expensive than the competition, but on the pro side, ours is 1.5X HP. Which one is better? A reviewer, who does the kinds of things you want to do and tests both products, can help by identifying whether the difference makes any difference for the work you want to do.

Customer feedback? How do you separate out the 7' tall guys who complain your table is too short from the 4' tall guys who complain its too tall? I suspect that most good manufacturers use focus groups and product testers who are known quantities and have given good feedback in the past in the design process. I suspect most also monitor user groups and the like to see where the "average" user comes out. I doubt seriously that any manufacturer attempts to isolate themselves from feedback.

Obviously, that doesn't always translate into a better product. Say, as a manufacturer, you get feedback that says you are losing sales because you a slightly more expensive than the competitor. You go to the design guys and say "find a way to shave $5 off the shelf price affecting performance as little as possible." They come back with "let's got with a plastic knob." You change the design, but now everyone says "we hate the plastic knob, go back to the metal knobs." But they weren't buying your products before, and now they are, because they are priced competitively. What do you do?

As far as your distaste for reviews, I *like* head to head comparisons. I like when the products are arrayed out, because you start getting a feel for what innovations one product has that the others lack. You get more objective data on certain performance characteristics. Head to head comparisons also mean there is a higher chance that something you have used is in the mix--hearing that MfrA's router has better dust collection than MfrB's router is sort of useless to me if I've never used either one. I'm in the market for a twin-screw vise, and liked the recent head-to-head in FWW. Frankly, I've never owned a twin screw vise, and have no basis for comparing one to another. Reading the reveiw gave me some things to look for--things that I read and said to myself--"hey, that makes sense, I'm not sure I would have really thought about that, but I could see how it could be annoying to have it another way."

The key is finding a source of reviews that you trust and with reviewers that do the same kind of thing you do. Its useless for me to find reviews of chopsaws in a magazine that caters to the building trades when my interest is in cabinetry--the trade offs for each are different, they want light weight, I want accuracy.

As a final matter, I really don't understand your last bit. "When a customer invests his money in a new tool it is because he's hoping that new technology will solve an existant problem that is unique to himself and maybe a few others. They're LOOKING for change: something new." I'd say 99.999% percent of the time you are wrong. When Joe Bagodonuts goes out to buy a phillips head screwdriver, its because he has a phillips head screw to put in, just like the other 99 million people with phillips head screwdrivers. The number of users who demand unique things from their tools has got to be a tiny fraction of the market.

[William Falberg]

Now that I read it again, my response doesn't make any sense to me either. I was bitter about something but I forgot what, now. Your post makes more sense so I'll " +1 " that. It's a way-complicated subject for me. I'm debating with myself, mostly. (Note to self: think before posting.)

[Brian D Anderson]

Hi John - I found it hard to give a negative review to a product from someone as nice as Tom who participates here. Time and motivation are factors too....I'm busy like most of you, and pick and choose my free time accordingly. I tend to be more enthusiastic in general when I truly like a product. This one didn't catch my fancy for long as it didn't work out as well as I would have like, so I set it aside and moved on. Had I purchased the Final Cut system as a consumer, I'd likely be motivated by an expense that didn't pan out. My comments about the Final Cut system in general, have been honest when the subject is brought up, but I was just never motivated to originate a critical post about it. I have seen other less than glowing comments about the Final Cut on various websites, so I figure not everyone is holding back, and the good, bad, and the ugly always comes out eventually anyway.

I wouldn't necessarily draw any sweeping conclusions about the population based on Brian's and my reactions....it's just the way it unfolded for me in this case, and it was similar to Brian's situation.

In general, I tend to take reviews very lightly for a variety of reasons, some of which have been mentioned here. They all boil down to someone's opinion, and what's important to the reviewer may or may not be as important to any particular buyer.

Exactly what he said. I was going to write a lengthy reply . . . but Scott said it perfectly.

-Brian