Are Audio magazine rating worth anything?

Are Audio magazine rating worth anything?

Postby Harry » Wed Mar 25, 2020 12:24 pm

I was riding down the road years ago and saw a sign at a gas station that said "Experts in foreign and domestic cars" which to me meant you are neither. For many years I have felt that way about Stereophile equipment ratings. Don't get me wrong, VPI has been well served for years by these ratings, it is just that when you have so many products in Class A you really don't get the feeling any of this is real Class A or what you should be following as a purchase.

There are 24 units in Class A with price tags that run all over the landscape from Hyundai to BMW. I know this industry and many of these are unavailable, very limited production or no production (the Continuum has been in Class A+ for years with zero production), the HR-X has not been made or reviewed in 10 years. If you have 24 tables, some with arms, some without arms, and they are all considered Class A you have basically diluted the Class A to the point of unsuitability.

There isn't one reviewer who sets up two turntables on two of the same platforms with the same cables leading out and the same cartridge. A perfect example is what we do when we compare an HW-40 to an Avenger or Prime Signature. We set up the same cartridge (usually an A-95 because they are so similar in sound piece to piece), set the levels on a test record, put them both on the same stand, using the same Nordost phono interconnect, and the same phono section, and then listen. When we say an HW-40 sounds better than a Prime Signature we know exactly what we are talking about and the differences are accurately presented. No reviewer does this, there scientific method is based on feelings not accurate testing by removing as many variables as possible.

I knew Harry Pearson very well and would visit him all the time for fun not for VPI issues. We would sit and listen to the Clearaudio reference with its 700 pound gyroscopic stand and then compare it (as an example) to a Classic 3 sitting on a 3/4" plywood base thee feet from the 18" woofers with a different cartridge!!!!!!!!!!!!!!!!!!!!!!!

This was acceptable with $500 tables and $50 cartridges, when you are talking about tables the cost of cars and some the cost of Aston Martins I think a little more rigorous method should be used. The consumer is literally screwed reading these things and after 40 years the fact that they cannot fix this is a disgrace. It's not that hard to do.

What you need for basic fairness:
1- A stand large enough for two turntables
2- 2 cartridges from Ortofon or Audio Technica because they are consistent in manufacturing.
3- 2 of the phono interconnects of your choice
4- A phono section that has two inputs for MC or MM, whichever you standardize
5- A test record which you can find on E-bay for $5 like the Command test record
6- Two turntables

Does this sound difficult, of course not. But if they do this they will have to make decisions based on the sound in the system and the test numbers. You know that wow and flutter, absolute speed, and dynamic speed stability actually matter and contribute to the sound of a turntable. This would be a breath of fresh air as opposed to personalities, business meetings with $1000 meals with $600 bottles of wine, and all sorts of other things I can't even mention but you can imagine.

So let's all pick up our latest Hi Fi magazine and read about a slight dip in the lower mid bass and a slight smearing of detail, or a tipped up top and have no clue - as compared to what?

HW
Harry
Admin
 
Posts: 1428
Joined: Mon Aug 25, 2014 10:38 pm

Re: Are Audio magazine rating worth anything?

Postby theeng » Wed Mar 25, 2020 4:47 pm

Harry,

As an engineer, I can appreciate your frustration and your observations. However, I propose that there are other factors at play; the least of which is the record itself. How clean is it really, when was it last played, and of course there are the Listeners own biases. When I was at CAF-2018, you had two HW-40's set-up with the only difference being one had then Lyra Etna, and the other the Soundsmith Hyperion. The same Louis Armstrong record was played on each. They both played back beautifully, but the Soundsmith made Louis Armstrong sound like he was playing in New Orleans - steamy & humid, while the Lyra sounded like he was playing in Aspen - brighter & dry. But which was 'right'; that can only be compared to the original tape.

So, with R2R now making a come back in albeit small and costly numbers, would it be better to compare Tables to a known standard - magnetic tape. But, of course, differences in the R2R playback are not without its own debate? Which brings up the paradox, there is no readily available absolute standard by which music playback can be compared independent of listener bias. There were in the past, industry standard records to test turntables tracking and other performance factors, but those are no longer manufactured and the remaining ones only infrequently used. Maybe selecting records pressed with the Mobile Fidelity UltraDisk One Step process could a better alternative for a standard since the precision of the mastering and pressing is at a very high standard. But, how much different is this from how reviewers currently operate by detailing how different records that they are familiar with and one assumes they like sound. But of equal weighting, is the room itself - which is generally accepted as the major factor in sound reproduction. Having seen some pictures of reviewers room, one has to take pause to the value of some reviews.

So, I agree with you that ratings are of little value, but with so much marketing being is assigned to the rating, good luck in changing that. However, that does not prevent VPI from advancing its own detailed comparative analysis between its own tables to better guide the buyer. If VPI was to perform this type of analysis, then buyers may better appreciate what they should expect from say a Prime to a Prime Signature to an Avenger to a HW40. There is a lot of forum discussion, but VPI could collected it with its own assessment and then detail/document it using the procedure you present. However, with so much riding on what the buyer's budget is, and with manufacturers racing to get a piece of the resurgent record market, and at the same time, trying not to exclude or otherwise alienate buyers, you are between a rock and a hard-place. You want the buyer to be satisfied with their purchase, want them to upgrade, but not dismiss very expensive products (and their diminishing return}, or those that desire to own them.

In conclusion, it is an imperfect world, and every time we try to standardize what is a personal bias and personal experience, in some small way we chip away at the essence and passion of just being human. The more I read of Artificial Intelligence (AI) and Artificial General Intelligence (AGI), what comes to mind is the Star-Trek nemesis the Borg, You will be assimilated, Resistance is futile. But, the internet gives us all a voice for now, and the VPI forum, and the VPI company site can be powerful tools to advance its own thoughts. Consider what Dr. Van-De-Hull did for FAQ - the phono document is many pages, that took a lot of work. Consider that maybe VPI on the corporate page can have a tab for the VPI Library; here will be reviews by magazines and other sources, and well written articles associated with vinyl record playback, and articles written by VPI. There is no lack of opportunities.

Just some thoughts.

Neil
theeng
Member
 
Posts: 56
Joined: Mon Dec 09, 2019 9:47 am

Re: Are Audio magazine rating worth anything?

Postby Golear » Wed Mar 25, 2020 9:27 pm

I think Arthur Salvatore (AS) went into great detail about how worthless the Stereophile rating system is.

Here are a few links:
http://www.high-endaudio.com/magaz.html
http://www.high-endaudio.com/reviewers.html

Its not that they can't do it, or that they don't know how to do a proper review. It's that the realities of the magazine industry don't allow for it.
Golear
Senior Member
 
Posts: 399
Joined: Wed Nov 30, 2016 4:51 pm

Re: Are Audio magazine rating worth anything?

Postby Mat » Wed Mar 25, 2020 11:16 pm

The frustration also comes when we flat out see contradictions that make no sense. To stick with the example, Stereophile identified the Classic Direct as a Class A+ product. Fast forward 5-6 years... the Classic Direct magically disappears since it has been "too long" when there are products that have been there much longer. The HW-40 which is identified as "better in every way . . . than the Classic Direct." and is knocked down to Class A... something doesn't add up, besides the fact we haven't advertised in awhile. :|
Mat Weisfeld
President
VPI Industries Inc.
User avatar
Mat
Site Admin
 
Posts: 1133
Joined: Mon Aug 25, 2014 5:19 pm
Location: Cliffwood, NJ

Re: Are Audio magazine rating worth anything?

Postby Mat » Wed Mar 25, 2020 11:20 pm

Golear wrote:I think Arthur Salvatore (AS) went into great detail about how worthless the Stereophile rating system is.

Here are a few links:
http://www.high-endaudio.com/magaz.html
http://www.high-endaudio.com/reviewers.html

Its not that they can't do it, or that they don't know how to do a proper review. It's that the realities of the magazine industry don't allow for it.


Golear, that was a great read! Thanks for sharing the links.
Mat Weisfeld
President
VPI Industries Inc.
User avatar
Mat
Site Admin
 
Posts: 1133
Joined: Mon Aug 25, 2014 5:19 pm
Location: Cliffwood, NJ

Re: Are Audio magazine rating worth anything?

Postby jmcox » Thu Mar 26, 2020 2:32 am

Wow, incredibly entertaining. Thanks Golear.
VPI Avenger/Transfiguration Proteus, VAS Nova Gold .25, VAS .30 MONO/Ring Clamp/SDS/EAR MC4 SUT/Manley Chinook SE phono/Primaluna Dialogue Premium preamp/McIntosh MC40 mono blocks/Ampsandsound Cornscallas/Kimber Select all cables/Furman Reference 15i
User avatar
jmcox
Senior Member
 
Posts: 166
Joined: Wed Jan 27, 2016 10:21 pm

Re: Are Audio magazine rating worth anything?

Postby Harry » Thu Mar 26, 2020 10:55 am

I remember reading that a quarter century ago and really appreciating what he was saying.

At the VPI house we have a DSA phono with three inputs and three tables set up: a Scout, a Prime, and a Prime Signature. We have the same cartridge in each table and the same length of the same interconnect cable. When you hear the three tables you are actually hearing the complete table against the other two with no differences other than normal manufacturing variables which are quite small.

About reel to reel tape, we have 7 tape machines but best of all an Ampex ATR-102 that was ordered for me by Chad from Acoustic Sounds from ATR Services so that when playing his tapes it will sound just like what he uses when making the records. It is an amazing tool and non of the reviewers have one so when they make comments about the sound of a table arm or cartridge they have no clue what the original tape that the record was made from sounds like.

The biggest difference between tape and vinyl is that tape never crunches, it just gets louder, with no cringing as you are listening plus you can listen at levels that would drive you crazy on vinyl. The reason we make tables the way they do is they have to compete with reel to reel tape and we do the comparison all the time. We did an Avenger Reference VS Reel to reel at the Montreal show a few years ago and it was quite revealing how close the table can sound.

Bottom line is the HW-40 sounds closer to the master tape on the Ampex ATR-102 than any other table we make with a stability and smoothness plus very wide dynamic range that is so easy for tape to reproduce. That being said the HR-X, Avenger, TNT are very very close, we are splitting hairs but that is what we do.

HW
Harry
Admin
 
Posts: 1428
Joined: Mon Aug 25, 2014 10:38 pm

Re: Are Audio magazine rating worth anything?

Postby Harry » Thu Mar 26, 2020 12:06 pm

I'mm retired so I can say things as I see them.

The Recommended Components list is a disgrace to consumers who actually have to spend money on this gear not get super accommodations or indefinite loans from underfunded or garage manufacturers. The interesting part was for most of that time the Continuum was not being made so why was it included??

***The Japanese Tech Das was deleted then came back to A+ with a new version for $150,000 with no tonearm because Mickey says it is now better for a 50% increase in price.
***The HW-40 Anniversary direct drive gets reviewed by Mickey and it is called better in every way than the Classic Direct for half the price and drops out of Class A+. WTF!!!!

2015 Class A+ - Continuum Caliburn - $200K
- VPI Classic Direct - $30K
2015 Fall Class A+ - Continuum Caliburn - $200K
- VPI Classic Direct - $30K
2016 Class A+ - Tech Das - $105K no arm
- VPI Classic Direct - $30K
2016 Fall Class A+ - Tech Das - $105K no arm
- VPI Classic Direct - $30K
2017 Class A+ - Tech Das - $105K no arm
- VPI Classic Direct - $30K
2017 Fall Class A+ - Tech Das - $105K no arm
- VPI Classic Direct - $30K
2018 Class A+ - Tech Das - $105K no arm
- VPI Classic Direct - $30K
2018 Fall Class A+ - Tech Das - $105K no arm
- VPI Classic Direct - 30K
2019 Class A+ - VPI Classic Direct - 30K
2019 Fall Class A+ - VPI Classic Direct - 30K
2020 Class A+ - Tech Das Premium - $150K no arm

I am really disgusted with this outcome. I challenge anyone to bring a Tec Das over, we will use the same arm on both, the same cartridge on both, and lets see which one sounds more like the master tape of Take 5 on my Ampex ATR-102. Better yet, one is more than 10 times the price of the other, it should not be close but it will be and I will lay odds the HW-40 wins.

HW
Harry
Admin
 
Posts: 1428
Joined: Mon Aug 25, 2014 10:38 pm

Re: Are Audio magazine rating worth anything?

Postby theeng » Thu Mar 26, 2020 12:27 pm

Harry,

Awesome stuff, but one problem, its buried here in this forum which has only limited visibility. However, if you put pen to paper (word processing actually) and wrote a White Paper similar to what Mike did for the HW-40, then it stands as a reference, challenged or otherwise. Then from the White Paper VPI can take that info and prepare a nice marketing document (pdf) that can be easily distributed, people can download. If you have Microsoft Word, you can put together something nice without having to spend any money other than your time.

Just some thoughts,

Neil

PS; All the details I have discussed about cleaning on this forum are now with a bunch more, documented in a 50-page document titled Precision Aqueous Cleaning of Vinyl Records. For 20-yrs I was the US Navy technical lead for precision cleaning of high pressure oxygen, air and life support systems. I have written 100's of pages of reports, specifications and standards, so this new document is just one more; now to figure out what to do with it.
theeng
Member
 
Posts: 56
Joined: Mon Dec 09, 2019 9:47 am

Re: Are Audio magazine rating worth anything?

Postby Golear » Thu Mar 26, 2020 12:50 pm

Harry wrote:Bottom line is the HW-40 sounds closer to the master tape on the Ampex ATR-102 than any other table we make with a stability and smoothness plus very wide dynamic range that is so easy for tape to reproduce. That being said the HR-X, Avenger, TNT are very very close, we are splitting hairs but that is what we do.

HW


That's a meaningful way to compare equipment. But how does one compare it to something else, like a Clearaudio? Magazines used to do that, and the manufacturers would invariably comment that the set-up must have been incorrect. They might say... "We use the Ampex ATR-102 with the Dikembe mod". And they'd have a point.

As far as the current magazines are concerned (and ever since the early days of Stereophile, IAR and The Absolute Sound) is that (many) magazines and reviewers pander to the manufacturers. The magazines get advertising revenue and the reviewers get gear for free or at very friendly prices. And this can come to hundreds of thousands of $. For example, Johnathan Valin got 17m of Nordost cable. If it was Nordost Odin, that would be $272,000. Not sure if that is correct, but I just did a quick lookup on the price of Odin. And then a "friend" cut it up and sold 1m lengths on Audiogon (or some such site). When the scandal blew up, Nordost said it was their policy never to ask for the return of cables given to reviewers. That's a tidy side income. It could even be the primary income. (Do they need to split the profit with the magazine?)

How can Nordost do this? Obviously, they have a very big markup. So their cost is nothing close to $272,000. May be it's a few hundred $.

I think the blame has to be placed on the manufacturers, like Nordost. They are the ones who provide gear to reviewers, and they are the ones who buy adverts. Manufacturers must insist on the return of all review equipment. The reviewers must gain nothing (other than the use of the gear for a certain amount of time.). What if the reviewer wants to buy it? Return the review sample to the manufacturer, then the reviewer can contact a dealer, like the rest of us.

(When a reviewer writes "I liked it so much that, yes by golly, I bought it and it has become my reference!" how do we know that it wasn't at a "very special price"?)

The magazines can receive adverts from magazines... but the magazines have to show what revenue they earned from each manufacturer. This is pretty easy to work out, but make it explicit for the reader. If one manufacturer accounts for a huge chunk of the revenue, that can be a factor. For example, it was obvious that Microsoft was running huge adverts in "PC Magazine". This would be important to know, for example, when PC Magazine did a comparison of Microsoft MS-DOS with Digital Research DR-DOS. Magazines should turn down offers of megabuck advertising from a manufacturer lest they lose their independence.

If there's a conflict of interest between the reviewer and the manufacturer, then say so openly. HP once said that he would not review Wilson speakers because Dave Wilson was a close friend. (It's also a convenient way out if HP really hated the sound of Wilson speakers.) Readers aren't dumb.

And finally, it's up to us, the audiophile community to demand more from the magazines and from the reviewers. Just stop visiting the websites, and refuse to read the reviews. Or do so, but just take what they write as press releases or marketing material. I haven't read Fremer's review of the HW-40 and don't intend to.

In order for me to take Fremer's review of the HW-40 seriously, he'd have to publish a review that says that one device is terrible/horrible/a joke, and the reasons for that. I would then try to hear it, and if I agreed, then I'd take Fremer seriously. And if he says something is good, then I'd want to hear it and make up my own mind, too. So on the balance, the review doesn't really help or hurt my decision. It's not a factor. (Here's one from me: The Einstein phono stage is terrible. Soundstage is flat as a pancake. No depth. You should still hear it though - it might work wonders in your system.)

What can a magazine do, then? Hi-fi is a complicated field. Tell me about set-up - what to watch out for. Warn me that the original Linn Sondek LP-12 can't run at 45 rpm. How does it like Stillpoints LPI? What's the effect if you put Stillpoints under the turntable? How fussy is it about the type of stand that it is on. Warn me that the motor block on a current VPI can be noisy and might need some kind of isolation under it. Can you place it close to a subwoofer? (In the case of a Linn, no. In the case of a Goldmund, I think you can put it ON the subwoofer and it will be fine). Don't come up with a judgement, just write about the experience.

And magazines can teach. For example, HP deserves credit for popularizing the concept of "soundstage". Ivor Tiefenbrun (of Linn) said there was no such thing. And in the early days, it was HP and The Absolute Sound in the US vs the British magazines (which followed the Linn line). HP should have stuck to that. He did say, years later, that the battle over soundstage was won, and that The Absolute Sound should try to raise the bar. He wrote about a whole slew of new concepts like "gestalt" and "continuousness". (Does anyone know which issue that was?). It was a work-in-progress, as these terms would have to be explored and pinned down. HP called on the other reviewers to write reviews based on these terms. Alas, they ignored him, and then The Perfect Vision distracted HP, and then HP got sick.... So we're stuck where we are now. Magazines serve up drivel because they don't know what to write about. Now that digital has been defeated, it's up to us to come up with the next frontier.

There's the whole field of cleaning LPs, too. And I've learned more about that from this forum than any of the magazines.

How about turntable set-up? Yes, we can buy some fancy LP set up gear. But we can also do every one of those tests with a $300 oscilloscope and a $50 LP. It's probably better, because the oscilloscope can be calibrated to a much higher degree. A "measurement guy" told me, "First you measure for open circuit, then you measure for short circuit, then you measure across a calibrated resistor, then plug all that into the equation....".

And magazines can cover mods. I think there can be very good discussions about solder, caps, resistors, circuit mods, etc. Arthur Salvatore's system doesn't have a volume control.... And they can talk about their rooms, and how to treat them.

It is driven by discovery. For example, scientists are scrambling to explain "Dark Matter" and "Dark Energy". In the same way, what if someone could hear something that they had never heard before, from a certain piece of equipment? Describe it, and let the word out.... (Here's one: forget about 300B and new old stock premium 12AX7 tubes. Russian tubes are better than anything available in the West. They run hot and they wear out more often but they're ridiculously cheap, so just buy 50 of them. US manufacturers haven't switched over/they're milking their existing designs ...so you'd have to DIY, or work with people to make custom gear.)

A healthy audio press is also of value, to the greats of the industry, like HW. What, exactly, was the HW-19? Years from now, a young whipper-snapper would find out that it is a true classic. Ditto for Arnie Nudell's speakers. And William Zane Johnson's amps (if you find an Audio Research SP-3, grab it and then mod it like crazy).

How to do all this? Return to the model that was followed by The Absolute Sound, early on. Subscription only. Few if any adverts. Equipment sourced anonymously. No interaction between reviewers and manufacturers. It's hard to do this, of course. What's to stop a reviewer from contacting the manufacturer and working out a side deal? May be every review should be commented on by the Editor. Or Editors. Or HW (except for VPI gear).

EDIT: The last line in my post was, "Or may be the identity of the reviewers should be secret". I think that's incorrect. There shouldn't be any secrecy.
Golear
Senior Member
 
Posts: 399
Joined: Wed Nov 30, 2016 4:51 pm

Next

Return to Harry's Corner

Who is online

Users browsing this forum: Google [Bot], NJdiyer and 7 guests

x