.jpg)
Put four converters to the test, in a pro studio with pro testers. See which one comes out best. Or is the result a foregone conclusion?
Recently reported in the audio press is a test of four A-to-D and D-to-A converters. These converters are in the first rank of professional equipment, hardly home studio gear.
The studio where the test was done is one of the best in the world. And the people doing the listening have the kind of experience over many years that anyone would die for.
So the converters were hooked up, whereupon the hooker-up - if that's a word - left the room so that no-one taking part in the test knew which converter they were listening to, not even the person in the room switching between them.
This kind of test is known to science as 'double blind'.
This means that not only do the people being asked to make the judgments not know what they are listening to, neither does the person conducting the test.
The person who does know which is which needs to be completely away from the experiment where they cannot possibly influence the outcome, however unconsciously.
The group decided that Converters B and D were both very good, one better on vocals, the other better on everything else.
Converter B is a product of the company whose representatives organized the test.
So the product that the organizers of the test clearly hoped would have won, actually did win. Well it came joint first. I'd have to say that if it's the best for vocals, then it is in first place as vocals are more important than anything else.
So this is the point in this article where you would expect rampant cynicism. Of course Converter B is going to win, if the manufacturer's representatives are organizing the test!
Well in fact we have no reason to believe that there was any underhandedness of any kind. There is no reason to think this is anything other than a genuine result.
However...
In science there has to be a clear distinction between a test that you can learn something from, and a test that is of dubious or perhaps no value.
And one of the indicators of a flawed test, to a scientist, is that the result of the test supports the objectives of whoever organized or paid for it.
It doesn't matter how fair minded you are and how scrupulously the test is conducted. There will always be the temptation that should the test go the 'wrong way', the results will not be published. So only the results of 'successful' tests would be presented to the public any other test reports would simply be filed in the trash can.
As you may have noticed, no-one has been named in this article. That's because we don't doubt the honest intentions of the people involved. We don't doubt the validity of their conclusions.
However, anywhere else we would wonder how many test results had been trashed because they went the wrong way.
Testing is important, but it is also important when evaluating the results of a test to know of any circumstance that could invalidate the test, or flag up a potential conflict of interest.
By the way, the manufacturer of Converter B had another of their products in the test.
It came last!
No comments:
Post a Comment