i  <esau

(un)sound science

Jim Lill is a musician from Nashville who conducts audio equipment A/B benchmarks with unusually high rigour and posts them on YouTube.

His most recent of these is on microphone preamps.

people are.

It's fair to say he's been getting a lot of feedback:🔊

A gallery of youtube thumbnails responding to Jim's video with titles like 'This guy is WRONG' and 'Jim Lill is so terribly wrong'

Criticism has converged on a handful of things:

  1. He's comparing a modern Focusrite 2i2 to a 1970s Neve 31105 that was specifically designed to be as transparent as possible. It's the other Neve preamps (like the 1073) that have special character.
  2. He says that none of the audio engineers he knows set their preamps' input gains to distortingly high values, but apparently that is a common thing that people do.
  3. He says "So Jim, are you saying all preamps sound the same? No. Some preamps are broken and some preamps are made so cheaply or poorly that their frequency response doesn't stay flat from 20Hz to 20kHz." which implies that he thinks all preamps that aren't cheap or broken do sound the same.
  4. He advertises a course of his in the video and asks that people send money to his PayPal.

It's plausible that he's overclaimed, and that the 31105 does something different under different conditions, or the 1073 does something special under normal conditions. Ideally, he partners up with a skeptical audio engineer who believes that preamps do sound different, and they agree upon a set of experiments that would confirm or falsify the null hypothesis through the magic of adversarial collaboration.

But at this point in the process of the Controversial Video lifecycle, it is far easier to record your hot take on it than it is to conduct good, measured science.

The attention economy thrives on controversy and there's basically nothing we can do about it. If it takes about two months to run and record a robust set of debunking benchmarks which might make you $200📉, but only an hour to record and upload a response which will make you $100, why wait?

A carnival sign that says 'You must have put this much effort into your response to post'

Overall, I think it's fine. This is a media ecosystem that is succeeding in incentivizing things like the original video in the first place💸, and I'm sure eventually someone with sufficient stubbornness and principles will make the counter-video that I want to see, even if YouTube wastes a lot of people's time with the cashgrabs in the interrim. I'm still hopeful that on the other end, the two sides of this debate come together to discover why scientific best practices exist in the first place.

And if I had to guess, I think a similar suite of controlled tests run on a 1073 will show that its secret sauce, if it exists, is also quite satisfactorily reproduced with a bit of EQ and saturation. Jim's core message of "you don't need expensive vintage gear to make good sounding recordings" will absolutely hold true, and probably also "expensive vintage gear's audio characteristics can be inexpensively reproduced to the standards of the average amateur musician."

That's what's actually at stake here in this debate. The people clutching pearls or insisting that they can score above 50% accuracy on an A/B test are doing something else.