Minidisc Test Lab
July 31, 2004
Two weeks ago I rushed out and bought the first Hi-MD Product available in the USA, the Sony MZ-NH600D Walkman. This is the third generation of Minidisc, following the Gen-1 standard 74-min high-quality Audio MD which spanned several iterations of improvement in sound quality using 292 Kbps ATRAC technology on 170 MB media, and the Gen-2 NetMD which introduced two new audio formats under the guise MDLP: LP2 and LP4 which doubled and quadrupled capacity on the old 170 MB media, respectively. Gen-3 Hi-MD introduces a whopping leap to 1 GB media on the same form factor (using the same 780 nm wavelength laser and 65mm diamater disk, but using fancier encoding and reducing the bit length from 0.59 um to 0.16 um).
This entry-level Hi-MD Walkman has no mic/line input. Instead, you record to it like Gen-2 NetMD with your computer over USB. This is accomplished with two software options: the bloated, buggy, and publicly hated SonicStage, or the lighter weight "Simple Burner" option which only records CDs straight to MDs. I have used both of these packages on my Sony MZ-N510CK NetMD Walkman, and I can first report an interesting flip on the software front: The long-awaited SonicStage v2.0 released with Hi-MD is far superior to earlier versions. On the other hand, I un-installed the new version of Simple Burner after 10 minutes because it was difficult to use and was crashing Windows XP. Sony apparently poured their resources into improving the bloatware, and let the previously popular lightware fall into disrepair. Ce la vie.
The new SonicStage is also Sony's answer to iTunes and Napster 2.0, serving as the client-side software interface to the Sony Connect music download service. Music from Sony Connect comes down as 132 kbps ATRAC Gen-2 LP2, which sounds quite good. But more on sound quality later. First, I have some results from the other new benefit of Hi-MD: using your MD device as a removable media USB data drive. This concept was tried years ago with an old "MD Data" 140MB format which flopped. Let's hope the 1 GB Hi-MD format lives a little longer and spreads a little broader in the USA. (All MD gear enjoys greater popularity in England, Japan, Australia, etc.)
Standard two-dollar 74/80-min MD disks formatted as Hi-MD hold 300 MB instead of 170 MB. My Hi-MD unit also came with a sample 1 GB disk. These high capacity blanks are not yet available in the general USA retail market, but are expected to cost $11 each. Regardless of capacity, Hi-MD allows music, data, or both to be recorded together on one MD. This makes the single-AA battery powered Hi-MD Walkman a curious competitor against the wildly popular USB Flash Memory Drives, with Hi-MD sporting the benefit of removable $2 media. Let's see how they compare.
DATA Read/Write Speed Test:
This test was conducted on a
Dell laptop with USB 2.0 in Windows XP Pro. I tried dragging three
folders of data onto a standard 74-min MD (Sony Gold) in Hi-MD mode, and
was surprised at how slow it was. So I got out the stopwatch and
measured the real time it took to transfer data. I ejected the MD
between tests to make sure nothing was cached. For the Write and Delete
tests, after the Windows copy dialog finished and disappeared, I waited
until the "Access" indicator on the MD unit quit blinking before
stopping the stopwatch. Deleting the files also took surprisingly long,
but keep in mind this is 700 FAT file unlink operations on
floppy-like media.
Then I performed the same drag/drop Windows file copy operation to a Lexar 256 MB USB Flash Memory drive. Sure it's hardly a fair race, but then again, the Flash Drive wasn't as fast as I expected. In fact, it was slower at deleting, FWIW. I can add some Zip-750 stats later for additional comparison. In any case, the point was to gather some real world speeds when using Hi-MD as a sort of 300 MB floppy drive. I will endeavor to also add a column on the 1 GB disk, in case it's any faster (because of data density).
| Hi-MD | USB Flash | Zip-750 | Firewire 160 GB HD | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Time | Speed | Time | Speed | Time | Speed | Time | Speed | |||||
| Read | 211 sec | 197 KB/sec | 26 sec | 1600 KB/sec | ||||||||
| Write | 533 sec | 78 KB/sec | 162 sec | 256 KB/sec | ||||||||
| Delete | 47 sec | 14 Files/sec | 68 sec | 10 Files/sec | 1 sec | 664 Files/sec | ||||||
Aug 1, 2004 (still being written): I have been conducting listening
tests over decent but not awesome headphones (audio-technica
ATH-M2X) the past few weekends. It's been a while since I've done
any critical listening on audio compression methods. First, major
props to all the software engineers out there who have worked so hard on
MP3, OGG Vorbis, and all these other audio compression techniques. I am
no expert, but I know enough about audio, perceptive encoding, data
compression, and software programming to know how frickin' amazing it is
for CD audio to be squeezed down by 5:1 or 10:1 and still sound decent.
ATRAC is Sony's baby, born (I believe) as the encoding technology behind Minidisc in 1992 and nurtured through many iterations. (ATRAC/MD sound quality saw major leaps in the first few years, then followed a natural tapering of results after ongoing engineering. In fact, if Sony had waited just one year to introduce MD in the USA, the improved sound quality probably would have avoided the unfortunate stigma that MD has never overcome in the USA market.) MP3 is the widely-accepted competitor, growing up this year into retail-ready MP4 aka M4A (mp4 audio). OGG with the Vorbis audio codec is the highly respectable open-source alternative. There are many others like Microsoft and RealAudio.
Not being math majors, what the general public may not understand is the exponential decay of improvement in sound quality as software engineers continue to pour sweat into these technologies. For example, it is simply not reasonable to expect that a lossy-compressed sound file at 128 kbps "now" will sound as good as 256 kbps did a few years ago. Such a two-fold accomplishment requires a 10-fold leap in technology. Granted, MP3 is ten years old, and such a leap has almost taken place. But another doubling of results might take 100 years.
Sony wants us to believe it only took 4 years. When I bought Hi-MD, I noticed they were touting ATRAC3plus, their latest generation audio compression software, as an MP3 killer. Specifically, their marketing materials bragged about ATRAC3plus (Gen-3) at 64 kbps maintaining "extremely high quality sound" and implied it was comparable to ATRAC3 (Gen-2) at 132 kbps. The Minidisc FAQ echos Sony's hope that this 64 kbps miracle sounds equivalent to MP3-128k or ATRAC3-132k. (See the above Sony Japan graph copied from the FAQ.) Keep in mind: the goal here is to store twice as much music on a memory sick, CD, MD, DVD, or whatever - with the same sound quality. The bottom line after my research on a variety of material: No such luck.
ATRAC is good stuff. While Sony unseating MP3 as the reigning champion (even with McDonald's as a partner) is about as likely as the MPAA rating graphic violence worse than nudity, there's nothing wrong with ATRAC3plus. It's on par with Vorbis and MP4 as a sophisticated 2004-era generation of audio compression. However, the above graph is complete fiction. Let's break it down.
The chart attempts to map out the three generations of Sony's audio compression, indicated in colored regions: ATRAC in green, ATRAC3 in yellow, and (missing its label) ATRAC3plus in red. (Why, oh why, didn't Sony call it ATRAC4?! Someone at Sony Japan is surely committing suicide over that one.) Diagonal lines map three comparitive ratings: "CD Quality", "MD Quality", and "FM Quality". Time goes to the right, and bandwidth goes up the left edge. The region below 64kbps is highlighted.
1. MD QUALITY? The ATRAC data point at 292 kbps (in green) is deceiving because they mark it as the 1992 initial debut ("ATRAC1 1.0"), which sounded like crap. EVEN THE AMERICAN PUBLIC REJECTED IT, so you know it was bad. Sony labels this as "MD Quality". Problem: ATRAC1 v2.0 in 1994 was very listenable as the new MD algorithm, and 1996's ATRAC1 v3.5 and 1997's ATRAC1 v4.5 were both phenomenal, eliminating the gap between MD and CD to even the most discriminating ears (for those willing to forget the past and give MD an honest listen). The bit rate for all these ATRAC1 generations is/was 292 kbps, which is still a challenge to work with, but like a breath of fresh air to any engineer trying to squeeze high quality out of 128 kbps.
2. MD MORE LIKE FM THAN CD? Therefore, back in reality, "MD Quality" quickly became "CD Quality" with the same core algorithm at the same bit rate. Sony is selling themselves short by simply labeling it "MD Quality", and this is shocking because they're still selling MD's in some flavor. You'd think they'd want to make the lines between "MD Quality" and "CD Quality" closer together. And, unlike most of this graph, that would actually be true! Instead, the "MD Quality" line runs closer to "FM Quality", which makes no sense. But the marketing engine doesn't care about making the past look bad, as long as it helps make the present and future look outstanding. A better graph would show sound quality up the left edge, instead of bit rate. Then ATRAC1-292k could swoop up the quality curve over time.
3. MAP MAY NOT BE TO SCALE. Check out the scale on the left side. Get out your ruler. First of all, it's an exponential scale, not linear! Call the marketing police: we're entering math textbook land! The scale doubles with each centimeter, from 64 to 128 to 256 to 512. Then we leap to 1400, a 3x instead of 2x change, but okay, there's no data up there and it just reminds us of raw CD speed. But at the bottom, we hit 48 instead of 32. No biggie? That's a 25% error! Sony is exaggerating the gap between 48 and 64 to make it look like more bandwidth savings. The truth: it's a 25% savings, not 50% savings like elsewhere the graph. And trust me: ATRAC3+48k sounds like shit. It's not worth 25% more quantity on the memory stick. It's a last resort if you're stuck on a modem, period. Sure, it may sound like less shit than other competing algorithms at low bit rates, but who cares? Who really sits there using up every last bit of their modem connection listening to live audio? Okay, phone over Internet, online training, whatever. That's not what I'm here to talk about. I want to enjoy listening to music. My goal in all this research, by the way, is simply to determine which of six choices to use when (a) ripping CD's to my hard disk library through SonicStage, and (b) which to use when I then copy that music to MD to listen to on the plane or while inline skating.
4. MAP MAY NOT BE TO SCALE (continued). The horizontal axis isn't any more consistent. The first inch takes you 8 years from 1992 to 2000. The second inch takes you 4 years to 2004. Who cares, you say? We'll get there.
5. DECEPTION ON "CD QUALITY": The first deception which follows from these inaccuracies is that only ATRAC3+256k in 2004 reaches "CD Quality". Few Minidisc afficionados would argue this title can be shared with ATRAC1v4.5-292k from 1997. Again, the past is desecrated to make way for the future. Hmmm. This marketing material has Japanese text, but it must have been made by an American. The Japanese have too much respect for the past in their culture. Maybe that's another dead Sony Japan engineer with his own sword in his gut. We're losing a lot of lives here, for the sake of selling ATRAC3plus.
6. DECEPTION ON "MD QUALITY": Now the sin grows larger, and your shocking gasp of horror should grow to match. We see a straight line from 292k down through 132k as time progresses. This is a reach, but I'll give the benefit of doubt to the masses with substandard hearing who seem to always be participating in these marketing studies. But then we continue this trend down through 64k. Bwoop! Bwoop! Bwoop! Red alert! Danger, Will Robinson! We have complete bullshit in sector three! Bwoop! Bwoop!
If you don't believe me, listen carefully to your favorite style of music at 64k. Ignore the hype; ignore me; try it yourself. You know what really did it for me? Loading up a 1GB Hi-MD disk at 64k with 35 hours of music and listening to it on various parts of a trip (usually on a plane or walking through airports) for perhaps 10 hours. I had believed this hype, and expected the 64k miracle to sound good - so it did. Around hour 7 or 8, I snapped like a cat who steps in peanut butter. It stopped being pleasant background noise and I realized I'd been had. "What is this shit? Eww! Get it off me!" I started licking my paws, flipping through tracks on the MD, looking for something that sounded better, but all I could hear was that awful metallic sound that creeps in with sound compression gone too far. It was on every track, from jazz to rock to techno, from guitar to drum to vocalist. Once you key into it, it's like learning any other skill, and you're ruined. Innocence is lost.
7. DECEPTION ON "FM QUALITY": ATRAC3+48k may sound just as bad as ATRAC3-66k. I can't stand to listen to either one long enough to tell. But I can stand to listen to FM radio all day. Go figure. It must be something about this whole digital world we find ourselves in, blindly convinced by marketing trash like this, that anything new and digital is better than the old analog ways.
8. MY FUTURE'S SO BRIGHT: Here's where this graph goes from inaccurate to laughable. The diagonal straight lines on this ATRAC "Road Map" (which aren't really straight because both axes of the graph are nonlinear, but that's mathbook talk, again) continue on down to the right, off to the fading unknown of 2006. (I told you we'd get here.) We don't even have to wait until 2012. The implication: sound quality will keep getting twice as good in half the time with half the resources. CD Quality will be attainable at 64k, and the others will approach ZERO kbps. Heh. Yeah. I'll see you in 100 years.
P.S. Bandwidth keeps growing, getting cheaper, and media capacity keeps increasing. There's not even market pressure for this fantasy! I don't know what Sony is smoking, but it's strong stuff.
CONCLUSIONS
It's fun to pick on the marketers and executives who make up these pretty charts, but what really counts is what you hear. On a scale from horrible to excellent, ATRAC3+64k sounds ... good. It certainly sounds better than I expected, given half the bandwidth of the ubiquitous 128k which has become the common "sweet spot" between size and quality. Okay, I would put ATRAC3+64k somewhere on par with FM radio. The stereo separation is good, not great. Dynamic range is noticably reduced. But most of all, the digital artifacts (that nasty distorted, metallic sound) are clearly evident at 64k, especially on layered, airy vocals like Enya, the jazz-rock piano licks of Supertramp, or the complex techno of BT. Even simple pop music like Brittany Spears sounds rough and "dirty". Bad Dirty like it was recorded with an old, worn-out microphone and the electric bass had cotton on its pick-ups; not Good Dirty like Brittany grinding her ass into the camera.
Unfortunately, it may be some time before I have any samples here on this web site to compare. But if you have access to any of this material, here's what to listen for as you fire up your best HiFi system, close your eyes, and focus on the music. The trick is to isolate different parts of the music. Start by listening just to the vocals for a minute; then switch to a particular instrument, like the drums, then the bass. As another method of breaking it down, don't pay attention to any particular instrument or vocalist, but instead focus on tonal ranges: low bass sounds, then midrange, then treble (cymbals, female vocals, etc). Of course, I should have started all of this with the standard disclaimer: judging compressed music is subjective, YMMV, yadda blah yadda. But in this case, I have a hard time even saying that. The audible distortion I am hearing at 64 kbps is blatant. I first noticed these problems while flying on a jet airplane, where the music in my headphones was barely louder than the ambient noise!
Conclusion: ATRAC3+64k is suitable for any scenario where you are not focusing on the music. If it is in the background somewhere, these differences are probably too subtle to grab your attention, except in the case of a song which you have spent hours comparing at different bitrates, and have learned every nuance through listening 100 times. :-) For me, the entire album of Supertramp:Cannonball has been ruined in this way. You may scoff at this being no big loss. :-P
This morning I took on a stronger challenge: Compare higher bitrate compression with the original CD source material.
Additional References:
OGG vs. MP3 vs. WMA vs. RA: Which
is the best low-bitrate audio compression algorithm? - From
1998-2002 - too bad ATRAC wasn't included - and the sound samples are
missing :(