Audio Taste Test: Who Beats The iPod On Audio Quality?

  • MORE

as best_pmp_audio_shIf you read our review of the Zune HD you may have noticed that we compared its audio quality to the Sony Walkman X Series. I did this after listening to the same song on both players at the suggestion of Jeffery Wilson (who reviewed the Sony). We both agreed that the Walkman had superior audio quality over both the Zune and the new iPod nano, which then led to the question: if other people knew this, or could experience it, would they still choose an iPod? Does sound quality matter to consumers? I decided to put it to the test.

We took the Zune HD, Sony Walkman X, iPod Nano and the Samsung P3 (another player with a high audio quality rating) to two music stores in New York City to ask employees and patrons which they thought sounded better. However, we didn’t let our test subjects see which players they were listening to. The only data they had to go on was how each sounded while listening with a pair of Bose QuietComfort 15 noise canceling headphones.

We created a playlist with 15 - 30 second snippets from 6 songs. The tracks were not only from different genres but also encoded at different bit rates so each listener would experience the full gamut on each device. Our playlist consisted of the following tracks in MP3 format:

Our first stop was Village Music World, a little store packed floor to ceiling with CDs and records -- audiophile heaven. After both employees and patrons listened to each playlist on the Zune, Walkman, nano and P3, we asked them to describe their auditory experience.

Testers felt that the Zune and nano had nearly identical sound quality but rated it as thin or flat. Everyone agreed that the Walkman and P3 had superior quality because the sound was fuller and richer, but our testers were evenly split on which they thought was slightly better.

Next we went to Other Music in the Village, a store famous for its amazing collection of CDs and LPs plus its knowledgeable employees. After listening to the players in the same order, Other Music employees had similar reactions. Zune and nano sound quality: thin; weak bass; felt hollow; flat. Walkman and P3: fuller, richer sound; more present bass; more “air” to the music. One Other Music employee rated the nano higher than the Zune, but preferred the Walkman overall.

Everyone we conducted the test on were iPod owners (past or present) and many rated the quality as good before the test. All but one was surprised to learn that the iPod was one of the devices they’d given a poor rating to. The only exception was John from Other Music; he stopped listening to music on his iPod due to audio quality. (He likes vinyl best.)

After the test was over, I asked each person which player they would buy if they were going to purchase a new one now. Everyone chose either the Sony Walkman X Series or the Samsung P3 over the nano or Zune.

We played the PMPs in the same order for each listener:

  1. ZuneHD
  2. Sony Walkman X Series
  3. iPod Nano
  4. Samsung P3


When deciding to purchase a music player, what do most consumers look for? The four players we showed people each have good functionality and a good to great interface. Though iTunes is convenient for both organizing and purchasing music, it's not the only kid on the block, anymore. Yes, Apple has the biggest selection. But when music lovers are looking to purchase new releases like Whitney Houston’s comeback album I Look To You, they can do so through or the Zune marketplace or even Wal-Mart’s website.

If a consumer feels that all else is equal, what’s left? Audio quality. And in that department, the most popular player and its most high-profile competition lag behind.

Does audio quality matter to you? If you could have a player that sounded far better for about the same price, would you choose it over the hot or popular product?

Image: secret by bandita on Flickr

Add a comment
  • Steve Says:

    Well, Testing is something that make yourself believable about that specific things. The share how you can do it.

  • Warren Peace Says:

    Re: "audio quality"

    Matthew, I hope you've learned a little more since 2009, because then you were a pedantic know-it-all who didn't know what he was talking about. How can you believe that all DACs sound the same? They don't, and in fact there is as much variation as there is in different headphones! Two players that share the exact same DAC chip rarely sound the exact same, due to differences in circuit design, varying amp sectio s, etc.

    I see you're in love with a narrow definition of the term audio quality, but you're wrong about that being the only accepable useage, you're wrong about differences in the file (compression, sample rate, etc) being the only or even the main driver of output quality, you're wrong about everything you said, really. Not a good look for a know-it-all.

    Then we have "Bobby Digital" bashing digital audio in favor of vinyl? Really? Digital is objectively superior for information storage, and I'm sorry you haven't found a DAC that provides the same kind of distortion that you like, but saying vinyl sounds better is just saying you prefer some kinds of distortion over clarity.

    As others have pointed out, this wasn't a rigorous scientific test, but it was never meant to be. As a quick and dirty test it succeeds at making its point.

  • Gå-Toola Says:

    Walkmans are awesome. Had both a Nano and a cheap Walkman and the overpriced iPod got bloody beaten by at least 3 times in loudness from Sony's cheapest player, still not even sounding bad.

  • Ignaswaran Says:

    Consumers are all NT Nader equal, Some are rich and some are poor and many fall between with most on the poor side. Therefore to those one device over another is more of which one can

  • larry Says:

    No. Customers do NOT care about the 'audio quality'. If they did, they would be listening at home to uncompressed music on their hi-fi. When the 'need' for portable audio arises, the player with the largest HD, allowing the most music to be transferred at an uncompressed/ or near uncompressed format, such as Apple Lossless, would come out ahead. The sad fact is that people want more songs, less expensive everything, and are happy with the tradeoff in sound quality.
    How many 'upgraded' earbuds or headphones do you see in peoples' ears??? I don't see many. I see the uncomfortable, acceptable (barely) sounding 'phones that came with the device they chose.
    I love my Apple Ipod(video, the old 30 Gig) and my new iPod Touch 32. They are both filled with AAC compressed music, most at 256 kps, and that is fine for what it is used for. Walks, yardwork, camping, etc. Anything else is listened to on CD or Vinyl at home where ALL portable music players are blown away. Headphones or speakers.


    it is true that bit rate is very important in these tests. if they all listened to the same song then that's ok since it's a constant. otherwise one person could be listening to a a cd quality song and the next person could have heard a 128 kbps mp3 which is complete garbage no matter how you slice it.
    either way mp3's are compressed and suck compared to vinyl!! you young whippershnappers don't even know what you're missing sonically!!
    peace laptop...

  • tonti Says:

    Very interesting results. I've listened to both a 2nd gen touch and zune. I would have to say both sound just fine and there's hardly a difference between the two. IMHO, when it comes to sound quality is the earphones that matter. Get a good one and it'll take care of the rest

  • Matthew Says:

    Although that this article can be used to show what consumers enjoy listening to more, the use of the phrase "audio quality" is seriously misused. "Audio quality" when discussed is strictly based on sample rate and bit depth, and these are completely dependent on the audio file... not the player. You could make an argument that the digital to analog conversion in the players very in quality, but that would be so minimal that the majority of users would not notice.

    As to what users are hearing differently in from each product is simply differences in the overall frequency balance that is independent to each player. This could be compared to using different amplifier and speaker set ups with the source (cd player, vinyl player, etc.) What each user is hearing are these differences: volume difference, and e.q. difference.

    Now this is a valuable article if one is wanting to know which player more people prefer. However, saying that the "audio quality" is better or worse can be deceiving.

    If you want to know what is actually happening, you could perform a test to see exactly what the differences in loudness and e.q. are coming from each player that you used in this article. Just because you set each volume to 50% doesn't mean that they are playing the same dB at all. Also, the Fletcher-Munson Equal Loudness Contours have proved that as the intensity level of sound increases human ears perceive a more balanced e.q. curve... basically that louder is "more full" among other descriptions. I really believe that this was most likely a key variable in the comparison that was performed.

    Sorry for my rant, it's just that many people are uninformed when they discuss "audio quality."

  • Nage Says:

    Should've thrown in a Cowon S9 or D2+ in there as well.

  • K. T. Bradford Says:

    All of the volumes were set to 50% and we turned equalizer settings to normal on those that had this option. As to bass boosting, we're aware that sometimes people think audio sounds better when there's more bass even when it isn't technically better. This is why we tried a range of songs - some heavy on bass some not. In some cases, a more present bass is what you need, though, and if a player doesn't deliver it doesn't deliver.

  • Bill Says:

    What controls were applied to these tests? Were volume levels matched? Louder always sounds "better". Were tests done to measure how flat each player was? Its entirely possible that the players with more bass were actually boosting the bass, isn't it?

  • J Says:

    Ive always felt like my iPOD touch was flatter than my iPOD nano, which itself is flatter than my iPOD Shuffle... bigger always doesnt mean better.

Back to top