Section from Metallica's Fade to Black (PCM .wav file compressed with FLAC)
For this sample at 128kbps encoding, the best encoder is clearly mp3enc. All other encoders show audible distortion (for example, on the vocal "s" sounds). LAME and bladeenc are very near in quality, with LAME being very slightly better (chosen every time in one consecutive run of ten blind random side to side comparisons). The quality of l3enc showed the most notable distortion. The distortion produced by LAME and bladeenc is similar to the distortion produced by l3enc, but the distortion is much more pronounced in the output of l3enc.
Section from Queenryche's Best I Can (PCM .wav file compressed with RAR)
Encoded with LAME 3.57 Encoded with mp3encdemo31
The different ways the encoders handle this sample at 128kbps is interesting (mp3enc with -qual 9, and LAME with -h). The most obvious difference is revealed in the 's' sound in "see". In ten blind random side by side comparisons, LAME was chosen every time as having frequency content closest to the original. (Several comparisons were blindly thrown out of the results and not counted due to distractions, and subsequent inability to grasp the difference for a particular set.) However, I did notice a slight distortion in the LAME output that I could not detect in the mp3enc output. This distortion is barely audible as a slight warble in "luminate", and allowed me to distinguish the LAME output from the original in five consecutive blind comparisons. (I was too lazy to do a run of ten.)
This example can clearly illustrate that mp3encdemo31 employs a low pass filter at 128kbps. This filter may have enabled mp3enc to remove audible distortion that LAME did not. Aside from the frequency limitation, the output from mp3enc does sound a bit more natural than the output from LAME, although very detailed listening may be necessary to observe this difference. This trade-off is interesting in that people who prefer preservation of the frequency content over distortion will prefer the output quality of LAME at 128kbps, and vice versa. One conclusion from this observation is that the correct solution is situation dependent. Lower quality speaker systems may even mask one or both disadvantages, due to distortion or frequency limitations.
As a side note, listening to mp3enc output at 160kbps clearly indicates higher frequencies that are not present in mp3enc output at 128kbps, but that are present in the original waveform and in the LAME output.
Also, physical objects placed near the speakers or the listener, and the position of the speakers and the listener, may have a dramatic impact on the perception of higher frequencies, so these variables should remain constant during comparative listening.