Simpson's Paradox

Kevin Drum has a post up about CAFE fuel economy standards. It's interesting but I wouldn't have written about it if he hadn't written "I think Bush's proposals would raise fleet averages: if you raise the standards for every category of vehicle, the average for all of them put together almost has to go up". As I wrote in the comments:

"For shame Kevin, you seem to have fallen victim to Simpson's Paradox which (and this is over-simplified) says: one cannot meaningfully average averages. Check out the examples at Wikipedia. They illustrate the fact that intuition (or as W would put it: "your gut") is no substitute for the evidence.

I have been fascinated by Simpson's Paradox ever since a math prof made a statement that just couldn't be true: "Hey Bill, you like baseball... what do you think of the fact that Player A can have a higher batting average against lefties than Player B, Player A can also have a higher batting average against righties than Player B, yet Player B has the higher overall batting average?"

After confirming that I had heard and understood him correctly, I replied confidently that, barring the possibility that there was some category of pitchers other than "lefties & righties", it had to be impossible. A few minutes at the chalk board (yes, it was a while ago :-) and I was wiser and more humble. Thinking is fun...!