Rather than include the four year-old Xbox processor in the Xbox 360, they used their expertise to develop a virtual environment for their Xbox, going so far as to license Nvidia's tech to get the games to work on AMD's new Xenos GPU. Of course, not every game would make the cut, only a fraction, but it was enough for most people, who really just wanted to boot up Halo 2 and call it a day.
In reality, while I whipped out Project Gotham Racing 2 or Jet Set Radio Future from time to time, I wouldn't have died if Xbox compatibility were out of the picture. It appears that Sony's taken that note and sung with it because the PlayStation 4 won't feature any PlayStation 3 compatibility at all. It's not even in the cards. One advantage of PC gaming is that no matter how old the game, people manage to get it to work (something Cody and I discussed on the recent N5 Show) while each new box is a gamble. That isn't to say that developers won't re-release old games, which is becoming more and more common, particularly with downloadable titles.