Some recent poking around inside Apple's third-generation TV unit show that there's actually a dual-core A5 processor inside, and not the single-core CPU listed on the spec sheet. It's possible Apple's just reusing iPad 2 parts that have a bad core, and simply disabling it. Further investigation shows that the chip was actually made using a 32 nm process, compared to the standard 45 nm found elsewhere in Apple products.
The last teardown we had seen of the new Apple TV revealed extra RAM to handle 1080p video playback. The third-gen Apple TV also introduced a new user interface, though apparently the venerable Steve Jobs wasn't a huge fan.
It's entirely possible that this new manufacturing process will actually be squeezed into the iPhone 4S to save space and improve battery performance, perhaps with an Apple A5X variant we heard about previously?
UPDATE: According to some digging around in a restore image, there are already plenty of iPad 2 models out there using the same processor.