I've driven the Altima SE 3.5 and the Camry SE V6, and a couple of small things stood out:
1. The manual mode on the Camry does not allow you to select and hold a gear, but rather simply specifies the maximum gear that the car will use. So moving it to "4" will not necessarily get you 4th gear if the car thinks you should be in 3rd gear, for example. I also found it annoying having to ratchet down to "1" from "4" at every stoplight, since it doesn't reset itself.
2. The leather seats on the Altima are extremely uncomfortable. They are VERY hard, and the leather also has this weird "puckery" look. YMMV on the comfort, though.
It sounds like you've done a lot of homework. Have you had a chance to test drive all three?