Recent videos from Famitsu:

YouTube - Persona 3 PSP Video 1

YouTube - Persona 3 PSP Video 2

We can see the female side will have new music, as well as new social links (Ken and Junpei revealed in the video so far). Also outside of Tartarus is a point-and-click type of deal kind of like Phantasy Star Portable, to help fit the game inside the little UMD I presume. The load times also seem much faster but I don't know if that's because they were playing on a PSP Go or not.

Edit: Oh yeah, and Elizabeth is replaced by a yet-unamed male version for the female side, called "Malizabeth" by fans for now.