I have a third-party memory card for all my PS1 games. It has 8 'sections' of 30 blocks and you cycle through the sections by pressing a little button on the top of the card. It's not organised at all though. When I go back to play some old games, I always end up spending 10 minutes cycling through and trying to load from each section.

I've only ever needed one PS2 memory card. It's just the standard black Sony one.

My PSP came with a 2GB Memory Stick, but it's not enough for all my games, so I use that one for my SNES, GBA and NES emulators and associated games. The 8GB SanDisk memory stick serves as my usual memory stick with all my PSP and PSX games.