Actually, you only have to buy a Stable if you're planning to breed Chocobos. You can get the Chocobo Lure Materia and go catch a Chocobo and ride it around all you like, but when you dismount, it will take off.

There's a farm near the Mythril Mines when you leave the Midgar area. If you go there, you can buy greens and Chocobo Lure Materia. You can ask the people there about how you go about catching a chocobo.

Did you really cross the swamp without a Chocobo? The Midgar Zolom is really tough, so that's quite a shock. Must've taken a lot of leveling up.