The trick is to make yourself known in the Garden. Meaning, the CC group must have heard of your reputation before it challenges you, 'cause they only challenge worthy opponents. So, you've got to play your way through the Garden to gain a reputation.
About 15-20 games with random people should do the trick.

My suggestion is to try and get as many cards up to Level 6 (all if possible, except for the first one of Level 5, (SPOILER)the PuPu Card which you can't get until the PuPu quest.). You'll get Level 7 and higher by playing the CC group. Whenever you beat one suit, try and get as many of his/her/their cards as possible, as they'll be highly useful in the following suits.

A good source of cards, as well as an easy player is the guy sitting on a bench outside the library.

Just a hint: (SPOILER)There's a member of the CC group, no-one will tell you about. He's the Joker. It's the guy in the Training Center, from whom you can buy items. But if you challenge him to a card game, he'll tell you that he's the Joker. He's at the screen with the 'platform' wearing a green jacket and a yellow cap. If I remember correctly, you can get the Leviathan card from him.

Good luck!