I actually had this same problem when I tried to learn Russian. I could speak it just fine, but when it came to reading and writing I was completely lost. Eventually I just gave up, because it wasn't all that important to me.

My friends who take Japanese classes tell me it helps to actually use the language, so reading Japanese books/manga and writing journals in Japanese would be a good start.