When it say's non-system disk, it means it doesn't have anything to boot from. If you just formatted the HD, obviously it can't boot from that. If there's no floppy or CD in, then there's nothing to boot from at all, and so you get that message. Buying a new HD won't help, since a new HD will have no system on it to boot either.
If it's not booting from floppy/CD when you do have one in, check that they're actually a bootable floppy/CD (the Win98 install CD should be bootable anyway), and as Baloki said, check that the computer is set to try booting off of either of them too in the BIOS. With a computer that old, you might also have to explicitly tell it to try booting from CD before trying the HD.