It's not legal to download ROMs under any circumstances, since the practice of serving them over the internet constitutes piracy. The only time that it's legal for ROMs to be downloaded is if the copyright holder has released them into the public domain (which basically will never happen with commercial console games) or if the copyright holder has expressly allowed permission for the ROM to be distributed. Making copies of games you already own is a legally grey area but it's never legal to distribute them.

Authorities are far more concerned with shutting down those distributing them rather than those downloading them though, so in reality it's a question of your own morality.