I wear gold and silver together a lot. I have a silver Pandora bracelet which I often layer up with other gold things... SHOOT ME!
Not but seriously, I don't think it's all that bad really, to wear them together.