Most musicians and anyone infact would probably relate colours and music (chords, keys, notes etc) because of the 'feel' or 'texture'. I'm currently doing some composition work on seasons and because of this I am relating chords and colours. So for summer I would use 'warm' notes, major keys, which to me are red, yellow, orange etc, then winter 'cold', blue, minor, diminished, etc.
I think things like this are greatly influenced by memory and the subliminal mind....mebbe :)