Try some of these:

The Tea Party - The Edges of Twilight, Transmission, Triptych
The Mars Volta - Deloused in the Comatorium, Frances The Mute
A Perfect Circle - Mer De Noms, 13th Step
Tool - Aenima, Lateralus, 10,000 days

I bolded my favourites from each band, though all of those albums are worth listening to, and so different from each other for each band that even if you didn't like one you may like the others.