No one has mentioned Superman 64 yet?

Maybe not the worst game I've played, but I'll give a mention to Super Mario Sunshine. Mario games pride themselves on precise controls, and controlling that water backpack was wonky at best. Some of the platforming difficulty was due to the awkwardness of the controls rather and that's just poor design.