But it would be so annoying to quickly pass the controller back and forth between two people.

Or can it be played on two seperate controllers?