Next time it happens try setting the controller down, nudging the analog stick just a teeeeensy bit and see if you can get it to stop, and then if it does, just don't touch the controller at all and see if it happens again.

If that works, then you'll just have to bump it back into place every now and then or just buy a new controller P: