I never said that the controls don't work well. I in fact mentioned that they were among the best (if not the best) touch controls in any game I have ever seen.
That said, I also pointed out that they do almost nothing that the buttons couldn't do equally well (we know this, because buttons control the top screen character, and offer essentially identical functionality). I also note that, though the controls work well, the swipes and presses can be tiring over long periods of time. A lot of people will say that this is mitigated by the game being on a portable/mobile device, and thus designed for relatively shorter, pick-up-and-play sessions, but it is still a valid point.
My vendetta against touch controls is merely that I have seen woefully few games execute them well. The lack of feedback and precision hurts them almost universally. The World Ends With You is the best game I have played which uses touch controls. However, while it's controls are solid, it has done nothing to me that indicates that touch controls have any real use that buttons do not, which means that even it does not do enough to justify their existence as a primary control scheme, in my mind.
Every control media has something that benefits it over the others. The mouse and keyboard gets the absolute pinnacle of analog precision. Buttons offer phenomenal ease of use, provide excellent physical feedback, and are great for binary precision. Analog sticks offer a vast range of input commands in both direction and magnitude.
Touch controls strip away the best features of all the other controls schemes, and offer an array of their own problems. The stylus (or your fingers, depending on the device) block visibility of the screen (some more than others). Lack of resistance and feedback can make touch controls overly sensitive and strip away precision. Any attempt at analog movement (such as the circle pad in Final Fantasy Dimensions) is crippled by not having anything to stop your finger's movement.
If touch controls want to be accepted as a valid core input system, they need to do something better than the others. Currently, the absolute best touch control setup I've seen has managed to draw even to buttons, but has done nothing to surpass them. And the other touch control games I've seen fall woefully short of even that mark.