You can wait for Charm to be illegal before you fight Ultima. Then it can't really do anything. :P