You can also right-click on My Computer, select Properties from the menu.
On the pop-up window, click the Automatic Updates tab.

You can turn off automatic updates there and they will stay off, even after you restart.

However, if you turn them off, you'll want to manually keep your computer up to date by visiting http://windowsupdate.microsoft.com/ regularly.