I doubt it's over heating as my room is fairly cold, and it's only in the first few seconds I turn it on.

I'll give the memory stick thing a try next time I turn it off. It seems once I get the thing in windows it's fine, but before then it's fair game to shut down whenever. So it's only during the booting process it shuts down (thus far).