Basically, if you start out with an uncompressed image, saving it as another format will perform a series of operations on the data to achieve maximum quality or compression.

JPG/JPEG is a file format based on storing the averages of regions of the image. It divides the uncompressed image up into blocks and fills the pixels in each block with the average colour of the whole block. This makes it suitable for graphics that have a very large number of colours, like a photograph or screenshot of a 3D video game. It doesn't support transparency, which is the main reason I don't use it. Every time you open a JPEG file, it decompresses and recompresses when you close it. The silly thing about that is that it uses lossy encryption and therefore each time the image is accessed it contains different data. Open/close a JPEG enough times and you'll end up with a blank, one-colour image. You have to do it quite a few times though.

GIF is palette-based. That means a portion of the file is dedicated to holding information about every colour that appears in the image. Not surprisingly, you don't really want to use GIF for anything that contains a lot of colours. If your uncompressed image has too many colours, depending on the quality of GIF it will lose quality in varying degrees: The editor will determine the most common colours in the picture and put them in the palette, then substitute all of the extra colours in the image with the closest match from the palette. GIF is only 8 bits per pixel (2 red, 2 green, 2 blue, 2 alpha) so it has a small file size making it good for use in the internets. It supports transparency and animation as well. GIF is suitable for web graphics and logos, etc. but it's not too bad if you don't need a high quality photo.

PNG is like GIF, only 24 bits per pixel, allowing for three times better quality, but has a larger file size. GIF supports up to 256 colours, while PNG supports up to 281474976710656. PNG lacks support for animation, but MNG, which is an extension doesn't. PNG has a far higher information:byte ratio than JPG thanks to a better compression algorithm.

TIFF is a flexible file format. You can pretty much do what you want with it. You can bung a JPEG into the data section of a TIFF file and it's just a JPEG by another name. You can use it for vector graphics, which means that instead of storing information about the pixels of an image it stores information about the shapes represented in the image. Naturally, that kind of detail is only useful for simple shapes, like logos. The plus side of that is that vector images are scalable in size without any loss in quality. It isn't very widely used though, which can be a bit of a pain.

Bitmaps are just that: a giant matrix of pixels. A bitmap contains information about every single pixel in the image. Usually, it's a 24 or 32 bit bitmap, meaning it has 3-4 bytes per pixel. When you're talking images of size 1000x1000 (1 million pixels), you have 3 million bytes, which is approximately 3MB. The data to file size ratio is not good enough for me personally, to make the tradeoff.