Could it be some "accelerator" software that your ISP uses? I know that some of those "accelerator" programs cache everything that they can when you first open a page so that they load faster the next time through. So maybe your pages are taking a whlie to load the first time through as everything gets cached?