I don't really believe that desert = astronomy and mathematics. The Twin Floods weren't always such a complete desert - and what about the astronomers of India, China, Mezoamerica? The Arabs were at the top of the game of mats in medieval times, but I'm sure there were reasons other than "desert-dwelling". For example their calendar, which is based on watching the moon - and which requires some mathematical prowess - could have given them the motive. And much of their stuff was also borrowed, given down by the Greeks and Indians.