Not necessarily. Random encounters scale to the average level of your active party with a minimum variable dependent on the map. Tynemouth Hill, for example, has a minimum level of 2.

Story encounters, too, can scale to the average level of your active party depending on your actions.