It's not the summoners and it's not the Temples - but you're still on the right track. It's not a person/creature or a specific object.