Yeah, physical strength isn't the prerequisite for lifting the hammer. Worthiness is. I actually rather enjoyed that scene, because it tells us that there could be more than one 'worthy' individuals at any one time.