Hm, "elf ears" are obviously just an invention by Peter Jackson.

The only real difference between Elves and Men are the facts that Elves are immortal, got better sight and hearing, and are lighter and fairer.
Period.