Here, you should check out this article. Under the microscope: Samsung Galaxy S4 vs. HTC One

More importantly, you should head down to the store and try both of them out. Mess around with them, hold them both in your hand to see how they feel and how you like them.

I've been a fan of HTC since highschool. I had the G1 in highschool (first phone with android operating system) and i've had an HTC Sensation for a few years now. I prefer the One over the S4 too, not only for aluminum body and the speakers, but browsing through the menus and phone feels so much smoother. I didn't really like the no SD card at first (like you said) but its really not that big of a deal, there are tons of other options to transfer stuff to your computer.

Like I said, i'd go try them both out and see for yourself, but I like the One better. As long as you stay away from Apple you should be fine though