There are a couple I can think of.

Shin Megami Tensei: Strange Journey - Most MegaTen games have a classic learning curve for RPGs, by which I mean the game starts off hard as hell but eventually gets pretty easy towards the end. Not this game, it just gets harder. The fair challenge of the game is the twisted dungeon design with warp points, hidden paths, rooms of total darkness and of course the high encounter rate. The real bitch are the bosses of whom many of them are luck based missions after a certain point in the game. This is the real issue, as many of them get overpowered special moves (and unlike previous entries, they lose them when you acquire the boss as a minion) and combined with other gameplay features, can get you killed pretty quickly with little in the means of countering. Normally the series is pretty fair. Not this game.

Demon's Souls - Trial and error gameplay, serious item management that will result in grinding if you get too careless, dying has more bite than later entries in the franchise, regular enemies that can often put up as much of a fight as some of the bosses, and some sadistic level design where half your deaths are dodging off a cliff to avoid a powerful enemy attack. Demon's Souls is one of the nastier entries in the Souls/Bloodborne franchise and the only one where dying in the tutorial is mandatory. The saving grace of the game is that most of the bosses are pretty easy, of course the ones that are not...

Zelda II: The Adventures of Link - A bad translation that makes solving many of the puzzles more difficult than they needed to, actual puzzles that require some weird game logic to even solve, enemies that drain XP, enemies with absurd stats and hyper-competent A.I., brutal bosses that all feel like they were designed to easily counter whatever Link can throw at them, a Lives system that automatically sends you back to the beginning of the game whenever you run out, and the Valley of Death which is filled with Castlevania/Mega Man style goddamn bats whose sole reason to be there is to knock your ass into instant death pits. I can see why many fans don't like it.

Final Fantasy III (Famicom) - High encounter rates, damage algorithms with RNG, battles against eight enemies at a time, limited save points, item management, some jobs are absolutely worthless, brutal enemies with a number of tactics at their disposal to make certain dungeons an uphill battle, brutal boss battles, vague instructions on where to go next, and the nastiest series of endgame dungeons in the series. Playing this game almost made me disbelieve this was made by the same people who gave us FFVIII and X.