There's like... Golf Story? And I recall there being a couple of racing RPGs back in the day. So it's feasible. But I do find it odd how we as a community use action-adventure as a catch-all, but the companies use RPG as a catch-all. I've noticed it myself

Similar to how originally there were debates on what games should be called fighting games or beat'em up games. Since in both instances you are fighting or beating up people. Even hack'n'slash got debated in the same sphere

It probably doesn't help that action-adventure games have recently been in bed with RPG mechanics. I remember Saints Row showing up in RPG lists on stores like XBL initially because of this. And people would probably happily argue that Assassin's Creed games may well fit into this category now. Zelda and Dark Souls aren't the only ambiguous titles anymore