It just that Final Fantasy went to fun mini-games like a whole entire theme park, to fantasy themed underwater football to..... fishing.
It just doesn't really put the Fantasy in Final Fantasy. It's not everyday you can go to a theme park, or breathe long enough underwater to play a game in a pool. So they feel like a rare and special experience with a element of fantasy in them. Fishing is just well, just that.