Well honestly, it's because most of the people who made all those awesome games in the 90s left Squenix in the early to mid 00s. That's why a lot of the franchises started to slump and sag. I agree that making a direct sequel began the fall cause it pretty much proved to Squenix that fans would buy a lousy sequel for the sake of nostalgia and fanfare. So we've got most of the talent leaving, the rest being utilized poorly, and Squenix learns they can easily make a buck by slapping the FF moniker on a title and make way more money than the game is probably worth. It's pretty much Disney post-90s Renaissance but before Pixar started to make them change around a bit.

I mean their best selling franchise is a giant fan-fiction crossover story that 97% of its fanbase bought either for the FF cameos or for the Disney nostalgia. KH's success is largely due to profiting on the merits of bygone eras. Yes, it has evolved into something that can stand on its own legs but Squenix's marketing of the series makes it pretty much impenetrable to all but a few who have the money and time to play every nuance of the series.

The most profitable business move Squenix has done in the past decade was buying Eidos of all things. FF has pretty much been diluted into a shadow of its former self, no longer really embodying the bold creativity of its early days. What was once a series helmed by a bunch of creative guys just doing what they felt was cool and to make the best damn game ever is now been transformed into a franchise that must be created for maximum profitability.

X-2 and the Compilation were large milestones in the fall, but I feel the FFXIII trilogy is pretty much the next major change. I guarantee that FFXV will probably only have a 1/3rd of it's plot explained with plenty of plot holes and loose ends just so Squenix can churn out sequels to tell the "whole story". Expect each new numbered entry to be transformed into its own mini-franchise within the large FF franchise. You'll probably never see another standalone title for a very long time.