I wasn't saying that The Answer was non-canon, I'm saying that the fact that the games are restricted to only showing a single route for the story is not indicative of only that route being canon. No, the games do not take into account everything you do, or every decision you make. That is not ATLUS saying that your story was invalid, that your experiences didn't really happen in the game's story. That's simply ATLUS working within the technical limitations of gaming.
That's the problem. Canon can only follow 1 path by it's very definition.

As of right now, we don't know the exact canon path for Persona 3 or 4. However, the male MC and The Answer are confirmed canon. These two things have been confirmed, and cannot be argued. The true endings for both games have also been confirmed as canon.

Other things such as how you did in school, which social links you completed, and who you dated, are left ambiguous on purpose(although Yukari as the main love interest is heavily implied in The Answer).