The first game I played with 'Final Fantasy' in the title was The Final Fantasy Legend. The first numbered entry I played was either FFI or FFIV. I'm pretty sure I gave FFI a try first but never beat it since it was very hard.

1. FFI
2. FFIV
3. FFVI
After that I played them in order of release in the US (other than XI), which I believe is:
4. FFVII
5. FFVIII
6. FFV (Anthology)
7. FFIX
8. FFII (Origins)
9. FFX
10. FFXII
11. FFIII (DS)
12. Somewhere in 2006 I started FFXI, no clue if it was before or after XII or III.
13. FFXIII
14. FFXV

For the Tactics games and the sequels, I also played them whenever they came out. Except for FFXIII:LR or FFVIIC, never bothered with those.