Setting aside the file system (other file systems besides FAT were possible on MS-DOS using redirector technology or via the shell approach (ie NetWare)), 8.3 was still a requirement due to the fact that a number of INT 21h functions used the FCB (file control block). (The FCB has hardcoded locations for filename and for the extension.) In particular functions 0Fh through 17h all used FCB(s). (See Ralf Brown's interrupt listRalf Brown's interrupt list.) So a program using FCB functions would not be able to specify a 9 character filename or a 4 character extension.
Additionally the Program Segment Prefix contained two FCBs, so any program examining PSP would need be able to interpret a non 8.3 filename.