TL;DR:
The computer, or rather its OS, has no idea about a directory, in fact not even what constructs a file. It can only open/read/write/close data streams from devices on the bus. Getting a directory is a clever hack of 'loading' a dummy program. This works since
- LOAD command is implemented as opening a data stream and loading what is returned into memory at BASIC start.
- LOAD does not perform any check on the program.
- LIST command does not perform any checks on syntax or line number sequence.
- LIST simply outputs what is in BASIC memory by stupidly transforming what is found
As result, a directory listing is quite literally a LISTing (*1).
As so often, Michael Steil did sit down and wrote a short analysis to directory listing on Commodore machines.
Details
What I'd like to know is how these pseudo-BASIC directory listings are produced. In particular, is the disk drive itself generating a BASIC listing from the file system and passing it to the computer?
Exactly.
The disk drive creates a pseudo-BASIC-program when the C64 reads from a file named "$".
Or is the main computer querying basic file system information from the drive and then constructing the BASIC listing itself?
No, the computer, or better the OS, has in fact no idea about a floppy at all. It can access data by using open/read calls send to a device. What is received back will be interpreted as a (program) file.
On the drives side the special file name of "$" is interpreted as command to read the directory(s) (*2) and send the data read back within the framing of "program code":
- Two bytes (offsetted) string length
- Two bytes number of blocks
- A string containing file name and alike (the quotation marks are part of that string).
- A NULL byte at the end
This gets repeated until
- Three NULL in a row (like end of line and next pointer being both NULL) to terminate it.
Also, is the listing really represented in the computer's memory as a (non-functional) BASIC program with line numbers,
Yes.
In fact, what is delivered only conforms to the rules of a BASIC program as far as the LIST instruction is considered.
or is it stored in a dedicated format that the LIST command detects and specially parses?
Again, the computer has no idea about things like directories, not even what floppy is, only devices that respond to open/read/write/close instructions on the expansion bus.
It is a rather clever hack to add the functionality of a directory listing to a machine that has no higher level functions beyond initiating and acting upon data streams on the (serial) IEC bus.
It wasn't until after the C64 that Commodore BASIC got a bit more sophisticated by adding disk commands; except they were for most parts just syntactical sugar on top of existing commands/workings.
If the directory is stored as a BASIC program, how is it that two lines can start with the same line number (as is the case when two files have the same size)?
Not only that, as shown, 'line' numbers even come in non sequential form. This is because the LIST command does not do any syntax or sequence checks. It simply starts at $801 and interprets all bytes as BASIC code. No syntax or sequence checks at all. So whatever bytes are found at the position of a line number is turned from their 16 bit value into readable (ASCII) decimal.
The same goes for outputting all 'program' text. here every byte will taken, compared against the token table and outputted as such, or if not a token, simply as its own value - which opens quite some tricks to add confusion - unless it's in quotation mode, but that's another story.
It may be worth to add a 'secret' trick to read a directory (hopefully) without destroying an existing BASIC program. Using a secondary address of '1' like LOAD "$0",8,1 will load the directory into the screen buffer. The data is stored exactly the way the drive delivers it, thus the display will look quite weird, due the binary nature, but at least file names are pretty clear visible (*3).
This works as the disk controllers create the pseudo program like for the original PET - including the program start address. While the PET had it's BASIC buffer starting at $401, the C64 uses this area as default screen buffer, having BASIC programs starting at $801. Normally, without a secondary address, the pseudo program gets 'relocated' to $801. With ,1 added it will be loaded to the address told by the 'file', which now happens to be the screen bufffer.
A 'tool' that comes very handy when keeping multiple versions of a program on a disk but having forgot what number the last saved version had (*4).
*1 - Wouldn't be surprised if that naming similarity made way to that idea :)
*2 - On classic dual drives (4040, etc.) using "$" alone will produce the content of both drives - unless qualified for a single drive ("$0" or "$1").
*3 - Since it's a byte for byte representation, tricks used to pretty up a directory entry, hide parts or (simple) protection schemes are visible right away.
*4 - Saving to 'Pn+1' worked great as GIT before GIT
