It was long assumed by FS drivers that all diskette and harddisk units operate in blocks of size 512 Bytes. How did this happen to be the case, historically? Why not the "neater" 1 KiB, say?
Sectoring on disks predates floppies. It was in widespread use on hard disks by the time floppies came around. Also, I would note that everything from 1 byte/word granularity to track-sized sectors has been argued for (and sometimes used) over the years on both floppies and hard disks.
Still, why did it settle largely on 512? First, it's just as neat as 1024. It's the previous power of 2. And it was picked for the same reason why memory sizes are a power of two; it's a nice round number for a binary computer.
Beyond that, as to why 512 instead of 128, 256 or 1024 or whatever, my impression is that it was a question of trade-offs:
Space efficiency on disk. Small sectors waste space for headers and spacing. Large sectors waste space to store a few bytes in a whole sector. 512 bytes turns out to be a reasonable compromise for a couple hundred files on a half megabyte of media.
Use of memory vs. speed. 1 KB was a lot of memory on systems in the late 70s and into the 1980s. Open a few files, add a tiny bit of caching so the system doesn't crawl, and you're using all your RAM just for buffers. On the other hand, if the sector size is too small, there is increased overhead, more seeking on disk, and it gets too slow.
512 byte sector became the de facto standard because they were the size of sector used by the IBM PC. Prior to the PC's dominance, other sector sizes were also common - notably, CP/M (which was the most popular microcomputer operating system prior to the release of the IBM PC) used 256 byte sectors.