For those of us who grew up with personal computers in the 1970's or 1980's, the experience was a very gradual migration from floppy disks to hard disks as the primary persistent storage. So, based on this experience, it is a bit counter-intuitive that the invention of the hard disk preceded the floppy disk by many years.
Other than the rather subjective "It was just an accident of history" answer, are there any objective technical reasons why floppies would have come after hard disks?
While Stephen Kitt's answer already hits the core, I believe it needs a bit more history, as direct access magnetic storage did start quite a while before the IBM 350.
First there were drums. Drums were huge cylinders with a magnetic surface, and a separate head for each track - something that would have been rather impossible with a disk. Drums are somewhat located at the the borderline between main memory and external storage. Except that the idea of external storage wasn't even formulated back then the way we see it today (*1). The drum was the main memory of early computers and the CPU was built around its timing. There was no memory to hold even a single track in direct access (*2).
The IBM 650 Magnetic Drum Data-Processing Machine of 1953 is a good example. It was built around a drum with up to 80 tracks holding 50 words of 10 decimal digits each, so lets just make that 5 Bytes, giving ~20 KiB in today's units.
Drums were in use way until the 1970s and gained sized up to 200 and more heads. while also enhancing data density per track, resulting in drives with several megabytes. On a drum, heads were neither flying nor movable. Each track had it's own head. Thus there was no seek time, no head settling, at maximum waiting for one turn to pass was needed. Which at 10,000 RPM (and more) wasn't much.
It wasn't fun to adjust heads on a drum. A little too low and the drum was dead (head crash); a little too high, and there was no usable signal. This is something that carried over to disk (see later).
While Univac added bigger (and additional) drums in 1956, IBM chose the lower-cost approach of a disk. Here only a small number of heads were needed and less electronics to handle a way bigger amount of data at lower cost. On the negative side, disks were incredibly slow compared to drums.
When IBM then added the 355 disk unit (a variant of the mentioned 350) to the 650, a controller was also needed (653) which housed the first core memory to store (part of) a track, so access within a single track could be sped up.
Unlike often assumed, the IBM 305 RAMAC wasn't so much a disk based system. The CPU was still based around a drum. The 350 disk just added a layer of storage; faster than tape, slower than drum. Due to the (almost) standard disk, the 305 could operate with a smaller and cheaper drum, while handling data sizes similar or even bigger than the 650, albeit slower.
From here on IBM walked the disk 'track'. While drums as main memory were replaced by core, drums as secondary storage vanished rather quickly, but not everywhere and not completely. In fact, DEC offered their faster PDPs (like the 11/45) with a drum as fast paging memory (*3) in addition to disks. To some degree drum plus disk was what hybrid disks (HD + Flash Cache) are today.
Early Disks used basically the same technology as drums with heads at fixed height. Part of the inherent adjustment problem was that they couldn't really go close to the surface and thus density was quite limited which again resulted in even lower speed. In comparison, the 650's drum did rotate at 12,000 rpm with about 2000 bits in one track (50 words á 10 decimal digits), that multiplies to roughly 400kbit/s transfer rate. The 355 disk rotated at 1,200 rpm (less than a 10th of the speed); data transfer rate was 8,800 characters per second - with 6 bits/character that's about 52 kbit/s, roughly an 8th of what the drum did. In addition there were only 3 head mounts (350 had two) with two heads each, all covering the same 100 surfaces. Track to track time (one track) was almost 100ms, surface to surface could reach 2s and above. So yes, faster than handpicking a punch card, but in no way even near how a drum performed.
But boy were they cheap.
In the end a way more capable technology was replaced by a cheaper, lower performing technology for fast storage. What followed was much like later small hard disks: a race for lower cost small drives, but also for more storage at the same cost. Just 5 years later the 353 reached a capacity of 16 MiB and a transfer rate of 8 Mbit/s. Much of this was possible due now self flying heads allowing a more than 10-fold storage density. Even access time was way down as every disk side got a dedicated head.
Next step were Storage Module Device disks - a series of disks modeled after the IBM 1316 disk pack, a 6 deep stack of (now) 14 inch disks. Exchangeable in less than two minutes and whopping 2 MiB per stack. By 1964 the capacity had grown to 7 MiB and many companies provided compatible devices and for at least until 1980 they became standard - sizes increased up and beyond 300 MiB per stack.
Exchangeable disk stacks were standard. Hard disks fell completely out of use (while drums were still made and used). It wasn't until 1973 that IBM changed the design with what was called the Winchester drive. And no, it wasn't meant as a fixed built-in in hard disk like we use today. It was still an exchangeable media, except that now the head mount (and heads) were part of the disk pack. The idea was to keep the disks away from dust by being completely encapsulated. Electronics, drive motor and head motor were still part of the drive, not of the disk pack. The main goal was again a higher density, not so much for faster transfer (but they didn't mind) but to make it cheaper. Capacity of these 'Data Packs' was 30 to 70 MiB.
It took a few more years until the improved drives were made into a non removable setup with a tenfold increase in storage (>300 MiB). While SMD drives did compete size-wise this first generation of 'new' hard disks were faster - but users were not really convinced. With 'traditional' drives they could store a multitude of data on a few drives by exchanging packs, while not a drive had a fixed size. Large, but fixed, and no way for easy backup and restore by just putting in a disk pack (*4).
IBM developed (*5) the floppy as a way to replace punch cards at data entry stations. Here cost was the main concern, as punch cards were cheap, proven and equipment was existing. Outside this process, there was no real need for a diskette like media. For program and data exchange tapes were perfect; they were fast and held large amounts of data with small reels being handy to transport. A floppy wasn't smaller, but stored way less data and required new drives.
It was the Minicomputer of the mid 1970s that made the floppy a basic media. While up to 1 MiB storage was negligible for mainframes, it was huge for minis. And while it was incredibly slow in mainframe terms (card readers were faster), it was an incredible gain for minis. And way cheaper compared to disks.
For the upcoming micros during the second half of the 1970s, the floppy was heaven sent. And that's why they started out with floppies - and mainframes didn't.
And here history loops: With minis and micros growning in demand, it suddenly became feasible to develop small (in size), small (in cost), small (in capacity) hard drives again. 1979 the first 8" HD with 'just' 10..60 MiB and in 1980 Seagate with the 5 MiB 5,25" ST506.
*1 - The terminology classic IBM systems use do reflect this until today, as disk is still seen not as a separate entity, but another level of system memory. If at all, tape does qualify as such.
*2 - The needed part of a track was sent directly into the corresponding registers as required by the instruction processed.
*3 - Guess why BSD's swap is traditionally on /dev/drum :))
*4 - A common backup scheme was to copy the whole data disks onto a new disk pack by reorganizeing the database, then store the old and continue work with the (now defragmented) new ones. In case of a deadly disk crash, the system could start from the old disk within minutes.
*5 - Developed is the key word here. While the original design was for a cheap way to load microcode for the Merlin, it was data entry that transformed it into a general purpose storage method.
Floppy disks are harder to get right than hard disks. Early hard disks were enormous; the IBM 350 used fifty 24-inch platters. They were also rather fragile and cumbersome (heavy and power-hungry). To go from a hard disk of this sort to floppy disks required a number of developments: in particular the ability to remove the platters (which came in 1962 with the IBM 1311), then reductions in sizes of all components while still providing useful storage capabilities: heads had to get smaller, actuators etc. had to get smaller, medium reliability with smaller quantities of material had to improve, storage density had to increase, and of course costs had to come down for all this to be accessible to mere mortals.
This matches the evolutions seen in general in magnetic-disk-based storage: from early fridge-sized hard drives to 2.5” drives, and on the floppy side, 8” disks in huge drives with large PSUs, then 5.25” disks in full-height drives, then 5.25” disks in half-height drives, then 3.5” disks in still smaller drives... Many removable drives fit into this continuum (SyQuest cartriges, Zip and Jaz, Compact-Flash-sized micro-drives...).
Some issues are specific to floppy disks. One well-known example is that hard disks have a head which isn’t in contact with the platters, whereas floppy drives’ heads are kept in contact with the magnetic medium, which leads to rather different constraints. Another example which was pointed out to me recently is an issue which came up during development of the IBM 33FD and its floppy disks: IBM discovered that the floppy enclosures warped, and eventually determined that using black enclosures was the only way to avoid this (at the time).
It’s worth noting that magnetic-disk-based storage was expensive for a long, long time, and other forms of storage were favoured for a long time — paper initially, then magnetic-tape-based (the latter both in large computers and micros). Early attempts in the IBM project which led to the invention of the floppy disk tried to use tape rather than disks (see Wikipedia’s page on the topic).
Floppy disks solve a different problem from hard disks: They're cheap, and they're portable. A floppy disk (even an eight-inch floppy) slips into a brief case or a file folder in a way that reels of half-inch magtape and punched card decks just will not do; and if somebody asks you for a copy of that report you're working on, you can hand them a floppy disk without being too worried about whether you'll ever get your media back.
Hard disks existed before anybody ever imagined that "that report you're working on" would be a file that they could open on a computer.
Expanding on the "cheap and portable" comment about floppies, an IBM paper on their history [1] gives a nice summary of the problem they were initially intended to solve:
The introduction of semiconductor memory created a need for a low-cost device to load programs and information. The requirement was for a read-only device and, at minimum, a serial tape-like medium would have been acceptable. It was recognized, however, that direct access to a section of data and the ability to read a record repetitively would be useful in some applications. [...] Other characteristics that were requested included a medium that could be replaced without tools within fifteen seconds and that was small and light-weight to permit inexpensive shipment.
So at least in the eyes of their inventors, floppies served a completely different purpose from hard disks (to the extent that IBM called them "diskettes" rather than "disks"). Interestingly they considered a variety of existing media, including dictation tapes, audio cassettes, and even 45rpm vinyl records, before deciding that something new was required.
I believe the first application for the floppy within IBM was distributing microcode (firmware, in modern terms) for the 2835 and 3830 disk storage controllers. Another paper [2] describes how their designers used the medium:
[...] a miniature microprogram load, read-only disk drive, using a small interchangeable plastic disk, was selected. Data are entered on disks at the factory, providing very good control over the data. The disk is mailable and disposable, so that microprograms in the field can be updated easily and efficiently. (pp. 14-15)
I think the comment about writing at the factory and reading in the field is important - it made floppies cheap enough to throw away, and the drives that read them could be installed in large number of shipped devices while only needing a few expensive writers at the factory (similar to the early days of CD-ROM, perhaps?). This also helps explain why floppies found their way into the price-sensitive home computer market well before hard drives did.
[1] James T. Engh, The IBM Diskette and Diskette Drive. IBM Journal of Research and Development, v25n5 (Sep. 1981), pp. 701-710
[2] G. R. Ahearn et al., Design Innovations of the IBM 3830 and 2835 Storage Control Units. IBM Journal of Research and Development, v16n1 (Jan. 1972), pp. 11-18
The first hard disk drives were huge. Thick platters were on the order of a meter in diameter. The drive enclosures were larger than a kitchen refrigerator. The magnetic domains (the bits) on the hard disk were large enough to see without a microscope.
It took awhile to miniaturize all that stuff, electronics, magnetics, including the R/W heads, get the bit density up, and figure out a magnetic substrate thin enough to use for something like a floppy disk that would hold a reasonable amount of bits of data.
But, just like Moore's law driving inventions that led to higher transistor densities, every couple years or so there was a roughly double improvement in HD magnetic bit density.
Cost. For most hobbyists, in the 1980s, hard drives were out of reach, until eventually the cost dropped to affordable levels.
I had a VIC-20 and C64 with cassette tapes, an Amiga 500 with twin floppies and later an Amiga 500plus which I bought a 40Mb hard disk and RAM expansion for, because it cost less than what I paid for the Amiga itself.
Fundamentally, they're both just magnetic storage media. Most of the challenges behind making platter-based media is common to both designs. You need to have engineered precisely moving heads, tracks/arms to move them, motors with very precise/steady rotation, etc.
The big distinction is that the "reader" is integrated with the media, in the case of the hard drive. This lets you have rigid fixtures that exactly ensure alignment of the reader relative to the media and a clean atmosphere free of potentially damaging dust.
Floppy disks lower cost and increase portability by off-boarding the reading part, and making the media removable. This adds a new level of complexity, because you don't have as tightly controlled of an environment as with hard disks. Now you need a mechanism for accepting and ejecting the media, some way of ensure alignment between these loosely coupled parts (literally, physically), so that your drive head reads the correct data.
Short answer - economics. There was a big gap to be filled in. Affordability and portability of the hard disks were the issues that floppy disks addressed. https://en.wikipedia.org/wiki/Floppy_disk
