A lot of 70s era microprocessors were packaged in DIP packages with 40 pins. This was a reasonably good fit for 8-bit processors: 16 address lines, 8 data lines, 2 power and clock are all absolutely essential, which leaves 13 lines for control signals, which is about what a processor needs. However, other, larger processors also sometimes used this form factor, including processors with 16-bit buses and larger address spaces, like the 8086, along with the General Instruments CP-1600, and National's PACE. In order to achieve this, the designers resorted to cumbersome schemes that multiplexed multiple signals onto single pins, making the chips harder to incorporate into designs (e.g. requiring an external latch to track the address while data is supplied on the same pins). Why did they do this, when they could realistically have used larger packages (e.g. some chips have been packaged in DIP-48 and DIP-56, which would have allowed plenty of room for all functions without multiplexing)?
Reasoning about the usage of existing packages
Adding a few hundred transistors for multiplexing is approximately free compared to buying production machinery for several millions - way before the first chip can be made.
Creating a new chip (family) is for sure a risky bet on the future and takes some investment. Keeping this investment down to a minimum reduces the risk taken.
Ordering, building, integration and ramp up of new production equipment is a quite large investment. Requiring this for a new chip might endanger the whole project, as management might not be inclined to spend that money on some fancy and uncertain new stuff.
So calling for new, larger packages not strictly needed to make a new chip would be a rather stupid move for engineers. Especially not if one can come up with schemes like multiplexing to make it work with the existing production environment.
History for larger packages
The driver for DIP 40 package weren't CPUs, but rather memories, as their increasing pincount made 300 mil packages (TTL like DIP) impossible, so the switch for 600 mil was needed. Similar for I/O chips. With the need for devices like 8155 or 6820 as whole families (and the proven success of early CPUs) new machinery for DIP 40 could be justified.
Multiplexing as an advantage
Also, when looking at a whole family like the 8080, multiplexing isn't negative at all. It becomes an advantage. A 8155/56 offers three 8-bit ports and a 256 byte RAM (and a timer) in one single DIP40 package. With non-multiplexed lines, only two ports would have been possible. Similar the 8755 offered 2 KiB EPROM (ROM in the 8355) in addition to two ports. With these chips it was possible to build typical (small, embedded) systems without any demultiplexing at all.
Connecting 8085+8156+8755 was a quite complete setup with 2 KiB EPROM, 256 bytes RAM, 41 bitwise I/O lines and a timer. The only external components needed is a crystal, a diode, one capacitor and three resistors. That's it.
Similarly, the 8088 (and 8086) could enjoy the same benefits due using the same multiplexed bus structure.
So on the whole, multiplexing is an advantage, and only comes with additional cost if one wants to build a maxed out system — which would be a pile of circuitry anyway — so a few demultiplexers won't hurt either.
Your question assumes that CPUs with more than 40 pins were a rarity in the 1970s, but this was common for early 16-bit CPUs. Both the TI TMS9900 and the Motorola 68000 had 16-bit external data busses, no multiplexing, and came in a 64-pin DIP package.
The thing that was common at this time was DIP packaging. And the number of pins on a DIP package was limited by simple geometry - a rectangular package needs a progressively longer lead from the corner pins to the semiconductor chip as the number of pins increases. This geometry constraint limited DIPs to 64-pins, practically. This limitation also helped drive the transition from rectangular DIP to the various square packaging that followed.
The Intel 8086 came in a 40-pin DIP and relied on multiplexing for its 16-bit data bus (shared with address lines). The WDC 65C816 used the same 40-pin DIP and multiplexing scheme. Neither of these 16-bit CPUs were particularly successful in these packages. The 65C816 was repackaged for the SNES as the CPU core in a QFP package. The Intel 80286 was the more successful 16-bit successor to the DIP 8086 (and 8088) in 1982, and by that time square packaging was replacing DIPs.
In short, the market did not want a 16-bit CPU in a 40-pin DIP package with external demux. Fortunately, DIP soon gave way to newer packaging geometries that supported far higher pin counts.
Coincidentally, I found this explanation reading through Microprocessor Interfacing Techniques 3rd Ed 1979 by Zaks and Lesea page 16:
The Standard Microprocessor System
Throughout this book, reference will be made to a “standard microprocessor.” The “standard” microprocessor today is the 8-bit microprocessor. Examples are the Intel 8080, 8085, the Zilog Z-80, the Motorola 8600, the Signetics 2650, etc. In view of the pin number limitation on DIPs (dual-in-line packages), the 8-bit microprocessor has become the norm. The reason is simple:
The Number of pins is limited to 40 (or 42) by economic considerations. Industrial testers required to test components having more than 40 pins are either not available, or would be extremely expensive. All standard testers will accept only up to 40 or 42 pins. In addition, naturally, the cost of the package itself increases rapidly over 40 pins.
Lead lengths inside a 64 pin package slowly get problematic, even at 1970s speeds. Even a digital system working at a few MHz requires you to pay attention to things like routing signal lines close to their return (or a ground plane), and keeping power bypass capacitors connected to the die with short leads. There is a good reason modern CPU packages started including some of the bypass capacitors when external signalling speeds went above ca. 50MHz (which is only ten times as much as a 5MHz 1970s CPU - and the modern package is still smaller than a 64 pin DIP. Including capacitors in a DIP package had only been done commonly with the expensive ceramic DIP packages in the 1970s. Being cavalier with these things is what probably accounts for many of the "gremlins" you tend to get in solderless-breadboard projects. The awkward electrical interfaces (asymmetric impedances, differences in rise time, sometimes low noise margins) you got when pairing NMOS and bipolar TTL technologies on a card did not exactly help.
Also, a LOT of industrial automation style equipment (and quite a selection of commercial computers) used the "Eurocard" form factor (which prescribed a PCB size of 10x16 centimetres). A 64 pin DIP, of course, does fit on such a card, but signal routing gets unwieldy.
Economical PCB manufacturing and design capabilities might also have played a big role in staying with DIP instead of just going to finer-pitch or quad-row packages - the DIP package was invented LATER than a predecessor of the SOIC package (the "flat pack" package, which was surface mountable and had 1.27mm / 20th of an inch lead spacing and can be made to match a SOIC footprint if you re-dress the leads).
Just to remind what a big deal physical tooling and wiring was...
... the Mostek 6502 chip intentionally had the same electrical pinout as the Motorola 6800, despite completely different design, instruction sets, and even endian-ness.
With 40-pin sockets, they were a commodity, not terribly more expensive than the same-width 24- or 28-pin sockets. (when 28-pin first came out, they were pricier than 40s, a problem we solved with a mitre box and a razor saw). However the larger (and wider) sockets were expensive enough for cost accountants to sit up and notice - they certainly cost more than a few latches (and their sockets).
As chip package sizes get ever smaller, like the Microchip Tiny84A, in its 4mm QFM package, I just see it as a strategy to get more raw circuitry, and more functionality at the cost of multiplexing the pins or vitualizing by IO pin mapping in software. There is a great article here: