Answering this for all kinds of connections is simply way too broad. But there are three basic facts:
- Connected devices have specific functions.
- Connecting devices of the same function is an exception.
- Straight cables are less effort/lower cost.
RS232 as the mother of all (modern) serial makes a good example. Typical uses are communication setup, or peripheral devices. For example on remote lines:
(Picture taken from Wikipedia Article about Data terminal equipment)
By wiring DTE and DCE accordingly a simple straight cable can be used. No need for any specially wired cable.
The same is true for peripherals, like connecting a computer to
- a printer,
- a tape reader,
- a tape punch,
- a keyboard
or any other of countless kinds.
In addition, when it cones to communication interfaces it is not just about switching RX and TX: there are, depending on the protocol used, up to 20 lines that have to be wired accordingly. Many in a way specific to the device.
By offloading this task to each device, having it internally wired as needed, again a simple straight cable can be used.
The only case were a crossover cable is needed is when two devices of the same type, usually DTE, are to be connected without a communication line in between - hence the name Null-Modem - as the cable nullifies the insertion of a modem line.
While this may seem nowadays almost a default task - like when experimenting with some single-board machine - it was the extreme exception when these interfaces were standardized. And it stayed that for most of the time since then.
Equally importantly, null-modem cables often do not just cross TX/RX lines, but as well shortcut some additional signals to emulate modem response. This makes them even more special to the situation and unfit to be used in any other setup.
The same works with LAN cabling. The moment we went to point to point connections, instead of a bus (yellow cable), straight cables became the simplest solution. Lowest priced, least chance of false application. It's always device to switch (or hub) and so on. Again the pitfalls of cabling get internalised and cost reduced.
The usage of 'twisted' cable is again restricted to very special situations, like coupling two PCs or patch connections in a data center.
In the end, it's been the same even since telephone days - here also cables were straight, connecting between two distinct sides. The unusual case of connecting two sets without an exchange in between did require a crossover cable.
And that's the basic issue: with an exchange in between, all changing happens there, so the cable can be simple and straight.