11

Old computer systems were supplied—by our present notion—with very little memory, thus conservation of both RAM and storage room has been tremendously important during those years of austerity. Meanwhile a multitask operating system ran numerous processes which often kept the same static data in memory (but separately). Until c. 1992 putting libraries into every executable file was a common practice and a simple Hello-Worldish binary produced on early Unix System V releases may easily take 64 KiB (compared to about 12 KiB for modern OSes); it slowed computing and wasted the room.

In mid-1980s memory mapping was already a known concept, multitasking was understood as the future (even for personal computers), and developers of the day couldn’t be oblivious to a great economy that placing one copy of a common library to RAM (instead of many separate copies) promised. In fact, Microsoft jumped on shared libraries from the beginning of Windows. What hampered introduction of dynamic linkers elsewhere?

10
  • 2
    Especially since the concept was used in Multics at the dawn of the multitasking/virtual memory age ...
    – davidbak
    Mar 27 at 15:58
  • 3
    I think your viewpoint on what was common when depends very much on the systems you were using. Thompson was long unconvinced of the merits of shareable library images. On the other hand, VMS embraced the idea (though V1.0 only permitted one per process, but that was just a temporary limitation). TOPS-10 had "shareable high segments" forever, as far as I know. Mar 27 at 16:54
  • 4
    Not entirely retro-computing. Dynamic vs static linking is an active debate today. Rust and Go prefer static linking. Mar 27 at 18:37
  • 3
    Microsoft was pretty much memory-constrained with Windows 1.x targetting machines << 1MB of RAM - They were practically forced to do something to save on scarce memory. That pressure was probably not so high for UNIX-y OSs that could expect a bit more elbow room (due to their target selling price and higher-end machines to run on).
    – tofro
    Mar 27 at 20:02
  • 4
    I don't have the stats as to what is 'usual', but on the code I work on, it's basically one process per container. Sure, the libraries are still shared objects, but that's just because that's what is available. I cast an envious eye at my Go brethren. It seems to me that how shared libraries work is, you call one routine, you get the kitchen sink in your address space. Maybe in a different forum we could discuss declining standards of modularity :-) Mar 28 at 1:58
19

I have to disagree, to some extent, with the framing of your question. While it is correct that limited RAM in early micros made it a valuable resource to conserve, it is not the case that shared code libraries weren't used to accomplish this. Shared code libraries were very prevalent in the early micros, and were generally embedded as ROM firmware, as a direct solution to the problem of RAM being limited.

If you think of the typical home micro with BASIC in ROM with this framing, it is clear that BASIC is just a shared run-time library. What it does is make possible many sophisticated (if somewhat slow!) applications with very modest RAM requirements. This is true whether the run-time is located in RAM or ROM, but is much more effective if you can ROM it and leave all the system RAM for the app.

Apart from BASIC, many systems included a full set of reusable application libraries in ROM. The original Mac, Atari ST, and Amiga all did this. On the Amiga, the code in the ROM was literally comprised of shared libraries in the modern sense of that term.

If you were to accept that shared code libraries have been a consistent feature from the earliest RAM-constrained micros, then your question sort of morphs into why did it seem to come later to the systems we use today, like Windows and Unix derivatives. That is a totally separate question, and I think it relates more to the unintended side-effects of this approach - colloquially referred to as DLL Hell. That is a problem best solved in the OS, but routinely solved with add-ons that succeed to varying degrees.

In defense of the early implementations that exposed this problem with dependency issues, perhaps, the problem size of doing dependency management well was often underestimated. Also, OS developers likely couldn't have predicted the proliferation of 3rd party shared libraries that became de-facto additions to the OS, but without the corresponding and disciplined revision/testing/release process.

4
  • 4
    Exactly what I expected to read :)) Just one litte remark about Windows. It was, like Amiga-OS, from the start designed to used shared libraries - and alike build from libraries. Basically as neat as AmigaOS, but turned into hell by less than careful programming. In some ways the Amiga was lucky that it had only a tiny number of developers. When compared to Windows that is. And the huge number of developers includes always a considerable number of less than disciplined ones. The same effect killed the upgrade path from DO by use of hardware access and pointer arithmetics.
    – Raffzahn
    Mar 27 at 16:46
  • Brian H: of course Ī̲ am aware of importance of ROM for microcomputer software. It is not a “typical home micro” that demonstrated technical ineptness, but various memory-protected OSes that ran on hardware like Intel 80386 and Motorola 68030. RAM wasn’t as precious at that point as for micros, but it was wasted by dumb software making that OSes not very impressive. Indeed, the problem of wasted computer resources persists, but a byte of RAM and a CPU cycle are immensely cheaper nowadays. Mar 27 at 16:56
  • 2
    @Raffzahn You are right. I think the prevalence of high-quality, 3rd party middle-ware for Windows, rightfully desired by app developers, contributed greatly to the dependency issues. When such things eventually showed up for the Amiga, like the excellent MUI framework, the same sort of issues would follow. Even today, the tight ecosystem of macOS acts as a constraint on this problem.
    – Brian H
    Mar 27 at 16:56
  • 1
    @IncnisMrsi I see that your question is more about the OS ability to enable the shared library technology in a useful and reliable manner. You are right. A lot of it does seem like "ineptness". I'd offer a small defense of the early implementations that they underestimated the problem size of doing dependency management well, and couldn't have predicted the proliferation of 3rd party shared libraries that became de-facto add-ons to their OS but without a corresponding revision/release process.
    – Brian H
    Mar 27 at 17:04
6

On the BESM-6 (a 1960s Soviet mainframe) the most widespread programming environment was dynamically linking by default. Directly loaded application executables were not typically pre-built; the "executing loader" would link the program in memory and jump to it. Overlays were dynamic as well.

For example (the compiler output is edited for brevity; note the loader printout after *EXECUTE)

                                                       27.03.21 M1
 Ф O P T P A H
   /16.07.73/
                PROGRAM MAIN
                PRINT 1
             1  FORMAT(’ MAIN’)
       2        CALL LOADGO(’FOO’)
       3        CALL LOADGO(’BAR’)
                END

                SUBROUTINE FOO
                PRINT 1
             1  FORMAT(’ FOO’)
       2        RETURN
                END

                SUBROUTINE BAR
                PRINT 1
             1  FORMAT(’ BAR’)
       2        RETURN
                END

           *EXECUTE
        MAIN       01000        SWRITT     03150        AHID/*   E 04372
        PROGRAM  E 01000        LUNMUN   C 03153        IOSKIP*  C 04377
        BCDWRIT*   01025        LUN*MON    03154        IOXGIVEM   04400
        FT*621   E 01025        OCTTDEC    03162        GIVEMASK E 04401
        BCDENC*  E 01031        TABWT/*    03214        IOXXRPCK   04430
        FT*611   E 01031        RWTB/*     03221        IOXXUPCK E 04431
        FT*571   E 01033        PRINT8     03304        MON*ITOR   04502
        FT*561   E 01036        PRINT80  E 03304        PLBEG    E 04511
        NEXTLET* E 01043        IOXXTTWT   03342        PLCLO    E 04517
        RK*      E 01060        TTPRINTD E 03346        ASAVE*   E 04525
        WSY*     E 01330        TTPRINT  E 03346        SAVE*HID E 04532
        IOCONT*    01513        TTPRIKS  E 03360        FOR*ALL  E 04547
        IOAC*    E 01545        /IP*     E 03426        OH*      E 04551
        FT*002   E 01560        IOXXFMR*   03442        OH*1     E 04556
        FD*642   E 01561        IOXXFMW* E 03443        ISO/GOST   04565
        FD*722   E 01561        IOXXER/M   04040        TSTATE*  C 04634
        FC*722   E 01561        IOXXER/O E 04041        *ICHECK* C 04635
        FC*642   E 01561        IOXXPKWT   04062        GIVELEXX   04643
        FC*002   E 01561        BCDPUN   E 04063        SAVELEXX E 04745
        IOEND*   E 01566        COLPUNC* E 04064        RD/BT      04765
        FT*003   E 01566        COLPUNB* E 04121        WR/BT    E 04766
        IOSVFR   E 02075        COLPUNE* E 04121        SDEC*      05005
        SUBPERR* E 02107        IBCDCTR* E 04126        *IOXLSW* C 05045
        STOP*      02511        IOXXLPWT   04226        CLEARLEX   05046
        BCDBEG*    02532        ERRIOM     04231          CBOБOД   05060
        FT2*     E 03041        ERRLUN   E 04311
        KONV1*   E 03054        HID/*      04367

 MAIN

        FOO        05070
          CBOБOД   05106

 FOO

        BAR        05070
          CBOБOД   05106

 BAR

What happens immediately after *EXECUTE is akin to what ld.so does after an executable starts. The LOADGO procedure dynamically creates an overlay and jumps to it (a la dlopen+dlsym+subroutine call) with caching, i. e. two consecutive LOADGO calls with the same name will just do the jump, otherwise an analog of dlclose will be done first.

Speaking of memory-mapped shared libraries: address space used to be scarce enough to discourage wastage due to page alignment of libraries, and not all ISAs allowed for efficient position-independent code.

4
  • For the last point: a MMU with logical addresses having at least 24 bits doesn’t make position independence a wish of terrible importance. You can stuff a lot of library code and r/o data to 16 MiB—both frequently used things and not so—and still reserve spacious (by 20th-century standards) compartments for program code and its data. Only parts of the library necessary in each moment of time should be actually present in RAM. Mar 28 at 13:49
  • @IncnisMrsi A related question would be why the concept of memory-mapped files, while invented in the early-to-mid-70s (TOPS-20), only appeared in UNIX-like systems in 1982 or later (SunOS). Speaking of 24-bit addresses, as far as I remember Xenix 286 which came later, it did not have shared libraries nor the mmap() system call.
    – Leo B.
    Mar 28 at 18:45
  • Perhaps features of Intel 80286 were insufficient to catch page faults in a reliable manner. Not a problem for Intel and Motorola CPUs made since 1985, but several more years passed till shared libraries became ubiquitous on UNIX-likes. Mar 28 at 18:51
  • 1
    @IncnisMrsi 286s don’t have pages, but their segmented protected mode is good enough for shared libraries, see OS/2 for example. (Many paging features can be implemented using segmentation on the 286, e.g. copy-on-write, remapping etc.; the main limitation on the 286 is the 64KiB maximum segment length.) Mar 30 at 12:46
4

The question seems to cover multiple areas at once, including

  • Dynamic linking at load time by the OS
  • Runtime linking controlled by the application
  • Shared libraries, in form of system-wide (or per-user) libraries

and

  • Shared code, either as preloaded by OS or
  • Shared code loaded by application when needed, but only loaded once

In any case, all of that was already in use, since the 1970s (*1), in mainframe software. Applications were not only developed in modules and linked to a static binary, but also only partially linked, deferring the final linking to load time. Some OSes did offer link loading as default program start. Here, an executable was not linked at all, but started from a library and linked while loading.

Since the linker was part of the OS runtime, any linking could be done later during program runtime. This included, of course, unloading of modules as well. Underlying this was always some library management system, usually also part of the OS.

In the early days of real memory operating systems there was no difference between shared and not shared, as sharing was just exchanging a memory pointer. With virtual memory this was supplied by the OS. Again, during load and later.

Last but not least were functions where the OS could be made to pre-load certain modules (including whole applications) at startup or by operator command. Quite handy in a multi-user environment with many users using the same applications, like editors, languages or other tools. The main program would be loaded only once at startup. Any user starting 'his' editor, would not load the whole application but only a stub opening local data storage (where the files go) but otherwise run the pre-loaded, shared code.

All of this was developed essentially with the beginning of multi-tasking and multi-user support, as RAM was always too small (*2), putting massive emphasis on space saving.

But the same is true for mainstream micro computers. Sure, OSes started out less sophisticated than mainframes 20 years before, but they soon caught up. This not only includes the mentioned Windows, but also Amiga OS, which was (like Windows) essentially a collection of libraries (*3). Well, and of course various unixoide OSes. Even before that, there was OS/9, offering quite sophisticated ways to share code at runtime. And that's on an 8-bit CPU.

Conclusion: I have a hard time to see the 'not use' of dynamic linking and shared code as implied in the question. The abilities were present all during the 1980s and easy to use. So the assumption about not being offered is wrong.

Looking closely, it seems as if the underlying question is rather why were single-user single-application OSes - like MS-DOS - the main environment used during that time. Right?

It's the classic question why Romans didn't have gas stations: A solution without a need.

Usage in a single-user single-program environment was about one application at a time (*4), which in turn was able to use the whole memory for whatever needed. More often memory was already too small for all functions of that single application, so overlay swapping within itself was needed, not sharing between parallel applications.

Bottom line: Never forget when looking back, that any solution, no matter how helpful it is nowadays, also needs a problem to be solved - often the answer is simply: the problem didn't exist back then.


*1 - Technically even before that, but, like with many other developments, it wasn't until the 1970s that an overall structure was established for generic use.

*2 - In 1970, a machine with 128 KiB was considered large. The largest (civilian) installation in Europe in 1972 was the information system for the 1972 Olympics in Munich, with an unbelievable 2 MiB of online RAM.

*3 - Brian H. may go into more detail.

*4 - Maybe enhanced with a few background helpers.

4
  • 2
    The question is not about MS-DOS (which wouldn’t gain from shared libraries due to necessary complexity of memory management), but about such systems as Unix System V and early Linux (until libc3). Mar 27 at 17:12
  • 2
    "MS-DOS wouldn't gain from shared libraries" - what exactly is the difference between "shared libraries" and IO.SYS and MSDOS.SYS - and any other TSRs ?
    – alephzero
    Mar 27 at 17:48
  • @alephzero: a true library is (if necessary) loaded upon execution of a program. Things that are loaded beforehand are shared (although for IO.SYS and MSDOS.SYS the call mechanism technically differs), but are not libraries. BTW system calls are a thing distinct from simply library calls in modern OSes. Mar 27 at 18:50
  • These days, some systems preload common shared libraries because (a) they assuredly will be needed, and (b) preloading at "preferred" address ranges reduces the need for per-process address fixups. Mar 27 at 21:43
2

Without virtual memory, you basically must have all library code loaded into physical memory at once to use dynamic linking. Systems using processors that supported virtual memory were uncommon before about the mid 1980s. But even if you had virtual memory, it was still problematic. Imagine working on MIT's Multics in 1972. You need to call the sine function. It's only a couple of dozen words, but you have to drag in a four kiloword page to get it. The whole machine only had a couple of hundred pages, shared among dozens of users. All this dynamic stuff was slow. There were big performance advantages to creating a self-contained statically-linked binary that only needed a few pages loaded rather than calling dynamically-linked functions spread over many pages.

You might think of virtual memory and dynamic linking as conserving memory, but they only work well when you already have plenty of memory.

3
  • Can you please clarify… do you claim that losses because of underused memory pages (so-called internal fragmentation) were a major annoyance in systems of 1970s? Mar 30 at 6:22
  • @IncnisMrsi In Multics they were. The biggest bottleneck was page faults. Statically-linked code that minimized fragmentation was a big winner: if you could get your whole working set into primary memory at once, you could pick up the abundant spare CPU time, and you'd keep it all in memory because of high utilization. You could save money that way, too, because the algorithm that charged for system utilization based on a model of how many page faults you'd get on an unloaded machine gave you excessive credit for heavy CPU usage with few page faults.
    – John Doty
    Mar 30 at 13:19
  • @IncnisMrsi The designers of Multics anticipated the problem: the GE 645 actually supported two page sizes, and the very earliest research prototypes used it. Unfortunately, all of the special cases needed in unpageable kernel code to deal with two page sizes made the unpageable code so large that it barely fit in the available memory, leaving very little room for user code.
    – John Doty
    Mar 30 at 13:24
1

With single-user (non-multi-tasking systems) systems, an application could use any memory that hadn't been allocated before it launched, and any such memory that an application didn't use would generally sit idle. While it was possible for applications to open a DOS shell, which could use any memory the applications didn't, it wouldn't be terribly common to have two or more programs in memory that shared a substantial amount of code.

There were a few situations where it was helpful to have some code such as communications "drivers" shared among applications, but under MS-DOS that was accomplished by loading a FOSSIL driver and then loading the applications that would communicate with it. In such scenarios, however, an application that wanted to communicate wouldn't dynamically link to the driver. Instead, such a driver would often set up an interrupt handler, allowing an application to set up some registers and trigger the appropriate interrupt.

Personally, I'm not a big fan of dynamic linking. If a calls to an application's printf are dynamically linked to an OS function by that name, that will make the application sensitive to how the OS function opts to process various constructs. If one had some code that expected the %p format specifier to output a pointer value as 16 hex digits preceded by 0x, and some other code that expected it to output 16 hex characters without a prefix, an implementation which chains to the OS function will only be able to support whichever program is expecting whatever behavior the OS implements. By contrast, if printf were statically linked, one could build each application with an implementations whose library printf function handled the %p function in the required fashion, and run both programs on the same OS.

6
  • Surely if behavior of %p has critical importance for the software, then static linking of printf is feasible (at the cost of many extra KiBs of memory). But Ī̲ don’t expect statically linked functions to conflict with dynamic linker even if names clash. Mar 28 at 10:35
  • 1
    The %p argument (no pun intended) seems weak, Any program that expects particular format was written without regard to the specification of %p. It'd take, what, 2 more lines of code to handle either? Mar 28 at 13:49
  • 2
    @another-dave: I used %p because it's clearly specified as something which different implementations may process in incompatible ways, none of which is unambiguously "best". Perhaps a more interesting question would have been whether using a sprintf format specifier%3.1f to output the value zero in a French locale will output a string as 0.0, suitable for feeding to C compilers or many other programs, or as 0,0 so as to be more recognizable to some people (but fewer programs). Do you think the answer would be improved by using the latter example?
    – supercat
    Mar 28 at 20:04
  • Nit-picking further: the program starts up in the "C" locale and remains there unless it takes specific action to change it. I think your answer would be better without the example. You could make the contrary argument about a shared lib automatically giving the standard behavior for the platform, which a static lib would defeat. Lest it not be clear, I agree that these days predictability general beats saving a few megabytes of (virtual!) memory. I'm just unconvinced by the example. Mar 28 at 21:05
  • @another-dave: I think it would be good to include some kind of concrete example. The question of how printf processes %l is perhaps the most problematical, since there is a fair amount of code which expects long to be the smallest type that's at least 32 bits (which is how it was often treated historically), but there's also code that needed a 64-bit type but opted to use long rather than int64_t or long long. While it should be possible for implementations targeting 64-bit platforms to process code in a way that would allow the size of long long to be specified on...
    – supercat
    Mar 29 at 17:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.