Transition from static to dynamic shared libraries

Question

A static library, also known as an archive, is one intended to be statically linked. Originally, only static libraries existed. Static linking must be performed when any modules are recompiled. (From the Wikipedia article on libraries (computing).)

When did operating systems transition from static libraries to dynamic libraries? Particularly interested in Unix and MS-DOS / Windows.

When using a GCC front-end with the -lfoo switch, libfoo.so or libfoo.a are searched for. If only one of these are in the search path it will select that, but if it finds both in the same search directory, it will prefer libfoo.so. Was this always the case, or was there a time when libfoo.a would have been the default if both were present?

Most modern operating systems can have shared library files of the same format as the executable files. This offers two main advantages: first, it requires making only one loader for both of them, rather than two (having the single loader is considered well worth its added complexity). Secondly, it allows the executables also to be used as shared libraries, if they have a symbol table. Typical combined executable and shared library formats are ELF and Mach-O (both in Unix) and PE (Windows). (From the Wikipedia article on libraries (computing).)

Was the change from static libraries to combined executable and shared libraries for Unix and Windows always like this, or was there a time when shared libraries were distinct from executables?

Answer 1

In the Unix world, dynamic libraries first appeared in SunOS, on top of its virtual memory infrastructure (which allows libraries to be shared in memory, and demand-paged). See Shared Libraries in SunOS for details. This was generalised in System V Release 4.

Windows has had dynamic linking ever since version 1.0, even though early versions didn’t ship any obvious DLLs (those appeared in 3.0) — the core operating system was effectively dynamic libraries disguised as executables, and device drivers (with a .DRV extension) were DLLs. Early versions of Windows avoided any paging requirement by loading everything in a shared memory space, with no protection. OS/2 also supported DLLs from the beginning.

MS-DOS has supported overlays since version 2.0, through interrupt 0x21 service 0x4B03, but that doesn’t implement dynamic linking — the overlays were relocated, but there was no provision for symbol resolution between the loaded program and its overlay. Some compilers or runtime environments provided their own support for dynamic linking. DOS/4G is one example, see How 3Dfx Voodoo 1 Emulator works for previous discussion; HX is another (with support for Win32 PE DLLs).

All the above used the same file format for dynamic libraries and executables, from the beginning.

Earlier operating systems supporting dynamic linking include OS/360 and Multics.

(I don’t know about your GCC question, I’ll update this later.)

Answer 2

Historically, the linking and sharedness of libraries are orthogonal. Shared libraries reduce the needed memory size. Dynamic linking makes it possible to update and relocate the library code without recompiling the application.

Modern systems only have two cases:

• Static non-shared libraries: When you link with a static library, the entire code is included in the application executable and is not shared with other applications. Though on many systems, the code will be shared between multiple processes of the same application.
• Dynamic shared libraries: Dynamic libraries share the storage on disc and also the memory at runtime between several processes. Dynamic linking fixes the call addresses to point to the actual location of library in the address space.

Historically there has been all combinations:

• Most early reusable libraries were distributed as source code. Compiled directly to the executable, this was equivalent to a static, non-shared library.
• Common routines included in ROM on early home computers were a form of statically linked shared libraries. Only one process was running at a time, but sharing the common routines reduced required storage space. If the ROM was updated, it was done carefully so that important link addresses remained unchanged.
• First Machintosh System software used a common dynamic loader for both code and graphical resources. It could swap out code segments that were not in use. Because there was no virtual memory addressing on the machine, address lookup for jumps between code segments was performed at runtime. There was only a single application running at a time, so this was dynamically linked non-shared library, though in some cases storage could be shared if multiple applications were on the same floppy.
• Dynamic shared libraries seem to have first appeared in Multics in late 1960s. It would seem logical that UNIX would have inherited this capability, but I haven't been able to verify whether there were any early UNIX versions without shared libraries.

With regards to Windows, it has always included the support for dynamic shared libraries, already since Windows 1.0. Early Windows programs usually only made use of the system-provided shared libraries and statically linked all other code.

You asked also about the difference between binary formats for executables and libraries. As far as I can tell, all systems that have supported shared libraries have supported a closely related format for executables. Many systems also support legacy executable formats, like .COM on Windows that dates back to MS-DOS.