Webservers: When did www.domain.net/dir/file.html first not point to $WWWROOT/dir/file.html but to something virtualized?

Question

If I'm not mistaken, the first webserver simply served all files and directories in a certain local directory (WWWROOT) in the identical hierarchy at a certain domain. This is in line how e.g. an FTP server acts. By contrast, nowadays many websites typically use generated pages served at arbitrary, well-designed points at the directory hierarchy below the domain.

When was the first time that the hierarchy presented by the webserver was virtualized like that?

Edit: I'm not talking so much about dynamically created content. CGI is indeed very old, but it used to be an actual files sitting in an actual location below WWWROOT and you would "call" it by adding that relative location to the domain the webserver was serving.

I'm more talking about the kind of URL schemes that e.g. Django talks about here: https://docs.djangoproject.com/en/3.0/topics/http/urls/ : As a part of setting up a website, you think of a URL scheme such that it is easily readable and then the webserver will server the files accordingly. They typically have nothing at all to do with files in a webroot.

As an example, take the URL that is currently displayed in my browser: https://retrocomputing.stackexchange.com/posts/15162/edit Is there somewhere a "15162" directory, containing a "edit" directory, containing an "index.html"? Most probably not. There probably isn't even a "posts" directory and all contents comes from a database, the pages being dynamically created.

That is great and gives you a high degree of flexibility, but that could only come about after someone realized: "Hey, I know www.domain.net/dir/file.html currently always serves $WWWROOT/dir/file.html, but in the end it is just a request to which a program (the webserver) needs to give a response which might, but need to be a verbatim copy of a file on disk." I'm referring to that idea of solving the problem right at the core, the webserver, and not by placeing a CGI script or Java applet somewhere in the directory structure below the webroot and then serving that in a special way.

Answer 1

From Writing Apache Modules with Perl and C (Doug MacEachern; Lincoln Stein, 1998, O'Reilly Media, Inc., 1999):

If you wanted to extend the functionality of the [CERN httpd] web server -- for example, to hook it up to a bibliographic database of scientific papers -- you had to modify the server's source code and recompile.

...

The earliest web API that we know of was built into the Plexus web server, written by Tony Sanders of BSDI. Plexus was a 100 percent pure Perl server that did almost everything that web servers of the time were expected to do. Written entirely in Perl Version 4, Plexus allowed the webmaster to extend the server by adding new source files to be compiled and run on an as-needed basis.

CERN httpd was first released in 1991. Plexus was released in early 1993. So it's reasonable to say that 1991 or 1992 someone did modify the CERN httpd to use virtualized URLs.

Answer 2

I'll have to look up the exact date but it was definitely pre-CGI. Very early on in the development of the web I wrote my own web server in perl that had not root directory. It would take the requested path and do a table lookup to find which subroutine should serve that "tree". I took it very much to heart that the path requested by a browser could be viewed as an abstract token. It may look like a file system path the web server did not have to ever map it to any directory or file in the system at all to serve the request.

This web server was the first at the Computer Science department at the University of British Columbia and quite possibly one of the first in the entire province.

The main problem with this approach was that modules for serving content (whether dynamic or static) could not be shared with other web servers. When the idea of a "Common Gateway Interface" was proposed I became heavily involved in the standardization effort as my web server was effectively a "CGI-only" web server.

Answer 3

In the early days a webserver did not know or care which hostname it served, it just served files (and optionally allowed for running external CGI scripts as the webserver itself couldn't). So, a single web root on disk for all hostnames recognized by the underlying host.

Also the http://xxx/~user syntax allowed for users to host their own pages in their home directory. This was naturally dynamically mapped.

The need for separate web roots came with the ability to support multiple non-identical hosts by a single web server which was added with HTTP/1.1 (which came out in 1999) - see https://en.wikipedia.org/wiki/Virtual_hosting