39

Originally, the FTP protocol connected back from the server to a client to actually transfer files through that new connections. 14 years later after the introduction of the FTP, the 'passive mode' was added to it, so that only client connects to server ever, however a need for multiple connections remained.

Compare that to HTTP or SSH protocols, where many things could be done through a single connection.

The NFS protocol (until the late NFSv4) has a rather chatty connection process, where the client first connects to the 'portmapper' (at a fixed port 111) to only get a port of the 'mountd' daemon, then connects to that daemon only to inform the server of a mount a client is about to do. After that, client returns to a 'portmapper' to get regular NFS port (2049), and proceed there with regular operations. The same kind of chatter happens when client wishes to use additional services like 'rquotad', 'lockd' or 'statd' -- each at its own port.

The funnier thing is that all but 'portmapper' (111) and regular NFS (2049) ports are also dynamic.

Then, in NFSv4 everything at last collapsed back into a single port (2049).

So my questions follows:

What were the design rationale behind not putting the whole protocol into a single connection, at a time those protocols were designed? What benefits emerged specifically from doing server-to-client back-connections in FTP?

Overall, what benefits got the designers planning the protocols in such peculiar ways?

  • 5
    (No time for an answer) Most important is a) not every (server) system could handle multiple connections from multiple clientes to a single port, thus b) dynamic port assignment, which in turn was as well meant to c) prevent the hassles of default ports, and would d) allow third party connection management. e) call back provides security against spoofing and in dial up settings and last but not least f) modularisation of software - _ So, there are quite some good reasons._ – Raffzahn May 13 at 21:58
  • 3
    At the time these protocols were designed all computers were fully on the net. NAT hiding first really came into play when home users only got one or two ip-numbers assigned. – Thorbjørn Ravn Andersen May 14 at 11:42
  • 1
  • 3
    Re: Originally, the FTP protocol connected back from the server to a client to actually transfer files through that new connections. 14 years later after the introduction of the FTP, the 'passive mode' was added to it, so that only client connects to server ever, however a need for multiple connections remained. Separate control and data connections were introduced in RFC354 (July 8, 1972), passive mode was introduced in RFC542 (August 12, 1973). – ninjalj May 14 at 22:24
  • 2
    @Raffzahn I know it's not up to your usual high standard, but just typing that into the answer box would've been sufficient. The only reason I haven't deleted your comment is that it still says stuff not covered by any of the answers. If you could convert this to an answer, that would be very helpful. (Answers as comments bypass our quality control measures, etc., so they're much worse than interesting tangents in comments.) – wizzwizz4 May 16 at 10:58
54

At least for FTP, the actual file transfer happened over a different connection to support a particular file transfer mode that isn't used much today. Suppose you have three machines, A, B, C, and you want to transfer a file from machine A to machine B. You are logged in to an FTP client on machine C. With FTP you can do the following:

        ┌──────────┐               ┌──────────┐
        │          │    Bulk data  │          │
        │ Server A ├──────────────▶│ Server B │
        │          │               │          │
        └──────────┘               └──────────┘
             ▲                           ▲
             │                           │
    Control  │                           │  Control
             │                           │
             │       ┌────────────┐      │
             │       │            │      │
             └───────┤  Client C  ├──────┘
                     │            │
                     └────────────┘

That means you can, from C, log in to FTP on both machine A and machine B, and with the right combination of PORT commands, you can set up a transfer from A to B where the actual file data does not go through machine C. This helped if the bandwidth between A and B was much greater than the bandwidth available to C.

| improve this answer | |
  • 11
    And this is explicitly mentioned in RFC 959 (and probably others). Another factor in the FTP design is that the control connection uses the Telnet protocol. – Stephen Kitt May 13 at 22:00
  • 3
    @StephenKitt: Thanks for the reference! I should have copied the diagram from there instead of drawing a new one. :) – Greg Hewgill May 13 at 22:01
  • 2
    Yes I actually implemented this once in the '90s when it was requested by users. One thing is that C never finds out whether the transfer completed or encountered a problem, from memory. – hippietrail May 14 at 3:35
  • 15
    @Nobody, not if you only have FTP access :) – ilkkachu May 14 at 9:48
  • 1
    @StephenKitt [ edit, I think it was control/data ] In ARPAnet, links were unidirectional, so it would naturally have been "one control" (from client to server") and one "data" (from the server to the client). tools.ietf.org/html/rfc33 page 3 – Vatine May 14 at 13:19
21

For FTP, I think there are two factors here:

  1. The absence of multiplexing multiple data streams over a single transport connection, and

  2. The server-to-client connection for data transfer.

The benefit of a dedicated transport connection for data transfer is that you can implement stream mode: the sender pours data bytes (and only data bytes) down the pipe, and the receiver empties the pipe into the file. End of stream is end of file. Simple.

The downside of having a connection per data stream is... not much. You're still using the same wire bandwidth. There's a few more bytes of kernel memory in use at each end to track the extra TCP connection.

The server-to-client connection only proved to be problematic later on in TCP life, when we had acquired NAT and firewalls.

In short, the original FTP design looks to me like they went for protocol simplicity, and I'd guess that's good for efficiency and for interoperability, when the OS world was much more heterogeneous.


With respect to portmapper, that's really covering up for a lack of a session layer in the usual TCP/IP stack. To connect to a remote program, you need to know its transport address, i.e., the combination of IP address and TCP (or UDP) port number. That in turn means either fixed assignments, or complex manual configuration, or someone to ask which arbitrary number is in use on that node at this time.

The portmapper takes the last-mentioned approach. Server gets arbitrary port assigned, registers it with the port mapper. Note that this was not specific to NFS, but rather it was a general RPC mechanism, RPCs being all the rage at the time.

As an aside, DECnet avoided this by having a thin session layer: a listening server got an arbitrary NSP (transport layer) port. The client never dealt with that port numer; it connected using a session layer identifier (either a small number, or a text string - the latter of course needs no central authority to adjudicate) which the system used to find the running server, or else start a new program. So you got both portmapper and inetd function built in, pretty much invisibly to both client and server (though some server-side config might be needed for the 'inetd-like' use case).

| improve this answer | |
  • 2
    FTP is old enough that it predates IP, the first FTP RFC that I can find is tools.ietf.org/html/rfc354 (July 1972), and the first IP RFC I can find is tools.ietf.org/html/rfc791 (Sep 1981). Eyeballing RFC 354, much f it is recognisable. – Vatine May 14 at 9:13
  • 6
    As HTTP eventually learned, there are downsides of having a connection per data stream. You have to go through the TCP handshake and 'slow start' for every transfer, just for one thing. That's fine when you're transferring a few large files from A to B, very inefficient when you need to transfer a few thousand tiny ones. (Probably the main reason rsync feels so fast.) – user1686 May 14 at 9:31
  • 1
    @Vatine Appendix III of RFC 959 has a nice list of all the FTP-related RFCs up to that point, starting with 114 in 1971. – Stephen Kitt May 14 at 11:36
  • 4
    As someone who writes communications code, the multiplexing issue makes the most sense to me. Separate socket connections are one way, possibly the most network-efficient way, to avoid problems with "in-band signaling." – snips-n-snails May 14 at 23:23
  • 1
    I think you meant "[...] or else start a new program." instead of "mew", the Pokemon. – Ismael Miguel May 15 at 14:46
18

(I originally posted this as an answer to the similar question "why we need two connections between the ftp server and the ftp client" at StackOverflow)


The decision to have separate control and data connections in FTP was taken at the Data and File Transfer Workshop at MIT on April 14-15, 1972.

RFC310 "Another Look At Data And File Transfer Protocols" was published on April 3 to prepare for the workshop. Some relevant information from that RFC:

  • The CPYNET protocol used on TENEX systems closed the control connection and opened a new one with possibly different byte-size. The selection of byte-size could be important for some computers, e.g: the 36-bit PDP-10.
  • Ad-hoc protocols on top of TELNET where the receiving process had to inspect every byte were considered slow. Using separate connections was suggested to avoid that overhead.
  • In the Data Transfer Protocol (equivalent to modern-day data connections in FTP), block mode was considered too costly just to provide control/data separation and EOF indication. Again, opening/closing a separate data connection was suggested as an alternative (which would also allow selection of an appropriate byte-size).
  • For FTP usefulness, efficiency was considered important, and again separate connections with perhaps different byte-size were suggested, noting the ambiguity that closing a connection could be either due to an EOF indication or an error.
  • For use in TIPs/IMPs (Terminal Interface Message Processors), some of which had no file system, and had devices listening on specific sockets, it was considered convenient to allow sending data to a specified socket.

RFC327: "Data and File Transfer Workshop Notes", published on April 27, briefly summarizes the discussions and decisions taken in the workshop. Speed and efficiency of file transfer were considered important, with byte-size and data format conversions being considered some of the most important factors affecting speed and efficiency. Finally, it was decided to use separate control and data connections. Other decisions were taken: the control connection would be a TELNET connection, the control connection would use ASCII human-readable commands and responses, and DTP (the Data Tranfer Protocol) would stop existing as a separate entity, and become the protocol used on the data connection of FTP.

Finally, RFC354: "The File Transfer Protocol", published on July 8, 1972, became the first incarnation of the FTP RFC to feature separate control and data connections. It used a SOCK command, instead of our familiar PORT and PASV commands.

Addendum

Inter-server file transfer (AKA FTP bounce/FXP) appeared on RFC542 "File Transfer Protocol for the ARPA Network", published on August 12, 1973, with the introduction of the PASV command.

Finally, RFC765 "file Tranfer Protocol", published on June 1980, was modified to use TCP instead of NCP, changing the SOCK command for the PORT command.

| improve this answer | |
15

For NFS (and portmapper), it's down to it being implemented as a SunRPC service. The whole idea is that you have your services randomly scattered on non-reserved ports, with the portmapper knowing where they are (by them registering on startup).

This allows you to have only one fixed port, gets around the problem of needing to coordinate with a global body to get your static port assigned, and things like that.

It has the drawback that you need to go via the portmapper for every "first connection" from a client, incurring at least a few round-trips delay before something useful can be done.

But it was also designed to work in a "LAN-type" environment, where the extra few roundtrips weren't that long. And with slower network speeds, to boot (10 Mbps would've been normal, 100 Mbps a luxury), so the extra time taken by the portmapper roundtrip is essentially lost in the noise.

| improve this answer | |
  • 4
    It also got you a rather primitive form of service discovery (you could call out to the protmapper on a given system to see what RPC services it's running without having to use any of them). Which is part of the other reason that such a design has fallen out of favor, because it makes life easier for bad actors who get onto the network. – Austin Hemmelgarn May 14 at 13:34
  • 1
    SunRPC is intended for LAN type environments but which has machines with different endianess but still thru the definition of the protocols (mount in NFS for example) being able to isolate the application programs from endianess and word size issues. – Stefan Skoglund Oct 11 at 13:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.