14

TCP/IP has some binary header fields which are affected by byte order, so defines 'network byte order' to settle the issue, specifically defines it as big-endian.

When was this decided? The earliest reference I've found so far is RFC 1700, but that seems to be dated 1994, and I would expect this matter to have been decided much earlier than that.

  • 3
    I'd expect answers to this question to relate the circumstances and "why" around this; YY-MM-DD hh:mm:ss.ddd isn't a very interesting answer. – wizzwizz4 Apr 11 '17 at 8:01
  • 4
    RFC 791 on the internet protocol speaks of byte order. September 1981. tools.ietf.org/html/rfc791#page-39 The earlier RFC 760 does not seem to mention it explicitly though the fact that the high byte of the address (the network part) is sent first was a bit of a hint. – George Phillips Apr 11 '17 at 8:05
12

As far as I can see, RFC 1700 doesn’t define “network byte order” as a phrase; it specifies the order of transmission of bytes (or octets) on the network, as done previously in the RFCs it obsoletes (going at least as far back as RFC 990 in November 1986). As I understand it, “network byte order” is just the order of bytes on the network, which depends on the network and should be documented in the network’s specification — it’s big-endian for IP, as documented in the IP specification, RFC 791, published in September 1981. (The initial TCP paper doesn’t concern itself with such issues.)

Endianness as a byte-order term goes back to Danny Cohen’s On Holy Wars and a Plea for Peace, published as Internet Experiment Note 137 in April 1980 and again in IEEE Computer Magazine in October 1981. That paper summarises the situation, pointing out that many existing communications protocols are little-endian (RS-232, Telex…) but that the ARPANet IMP was mostly big-endian, and that communications on the ARPANet were big-endian.

  • 1
    You are correct in your definition of network byte order. It's not called that in the initial IP RFC, but that's what it is. Therefore, for IP, it was decided some time between the initial descriptions of TCP in 1973/74, and the publication of RFC 791. Interestingly, the IMP specification uses 16-bit words, and mentions DEC machines, which, if I remember correctly, were big-endian. I believe a lot of TCP/IP work was done on DEC machines as well. – Dranon Apr 11 '17 at 14:42
  • @Stephen Your answer leads to the question, why was the ARPANet IMP big-endian? (IOW I don't think we've reached the bottom of the rabbit hole..) – snips-n-snails Apr 12 '17 at 5:48
  • 1
    @traal there’s a fair chance that’s just what the engineers at BBN preferred, or were used to. They developed on PDP-1s, which were 18-bit word machines (so byte endianness doesn’t apply, only bit endianness), but for DDP-516s which were 16-bit machines; I imagine the technical documentation for the latter would help ;-). Interestingly, the latest Annals of the History of Computing has an article on an early encryption device for IMPs, and includes a potted history of ARPANet. – Stephen Kitt Apr 12 '17 at 7:23
  • @StephenKitt I couldn't find the original 1969 version, but I did find a 1976 revision of BBN Report No 1822, which defines the IMP. From my brief skimming of it, there aren't really any endianness issues, though, because of the larger word size. – Dranon Apr 12 '17 at 14:04
6

For TCP/IP, the matter was "decided" the first time an IP packet was sent. The order of bits and bytes put onto a wire (or a radio link) by the sender had to match the order they were processed by the receiver. If the receiver processed them in a different order, chaos ensued.

Since all communication involved at least two parties (a sender and a receiver), the appropriate order was established by convention. If you followed the convention things worked; if you didn't then things didn't work.

Eventually someone saw fit to actually specify the conventional order in an RFC, but before that you could figure out what the correct order was by trying to communicate with another already-working system. I would not be surprised if repeated failures due to incorrect assumptions is what led this to being included in RFC 791, when it had been unstated in RFC 760 and IEN 123.

Keep in mind, it was not typical to write the RFC/IEN first, then build a system afterward. Rather, you built a prototype first, refined it through multiple iterations, and then when you finally had something that worked well enough to share with others, you would document it in an RFC or IEN (although sometimes you'd do so even if it wasn't working "well enough" simply to meet a commitment).

  • You seem to assume that there was no networking before the ARPANET. This is not the case. It's likely that the developers of ARPANET followed the same conventions of whatever underlying physical network(s) they were using. – JeremyP Apr 12 '17 at 9:26
  • I was not assuming that; if you can point out where I implied it I will edit in a clarification. I doubt the underlying physical networks would have imposed any restrictions on byte order. They undoubtedly would have had conventions regarding bit order but that doesn't matter as long as you get the same bytes out the other end. For example, RS-232 UARTS impose a bit transmission order (LSB first) but no byte order -- bytes come out the other end in the same order you sent them. Likewise, it is easy to bridge between Ethernet (LSB first) and Token-Ring (MSB first) without any byte swapping. – Ken Gober Apr 12 '17 at 13:33
  • Talking about the RFC that defines the Internet Protocol kind of implies that. Of course the underlying network does not have to influence the byte order of IP (because IP is just a blovk to it) but the engineers who made the first version of ARPANET might have been influenced by what they were working on top of. – JeremyP Apr 12 '17 at 13:53
  • Ah, since the question asked specifically about TCP/IP network byte order, I thought it implied that my answer applies to IP only. I'll edit in a clarification. – Ken Gober Apr 12 '17 at 15:25
  • +1 for noting that RFCs have usually codified/refined something that has at least a proven prototype implementation. At least, the good RFCs have. – Wayne Conrad Apr 12 '17 at 22:45
3

Network packets are formed once but read during transmission multiple times. Therefore it is worth optimizing reading of multi-byte values. Reading a big-endian unsigned number consisting of N bytes (where N bytes is no more than the native word size) is

cnt = N;
val = 0;
while (cnt--) val = (val << 8) | getbyte();

whereas reading a little-endian number requires a more complex code, something like

val = 0;
for (cnt = 0; cnt < N; ++cnt) val |= getbyte() << cnt*8;

Therefore reading a value transmitted as a big-endian byte sequence is more efficient.

  • I don't see how this answers the question. – Chenmunka Apr 11 '17 at 8:46
  • @Chenmunka This relates to the "why" around the question, see the wizzwizz4's comment. Given the optimization criteria, there was no need to "decide" anything by fiat, the format follows naturally from the requirements. – Leo B. Apr 11 '17 at 8:50
  • That assumes you’re reading from the network one byte at a time; it’s more efficient to read multiple bytes at a time if you can. I’m not convinced by the “follows naturally” argument; there are little-endian communications protocols (some older than IP). – Stephen Kitt Apr 11 '17 at 11:25
  • 4
    I'm not convinced this has anything to d with it. I think it's more likely that whoever designed the protocol that popularised network endian was using a big endian machine. – JeremyP Apr 11 '17 at 13:08
  • 1
    Note that this is not how it is implemented e.g. in Posix: You always read the complete packet, and use htonl etc. to convert from network order to host order or vice versa. And I seriously doubt there was ever any hardware where you really read packets one byte at a time. – dirkt Apr 11 '17 at 15:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.