50

From the the two methods of encoding 8-bit data as human-readable ASCII, for a time, uuencode format was more popular. USENET 'binaries' groups were filled with uuencoded posts with whatever goodies were shared. The format was quite robust, insensitive to line breaks (if your mail program reflowed the text, for uuencode you could still decode the file) and the uuencode/uudecode programs were quite user-friendly.

Base64 was not nearly so well liked. Some people would post base64-encoded binaries, arousing mild ire from these, who didn't have decoders. It was sensitive to formatting and white spaces. I'm not entirely sure, but I think it generated a little bigger output too.

Then I was off-the-loop for a time, and when I came back to the Unix and Linux world, uuencode was dead, and wherever 7-bit was still needed, Base64 ruled, and rules to this day.

What happened? What events led to base64 winning the format war?

  • 3
    uuencode IIRC had leading/trailing whitespace as being meaningful, which could cause issues AFAIK. – Muzer Jun 1 '17 at 14:59
  • 1
    uuencode was a de-facto standard that never made it into any RFC. Base64 was a real standard that made it into the earliest MIME RFCs and ended up in a dedicated (RFC3548, RFC3648) one. – tofro Jun 1 '17 at 17:35
  • 1
    @tofro somehow I think if the MIME designers had wanted UUencode, they’d have added it to the RFC ;-). – Stephen Kitt Jun 1 '17 at 18:32
  • 4
    Actually, reflowed uuencode, although theoretically recoverable, was not handled well by the tools. Conversely, the standard Base64 decoding algorithm doesn't care about spaces or line breaks at all. – Leo B. Jun 1 '17 at 23:22
  • 1
    Base64 is not sensitive to whitespace formatting at all as whitespace is specified to have a very specific meaning in base64: it means ignore this byte and read the next byte. – slebetman Jun 3 '17 at 21:13
36

I’m not sure about specific events, but I think the main reason Base64 “won” is that it’s one of the binary encodings supported by MIME, and MIME took over.

So perhaps the question then becomes two-fold:

  • Why did MIME pick Base64 over UUencode? Possibly because Base64 is actually more resistant than UUencode: it only uses alphanumeric characters plus two other characters to encode content (‘+’ and ‘/’ in MIME), and one character for padding (‘=’).
  • Why did MIME become the dominant mail/news content wrapper? I guess it boils down to convenience, especially once most MUAs and news agents supported it (ah, the days of slrn and Forte Agent...).
  • I'm actually curious why the "URL-safe" version isn't just the version, or at least why / was chosen instead of a safer non-path character. – Nick T Jun 1 '17 at 22:41
  • 4
    @Nick MIME was invented in 1990 or 1991, and codified in 1992; URLs were invented in 1992 and codified in 1994. So URL-safety didn’t exist when MIME was invented. (That doesn’t answer the non-path aspect of course since file system paths already used /.) – Stephen Kitt Jun 1 '17 at 22:47
  • 2
    @RossRidge SMTP is a line oriented text based protocol. Even if the whole mail system was 8-bit clean, you'd still need to encode binary files. For example, if a mail server sees the sequence 0x0d 0x0a 0x0d 0x0a 0x2e (a blank line followed by a .) it "knows" that is the end of the message being sent. Such a sequence embedded in a binary file will cause the message to be terminated at that point. – JeremyP Jun 5 '17 at 9:02
  • 2
    @JeremyP The whole mail system is 8-bit clean. You don't need to encode a text message that isn't plain 7-bit ASCII text with the horribly inefficient base64 encoding or even the quoted-printable anymore. The only limitation on the body STMP messages is the 1000 character line length limit. The "dot" problem you mentioned is a non-issue. It's solved by the same dot-stuffing algorithm that lets you send a plain text message with a single period on a line. In practice binary files still need to be encoded (because of poor support for RFC 3030's BINARYMIME), but not most text messages. – Ross Ridge Jun 5 '17 at 17:29
  • 2
    @RossRidge SMTP is a line oriented text protocol. Mail servers are at liberty to insert line feeds if they wish to shorten lines. All "8 bit clean" means is that eight bit characters will not lose the top bit when being transferred, it does not mean that you can safely transmit a binary file without some encoding. – JeremyP Jun 6 '17 at 8:58
20

The problem with uuencode is that the format was not robust in the face of some of the really crufty mail software and gateways into and out of proprietary non-SMTP and non-ASCII mail systems of the day. Just to liven things up further, there were multiple EBCDIC variants which had different code points for some ASCII characters used by uuencode, opening up another route for data corruption. For example, the character $ has code point 74 in code page 285 used in the UK, but code point 91 in code page 037 used in the USA.

This corruption would have been one of the driving forces behind the design of MIME, and its character set would have been carefully chosen to minimise problems with such gateways.

  • 1
    Perhaps you could explain why the variation between character assignments in various EBCDIC code pages is more of a problem for uuencode than for base64? – John Bollinger Jun 1 '17 at 21:22
  • 2
    @JohnBollinger: Adding up the letters a-z and A-Z, along with the digits 0-9, yields 62 characters. Thus, base64 only requires two more characters to get to 64, and finding two characters that behave consistently on all systems isn't too hard. – supercat Jun 1 '17 at 21:26
  • 2
    @JohnBollinger, UUEncode uses the upper-case letters, the numbers, and a wide range of punctuation. Base64 uses the upper- and lower-case letters, the numbers, and two punctuation characters. Looking at a selection of EBCDIC Latin-1 code pages, the letters and numbers have consistent code points, but the punctuation code points vary wildly. – Mark Jun 1 '17 at 22:30
  • 4
    @JohnBollinger You are misunderstanding the problem. I uuencode a binary into ASCII text containing a dollar sign. I send that to some gateway that understands it is getting ASCII and converts to EBDIC. The gateway then forwards to another EBCDIC machine that fails to handle the code-page conversion (so the dollar has been mangled to something else). Finally the second EBCDIC machine converts back to ASCII and sends the text to you. You try to uudecode and it all goes horribly wrong. – Martin Bonner Jun 2 '17 at 11:29
  • 4
    @JohnBollinger Base64 is not "predicated on the ASCII mappings", except vacuously in the fact that the characters exist at all (vs uuencode which uses the low six bits of a set of 64 consecutive ASCII characters), and the earliest formal description of, RFC 2045, does explicitly mention EBCDIC. And, in fact, "This subset has the important property that it is represented identically in all versions of ISO 646, including US-ASCII, and all characters in the subset are also represented identically in all versions of EBCDIC. " – Random832 Jun 2 '17 at 13:57
12

Base64 is slightly more compact as it does not use a character indicating line length at the beginning of each line:

% dd bs=1k count=1024 < /dev/urandom | uuencode /dev/stdout | wc -с
1444736
% dd bs=1k count=1024 < /dev/urandom | uuencode -m /dev/stdout | wc -c
1421440

Overall, Base64 is about 1.5% better.

  • 1
    this is not an answer. this is a comment – Jasen Feb 24 at 22:35
5

Some of the reasons base64 was disliked was because uuencode stored the original file name and file mode of the encoded data. Also, uuencode had been around longer and was more established, which meant that many people had a uudecode program available but they did not have a base64 decoder. Keep in mind at that time, many people were using systems that did not have a C compiler (the C compiler was often sold as an expensive add-on if it was available at all, and this was before GCC was widely available) so acquiring and compiling their own base64 decoder was a significant effort.

But in certain contexts you didn't need a file name or mode (e.g. inline encoding of the body of an email message), and uuencoded data was particularly vulnerable to corruption because at that time it was not uncommon for a mail gateway somewhere along the line to insert an unwanted newline somewhere within your message, or for character set translation to corrupt something. The extra newlines were usually easy to fix, and the uuencode format made it easy to see where they had occurred, but corruption due to character set translation was much harder to fix (sometimes impossible without trial-and-error testing). Base64 encoding solved these problems and was therefore a better choice for use within the MIME email encoding standard.

The decline in popularity of terminal-mode access compared to GUI access is what really killed uuencode. Users who were using graphical email clients on a PC or on a Workstation or X Terminal, found base64-encoded MIME attachments more convenient than uuencoding, and web browsers allowed you to download files without needing any encoding at all (shifting the common method of binary file transfer away from mail and news, towards the use of FTP and HTTP instead). Uuencoding is still an easy way to send a file when both the sender and receiver are using text-only terminals and can't use FTP, but today this is almost never the case.

1

I can't definitively say whether it is cause or effect so am somewhat chancing my arm by promoting it to an answer but: the only way of forming a data URL (i.e. one that has the data directly within it†) is as base64.

Since all moderately substantial application environments supports URLs, even if they don't explicitly support base64 encoding and decoding then they at least support decoding just by forming the data URL. So it's just really easy for developers to support.

Therefore I think base64's usage in URLs may have contributed to its ascension, in the same way that its use in the IBM PC helped the x86 — it's not where the thing came from or why it was designed, but it led to a substantial propagation.

† e.g. this tiny document icon that I cribbed from this site, which doesn't identify a remote resource but itself contains a local resource. You might need to copy and paste it into your browser bar if yours is anything like mine, as trying to follow it like a link from here inevitably leads to the error that it's not a functioning link. Which is the point.

  • I'd say Base64 was already en vogue (for e-mail) when nobody even knew what an URL was supposed to be... – tofro Jun 1 '17 at 16:28
  • The relevant RFC is from 1998 — tools.ietf.org/html/rfc2397 — when do we think uuencode lost dominance? – Tommy Jun 1 '17 at 16:58
  • 1
    Base64 encoding is already defined (even if the name wasn't established yet) in RFC1421 from 1993 – tofro Jun 1 '17 at 17:31
  • I think I've partly misread you, for which I apologise. I read it as being that base64 had ascended before being incorporated into URLs. Which is accidentally a corollary of your point, but I'd just plain misread. You're just saying: its ascension into URLs is consequence, not cause. Which is unarguable if you're dating it to pre-1998. – Tommy Jun 1 '17 at 17:45
  • 2
    You've got "cause" and "effect" backwards here. Base64 (RFC 1421) pre-dates URLs (RFC 1738) by almost two years, data: URIs (RFC 2397) by five and a half years, and widespread use of data: URIs by a decade or more. – Mark Jun 1 '17 at 22:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.