Hot answers tagged

45

The designers of Unicode did consider whether 16 bits would be sufficient, and decided that they would be, based on their assumptions: Unicode gives higher priority to ensuring utility for the future than to preserving past antiquities. Unicode aims in the first instance at the characters published in modern text (e.g. in the union of all newspapers and ...


39

The new additions mentioned are mostly to be found in the new Symbols for Legacy Computing block (PDF link) covering the 1FB00–1FBFF codepoint range. This block includes: a large number of BLOCK SEXTANT characters like 🬥 BLOCK SEXTANT-1236 ("The term 'sextant' refers to block mosaics divided into six parts." Also note because these definitions are new, ...


22

Cultural factors played a role. Western creators of Unicode expected that unifying East Asian ideographs would be similar to unifying Western fonts (roman, italics...) and that they get to decide where the line between modern and unusual characters lies. This proved to be too optimistic. For the unification, the devil was in the details, but also in the ...


14

Unicode represented a compromise among a number of competing objectives. Probably its biggest failing is its presumption that a single storage format could be suitable for string storage, string concatenation, text rendering, and text editing. It sacrificed considerable storage density so that text editing could be performed without having to be aware of ...


12

But would it have been conceivable for the inventors of Unicode to forsee that 16 bits would not be sufficient, to think of UTF-8 at the beginning, and only specify UTF-8 and UTF-32 as valid encodings for Unicode? Is there any history to suggest that perhaps some people were more forward thinking but their views where disregarded? The problem was ...


11

The character set is known as "Mac OS Roman". The official mapping from Mac OS Roman to Unicode can be found on unicode.org. While the character set is similar to Windows-1252 and ISO-8859-1, having many characters in common, it's not a direct match for either. All of the characters are in the Basic Multilingual Plane (BMP). It's worth noting, however, ...


10

From a more practical point of view, the program to convert between different character sets is called iconv. It can be used on the command line from the OS X (Mac) terminal, as well as in most Linux distributions. The most popular implementation is GNU libiconv, providing support for hundreds of different encodings, including MacRoman and the other Mac ...


7

Yes, absolutely, if not for some big players in the Unicode consortium heavily pushing an agenda for a 16-bit fixed-size character set early-on. On the ISO 10646 side, it was always assumed that a 16-bit character set would be insufficient. They had other things wrong that kept them from gaining traction though, and in 1990 a unification with Unicode was ...


6

TL;DR: UTF-8 and any other variable-length encodings are good "external", "exchange" format, but bad "internal processing" format. You start from a very wrong premise that "UTF-16 was kind of a mistake" and "UTF-8 is the best solution for pretty much all applications". It isn't. UTF-8 is very convenient in that you don't need to think about new code points—...


3

I posit that the original use of control characters was strictly for non-textual usage. In other words, things that don't print. In ye olden times, that was the primary way to communicate anything other than actual text to: Terminals - initially printing, e.g., ASR-33, and later video terminals, e.g., ADM-3A, VT-100, etc. Printers Modems - to control the ...


2

One example of the classic ASCII control codes being used today would be ETX and EOF. On a Unix-like system in the terminal, Ctrl-C is ETX and will normally quit the program, and ctrl-D is EOF and is often used to mark end of input to a program.


1

It could have been avoided if Unicode had taken a more reasonable path, but being Design by Committee, that was practically guaranteed not to happen. If desired, the BMP (= 16 bits of Unicode) plus some metadata, would have been way enough for everybody, forever. Now, I said "reasonable path" above, but what is reasonable? That's of course debatable, and ...


1

Since the BBC Micro had a Teletext chip (SAA5050) and used it for its MODE 7 and the Teletext character set was added (I believe in that revision), it had its character set added, although, maybe just be coincidence ;)


Only top voted, non community-wiki answers of a minimum length are eligible