2

I asked this question on Super User, but the community at the site told me to redirect to this site. If this is again an off-topic question, do not blame me please!

I am sorry if the control characters are actually super useful nowadays; I am not a computer technician.

I am studying Unicode characters. More specifically, I am learning when and how to use them. But the control codepoints keep bugging me.

What are even contr...?

By "Control characters", I mean the set of valid ASCII/Unicode characters which do not render anything, but instead were (or are) used to control the host device.

So what?

Indeed, some control characters, like the Linefeed (LF) and Tab (HT) are still used today commonly for organizing text, but what about the others? Where can I use the Device Control 3 (DC3)? Can I even use the Negative Acknowledgment (NAK)?

It would be great if someone can write a detailed usage definition for each control character (If it is useful nowadays).

Thanks!

7
3

I posit that the original use of control characters was strictly for non-textual usage. In other words, things that don't print. In ye olden times, that was the primary way to communicate anything other than actual text to:

  • Terminals - initially printing, e.g., ASR-33, and later video terminals, e.g., ADM-3A, VT-100, etc.
  • Printers
  • Modems - to control the modems, not just to pass text through to terminals & printers
  • Any device with no capability for out-of-band control

The last is the key. In-band = control using the same transmission of data (typically, but not always, 8-bit bytes) used for the content. Out-of-band = control using some separate mechanism.

A typical example of out-of-band control is disk drives. An ST-506 drive had 3 cables: power, data, control. The drive would know when to read vs. write vs. move not based on the data cable but based on the control cable. Well, a little more complicated than that, but the point is that it wasn't arbitrary characters in the data stream that determined when to perform different actions.

In-band control is a must when you only have one transmission path, unless your data is encapsulated inside something else, a packet. In the olden days, terminals, printers and similar devices were often a 3-wire system - send, receive, ground. Sometimes they would include hardware handshaking, but that was typically limited to start/stop and possibly answer/hang up (e.g., CD on a modem) or equivalent. Additional control, such as when to move the platen (CR, LF, VT), ring a bell (BEL), or sometimes more complex operations, were sent as Control Codes. Any other method might work locally, but would not work with a remote terminal or printer, transmitted over a modem, etc.

So what has survived?

The obvious ones are CR, LF and TAB. BEL is also still supported in many devices. But there is one more that is still very often used: ESC. While the average user never sees it, a typical non-GDI (don't get me started about GDI printers...) printer relies heavily on Escape Codes - traditionally ESC followed by a manufacturer-specific sequence of text. PCL is still huge, in almost every non-GDI laser or inkjet printer, but there are other variations still commonly used in receipt printers, label printers and other devices.

ANSI escape codes are supported by many common terminal programs & operating system shells, even though the transmission process now is almost always either a directly attached device (memory mapped screen/video card) or a TCP/IP network connection.

On the other hand, codes related to flow control and some other actions are now handled out-of-band. Typically this is by encapsulating the textual data inside a packet, with transmission issues handled outside the packets (e.g., TCP/IP transmission) or other "control" information handled in a defined structured manner inside the packet. While out-of-band transmission of control information does complicate the process a bit, requiring either a second transmission path or packets of some sort, it allows for transmission of any data string, which was a real issue in the early days. For example, if you wanted to control a Hayes-compatible modem while connected, you would pause for a short time, send +++, then pause again, and the modem would switch from transmit/receive modem to control mode. Great if you wanted to do that. But a potential path for problems if you didn't want to do that. Murphy's Law says that any arbitrary string of characters + timing will occur when you least expect it.

4
  • I believe the Hayes modem pause was intended to be an out-of-band mechanism (which it was) since if someone actually tries to transmit the string "+++", there will be other data before or after it. "+++" by itself is not a valid packet. – user253751 Nov 13 '20 at 15:37
  • @user253751 Exactly. But it was possible for it to happen in other ways than designed - i.e., either unintentional or intentional but maliciously. For example, a remote user communicating on a full duplex (destination echoes all normal text) with their (remote side) modem set to ignore +++ could cause +++ to trigger the other (host system) modem to drop into control mode! It was definitely not foolproof. – manassehkatz-Moving 2 Codidact Nov 13 '20 at 15:46
  • Ah, yes, if you are communicating with something other than IP. – user253751 Nov 13 '20 at 16:50
  • 2
    And then there was the system at University of MD when they started using video terminals under 3270-emulation on the IBM mainframes. They had some "express" terminals with a 1-hour limit. But if you knew the right control-code-style tricks, you could bypass that and stay for longer...not that I would ever do such things (cough cough). – manassehkatz-Moving 2 Codidact Nov 13 '20 at 16:57
2

One example of the classic ASCII control codes being used today would be ETX and EOF. On a Unix-like system in the terminal, Ctrl-C is ETX and will normally quit the program, and ctrl-D is EOF and is often used to mark end of input to a program.

2
  • What I was thinking! Great examples. – Toby Speight Nov 13 '20 at 17:08
  • 1
    The majority of Unix application programs don't do anything special with ^C or ^D. If you type a ^D in a terminal session from which a program is taking its standard input, and the program thinks that the standard input has come to an end, or if you type ^C and the program receives a SIGINT, that's usually because the shell set the program up with an "interactive" line discipline, and the line discipline intercepted the ^C/^D. Put those same chars in a file, and pipe it to the same program, and the program will just read them as normal chars. – Solomon Slow Nov 13 '20 at 17:39
0

IMO, ASCII control characters ceased to be meaningful around the same time when people stopped thinking of ASCII as a code for sending telegrams, and started thinking of it as a code that was used for storing text in computer systems.

The same question that you're asking today—what do most of these even mean?—is the same question that I asked when I first learned about ASCII more than 40 years ago. The answer that I got back then only grows stronger with time: They're obsolete.

5
  • Really? Try writing a formatted document directly to a laser printer (not through some "driver") without control codes. – manassehkatz-Moving 2 Codidact Nov 13 '20 at 16:59
  • @manassehkatz-Moving2Codidact: even then, chances are you won’t be using those characters in their ASCII meanings. (I happen to have done just such a thing: it was a portable Bluetooth receipt printer. The control codes were assigned pretty much randomly.) – user3840170 Nov 13 '20 at 17:24
  • @manassehkatz-Moving2Codidact, I interpreted the question to specifically be about the ASCII standard, and not generally about the languages or protocols that are used to control modern devices. What does ASCII "Start of Heading (SOH)" mean to a laser printer? What do "Data Link Escape" (DLE)" and "Unit Separator (US)" mean? The byte stream that you send to a printer contains non-text codes that control the printer, but those codes are not generally interpreted in the context of the original ASCII standard. – Solomon Slow Nov 13 '20 at 17:24
  • A different way to look at it is technical language in general: Is "carbon copy' obsolete simply because almost nobody actually uses carbon paper to make copies? No. The terminology remains useful even if the actual usage has changed. Similarly, ASCII "ESC" may be used now differently than originally intended, but that doesn't make it obsolete. – manassehkatz-Moving 2 Codidact Nov 13 '20 at 17:27
  • 1
    @manassehkatz-Moving2Codidact, agreed. There are a few special cases. Most notably, the use of HT, FF, and LF/CRLF in pre-formatted text. That's why, in my youth, I asked, "What do most of these even mean?" – Solomon Slow Nov 13 '20 at 17:47

Not the answer you're looking for? Browse other questions tagged or ask your own question.