10

In a parser library I am maintaining, I stopped recognizing singular Carriage Return characters as line endings to reduce complexity in the tokenizer's position tracking code, a perennial source of bugs. At present only CRLF and LF will increment the line and reset the column position, and at a future version I am considering stopping recognizing them by default and fail the parsing.

My question is what impact would such deprecation have? Is the CR line ending still prevalent on new applications or should it be considered legacy?

I surveyed my development tools and found that Visual Studio Code does not offer changing the line endings of a file to CR, while Visual Studio for Windows and the JetBrains IDEs give such option. The F# compiler fails compiling a source file with CR line endings, while the C# compiler can.

15
  • 9
    why not 1) convert CR+LF to LF 2) convert CR to LF 3) profit and forget about this CR crap? Sep 30 at 19:14
  • 6
    Not really sure what kind of complexity this is about. After all, this can only occur in 4 variations which can be handled by something like 4-9 assembler instructions (depending on CPU)
    – Raffzahn
    Sep 30 at 19:26
  • 6
    I mean load in memory, perform the transformation and pass the result to the parser. Some pre-processing. But of course don't change the original text Sep 30 at 19:38
  • 8
    I think this is technically off-topic, much like <retrocomputing.stackexchange.com/questions/16897>. Otherwise though, is it really such a big problem to maintain 1-byte lookahead? Sep 30 at 20:05
  • 4
    @TheodoreTsirpanis I'm really not sure what your 'position tracking' is supposed to be. If this is a tokenizing parser like it seems, then I'm not sure what position there is to maintain that wouldn't comply with parsing sequences of one or more CR, CRLF, LFCR and LF. But I'm quite sure we're way past a historic question. Maybe try an old fashioned way: take a punch card (if not present, mundane paper will work as well), draw up the few possible sequences and write the desired result beside. Add block breaks to see what state information may be required. It'll become quite clear what to do.
    – Raffzahn
    Sep 30 at 20:50
10

If your parser library is not designed to run on Classic Mac OS, there's no reason whatsoever to support a bare CR as a line ending. Modern macOS has only ever supported them insofar as some of its files might have originated on Classic Mac OS. Nobody uses them anymore and people who have got such files know that most of the tools they use will barf on them.

As a rule, in parsing code, I tends to use \n as the line ending and treat \r as optional white space.

2
  • 5
    When I'm using screen as a serial console, CR is exactly what I get on the wire when I hit the enter key. So I'm not sure that "nobody uses them anymore".
    – Rodney
    Oct 2 at 10:43
  • @Rodney OK nobody except people who have a need to process raw terminal input.
    – JeremyP
    Oct 2 at 12:27
9

The pbpaste command used to generate CR line endings up until Mac OS 10.6, at least. With Mojave and Big Sur, however, it's long gone.

MS Office on Mac used to be a dreadful emitter of CRs. It's now moved to CRLFs on CSV exports and text copied from Office apps and pasted using pbpaste. I have been unable to emit CRs on other modern Mac apps, so it's effectively obsolete as a convention.

1
  • 2
    Maybe I'm behind the times, but the versions of Office on the Macs I use still emit bare CR's when writing CSV, so this is still very much a concern for me! Oct 1 at 16:46
6

Is the CR line ending still prevelant on new applications or should it be considered legacy?

As usual it all depends on the environment your software is used in. If you're sure that all input will be [CR]LF delimited, then narrowing it down might work fine.

If it's guaranteed that all data will be produced by Mac Software less than ~5 years old, chances may be good. Just make sure it's never used with old(er) data. Data is not just produced by Macs, but many other systems as well.

My question is what impact would such obsoletion have?

As you already assume, it may break compatibility in some Situations. Chances for this to happen is all within the use szenarios of your library. Do you know them all?

Keep in mind, Data can come from many systems (not just ac or PC) and is of varying age. It's not uncommon to parse text created on some odd late 1970s system, as it my contain important business numbers.


So far for the 'how common part. Talking about design decisions of your new software is rather off-topic here, so readers may ignore the following.

Personally I would not only start to think about dropping either format. One of the most important aspects of data handling libraries is resilience against 'unusual' input.

Or, as the saying goes: Be gentle to any input and strict with output.

The other guideline here might be: Never optimize more than necessary.

Last but not least: Be prepared to enjoy many bug reports and feature request from upset users who happend to have an older file and do notunderstand why it can be opened in an editor but fails with your library.

Turning CR/LF sequences into line end tokens requires usually only a very simple state machine (*1), able to be implemented in less than a dozend assembler instructions. Not exactly a huge performance killer or complicating issue at all.

Now, if you're software design is already settled to a very pecific and rather limited approch, as the addendum suggest, then why ask at all?


*1 - Rewrote that line 3 times, hesitating to call it a state machine at all, as it's just a few IF clauses.

9
  • 2
    Many legacy text files use mostly CR+LF line termination, but use CR-only line termination to accomplish overprinting. Treating a bare CR as equivalent to a CR+LF will likely strip out what had been semantically useful information.
    – supercat
    Sep 30 at 20:19
  • 3
    @supercat but they are a tiny subset of data and a very special case. After all, if the parser is about print data, then there is way more to be bandeled, not just CR/LF, but BS and anything from DC1..4 all the way up to ESCape sequences. So no, not really an issue to think about for a generic parser.
    – Raffzahn
    Sep 30 at 20:52
  • 1
    re It's not uncommon to parse text created on some odd late 1970s system especially in RC SE :-) Sep 30 at 22:33
  • 1
    @supercat CR-only as EOL was standard in Macintosh all the way through Classic days. It's slowly gone away through Mac OS X.
    – scruss
    Sep 30 at 23:12
  • 1
    @supercat: It is 2021 and Unicode has been standardized and widely-implemented for well over a decade at this point. If you're still using CR-overstrike in lieu of proper Unicode combining characters (or precomposed characters), then IMHO any compatibility issues are your problem.
    – Kevin
    Oct 3 at 23:11
1

Terminals can process CR and LF a number of different ways, and the way systems stored text files was often a result of the kind of terminal to which they were most commonly attached.

CR may either reset the cursor/carriage without advancing a line, or may both reset the cursor/carriage and advance a line.

LF may advance a line without resetting the cursor/carriage, advance a line while resetting the cursor/carriage, or--on systems where CR advances a line--do nothing.

If memory serves, DEC terminals like the VTxx series (not sure about their predecessors) treated LF as a combination reset cursor/advance line. Later terminals that tried to be DEC-compatible would imitate this behavior, but I don't know of earlier terminals that behaved that way.

An advantage of having CR act as a newline (reset carriage and advance paper) is that terminal I/O could use the same line termination character without translation on input or output. As a consequence, many microcomputer systems and languages used to program them used a bare CR as a line separation sequence on input; some would expand CR to CR+LF on output, and accept CR+LF on input, but what they cared about was the CR and not the LF. While the Classic Macintosh was the longest-enduring system that did so, it was hardly unique. Historically, the Unix bare LF was the oddball. Indeed, many programs for MS-DOS and Windows would accept text files which used bare CR, but choke on those that used LF alone.

Program that may be used to process "vintage" text files should treat any CR as a line ending, and also treat any LF which isn't immediately preceded by a CR likewise. If one is using a language that supports input-stream pushback, this may be accomplished by having the read-character routine take the next character, see if it's a CR, and if so peek at the next character. If the next character is LF, return it. Otherwise, push back that character and return LF.

2
  • re If memory serves (...) -- VT100 setup option. Default was that code 012 was "linefeed" (a vertical format effector) but there was an option for it to be treated as "newline". If memory serves :-), this is was due to different definitions in ASCII and ISO. Also, it's possible Flexowriters had "newline" and not "linefeed", so that might be one of the origins of the confusion. Oct 1 at 16:56
  • VT100 User Guide - page 1-8, 3rd block of options in SETUP B. Page 1-14 describes the option. It also affects what the RETURN key does, a fact I had forgotten. Oct 1 at 17:03
0

The MATE Desktop's default GUI text editor Pluma (which is a fork of GNOME Gedit so I assume this applies to both) still supports 'Mac OS Classic' 0x0D newlines on the Save As... dialog.

Not an indicator of prevalence, but it's still available in a current text editor, so it's conceivable you could encounter it on a Linux-based classic Mac cross-compilation set up, but that's niche.

-2

The CR LF (Carriage Return, Line Feed) is the (standard!) ASCII way to tell a printer to start at the beginning of a new line. Unix is a rascal, they used just LF (aka '\n') as line ending (when your memory is measured in KiB, each byte counts). Here Microsoft (and original Mac) went with the standard...

Here on Fedora Linux I've got a pair of programs, unix2dos and dos2unix that translate line endings back and forth. You can get their sources at https://waterlan.home.xs4all.nl/dos2unix.html

7
  • 1
    Doesn't seem to address the actual question. Oct 2 at 19:32
  • The question asserts that the original Mac used CR only, not CR+LF or LF only. So dos2unix would be of no help. Oct 3 at 3:05
  • Here Microsoft (and original Mac) went with the standard -- no.. Mac went with CR, Unix went with LF, DOS went with CRLF. ANSI standardized on CRLF LONG AFTER Unix, Mac and Windows were developed. So Unix was not the rascal - the committee that decided to favor Windows was
    – slebetman
    Oct 3 at 10:50
  • @MarkRansom - the dos2unix linux package typically includes the commands unix2mac and mac2unix that work with bare CRs
    – scruss
    Oct 3 at 16:13
  • 2
    @slebetman: The ASR-33 teletype that used CR+LF predated the invention of Windows by many decades, and the idea that homing the carriage would advance the paper goes back a long time before the idea that advancing the paper would home the carriage. Although the codes to home the carriage without advancing paper and home the carriage with advance are more generally useful than one to advance paper without homing the carriage, and making what had been the advance-paper code home the carriage as well was certainly useful, it went against precedents that were already well established.
    – supercat
    Oct 4 at 14:51

Not the answer you're looking for? Browse other questions tagged or ask your own question.