8

Before I knew about line endings, I used to open small executables in Notepad and ruin them completely. However, some more complex executables (e.g. Paint Shop) were too big for Notepad, and I was asked to open them in WordPad. This completely blew my mind; how could a simple text editor not have space to open something that a more complex editor could open?

Later, it occurred to me that, even in a really big text file, lines at the beginning of the file could change in length without slowing down the whole computer. And (with some help from the Ctrl+A Ctrl+C Ctrl+V Ctrl+V trick) I discovered that Notepad appeared to add a line break when the line was really long, though moving the cursor back and forth showed no evidence of a real line break existing. I can only speculate why this could be.

How does Notepad store the text buffer in memory, and why do these effects occur?

Improve this question
1
  • 1
    This seems to be something to do with the EDIT control used by Notepad, but the MSDN documentation isn't very helpful.
    – wizzwizz4 ♦
    Apr 29, 2018 at 8:38

1 Answer 1

Reset to default
17

Notepad (at least originally) was implemented as a simple wrapper around the Windows EDIT control. EDIT is not really designed to handle large amounts of text -- it stores text in a single block of memory allocated via LocalAlloc (which, at least for 16-bit versions of Windows, means that it can't handle more than 64K of text in a single control, and in real-mode windows requires a contiguous block of memory to store all of it). Even in 32-bit Windows, where the memory management no longer restricts it to 64K backwards compatibility does, because the messages that are sent to it or received from it in order to allow the application to control it were only designed for the 16-bit system, and were never modified for 32-bit applications (Microsoft wanted Win32 to be source code compatible to Win16, so the existing messages had to retain the same layout[1]).

The line breaking issues you noticed are probably due to the fact that while it can support (nearly) 64K of text, the Windows EDIT control can't handle more than 32767 characters on a single line. Copying and pasting a larger line can exceed this limit, but the line breaking algorithm seems to rely on signed integers[citation needed]. At a guess Notepad is reading lines out of the editor and storing them to the file one by one, and therefore gets a phantom line added where there is none in the buffer because the requested line is longer than the limit so is split into two by the edit control to pass the data back.

More advanced editors like Write and its successor WordPad don't store the entire content in a single block; they typically used a linked list of lines, so were able to handle larger files even under 16-bit windows, and don't have backwards compatibility restrictions that prevented them from changing implementations when moving to 32-bit.

[1] - citation needed - I know there's an article about this on Raymond Chen's blog but I can't find it right now.

Improve this answer
8
  • If it stores it all contiguously, how is adding and removing characters near the beginning so fast?
    – wizzwizz4 ♦
    Apr 29, 2018 at 10:01
  • I haven't been able to confirm it, but I believe it uses a gap buffer.
    – Jules
    Apr 29, 2018 at 10:16
  • 1
    The backwards-compatibility angle didn’t limit things for long (well, a decade at most): XP Notepad can open 32MiB files (I haven’t checked Windows 2000 or NT).
    – Stephen Kitt
    Apr 30, 2018 at 11:47
  • 1
    @StephenKitt The backwards compatibility issue with the control is still there. At some point, the edit control was extended to allow the application to allocate buffer memory for the control, and at some point Notepad was rewritten to make use of this. But, by default, the control still allocates its own memory and follows its original behavior, and applications that don't know about the extended functionality thus get the original behavior.
    – Dranon
    Apr 30, 2018 at 15:20
  • 1
    @wizzwizz4 For the 64 kB maximum, even on a 12 MHz 386, inserting a character with simple memcpy() algorithm takes only 5 milliseconds - with screen frame interval of 16 ms or more, such latency will not be noticeable.
    – jpa
    Mar 16, 2022 at 15:42

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .