6

Before I knew about line endings, I used to open small executables in Notepad and ruin them completely. However, some more complex executables (e.g. Paint Shop) were too big for Notepad, and I was asked to open them in WordPad. This completely blew my mind; how could a simple text editor not have space to open something that a more complex editor could open?

Later, it occurred to me that, even in a really big text file, lines at the beginning of the file could change in length without slowing down the whole computer. And (with some help from the Ctrl+A Ctrl+C Ctrl+V Ctrl+V trick) I discovered that Notepad appeared to add a line break when the line was really long, though moving the cursor back and forth showed no evidence of a real line break existing. I can only speculate why this could be.

How does Notepad store the text buffer in memory, and why do these effects occur?

  • 1
    This seems to be something to do with the EDIT control used by Notepad, but the MSDN documentation isn't very helpful. – wizzwizz4 Apr 29 '18 at 8:38
17

Notepad (at least originally) was implemented as a simple wrapper around the Windows EDIT control. EDIT is not really designed to handle large amounts of text -- it stores text in a single block of memory allocated via LocalAlloc (which, at least for 16-bit versions of Windows, means that it can't handle more than 64K of text in a single control, and in real-mode windows requires a contiguous block of memory to store all of it). Even in 32-bit Windows, where the memory management no longer restricts it to 64K backwards compatibility does, because the messages that are sent to it or received from it in order to allow the application to control it were only designed for the 16-bit system, and were never modified for 32-bit applications (Microsoft wanted Win32 to be source code compatible to Win16, so the existing messages had to retain the same layout[1]).

The line breaking issues you noticed are probably due to the fact that while it can support (nearly) 64K of text, the Windows EDIT control can't handle more than 32767 characters on a single line. Copying and pasting a larger line can exceed this limit, but the line breaking algorithm seems to rely on signed integers[citation needed]. At a guess Notepad is reading lines out of the editor and storing them to the file one by one, and therefore gets a phantom line added where there is none in the buffer because the requested line is longer than the limit so is split into two by the edit control to pass the data back.

More advanced editors like Write and its successor WordPad don't store the entire content in a single block; they typically used a linked list of lines, so were able to handle larger files even under 16-bit windows, and don't have backwards compatibility restrictions that prevented them from changing implementations when moving to 32-bit.

[1] - citation needed - I know there's an article about this on Raymond Chen's blog but I can't find it right now.

  • If it stores it all contiguously, how is adding and removing characters near the beginning so fast? – wizzwizz4 Apr 29 '18 at 10:01
  • I haven't been able to confirm it, but I believe it uses a gap buffer. – Jules Apr 29 '18 at 10:16
  • I think that's probably right; when I printed out a block of memory it looked a bit like that. – wizzwizz4 Apr 29 '18 at 10:31
  • Re your [citation needed], possibly this article? blogs.msdn.microsoft.com/oldnewthing/20161229-00/?p=95045 – Greg Hewgill Apr 29 '18 at 23:55
  • 1
    The backwards-compatibility angle didn’t limit things for long (well, a decade at most): XP Notepad can open 32MiB files (I haven’t checked Windows 2000 or NT). – Stephen Kitt Apr 30 '18 at 11:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.