An idea about memory management [closed]

Question

The other day, I had an idea. This conclusively proves that people smarter than me already thought of this decades ago. What I'm wondering is... Is this a good idea? Was it ever implemented in commercial hardware? Did it work out? (Presumably not...)

Pretty much every modern OS maintains the illusion of "processes". Each process is isolated; in particular, they each have their own virtual address space. The hardware knows nothing about any of this; all it knows is that somebody keeps overwriting the register file, and changing the page table pointer, causing a (presumably expensive?) flush of the translation look-aside buffers.

Now suppose I'm writing an application, and I want to add database facilities to it.

• I can use something like MySQL, which runs as an external process.
  • Because it's an external process, it's isolated from my application.
    • A bug in MySQL cannot corrupt my application's memory.
    • For that matter, a bug in my application can't corrupt memory belonging to MySQL either.
  • Because it's an external process:
    • Need to use some kind of IPC mechansim to talk to it. (Unix domain sockets, actual TCP/IP, etc.)
    • Requires a system call into the OS kernel.
    • Requires a context switch.
    • Requires changing virtual address space.
• I can use something like SQLite, which is a shared library.
  • Because it's a shared library, there is no isolation. It's loaded directly into the address space of my application.
    • Bugs in SQLite totally can corrupt my application's memory.
    • Bugs in my application can corrupt memory belonging to SQLite.
  • Because it's a shared library, you can communicate with it by a normal subroutine call.
    • No system calls.
    • No expensive context switches.
    • No expensive address space changes.

Summarising:

• You can use an external process, which is isolated, but slow.
• You can use a shared library, which is fast, but not isolated.

[I'm aware there are lots of additional differences; that's not the focus of this discussion.]

My idea:

• Divide each process into what I'm going to call "zones".
• Each region of the virtual address space belongs to exactly one zone.
• The "current" zone is the zone that the Program Counter currently points to.
• Only memory in the current zone is writable.

(I'm imaging each memory page to have read/write/execute permission bits, but for any page outside the current zone the write-enable bit gets ignored.)

In this way, you could load a shared library but put it in a separate zone.

• Communication is still just normal subroutine calls.
• No expensive context switches.
• No expensive address space changes.
• Code in one zone can't corrupt memory in a different zone.
• Data pointers and code pointers still work across zones.

In short, we have speed and isolation.

There are several ways you could potentially implement this. For example, each page table entry could have a zone number associated with it. Or maybe you have a separate zone table giving the start and end [virtual] address of each zone.

You could allow each page to have arbitrary permissions for each zone. But that seems like it would add too much overhead. Or perhaps you could have just two permission bitmaps per page; one for the "owner" zone, and one for everyone else. There's several ways you could design it.

Is any of this a good idea? Was it ever implemented on any commercial hardware?

[The fact that modern computers don't do this suggests that it's either a terrible idea or it just never caught on.]

Answer 1

This is not exactly on target. But then, the focus of your question was somewhat unclear. Back in 1971-72, I worked with four other people to build a language called Muddle, or later MDL. This was at project MAC, MIT. BTW, I was perhaps the least important person on the team.

Muddle was a derivative of Lisp, but it had several experimental features in it. One of those features was scheduled processes within the scope of a single memory space. The operating system viewed a user running Muddle as a single process with its own space, but that is not germane to this discussion. The muddle executive had its own mini scheduler inside of itself, and it could timeshare the processing resource within quasi concurrent sub processes. The muddle programmer didn't have direct access to memory, and this almost eliminated the rogue process problem.

So what could be done with this feature? Well, one way it was used was to implement the parallel AND and the parallel OR. The idea is this : supposing it takes a half hour to evaluate A, which evaluates to TRUE. Suppose it takes a few milliseconds to evaluate B, which evaluates to FALSE. A AND B takes a half hour, but B AND A takes milliseconds. This is kind of unfortunate.

So the parallel AND launches two scheduled processes, one that evaluates A and the other that evaluates B. If one of them comes back quickly with FALSE, the other process can be killed. Now Muddle doesn't require a page map refresh when it switches between processes A and B. It's all one big process as far as the OS is concerned.

A search on MDL programming language should yield some results. I'm not sure how relevant all of this is to the question you are pursuing, but here it is. MDL remained in use for about ten years. some of its features found their way into Common Lisp. And languages like Java may have inherited or reinvented some of the same concepts. I don't know. If MDL has a place in history, it's mostly because an interactive fiction game called Zork was originally written in MDL.

Answer 2

[This is not only rather off topic, based on modern usage and theoretical in nature, it contains also way to many questions, based on way to many and narrow assumptions to give a detailed answer.

Assuming it to be moved soon, I would like to give a few basic hints to rethink its trajectory.]

TL;DR:

1. It's solving a non-issue for most existing systems as such communication doesn't have the imagined impact with the remaining being mitigated.
2. If you really keen onto maximum IPK performance, a look at L4 will be the way to go

Historical

Historically the need for separation did not exist. Different processes were simply loaded into main memory at different address, IPK was done either by direct calling (linkage done during load) or using very slim OS functions allowing to queue calls/messages.

On classic 360 (real mode) protection was done by assigning page keys to compare page owner to program counter. More than enough to catch anyone writing outside it's own pages.

Virtual Memory

Most important there is not one single way to do virtual addressing. And even when going with today's most prevalent idea of a two stage page directory/page table structure and a TLB, the assumption of 'high cost' - or notable cost at all - is not true.

First point here, the TLB (*1) is not the page table(s) but only a cache intended to speed up repeating access. That is page table entries of the last n pages accessed are held on chip to allow address translation without penalty of accessing the page data in memory. If a page entry is not in TLB the MMU needs to walk the page table(s) and load exactly that entry(*2). How many level there are and how large those structures are depends on CPU family and implementation.

Here it depends on the system used, but let's go with a basic approach. Let's assume (*3)

• a 32 Bit address space
• a two level structure
  • a page directory translating the top 10 bit
  • multiple page tables translating the next 10 bit
• each table thus holding 1024 entries
• each entry being 32 bit
• each table thus being 4 KiB
• each page thus as well being 4 KiB (the remaining 12 bit)

So if a process to be activated he MMU will have to read two 32 bit memory words to get the address where execution starts/continues. and only another one if it goes past that 4 KiB code page (two if it jumps more than 4 MiB away). Same goes for data access. All of that maybe further smoothed out by read buffers and generic memory caching.

So TLB flush, the start of your whole assumption isn't a big issue at all. The instructions to switch may take up way more time than that.

Memory Usage

Not everything has to be in another address space or OS address space or run under OS privilege. Processes can of course share code and data.

In case of your database example the code necessary for data base access can quite well be inside the address space of the user page but still protected against write from your other code parts

Shared memory for code and data as well as memory mapped data is an awesome tool for performance.

At least with mainframe OS it was quite common to have not only OS code within the user address space to save going thru the call interface, but as well (user specific) data like buffers. Worked quite fine with very little chance to screw it - and if so it did only cause problems to that single user program, not anyone else.

Call Interface the Way You Like It

Having a direct/fast call interfaces to OS/services/other processes isn't a new idea. Already certain Honeywell machines implemented 'Call Gates' as a way perform OS calls.

But no need to look that far back, as Intel implemented them in their 286 design (*4). Codewise an OS call - or call to some service like an SQL server - thru a call gate is standard FAR call with no difference from any other. On execution the CPU will note during standard memory check that this 'magic' descriptor (segment) points to an extern target and checks accordingly, doing a fault if the service is not allowed for that (user) process.

If successful it switches over to the other process and may even transfer parameters from user stack to service stack.

---

*1 - That is, if a TLB exists at all (*2). There are also systems with fixed tables of various structures in Hardware and many other solutions.

*2 - Which is the reason why IBM was so reluctant to add virtual addressing to the /360. Without a (somewhat large) TLB each memory access to a different page would incur an additional memory access to load the page table entry for that page (plus optional one or more directory level accesses).

None of the engineers could imagine why on earth someone would want to add a feature that would slow down their machines by a notable amount, a feature with no obvious advantage to existing customers or setup to be seen.

*3 - Yes, this is somewhat similar to the paging used by 32 bit x86, which due it's very basic nature fits well. Other systems may use different number of levels and/or different entry sizes, not changing the basic workings discussd here.

*4 - I did already mention that I think the 286 was a real great design, didn't I? A clean 16 bit system with no backpack tacked on, developed very much with Unix in mind.