30

John Carmack is credited with making fast-paced arcade games like Commander Keen possible on an IBM-PC that had no specialized graphics controllers suited for those, thanks to the "Adaptive Tile Refresh" routines he is credited to have invented.

However, for me this seems like a very US-centric view. In Europe, many home computers were very common in the 1980s that also didn't have any specialized graphics controllers suited for fast arcade games, like the Amstrad CPC or the ZX Spectrum. They had no hardware sprites and no (fine 1-pixel-wise) hardware scrolling, and processors slower than even that of the original IBM-PC.

And yet, many action games and arcade conversions came out for those. Granted, often jerky and with low framerate and reduced viewport, but to be able to run at all it really seems they used very similar techniques, i.e. not updating or rewriting the complete frame buffer, but only parts of it (i.e., "adaptive").

So should the myth be buried, or did John Carmack really come up with something new and fundamental, and if so, are there examples where his routines are ported and applied to newer released games developed by retro-enthusiasts for those machines?

5
  • 11
    Where is it said that he invented the technique?
    – Justme
    Nov 1 '21 at 18:26
  • 4
    @Justme, well, e.g. that linked Wikipedia article says "Adaptive tile refresh using hardware scrolling made its first appearance in an unreleased test game dubbed Dangerous Dave in Copyright Infringement, implemented by Carmack." and then goes on with mentions like "Carmack's adaptive tile refresh", "Carmack used these [EGA] capabilities", "Carmack's code", and "final part of Carmack's technique", so even if it didn't say Carmack invented it, it also doesn't mention anyone else.
    – ilkkachu
    Nov 2 '21 at 11:18
  • @ilkkachu That is what the article says, and it leaves room for interpretation. I don't deny that he might have been the first to implement, refine and commercialize the technique on an IBM PC that had an EGA video adapter. But did he invent the technique, the article does not say, but it does say that the technique is based on how large images were scrolled in the 1970s. EGA was just the first video card with enough hardware features to make it even possible to implement it on the PC platform.
    – Justme
    Nov 2 '21 at 12:18
  • It sounds like you'd be surprised if he did. Why is that? John Carmack is a legendary programmer.
    – user91988
    Nov 3 '21 at 0:07
  • Could ask him yourself :-) - superuser.com/users/131344/john-carmack Nov 3 '21 at 15:27
54

John Carmack almost certainly was the first to use the hardware scrolling capabilities of the EGA specifically, together with efficient tile and sprite drawing and erasing algorithms to create a slick, full-screen scrolling 16-colour console-like game engine on the IBM PC. "Adaptive Tile Refresh" was one part of this. He did so through experimentation, reading documentation and learning about the EGA and drawing/masking/rotating techniques through Michael Abrash's articles online, in Dr. Dobbs Journal and in the book Power Graphics Programming.

(And note that Mega Man (1990) for DOS came out very close (I believe 4 color CGA), and other earlier DOS games like License to Kill (1989) and Spy Hunter (1984) had scrolling as well - this wasn't in all directions and probably a full-screen redraw. I've also found a Quora post attributed to the developer of the DOS port of Golden Axe that says that game predates Keen.)

--

No, Carmack wasn't the first person ever to implement a system where tiles upon a moving background are redrawn row-by-row, or to devise algorithms for efficiently erasing and drawing blocks of tiles or sprites based on their motion (or lack of).

I've opened another question to discuss what parts of the Keen engine 'Adaptive tile refresh' refers to - I believe it's an optimization available when using hardware scrolling on a software-drawn tilemap to reduce the frequency and number of tiles that need to be redrawn as the screen scrolls. Looking at Chocolate Keen (reverse engineered from a disassembly of Keen 1-3) sprite handling also falls under ATR - when a sprite is drawn, it invalidates all tiles it overlaps on the tilemap on that display page, which are then redrawn on the next tilemap redraw (that is, refresh) pass.

As you stated in your question, drawing only incoming tiles on the edge of a scrolling tilemap would be a well-known technique for arcades or consoles where hardware scrolling was available, regardless of whether they used a bitmapped/bitplaned display or a tilemap. The Sega Master System and Intellivision (see Advanced Dungeons & Dragons) both have tilemap layers that are exactly the width of the displayed screen, so there's no other way to implement scrolling other than to draw the oncoming edge into the one-tile-wide region that becomes available by setting a flag to mask away one column of the screen with the border colour.

The ZX Spectrum has no hardware scrolling or sprites, only a 1-bit unscrollable display, chunked up in to 8x8 regions colored by 'attribute' values. Less-experienced ZX Spectrum programmers writing arcade ports such as Space Invaders-clones would clear the entire screen and draw the sprites, but more-experienced programmers would erase and redraw any moving elements or even redraw only their edges, or use XOR to draw-undraw elements if the resulting visual garbage when sprites overlapped other sprites or the background was acceptable. (XOR was known - mentioned in this 1976/8 patent, ZX Spectrum BASIC's drawing primitives expose XOR drawing mode as the statement OVER n.)

In short, people were familiar with similar tile and sprite drawing techniques or able to come up with their own algorithms suited to the target platform all through the 80s.

You say the PC had 'no specialized graphics controllers' but in fact it's the EGA card's PEL Panning register that made Keen's fine scrolling possible. The CGA has a configurable start address value which allows page flipping, vertical motion and coarse horizontal motion, but the EGA has an additional fine-panning register allowing motion within a memory byte/word - together these allow panning to any single pixel in the video RAM. The same registers are used in VGA Mode X (and a similar mechanism is used for scrolling on Amiga chipsets).

Carmack's innovation was to devise 8086 real mode algorithms like the ones in use on other systems for tiles and sprites, and having the engine be stable and fast across multiple competing first- and third-party EGA card implementations and monitors*. To get a full engine in place would need lots of experimentation and iteration. Adaptive Tile Refresh would be one useful part of this.

EGA programming is obtuse: bitplane selection, barrel shifting - lots of fields that can be set to ensure bitplaned graphics land in the correct 'slot' within a byte, etc. Descriptions of adaptive tile refresh also note that the drawing buffer is wider than the displayed region to allow for an off-screen region to scroll into (similar to the Sega example I mentioned), this means Carmack would have to also set the register containing the address increment per scanline (also known as the 'stride') to a non-standard value, and so on.

You asked Did Carmack figure all of this out on his own?

In the foreword to Michael Abrash's Graphics Programming Black Book, Carmack writes:

I learned the dark secrets of the EGA video controller [in Abrash's Dr. Dobbs articles], and developed a few neat tricks of my own. Some of those tricks became the basis for the Commander Keen series of games, which launched id Software.

Romero:

[Power Graphics Programming] is the book I gave @ID_AA_Carmack to read before he started on Catacomb PC in 1990. This led to Commander Keen scrolling breakthrough.

Asked on Twitter 'Did you know any of the antecedents from other machines?'

Carmack:

no, I didn't. Hard for people today to appreciate how scarce info was back then; Foley&Van Damn[sic] was no help for low level hacks

(Foley & Van Dam refers to the textbook 'Fundamentals of Interactive Computer Graphics', 1982 )

I had hacking/phreaking files from BBS, but never any good "real" programming info

So although tile refresh strategies for scrolling games on other systems existed previously, Carmack believes he came up with the complete strategy used in Keen in isolation.

*On VOGONS, you can find threads of people studying patching all the Keen games still, as certain titles misbehave in certain ways on certain cards even within a trilogy where one might expect the games to be the same.


I'm guessing the OP was an Amiga owner (like me) familiar with Ballistix and Shadow of the Beast (1989, both before Keen). Both these games have full-colour 50Hz gameplay with scrolling and (apparent) sprites. From the perspective of the 'home micros' (ZX Spectrum, C64, Amiga, Atari ST, etc.) as opposed to the business-focused IBM PC, the confusion as to what the fuss is about is understandable. In the PC world in 1989, EGA scrolling was 'cutting edge' stuff, though Apogee's claim that Keen "Makes an IBM outshine an Amiga" with its chunky world, beepy sounds and gaudy colours is a little bit ambitious. :)

4
  • 3
    You didn't need to be an Amiga owner (although I was one too :). Andrew Braybrook had a fully parallax scrolling landscape in Uridium in 1986, although the C64 had a better system for sprites. But the C16 could only do 8x8 character-size blocks, like EGA, and coders still managed smooth scrolling for games like Skramble in 1984.
    – Graham
    Nov 2 '21 at 8:55
  • thanks for your elaborate answer, i'm marking it, but more because of your third paragraph. I also saw your new linked post. Indeed I guess it depends on what the exact definition of the term is, and if it relates to the EGA hardware specifically. Developers for other systems also used a lot of hardware tricks, and I don't feel it's that important if there's scrolling in hardware or software, the point is either way to handle redraws and "refreshes" intelligently, i.e. just partially, and "adaptively".
    – scrollbear
    Nov 7 '21 at 19:35
  • FWIW, I rejected a pending edit to this answer from @German because in part it changed it from English to American spelling. There were other good changes in there, but I don't think it good form to change the author's preferred and correct spelling Nov 8 '21 at 22:43
  • Thanks for the polite notice. :) looks like someone else approved it. No problem
    – knol
    Nov 9 '21 at 8:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.