TL;DR
Did historical sprite systems provide unrestricted positioning and overlap
It wasn't unlimited and unrestricted, but limited by chip resources or memory bandwidth - or in case of inbetween systems by both.
because the designers believed this was very valuable in reducing game development cost?
No. Keep in mind, they often crippled machines quite hard to save a few gates.
[...] and once you are implementing sprites at all, the cheapest implementation provides unrestricted positioning and overlap more or less automatically?
Yes, but:
With off-chip sprite data the number of sprites is only limited by the RAM addressable (the way chosen to do so), but the number of sprites in a given time frame (like a line) heavily depends on the memory bandwidth available. The 9918 for example allows 32 sprites, but only 4 of them may share any given line.
With on-chip memory the main limitation is the amount of chip space than can be afforded for storage - not much back then. Prioritization and access effort grows linear with the number of sprites available, but delivers free overlapping of any number of sprites
Well, and then there is the "inbetween" selected by some manufacturers:
On-chip memory holds only management information (state, coordinates, etc.) but display data is held in off-chip RAM. The examples were Commodore's VIC systems. This combines the disadvantages of both. While it does allow full range checking, the number of sprites displayable within a line is limited by RAM access available.
The number of sprites only scales up to how much data can be read within a line. After that, additional measures for priority management would be needed, increasing cost while delivering a diminishing benefit.
The Long Read
[Prefix: No need to be a hardware buff - the whole issue in hardware is much the same as in software. It's about optimized data structures and the time needed to process the data, and the more straight forward processing is, the more can be done in a given time. In contrast, hardware can parallelize certain tasks already at low/simple level, gaining performance in areas software can't follow]
To start with, it's important to keep in mind that sprites are not something great to have, but rather a way to position objects on a pixel base on systems that lack a memory (and processing) bandwidth to do so in plain bitmap. Still, why being a crouch to help around this, the are as well limited by the same mechanics - just, due different access patterns, in a different ways than the main CPU.
The basic process for a controller with multiple sprites is to decide which sprites to display on a line and in which order. This means going thru coordinate storage and comparing to find overlap and sequence within a line, then moving relevant data on bit level into an output register, mixing them, only keeping the highest priority one in case of collision and outputting it.
In general there are two major hurdles:
- Memory access within a certain amount of time and
- memory addressable when doing so.
Memory to hold sprite data can be used in two ways:
- General purpose screen memory.
- Specialized on chip memory.
While sprite controllers using general purpose memory do allow a large, almost unlimited number of sprites to be handled, their operation is rather slow because each item has to be fetched over and over again.
The very common 9918 family is probably the best example here. Despite being one of the very first designs, it can handle a whopping 32 (monochrome) sprites within a picture (*1). But not more than 4 of them can be (have parts) within the same scan line. This is simply due the fact that there isn't enough time to check all coordinates and fetch all display data to do so.
The most restricting factor is RAM bandwidth, as it defines how many checks can be made.
[Collision detection, BTW, is just a side effect of drawing them on a line, when data is touched for drawing its coordinates needs to be checked when to insert and in which order. This automaticly implies a check on pixel level, all needed in addition is a comparison on each pixel when drawn if there were two sprites active.]
Sprite controllers with dedicated, on-chip memory (registers) in contrast can reduce this stress and offer faster access or even parallel processing.
This comes at a cost of making (relatively) large on-chip memories. An 8x8 monochrome sprite needs 64 bits of storage. With decoding and buffers its safe to assume 10 transistors per (static) bit. In addition at least another 18-24 bits are needed for coordinates and control storage, resulting in more than 500 transistors (*2, 3). So 8 sprites would already eat up the whole budget of a 6502 type chip. Just for storage, without any display logic at all. Using the 6845 as base, it will be about the same - and we still haven't the most important part added: selection and priority logic.
So first the most restricting factor for sprites is RAM size, but selection and mixing will add as well.
First step to placing a sprite is selecting if one (or more) of them is to be inserted on the actual line. This can be done by:
Cycle thru all coordinate storage, find all the sprites that match the line and merge them. This is essentially the same as with off-chip storage. Except it can of course be done faster and storage can be organized in a way to allow more direct access.
Using tagged addressing to for sprite line. This means instead of some comparator looping over all sprite line entries to see if it's the actual line, each sprite gets a separate comparator for each of it's lines, firing if this line is the same as the actual one. This is very fast. Results are gained in parallel and basically instant. It does allow (in theory) an unlimited amount of sprites. Except, it's extremely costly. A comparator needs an effort similar to a storage cell. Adding one to each sprite line effectively doubles the chip size needed.
Using a tagged and indexed addressing. Here only the top line coordinate of a sprite is held in tagged memory. When its comparator fires, a counter is started with the sprite height marking it over the following lines as active.
Mixing is again done in a similar way, but now according to horizontal (pixel) position. Again each sprite needs to be checked when to start supplying pixels. Again a tagged construction will give the best results. But now the addressed pixels need as well to be selected and mixed according to priority.
The most versatile solution is to give each sprite a read out register. Whenever the pixel position equals the X coordinate of a sprite its active line is copied and addressed as the pixel stream advances. To address a dedicated pixel counter or a shift register can be used. Either way, this needs as much read out circuitry as there are sprites.
Additionally a priority logic is to be applied to see which of the pixel streams (background and all selected sprites) takes precedence. Such a logic is a rather simple network of switches. Simple, but space consuming as all of them have to be connected. Here lies a further way to spend (or save) on chip estate: the network gets way more simple if the assigned priority is fixed with each sprites storage. Variable priorities require a lot more connections.
[Visible or not is BTW a simple addition, as it just means sending 'transparent' thru the network during the whole sprite duration.]
Now, at first sight there might be a way so save some readout/shift register, by having them global and only load them with the top priority overlapping sprites. It would save on the registers and even more on the priority logic, as it now only needs considering the smaller number of high priority sprites.
Then again, the original priority logic is still needed to find the multiple top priority among all sprites - plus somewhat more complex logic is needed to load, reload, and eventually even rearrange the registers on the fly.
As a result, reducing the number of active sprites will not save but increase cost.
Bottom line: The main cost for on-chip sprites is storage, everything else that follows is linear. There is no gain in restricting functionality, making numbers of sprites limited by RAM available.
Are Sprites a Good Idea at all - or why did they vanish?
The underlying mechanics can easily be seen by how (2D) graphics evolved.
First there was character based display, when moving only one byte per 64 pixels was already a considerable task.
Next the definable characters were added to allow (comperable) high resolution graphics. Kind of a compromise between full scale bitmap and character based (indexed) graphics, keeping memory bandwidth needed for manipulation mostly down.
About the same time sprites were added to allow fast manipulation of a few pixel wise moveable objects.
When memory not only became large enough to house full size bitmap, but as well fast enough to allow in time bitmap manipulations, sprite based controllers got replaced by specialized DMA engines called blitters. These could perform full bitmap manipulation faster than the still rather slow main CPU, while getting rid of all sprite based limitations.
Finally, when main CPUs got fast enough to not only handle its program, but as well sufficient data in time, all of this got abandoned in favour of plain bitmap.
Sprites always have been a substitute to circumvent (memory) speed issues.
*1 - Of course this number can be even further increased by exchanging them on the fly.
*2 - Motorola's 6810 128 Byte RAM had ~7000 transistors, or ~7 per bit.
*3 - Yes, this can be reduced by using dynamic storage, which would save a lot on the storage side, but at the same time needs refresh logic. Refresh logic doesn't scale with size. So while it is almost insignificant with large memories, it doesn't do well for small amounts.