23

"Full-motion video" sequences in PC and console games in the 1990s were bad-looking, in spite of taking huge amounts of storage space. Same thing with random short video clips that I obtained on a burned CD-ROM in the late 1990s. Huge files with really bad quality.

It was also common to see gigantic WAV files with no compression at all. This in a time when disk storage was measured in megabytes, and you ran out extremely quickly. This has long puzzled me.

Why, when storage was the most expensive, was the video and audio the least compressed? You would think that they would have done what hardware modems did: figured out ever more intricate ways of pushing more bits faster through the telephone copper wires using novel, hardware-based compression techniques, in numerous different versions over just a few years.

Not to mention those ultra-wasteful BMP files which had zero concept of compression, basically the extreme opposite of a PNG.

You would think that this would have necessitated the invention of very clever formats and algorithms, in order to store anything to speak of on the small storage medias and slow Internet connections of the time. Yet this was apparently not the case.

How can this be explained?

25
  • 103
    There was not enough computational power to allow compression we have now.
    – Vlad
    Nov 6, 2022 at 20:44
  • 18
    The PCs were not fast enough to decode clever audio and video compression in real time. And there were hardware cards you could add to your system to play DVDs if you wanted to do that.
    – Justme
    Nov 6, 2022 at 20:52
  • 11
    @manassehkatz-Moving2Codidact PNGs are very compressed, but lossless unlike JPEGs.
    – Romalis
    Nov 6, 2022 at 20:57
  • 23
    An algorithm is like a good idea: unusable if you don't have the means to execute it.
    – DKNguyen
    Nov 7, 2022 at 5:10
  • 17
    Just having video on a personal computer was impressive. There was also a goal in filling the new gigantic CD-ROM media so it was more difficult to pirate (and these days modern SSD's read the amount of data on CD-ROM in a fraction of a second) Nov 7, 2022 at 8:12

6 Answers 6

67

TL;DR Computer processing power/speed/cost and storage density/cost have been moving at roughly the same pace for 50+ years.

There are of course variants, where either CPU speed has increased much faster than storage capacity or where storage capacity has increased faster. But generally speaking, they both have come a long way.

Keep in mind that the early days of compression in modems used CPU chips comparable to those in computers, but the speed of the modem was relatively low. Even a 56kpbs modem, basically the top of the line in analog modems, only had to process - send/receive/compress/decompress - at roughly 57,600 bits per second. That is comparable to the speed required for high-quality audio, and far less than that required for video. Video at 640x480 at 8 bits/pixel is already in the megabits range, and bump it up to 30 frames/second and it is over 70 megabits per second! That's more than 1,000 times a 56kpbs modem.

For some uses, compression has long been practical. File compression was already common in the mid-1980s. But running a command to compress/decompress a few megabytes when needed is very different from compressing/decompressing a video data stream in real time - i.e., fast enough to read from a hard drive, decompress and output to a display without noticeable delays.

To put it another way, compression was the poorest when storage space was the costliest because CPUs that could do the job were either the costliest or simply not available. Yes, a supercomputer might have been able to do the job, but not an ordinary 8-bit or 16-bit computer.

12
  • 13
    In addition, decompression of a highly compressed data stream can require significant amounts of memory. Nov 7, 2022 at 9:51
  • 12
    Just to give an example: My Amiga 1200 home computer in 1996 was not able to play MP3 files because its 14 MHz CPU was not fast enough - it could only play MP2 files. Yes, there was an MP2 format that was lower quality but also faster to decompress. The same computer took several seconds to decode and display a 1000x1500 pixel JPG image. Nov 7, 2022 at 10:23
  • 14
    The OPs question contains dubious claims, like "CD-ROM was an expensive storage medium". Exactly the opposite was true - suddenly you could deliver hundreds of megabytes of data on a mass-duplicated plastic disc. What to even do with so much data? The question also assumes that CD-ROM producers knew what they were doing, or that they cared beyond getting something working. Or that compression software was available to anyone who might be tasked to add video and audio on a CD-ROM. Or that it could be used without any extra cost. Did the video even look that bad to contemporary eyes? Nov 7, 2022 at 13:14
  • 4
    Not just video — even still images took significant time to decode and display on early 16-bit* home machines. (IIRC, it was significantly quicker to load an uncompressed image from floppy disk than to load a compressed one and decompress it. Even more so from hard disk, once they became common.) (* IIRC, there weren't any common compressed image formats for 8-bit machines, probably due to the limtations and variety of their display modes as well as OSs. Floppy-based programs would load images uncompressed; tape-based ones tended to draw images programmatically.)
    – gidds
    Nov 7, 2022 at 15:18
  • 6
    @MichaelBorgwardt Hell, even on a ~200MHz Pentium circa 1998, MP3 decoding in real time still took a sizeable chunk of CPU time (maybe 20% or so) which meant that MP3s were generally not usable in CPU-hungry games. This was also before common ISAs gained special signal processing oriented instructions. In 1996, only those few lucky and/or rich ones to own a Pentium or a fast 486DX could even dream of decoding MP3s in real time at all.
    – JohannesD
    Nov 7, 2022 at 22:48
42

You mean beside that all these algorithms had to be designed, implemented, rolled out and accepted first? Which of course would only happen as a need was developed to use them?

Well, that may leave the major point of all of them requiring large amounts of computing power at the decoding stage - not to mention encoding as well.


In the end it was the usual timely interaction of demand and supply.

(Here one focusing on popular use, not invention or use in professional systems - after all, sound compression was available since the 1970, likewise bitmap, as for digital fax machines.)

At the beginning there were neither

  • large amounts of data to be compressed, nor
  • storage to hold it, nor
  • any video hardware to process the amount of data video would for example require
  • an environment for standardized solution, and especially
  • neither memory nor processing power to do anything but byte counting

So during the 1980s there was simply no need and no way for complex compression to exist. Games and alike used data specific programmed methods, while text played with straight reduced data types (6 bit instead of 8 bit character data - like Zork and others).

All that was meant to squeeze a bit more out of standard media at the time, which where floppy disks anywhere between 100 and 500 KiB during most of that time. The latter being expensive, rather professional systems.

In the mid 1980s systems with 700 to 1200 KiB per disk became more available (Atari ST, Commodore Amiga, PC-AT), seemingly huge for a few years, but still not really good to even hold a few seconds of low resolution video - even with today's level of compression. While this was also the time hard disks became more affordable, data transfer was still bound to diskettes (*1).

Then again, the mid to late 80s were when first text compression became widely available. Programs like SHA's ARC, PKWARE's ZIP or LHarc popped up, allowing to compress text fairly good and take out at least some slack of other formats.

At the same time something so common we hardly notice made inroads: TIFF and ultimately GIF as a way to compress drawings made its way onto our computers. TIFF notably for adapting the professional RLL compression used in FAX machines.

It was, as you already mention, the CD that finally opened the exchange of large amounts of data in easy to handle chunks. Suddenly exchange size jumped from around 1 MiB to 650. That's a size were it suddenly became possible to have (more or less) high quality video. At least several minutes.

It was the Sound Blaster Pro with its (comparably) cheap Mitsumi drive that enabled CD-ROM usage for PCs in 1991. Of course it took a few years until they became so widespread that it was worth for content creators to produce for CD.

The Video-CD of 1994 that finally introduced Video to PCs, using MPEG-1 compression - albeit at a rather reduced resolution of 288p (352x288). Except, most PC were not really fit to handle decompression. It wasn't until late 486, early Pentium machines that they could do so. Unless one was willing to buy a dedicated MPEG decoder card.

It was a time when compression started to focus on audio and video.

What followed was a cycle of

  • higher performance processors
  • dedicated decoder circuits in graphic cards
  • larger media
  • better compression
  • higher resolution

25 years of rinse and repeat later playing online full screen HD video is a standard task for next to every up to date PC.


Bottom line: Today's compression wasn't invented on a single day. It was an ever increasing cycle to improved hardware, software and media to handle ever more data and improved quality.


*1 - No, modem transfer wasn't really a thing. even with top end PEP modems a Megabyte is still some 10 minutes of transfer time. With common 2400 bps modems it was more than an hour.

13
  • 3
    And development continues. The standard compression techniques of today will be retrocomputing soon.
    – gerrit
    Nov 7, 2022 at 7:18
  • 1
    @supercat My guess would be resilience against bit errors. Nov 7, 2022 at 8:13
  • 3
    It's worth noting that the theory of audio or video encoding was developed in the 1940s by Claude Shannon. It would be a few decades before the practice caught up with the theory. Nov 7, 2022 at 11:39
  • 1
    Still not really good to even hold a few seconds of low res video is a bit of an over-statement. With codecs like h.265, you can get down to very low bitrates and still have usable video, like 720x480p30 at 134kbit/s. It doesn't look great, but ok for some DVD extras for example. With audio, a minute of video fits in 1.5MiB. (And the audio is 64kb/s AAC, could be much lower like 10k or 16k Opus.) That's low enough bitrate you might even stream the data from a floppy in real time without much pre-buffering, and can be lower. A shorter clip at higher bitrate would of course look better. Nov 8, 2022 at 7:35
  • 1
    Of course that's just an unimportant nitpick as far as retrocomputing is concerned, h.265 takes significant CPU horsepower to decode, and plenty of memory or cache bandwidth, and space for the reference pictures. Pure software decode with ffmpeg (without screen display) with -threads 1, Skylake at 3.9GHz decodes 1024x576 h.265 at ~630 fps, so 15fps playback would take a ~100 MHz Skylake core, with AVX2 and fast caches, not including screen output. So fun fact, modern video codecs are usable at surprisingly low bitrates, where older would have to drop frames and be massively blocky. Nov 8, 2022 at 9:01
23

Compression == Expensive

To get an idea of what was considered an acceptable cost for compression, you should take a look at the PCX format, which was common for games and graphics programs of the DOS era. It uses "Run-Length Encoding" which just says: "if the next sequence of bytes repeats, replace the sequence with a count and the repeating byte". Obviously, that only works for the most trivial or low-color images (imagine a background with a solid color). It has the advantage that both compressing and decompressing can be done on the fly at nearly zero cost.

If you look at GIF, you see that it only supports up to 256 colors. That's because the LZW compression scheme that it uses would not be effective on a full 24-bit color image. In a world where 8-bit color was common, GIF offered a reasonably quick to decode image compression scheme.

But as soon as you switch to JPG, the complexity ramps up dramatically. It basically uses Fast Fourier Transform to compress the image data, and this essentially requires a significant number of multiplications over every pixel in the image both for encoding and decoding. This is why downloading a JPEG over the early public internet was a painfully slow process: the transfer of bytes was slow, and the rendering itself was slow (just try opening a bunch of JPEGs rendered as thumbnails on a 90's era desktop).

Now, when you consider that MPEG is basically JPEG applied 30x per second, you begin to see the complexity and computational intensity involved. Of course, MPEG is even more complex than stringing a bunch of JPEGs together. It takes the difference between successive frames and compresses that, which is extremely computationally intensive. While the decompression is much simpler, it still requires a lot of CPU cycles. You have to remember that in the 90's, CPU clock speeds were still measured in MHz, and performance was counted in MIPS. If you wanted even a 10 FPS video, each frame had to render in 100 ms. If you have a machine that can put out 100 MIPS, you have to render each frame in 1 million instructions or less (definitely less if there's audio). It will take dozens to hundreds of instructions to recover each pixel via a decompression algorithm, which means your video has to have much less than a million pixels...more like on the order of tens of thousands. And that's why the videos are tiny 320x240x8 or smaller.

You have to recall that most CPUs were single-issue, with the Pentium being one of the first major consumer chips to have multiple instruction pipelines. Graphics cards didn't come with massively parallel 3D render pipelines like they do today. They mostly contained video RAM for the frame buffer and enough compute to push bytes around between the CPU and VRAM. OpenGL and Direct3D was almost always rendered in software.

What is surprising is not that the video was so bad, but that it was available at all, given the nascent maturity of the technology at the time. It all had to start somewhere. And for home computing, full-motion video basically started in the 90's. It should be no surprise that the first such video did not pop out fully formed at 8K resolution running on curved 120" 120 Hz displays backed by 32 GB of VRAM and hundreds of 3D render pipelines. Just like the first production cars did not start out shipping with Bugatti W8 1000 HP racing engines and quad-cylinder ceramic disc brakes.

7
  • 1
    Doesn't JPEG use Discrete Cosine Transforms?
    – wizzwizz4
    Nov 7, 2022 at 18:16
  • 7
    @wizzwizz4 DCT is morally equivalent to FFT/DFT: en.wikipedia.org/wiki/Discrete_cosine_transform Nov 7, 2022 at 18:58
  • 3
    The OP wasn't expecting 8K rez @120Hz, they were hoping for stuff like ffmpeg -c:v libx265 -preset slower -b:v 100k -c:a libopus -b:a 12k (Try it with -vf scale=640:-2 or 1024:-2 to downscale to horizontal 640 or 1024; it's surprisingly watchable for such an extremely low bitrate). So with a fast floppy disk reader that can manage somewhat over 112kbit/s, you could stream a minute of video in real-time from a floppy disk in real time at, at 640x360p25 or 30 for example, if you had the CPU or GPU power to decode in realtime. (Like an 800KiB file for a minute of video). Nov 8, 2022 at 8:30
  • (h.265 can use such large DCT blocks, up to 32x32, that higher resolutions can be better even if the bitrate is way too low to make each pixel look like the source. Surprisingly, 1024x576p25 looked less bad than 640x360p25 in a random video I encoded. Everything is blurred, but some large scale details are a bit sharper at higher rez. Lowering the frame-rate helps significantly, leaving more bits per pixel per frame at fixed bitrate, even though similarity between consecutive frames is lower.) Nov 8, 2022 at 8:36
  • 2
    @PeterCordes I've just tried, and with two USB floppy drives, it's possible to read each video segment from alternate floppy and play them in the same window, seamlessly one after the other, swapping floppies as you go. Now if only we could have an i3 back in 1995 :) Jan 21, 2023 at 0:56
6

CPU/RAM decode performance, and engineering time to develop better compression algorithms and tune them. e.g. zstd in the last few years is like 10x faster than gzip, but compresses about as well. Great example of how new ideas can lead to better efficiency, separate from just ways to throw more compute horsepower at the problem. (Although that does help massively for multimedia.)

For lossy compression of audio, lots of research was done on what sounded less bad to humans. That took time. MP3 was the first format that was "good enough", using DCT and quantization like JPEG for images, but also using models of human hearing to figure out which sounds the ear / mind wouldn't tend to notice anyway, and could be dropped without affecting perceptual quality. That "psychoacoustic" optimization only really affects MP3 encoders; I don't think that makes decode any slower. (MP3 being slower than MP2 is due to other factors; I don't know the details.)


For video, throwing around a significant number of pixels every frame (the result of decompression) gets expensive in memory bandwidth and space, even if you have dedicated decode hardware. e.g. 1024x576 at 24bpp (RGB) is 1.69MiB just for one decompressed frame, and a decoder needs to store multiple.

(Assuming square pixels, pixel count increase with the square of width or height, so lower rez saves a lot of work, and space.)

But HW decode is an unlikely assumption; I think most people wouldn't want to pay a lot extra for the capability to play back the occasional video, especially before the CD era when storage finally got big enough and fast enough. And make no mistake, video decode hardware that fits in a small block of an iGPU these days might be pushing the limits of technology in the early 90s, and wouldn't be cheap, especially with enough RAM.)

Modern video compression formats (like h.264 and h.265) are quite expensive in CPU even to decode, since they do things like large DCTs (16x16 for h.264 high profile or 32x32 for h.264), and copying blocks of pixels around as references. Also deblocking filtering even inside the "loop", i.e. to produce the picture that can be a reference for later frames.

(In-loop deblocking was one major feature that set h.264 apart from earlier video codecs like DivX / Xvid (mpeg4 part2). Some decoders, like ffmpeg, have a fast-decode option to skip loop-filtering, producing error accumulation until the next I-frame but can speed up software decode some. The higher the bitrate, the less aggressive filtering the encoder would normally request, so it could still be watchable.)

Dedicated hardware decode would help, if even that was doable at the time transistor densities. But probably not, even if practical it might cost a significant fraction of the whole computer, especially with the amount of fast memory that would be needed for holding reference frames.

Modern video formats work by predicting the pixels of one frame from a nearby similar block in an earlier frame. (If it's very similar, maybe just code a motion vector, otherwise that + a residual to be added). They can ask the decoder to hang on to some number of previous pictures, like maybe 4 or so is a common number, and reference any of them with any macroblock. Taking advantage of redundancy between frames (temporal) is huge for quality per bitrate, instead of coding all the details of each frame from scratch, especially if it's a complex but static background, and especially at lower bitrate where you're prepared to accept some artifacting. But it requires more memory in the decoder. And requires motion-search in the encoder to find good candidates out of a huge number of possibilities. Encoding doesn't have to be real-time for this use-case, but more than a few days to encode a few minutes of video would be problematic.


Since other answers mentioned floppy disks being inadequate even for modern compression standards, I though it might be fun to see if that's true. According to an SO Q&A, a good floppy drive could read at maybe 100k to 250kbit/s, presumably talking about 1.44MB 3.5" high-density floppies that were the last mainstream format. So I aimed at the lower end of that bitrate.

With codecs like h.265, you can get down to very low bitrates and still have usable video, like 720x480p30 at 134kbit/s. It doesn't look great, but ok for some DVD extras for example. With audio, a minute of video fits in 1.5MiB. (And that's 64kbit AAC, not like 12k Opus.)

h.265 at 100kbit/s at 1024x576p 15fps - usable quality but decode would take a single core 100MHz Skylake or 400 MHz Core2

If you assume all the engineering time and effort had been put in to develop modern (2013 to 2022) video codecs decades earlier than happened in practice, it would take more horsepower than CPUs had at the time.

Modern video codecs are usable at surprisingly low bitrates, where older would be massively blocky and maybe have to drop frames. It doesn't look good, but 100 or 200 kbit/s (floppy disk bandwidth) is better quality than old 240p MPEG1 videos at 800 kbit/s.

I extrapolated from timing ffmpeg -threads 1 -i foo.mkv -an -dn -f null - at about 630 fps on my i7-6700k desktop with dual channel DDR4-2666, running Linux with ffmpeg n5.1.2. That decodes the video as fast as possible, but doesn't display it anywhere or do anything with it. (It doesn't even convert to RGB, just leaving the decode result as subsampled YUV 4:2:0. It's plausible that video hardware can do that conversion or use the format directly, in an alternate universe where more dedicated HW support for multimedia stuff existed)

15 / 630 * 3900MHz is about 93MHz. Round that up to 100MHz, and that's ignoring any CPU time to copy the decode result to video RAM, or decode audio, or run a UI.

Intel released Skylake in 2015. It's a 4-wide superscalar out-of-order CPU that can get a lot done in one clock cycle. Especially with AVX2, processing SIMD vectors of 32 bytes per instruction, with 3 execution units that can handle AVX2 integer instructions. Quite a lot of person-hours have gone into hand-written vectorized asm to accelerate ffmpeg's software-decode of popular video formats.

Skylake also has large fast caches, like per-core L2 cache of 256KiB, and mine has an L3 cache of 8MiB. Data paths of 64 bytes between L1d and L2, and 32 bytes on the ring bus between L2 and L3. In the mid 90s, having a 64-bit external bus and access to cache with that width was a big deal (P5 Pentium). Skylake load/store execution units are 256-bit wide. (External DRAM is still only 64-bit wide, but dual channel, and caches often insulate the core from DRAM.)

Single-threaded software decode on my system runs at about 630 to 635 frames per second, with Linux's energy_performance_preference hardware P-state tuning setting set to balance_performance. This makes it only turbo up to 3.9GHz, not the full 4.2GHz the CPU is rated for. (But the fans stay silent.) So lets say you were aiming You might aim for decode at say 15 frames per second, to play back at that frame rate. 15 / 630 * 3900 = 92.8 MHz, assuming linear scaling of decode performance with clock speed. (Which should probably be true if you slowed down the DRAM in step with the core clock, but kept the CPU core and caches the same size so they'd manage the same instructions-per-clock. i.e. totally impractical for the 90s.)

So with the complexity of a Skylake, you'd still need to hit 100MHz to decode that video in real time. With lower complexity, you'd need to bump the clock speed up significantly. Lower rez would help but wouldn't be that much of a factor.

On a 2.4GHz Core2Duo (E6600 Conroe from 2006) with DDR2-533 DRAM, I got 92fps for the same test. (It has SSSE3 and 128-bit load/store data paths, but slower pshufb.) 15/92 * 2400 MHz = 391 MHz. So you might need a 400 MHz Core 2 to decode that video.

Lower resolutions are faster

Decode cost seems to scale linearly with number of pixels, e.g. on my Core2 with 360p (240fps) vs. 480p (117fps) vs. 576p (96 fps) for very low bitrate h.265 on a single core. So lower resolutions are much cheaper, since keeping square pixels, pixel count goes up with height-squared (or width squared).

This makes sense: every macroblock of the output has to be handled by copying from the reference pixels somewhere and then applying the iDCT residual if there is one. (Most blocks are inter-predicted from a previous frame or intra-predicted from earlier blocks in the same frame, but some are pure I blocks that are just inverse DCT). Anyway, each output pixel needs a value, and that has to come from some copying + processing work that scales approximately linearly with number of pixels, especially with 8x8 or larger blocks so SIMD can mostly work to full effect.

And h.264 is about 4x faster to decode than h.265 on Core2; some of that may be better software optimization for CPUs without SSE4 or AVX2 in h.264 decode vs. h.265. That CPU was already obsolete when h.265 was new, but was current when h.264 was new. (And such optimizations plus parallelization were actually needed for good software decode of 720p or 1080p video, especially in the days before GPUs commonly had fixed-function HW decode blocks, including on even older CPUs like AMD K8.)


Encoder settings

ffmpeg -i foo.mp4 -c:v libx265 -preset slower -b:v 100k -c:a libopus -b:a 12k -vf fps=10,scale=-2:576  '/tmp/foo 1024x576p10 x265.mkv'

-preset slower tells it to use a lot of CPU time finding better quality-per-bitrate ways to encode. The -2 for horizontal size specifies that the width should be calculated from the height, keeping the aspect ratio, but rounding to a multiple of 2.

h.265 can use such large DCT blocks, up to 32x32, that higher resolutions can be better even if the bitrate is way too low to make each pixel look like the source. Perhaps surprisingly, 1024x576p25 looked less bad than 640x360p25 in a random video I encoded at 100kbit/sec. Everything is blurred and distorted, but some large scale details are a bit sharper at higher rez. (Each DCT block has fewer bits, but conversely it's smaller so distortion at that scale is less visible when the whole image is larger. DCT transforms the information in the image from spatial domain to frequency domain, like an FFT, so quality at a fixed bitrate can be quite similar over a range of resolutions, dropping as you get too low and everything blurs, or you get to high and overhead costs more of your bits leaving fewer for details. And bits per block gets so low that quantization has to get really aggressive.)

Lowering the frame-rate helps significantly, leaving more bits per pixel per frame at fixed bitrate, even though similarity between consecutive frames is lower.

Dropping the frame-rate down to 10 fps improves quality vs. the original 25 or 30 fps, leaving more bits for each frame.

200kbit/s at 854x480p15 is of course a lot better than 100kbis/s, getting to the point where it's possible to not be constantly distracted by the amount of blurring, blockyness, and other artifacting, i.e. that one can kind of watch the video.

h.264 is quite a lot worse at this bitrate. When they claim that h.265 can use half the bitrate of h.264 for the same quality, I think they're probably talking about really low bitrates, like video-conference use-cases or lower, so quality is nowhere near transparent. At high quality, there's no substitute for bitrate; texture details just don't get preserved by any currently available fancy codecs even at high settings. x265 is somewhat better than x264, but not twice as good. But when codecs are already having to accept compromises and make detailed surfaces look like smooth plastic, the new coding tools in h.265 are more useful I guess.


Encode is barely real-time even at 10fps at 576p on a quad-core Skylake, when I use a very high quality-per-bitrate setting (-preset slower with x265 3.5). On my old Core2Duo E6600, 1080p video encoded at multiple seconds per frame with x265 -preset slow, about 40x slower than my quad-core Skylake.

And it takes a lot of RAM to do a good job encoding, keeping lots of frames around to search. Given the coding tools of a modern codec like h.265, encoders that can use them to get a decoder to produce good pictures are still necessary. With technology at the time in the early to mid 90s, I suspect you'd be running into very serious encoder limits in how much quality they could achieve given RAM size limits, even if you were willing to wait days to get a 3 minute video encoded.


TL:DR

This answer was meant to be just a data point about decode horsepower required to play an h.265 video in real time. That's why I didn't also time h.264 or divx, which are more reasonable in terms of CPU requirements.

The other key point I'm making is that I think even hardware decode with custom ASICs would be impractical for 576p or 480p h.265 in the early 90s, with chips the size and clock speed of contemporary RISC CPUs like MIPS or Alpha. Maybe I'm wrong about that and it wouldn't need a chip running hundreds of MHz, but I think it would still be pricy enough to have small market demand, not something that could just get tucked into existing video cards for minor extra cost.

And that algorithms / formats take time to develop, as we gain experience with what works (i.e. what encoders can use effectively while still encoding fast enough to be usable. Offline encoding is fine for many use-cases, even 10x slower than real time. But hundreds or thousands of times slower than real-time is a problem for turn-around times from edit to release. And again, an encoder has to be capable of looking at lots of frames at once to take advantage of some of the capabilities of modern formats.)

3
  • 2
    "100 Mhz SkyLake" - so that sounds pretty much like the first TriMedia core of Philips, which was a dedicated VLIW videoprocessor back in the 90's. Considering that we had sound cards with CD connectors back then, a GPU card with a DVD connector and a TriMedia processor sounds like something that would have made sense. Just bypass the CPU entirely.
    – MSalters
    Nov 8, 2022 at 14:51
  • @MSalters: Yeah, en.wikipedia.org/wiki/TriMedia_(mediaprocessor) says it was pretty capable, 5 to 8 issue, and with SIMD (of some unstated width). And up to 128K data cache. Oh, but that was by model year, with the 1997 model having 16K of cache. It wasn't until the 2005 model that it was stated to be h.264 capable, with the generation after that getting specific features to help accelerate decoding the bitstream (probably meaning CABAC, the lossless final stage that encodes they symbols, kind of like a final GZIP of the sequence of what-to-do-with-each-block messages.) Nov 8, 2022 at 15:33
  • 3
    There's the obvious time machine problem of H.264 being introduced only in 2004 :) My point was that companies like Philips did develop dedicated processors which were quite capable of decompressing contemporary video formats. The die shot does show the cache problem; about half the processor is dedicated to cache and that's just 16+32kB.
    – MSalters
    Nov 8, 2022 at 16:09
4

Expanding on Raffzan's answer in one regard:

You mean beside that all these algorithms had to be designed, implemented, rolled out and accepted first? Which of course would only happen as a need was developed to use them?

I am not an expert or historian, but every compression algorithm takes significant R&D money and time. For media usually this is some industry group trying to commercialize it (DTS, AC3, H264, and now H265 and AV1). That means there generally needs to be market speculation - Skype invested in Speex which became Opus. Netflix and Google invest in AV1. Broadcasters like H265 because they can fit more stuff in airwaves and over cable/satellite. H265 allows 4K HDR to fit on a 100GB Blu-Ray, which the movie studios like. DTS and AC3 allow home surround and theater surround in an acceptable format (uncompressed it would be 1-3 CDs in PCM format). MP3 seems to be a notable exception, being developed out of a research institution. (edit: See comment)

They all build on the lessons learned before, as well as take advantage of advancements in computing. You couldn't build AV1 in 1985 because it learns lessons from H264 and probably MPEG2 and everything that came before. Additionally, modern compression seems to use parallelism substantially. This was not possible in consumer hardware in the 80s and 90s.

You also assume a linear progression from "bad compression" to "good compression" but there are many more variables at work. I can't find a source, but I recall around 2016-17 that AV1 was targeting a 2x increase in decode complexity, and a 10x increase in encode complexity. For the major media streamers investing in it, this was an acceptable tradeoff for the streaming era. Only enthusiasts compress video at home, and probably 99% of all media consumed is sent from a central location. Such a tradeoff would've been bizarre in the home media era, before cloud computing centralized compute farms and made them available to megacorps as well as any startup with enough cash.

You see a microcosm of the fight between compression and processing every time a new format is taking hold. When scene release groups switched to h264 rather than xvid it obsoleted a lot of hardware and required more juice to compress and decompress/play.

Also, something like H265 which works best at >=4K resolution requires there to be 4K video to compress. No such cameras existed until the 90s and were not widely used until the 2010s. Scanning still images to 4K would've followed a similar, but accelerated timeline (Disney scanned Snow White in the early 90s).

1
  • 2
    re MP3: Fraunhofer only offered the player and a throttled version of the encoder for free. You had to pay to get a (legal) version with full encoding speed, and the patents protecting the software were in force until 2017 (more or less).
    – ccprog
    Nov 9, 2022 at 23:30
0

As other answers say, it's all about compute at the time.

When the original Command & Conquer came out, a high-end gaming PC would have a Pentium 100; the equivalent system today would have a Core i9-12900.

At best, that Pentium 100 did 2 instructions per clock, operating on 32 bits per instruction; if you limit to 256 colours, and you assume that a skilled programmer is (a) sustaining peak operations, and (b) fitting 4 pixels worth of compute into a single 32 bit operation (via "SWAR" type techniques), you get a peak processing rate of 800 million pixels per second. A 320x240 screen at 15 FPS is 1.15 million pixels per second, giving you roughly 800 operations per pixel.

The efficiency cores on the i9-12900 have AVX2, and can thus do 8 pixels at 32 bits per pixel in parallel. Those cores can sustain 1 instruction per clock on AVX2 code, and can sustain at least 1.8 GHz clock speed, for 14,000 million pixel sized operations per second - or 18x as many operations per pixel per second, with pixels 4x larger than the Pentium was handling.

And the efficiency cores are the slowest part of the chip for video decode; the performance cores, or the GPU, can do more pixels per second than the efficiency cores can (if you'd prefer to think about older CPUs, the i3-4025U is more capable per core than the i9-12900 efficiency cores).

So, by choosing the slowest compute option on an i9 for video, I get to the ability to do 640x480 at 60 FPS in 16 million colours, with a codec whose decompression complexity is comparable to one that requires your entire compute on a similar cost 1995 PC to do 320x240 at 15 FPS in 256 colours. I still have 7 spare efficiency cores, 8 performance cores, and an idle GPU to do other things; using all the cores, I'd have enough compute to do 1920x1080.

This is fundamentally what limited the quality of FMV in that era - you simply didn't have the compute to run a better decoder. And bear in mind that the numbers I've used are the absolute best case for early 1995, and the worst case for today - being realistic about what a 1995 PC could do would bring those numbers down, while being realistic about what even a 2015 PC could do would bring the numbers for today up.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .