What is the most accurate way to map 6-bit VGA palette to 8-bit?

Question

AFAIK VGA mode 13h palette has only 64 possible colors (6-bit) per channel.

One obvious way to map those 64 colors to 256 colors is to multiply them by 4 (since 4 * 64 = 256):

8_bit = 6_bit * 4;

This is however not accurate since biggest value is 63 and 4 x 63 = 252 not 255.

Some online ressources I found suggest to do the following:

8_bit= (6_bit * 255) / 63;
8_bit= (6_bit << 2) | (6_bit >> 4);

This is much better (63 is now mapped to 255). However those are not equivalent (it will not give the same result, for example 48 is mapped to 194 and 195 respectively).

I am wondering if someone knows how VGA hardware works (how those 6-bit values are mapped to analog signals by the DAC). The last formula seems the most plausible since it's fairly easy to implement in hardware.

Answer 1

The most accurate way of course depends on what is the purpose. No way is perfect and there is no one true method for all purposes.

First of all, how the VGA works, is that it used the INMOS G171 RAMDAC. Now, as it is a real world IC with real world tolerances, it is not that accurate, as it can have full scale error of 4% and 2% error between different channels, there are non-linearities, and even then, the other real world components around it that set the reference current have tolerances and non-linearities too.

But in an ideal world, the DAC simply has 64 ideal levels that are equally spaced and they are set to correspond to voltage levels 0V and 0.7V. At least on paper. The white level may not in theory be exactly 0.7V so it does not really matter what voltage it is in practice if it has some tolerance.

Now, if that signal is fed to a digital monitor (or TV) with VGA input, it has to sample the analog voltages and convert them to digital - with the imperfections of both the source and monitor being accounted for. Therefore, when VGA card outputs level of 63 for white, there is no guarantee that an 8-bit monitor samples it as 255 or 252, it may be that 63 is below 255 but it may also be that even 62 is above 255 so the information is lost. So, like said, the exact value is not that important.

When time passed on and PCs started to have 8-bit RAMDACs, they were still compatible with 6-bit colours, and could be set to just use 6 bits for VGA modes instead of 8 for VESA modes, and they just left the 2 LSBs as zeroes. So if a BT476 RAMDAC was set to output exactly 0.7V for 8-bit code 255, the 6-bit code would be 252 as 8-bit code and thus voltage less than 0.7V.

If you simply want to make an emulator or just a screenshot and convert 6-bit colors to 8-bit colors, just multiply by 4. So what, if white is (252,252,252) instead of (255,255,255). The same thing is done in modern digital video processing too, to convert 8-bit values to 10 (or 12) bits, e.g. 8-bit white level is 235 and 10-bit white level is 940. This guarantees that the step between each level is equal in size: 4, which is most accurate in that sense, and it is also easy to convert from 10-bit to 8-bit by just ignoring the two LSBs.

If for some reason you must map a (0..63) range to (0..255) or (0..100% for any bit depth) then any method is fine. It just means you get 100% white if you want, but steps are not always equal, as step is either 4 or 5. There is no reason to target for 255 anyway. Whether you truncate, round, or just use the 2 MSBs as the expanded LSBs, it only changes where the jump-of-5 codes happen, so any one of them is good enough and none of them is the ultimately best method.

Perhaps best to use the simple multiply by 4 method, as that is how cards with 8-bit DACs emulated a 6-bit DAC.

Answer 2

Justme detailed some of the reasons that the exact conversion method probably doesn't matter. Another reason is that you are probably going to interpret the 8-bit output according to the sRGB standard, which postdates VGA by a decade. I'm not so sure that VGA monitors of the late '80s had primaries or transfer functions that were very close to sRGB, or each other.

All that aside, in principle, to scale a 6-bit channel linearly to 8 bits, you should multiply by 255/63 = 85/21 and round to the nearest integer, which in a C-like language with truncating or floor division works out to

eight_bit = (six_bit * 85 + 10) / 21;

or equivalently

eight_bit = (six_bit * 259 + 33) >> 6;

Likewise, when converting from 24-bit to 48-bit color you should multiply by 65535/255 = 257, not 256, and when converting from 24-bit sRGB to a linear colorspace, you should divide by 255, not 256, to get a properly normalized value to plug into the transfer function.

Answer 3

[Preface:That question isn't really related to retro computing but hardware in general, it might thus be moved]

TL;DR:

Absolute brightness on screen is not defined by the signal, but by the screen's brightness setting (*1). For all practical means it's only important that the steps used are of equal distance and in total reasonably close to the desired range (*2).

For real displays it's a non-issue, thus no one would have cared.

The situation may be different if this is about intermediate handling - which the question doesn't seem to be about.

---

One obvious way to map those 64 colors to 256 colors is to multiply them by 4 (since 2 * 64 = 256):

Yes, that's the obvious and also usual approach. Although any hardware designer would see this rather as shifting than multiplying.

This is however not accurate since biggest value is 63 and 4 x 63 = 252 not 255.

True as well, but a real world application usually wouldn't care, after all, the maximum error is not only just 1.1% (3/256) but also linear, that is the error is continuous, increasing monotonously over all values, thus not changing any relations. This is most important as human vision is quite receptive to relative difference rather than absolute values.

Not to mention that absolute brightness is not controllable by the source signal, but only provided by the screen. That means it's only important that each of the encoded steps is of the same size.

I am wondering if someone knows how VGA hardware works (how those 6-bit values are mapped to analog signals by the DAC).

Well, that's rather simple: A VGA compatible output signal is 'encoded' as voltage of 0.0 to 0.7 Volt at 75 Ohm. That is 0.0 Volt for no colour, 0.7 Volt for maximum level. With 6 bits on the input side this means the voltage level is subdivided into 64 levels.

Some online resources I found suggest to do the following:

8_bit= (6_bit * 255) / 63;
8_bit= (6_bit << 2) | (6_bit >> 4);

Both are great approximations - if one really wants to close in on that 1.1% (*3).

This is much better (63 is now mapped to 255). However those are not equivalent (it will not give the same result, ...

Both reduce the error to about 0.4% (*4), but at different points. That's because the first is an arithmetical approximation, while the second is a mapping in 4 groups. The arithmetic distribution is rather continuous, while the mapping distributes the error in discrete intervals. They are overall the same, with a similar result, but not of course not the same.

... for example 48 is mapped to 194 and 195 respectively).

Well, it's about spreading, making three of the 63 steps 5/256th instead of 4/256th.

Arithmetic Mapping	Group Mapping
20/ 80	15/ 60
41/165	31/125
62/250	47/190

The last formula seems the most plausible since it's fairly easy to implement in hardware.

Yeah, as the mapping

 0..15 ->   0.. 60   (or In * 4 + 0)
16..31 ->  65..125   (          + 1)
32..47 -> 130..190   (          + 2)
48..63 -> 195..255   (          + 3)

is simply done by routing:

In1 -> Out1 -> Out7
In2 -> Out2 -> Out8
In3 -> Out3
In4 -> Out4
In5 -> Out5
In6 -> Out6

No electronics needed :))

(In1..In6/Out1..Out8 are bits in decreasing value)

---

*1 - For practical purposes the brightness setting of a screen can be seen as an analogue mapping of the input signal toward the screen.

*2 - Being close is all what counts, not how close, as all analogue circuitry is created to work best with a value around that 0.0..0.7 Volt range.

*3 - Since brightness can easily differ by 20% between screens, not to mention different curves of adaption, looking for where to place those 3 uneven steps is in reality rather useless.

*4 - By being quantized that's the lowest possible error anyway (1/256).

Answer 4

I would use simple linear search to construct a conversion LUT with minimal error. Here is simple C++ code for conversion from 6-bit to 8-bit:

uint8_t LUT[64];

int i,j,jj;
float ai,aj,d,dd,di=1.0/63,dj=1.0/255;

for (ai=0.0,i=0;i< 64;i++,ai+=di){ jj=0; dd=1.0; // all 6-bit values
 for (aj=0.0,j=0;j<256;j++,aj+=dj)               // all 8-bit values
    {
    d=fabs(ai-aj);           // compute difference
    if (dd>d){ dd=d; jj=j; } // remember smallest difference
    } LUT[i]=jj; }           // store it to LUT

It's possible to optimize it a lot by search around interpolated value instead of full search or use binary search. However I see no point in it as the runtime is pretty fast and in final usage I would use a hard-coded LUT anyway ... Here is the hard-coded result from the code above:

uint8_t LUT[64] = 
   { 
   0, 4, 8, 12, 16, 20, 24, 28, 32, 36, 40, 45, 49, 
   53, 57, 61, 65, 69, 73, 77, 81, 85, 89, 93, 97, 
   101, 105, 109, 113, 117, 121, 125, 130, 134, 138, 
   142, 146, 150, 154, 158, 162, 166, 170, 174, 178, 
   182, 186, 190, 194, 198, 202, 206, 210, 215, 219, 
   223, 227, 231, 235, 239, 243, 247, 251, 255 
   };

So convert like this:

8bit_value = LUT[6bit_value];

As you can see it's mostly incrementing by 4 except a few times (at indexes: 11,32,53) where it increments by 5 so it's possible to convert this to an equation something like:

8bit_value = (4*6bit_value) + ((6bit_value+10)/21);

Which leads to the same results ...

Another option is to use:

8bit_value = ((6bit_value*255)+32)/63

As @RichF suggested in his/her answer, however (s)he was missing the rounding +32 term before the division. Anyway this also leads to the same results ...

Answer 5

The general formula, as given by OP, is actually the most exact:

This formula describes the (obvious) idea that the most black color component keeps being the most black, and the most saturated component keeps being such too.

Obviously enough, the intermediate values are non-integer. If we can multiply and divide, the things are easy -- do the math, round to nearest integer and you're done. If we can't or we don't want to multiply and divide, however, other approaches must be explored.

Let's play a little with the formula and try to get rid of the division:

Now we can see, that the second OP's option (that is, v8=(v6<<2)+(v6>>4)), corresponds to the first two terms in the infinite expansion without round to nearest, the same as @Raffzahn proposal.

However, taking more terms and doing proper round to nearest would make the conversion more accurate. To demonstrate that, here's a simple C program doing conversions within several different numbers of terms:

#include <stdio.h>
#include <math.h>

double exact(int v6)
{
        return ((double)v6)*255.0/63.0;
}

double shr6(int v6)
{
        return (double)((v6<<2) + (v6>>4));
}

double t3(int v6)
{       // implements v6<<2 + v6>>4 - v6>>6 then round to nearest
        return (double)(((v6<<8) + (v6<<2) - v6 + 32)>>6);
}

double t4(int v6)
{       // implements v6<<2 + v6>>4 - v6>>6 + v6>>10 then round to nearest
        return (double)(( (v6<<12) + (v6<<6) - (v6<<4) + v6 + 512 )>>10);
}

void main(void)
{
        double err_shr6=0, err_t3=0, err_t4=0;

        for(int v6=0;v6<64;v6++)
        {
                double r_exact = exact(v6);
                double r_shr6  = shr6 (v6);
                double r_t3    = t3   (v6);
                double r_t4    = t4   (v6);

                printf("v6=%2d, exact=%7.3f, shr6=%5.1f, t3=%5.1f, t4=%5.1f\n", v6, r_exact, r_shr6, r_t3, r_t4);

                err_shr6 += (r_exact-r_shr6)*(r_exact-r_shr6);
                err_t3   += (r_exact-r_t3  )*(r_exact-r_t3  );
                err_t4   += (r_exact-r_t4  )*(r_exact-r_t4  );
        }

        printf("shr6 mean squared error: %f\n", sqrt(err_shr6/64.0));
        printf("t3   mean squared error: %f\n", sqrt(err_t3  /64.0));
        printf("t4   mean squared error: %f\n", sqrt(err_t4  /64.0));
}

Its output is:

v6= 0, exact=  0.000, shr6=  0.0, t3=  0.0, t4=  0.0
v6= 1, exact=  4.048, shr6=  4.0, t3=  4.0, t4=  4.0
v6= 2, exact=  8.095, shr6=  8.0, t3=  8.0, t4=  8.0
v6= 3, exact= 12.143, shr6= 12.0, t3= 12.0, t4= 12.0
v6= 4, exact= 16.190, shr6= 16.0, t3= 16.0, t4= 16.0
v6= 5, exact= 20.238, shr6= 20.0, t3= 20.0, t4= 20.0
v6= 6, exact= 24.286, shr6= 24.0, t3= 24.0, t4= 24.0
v6= 7, exact= 28.333, shr6= 28.0, t3= 28.0, t4= 28.0
v6= 8, exact= 32.381, shr6= 32.0, t3= 32.0, t4= 32.0
v6= 9, exact= 36.429, shr6= 36.0, t3= 36.0, t4= 36.0
v6=10, exact= 40.476, shr6= 40.0, t3= 40.0, t4= 40.0
v6=11, exact= 44.524, shr6= 44.0, t3= 45.0, t4= 45.0
v6=12, exact= 48.571, shr6= 48.0, t3= 49.0, t4= 49.0
v6=13, exact= 52.619, shr6= 52.0, t3= 53.0, t4= 53.0
v6=14, exact= 56.667, shr6= 56.0, t3= 57.0, t4= 57.0
v6=15, exact= 60.714, shr6= 60.0, t3= 61.0, t4= 61.0
v6=16, exact= 64.762, shr6= 65.0, t3= 65.0, t4= 65.0
v6=17, exact= 68.810, shr6= 69.0, t3= 69.0, t4= 69.0
v6=18, exact= 72.857, shr6= 73.0, t3= 73.0, t4= 73.0
v6=19, exact= 76.905, shr6= 77.0, t3= 77.0, t4= 77.0
v6=20, exact= 80.952, shr6= 81.0, t3= 81.0, t4= 81.0
v6=21, exact= 85.000, shr6= 85.0, t3= 85.0, t4= 85.0
v6=22, exact= 89.048, shr6= 89.0, t3= 89.0, t4= 89.0
v6=23, exact= 93.095, shr6= 93.0, t3= 93.0, t4= 93.0
v6=24, exact= 97.143, shr6= 97.0, t3= 97.0, t4= 97.0
v6=25, exact=101.190, shr6=101.0, t3=101.0, t4=101.0
v6=26, exact=105.238, shr6=105.0, t3=105.0, t4=105.0
v6=27, exact=109.286, shr6=109.0, t3=109.0, t4=109.0
v6=28, exact=113.333, shr6=113.0, t3=113.0, t4=113.0
v6=29, exact=117.381, shr6=117.0, t3=117.0, t4=117.0
v6=30, exact=121.429, shr6=121.0, t3=121.0, t4=121.0
v6=31, exact=125.476, shr6=125.0, t3=125.0, t4=125.0
v6=32, exact=129.524, shr6=130.0, t3=130.0, t4=130.0
v6=33, exact=133.571, shr6=134.0, t3=134.0, t4=134.0
v6=34, exact=137.619, shr6=138.0, t3=138.0, t4=138.0
v6=35, exact=141.667, shr6=142.0, t3=142.0, t4=142.0
v6=36, exact=145.714, shr6=146.0, t3=146.0, t4=146.0
v6=37, exact=149.762, shr6=150.0, t3=150.0, t4=150.0
v6=38, exact=153.810, shr6=154.0, t3=154.0, t4=154.0
v6=39, exact=157.857, shr6=158.0, t3=158.0, t4=158.0
v6=40, exact=161.905, shr6=162.0, t3=162.0, t4=162.0
v6=41, exact=165.952, shr6=166.0, t3=166.0, t4=166.0
v6=42, exact=170.000, shr6=170.0, t3=170.0, t4=170.0
v6=43, exact=174.048, shr6=174.0, t3=174.0, t4=174.0
v6=44, exact=178.095, shr6=178.0, t3=178.0, t4=178.0
v6=45, exact=182.143, shr6=182.0, t3=182.0, t4=182.0
v6=46, exact=186.190, shr6=186.0, t3=186.0, t4=186.0
v6=47, exact=190.238, shr6=190.0, t3=190.0, t4=190.0
v6=48, exact=194.286, shr6=195.0, t3=194.0, t4=194.0
v6=49, exact=198.333, shr6=199.0, t3=198.0, t4=198.0
v6=50, exact=202.381, shr6=203.0, t3=202.0, t4=202.0
v6=51, exact=206.429, shr6=207.0, t3=206.0, t4=206.0
v6=52, exact=210.476, shr6=211.0, t3=210.0, t4=210.0
v6=53, exact=214.524, shr6=215.0, t3=214.0, t4=215.0
v6=54, exact=218.571, shr6=219.0, t3=219.0, t4=219.0
v6=55, exact=222.619, shr6=223.0, t3=223.0, t4=223.0
v6=56, exact=226.667, shr6=227.0, t3=227.0, t4=227.0
v6=57, exact=230.714, shr6=231.0, t3=231.0, t4=231.0
v6=58, exact=234.762, shr6=235.0, t3=235.0, t4=235.0
v6=59, exact=238.810, shr6=239.0, t3=239.0, t4=239.0
v6=60, exact=242.857, shr6=243.0, t3=243.0, t4=243.0
v6=61, exact=246.905, shr6=247.0, t3=247.0, t4=247.0
v6=62, exact=250.952, shr6=251.0, t3=251.0, t4=251.0
v6=63, exact=255.000, shr6=255.0, t3=255.0, t4=255.0
shr6 mean squared error: 0.345033
t3   mean squared error: 0.287384
t4   mean squared error: 0.286086

You can play with it interactively here: https://godbolt.org/z/sMr5azYM8

From the results we see that:

• The "shr6" method, that is, (v6<<2)+(v6>>4) gives the worst results,
• "t3", the three-terms method with explicit round-to-nearest is significantly better,
• "t4", four-term method, gives exact integer result.

The similar approach could be used for any other conversions of color depth.