Complete reedit due to new information and 30K limit
Ok I tried your new files. the files without extensions should have *.HPI
extensions (HP Image) and should be decoded to files first. They have HFSLIF
signature at start.
To decode them the easiest is to use emulator. However to do this on your own we need either decoding table + fileformat description or binary+ASCII of the same program so we can infer the file format.
After some googling I found this:
see there the part Background: The LIF File System it looks like the files you linked matches their fileformat description. Here start of FW_UPGRADE in hex:
00000000: | 80 00 48 46 53 4c 49 46 00 00 00 01 10 00 00 00 | € HFSLIF
so:
8000 magic
4846534c4946 volume label "HFSLIF"
00000001 directory start
1000 LIF ID
0000 unused
...
So now its just the matter of decoding the DISC/file image into binary and try to detokenize. btw at the same page is also their utility to do this for download...
So the non ASCII files are DISC images not files !!! I tried the hpdir
utility from the previous link and this:
hpdir -c -extract FW_UPGRADE.htm
will extract the PROG file from DISC image into file WS_FILE.PROG
which is a bit smaller than the original DISC image.
However It looks like there is absolutely no tool to convert the binary into ASCII (all just extract files from the DISC images) nor I can find any TOKEN code table anywhere (with codes not just Keyword listing) so the only way now would be to reverse engineer the encoding. For that its necessary to have ASCII and Binary files of the same program containing all the Keywords and data representations (integer,float,string ...) Which we have now.
At the end of HP9000 BASIC 5.0 Language Reference Vol2 (thx to wizzwizz4) I found table of BASIC keywords supported by HP9000
Sadly without the tokenisation codes :( so a cross-match or inferring is needed (some commands are recognizable like PRINT,REM
others need some reversing or pairing with known BASIC ASCII source ...
Here actual FileFormat summary:
-------------------------------------------------------------------------
// file start
// 16 bit: 0000 FFFF size 4000 0007? 00 00C2 00C2 00 FFFF 0000
// 32 bit: 0000 FFFF size 4000 0007? 00 000000C2 000000C2 40 0000 FFFFFFFF 0000
// 16bit line encoding examples
[02 BA 01 B3] RETURN Model$
[B5] 2 [00 02] RP [02 FB BA 01 B3 8E] RETURN "RP"&File$
[B5] 2 [00 06] 46020A [BA 01 B3 8E] RETURN "46020A"
[B5] 2 [00 05] OTHER [BA 01 B3] RETURN "OTHER"
[81 BA 01 B3] RETURN 1
[80 BA 01 B3] RETURN 0
[84 BD] # [B3] WAIT 4
[85 BD] # [B3] WAIT 5
B5 31 02 80 B5 18 3F D3 33 33 33 33 33 33 BD 24 02 B3 BEEP 640,.3
[B5] 1 [00 C8 B5 18] ? [E0 00 00 00 00 00 00 BD] $ [02 B3] BEEP 200,.5
[8B B5 18 8E] ? [E0 00 00 00 00 00 00 BD] $ [02 B3] BEEP 100,.5
-------------------------------------------------------------------------
all offsets: are in decadic !!!
-------------------------------------------------------------------------
[16 bit Header]
08: DWORD start of last BASIC line
12: Starting section
17: DWORD 00C200C2
38: DWORD start of first BASIC line
46: DWORD start of variable table
[16 bit Section]
00: BYTE ID 40/48/C0/C8
03: BYTE intendation (probably ORed with another flags in higher bits)
04: WORD line umber (0 for starting section which is not code)
14: WORD offset to next section
28: WORD offset to next line (inside section)
??: code
[16 bit line]
00: BYTE ID 40/48/C0/C8
01: BYTE length of pevious line (0 for sections)
02: BYTE length of this line (0 if section)
03: BYTE intendation (probably ORed with another flags in higher bits)
04: WORD line umber (0 for starting section)
06: code
-------------------------------------------------------------------------
[32 bit Header]
08: DWORD start of last BASIC line
12: Starting section
17: DWORD 000000C2
21: DWORD 000000C2
[32 bit Section]
??
[32 bit line]
00: BYTE ID 40/48/C0/C8
01: BYTE length of pevious line (0 for sections)
02: BYTE length of this line (0 if section)
03: BYTE intendation (probably ORed with another flags in higher bits)
04: DWORD line umber (0 for starting section)
08: code
-------------------------------------------------------------------------
[code]
each line ends with B3 or B3,8E all codes are in [hex]
first byte:
00 - label: (index of label)
<01,30) - index of variable + 1
<80,8E) - constant <00,0E)
any? byte:
B6 - ! comment (B6,8E,BYTE size, char string[size])
30/07,D4 - Cprint (BYTE parentis,BYTE ?,WORD size+1,char string[size])
BC,01,0B/0F - REM (BYTE size,char string[size])
B9 - END IF
95,B3,8E - GOSUB
9B,A5 - PRINT
B8,8E,00,00 - ELSE (WORD ??)
BA,00 - RETURN
BA,01 - RETURN
BC,02,12 - DATA (BYTE size,char string[size])
BC,06 - REAL (index of variable) ends A4
BC,07 - INTEGER (index of variable) ends A4
BD,1C - STOP
BD,05 - END
BD,06 - FNEND
BD,0D - OFF ERROR
BD,17 - PAUSE
BD,23 - WAIT
BD,24 - BEEP
BC,02,12 - DATA (BYTE size,char string[size])
BB,03,0B - REPEAT
BB,03,15 - END SELECT
BB,1F,22 - CLEAR ERROR
BB,1F,40 - CLEAR SCREEN
BB,03,10,8E,00,00 CASE ELSE (WORD ??)
not first byte:
00 - string operand (BYTE size, char string[size])
09 - n-dimensional array variable declaration (WORD: i0:i1,i2:i3,... BYTE A4)
B5 - signals combination of more operands
90 - operand = operand (operands are stored before)
-------------------------------------------------------------------------
As you can see there are 2 file-formats 16 bit and 32 bit (sadly I got just single 32 bit file and without ASCII one of the first linked firmwares). You can use the 00C200C2/000000C2000000C2
pattern to detect it.
The code is comprised of sections and lines. First section (line number = 0) does not have any code in it but it contains variable table and additional still undetermined stuff.
All other sections contain lines. The last section has negative offset to next section.
Here current decoder C++/VCL:
//---------------------------------------------------------------------------
AnsiString decode(AnsiString file) // read HP9000 *.PROG file and convert it to text
{
// view config
bool _enable_wordwrap=true; // turn word wrapping on/off
bool _enable_hexview=true; // turn hexview on/off
// view end columns
const int _sz_lin=13; // adr: lno
const int _sz_bas=140; // adr: lno WAIT 4
const int _sz_hex=140+7+(32*3); // adr: lno WAIT 4 | adr: C0 0C 0A 23 ...
const int _ops=32; // max operands
// variables
DWORD dw;
BYTE *dat=NULL,c,*cmd;
int a0,a1,a,adrv,adrb,adre;
int i,l,_hex,_txt,_ok,_beg,_16bit=0,_32bit=0;
int hnd,siz,adr,adr0,adr1,adrh,len,lno,tab,ops;
bool eof,eos,sos; // end of file, end of section, start of section
AnsiString lin,txt="",op[_ops],cm,s,slin,sbas;
for (slin="",i=0;i<_sz_lin;i++) slin+=" "; // code intendation
for (sbas="",i=0;i<_sz_bas;i++) sbas+=" "; // hex view intendation
// load file to memory
hnd=FileOpen(file,fmOpenRead); // open source file
siz=FileSeek(hnd,0,2); // get its length
FileSeek(hnd,0,0); // and point to start again
dat=new BYTE[siz]; // create buffer
FileRead(hnd,dat,siz); // load to file into it
FileClose(hnd); // close file
// detect 16/32 bit
dw =dat[17]; dw<<=8;
dw+=dat[18]; dw<<=8;
dw+=dat[19]; dw<<=8;
dw+=dat[20];
_16bit=(dw==0x00C200C2);
_32bit=(dw==0x000000C2);
// adr of last BASIC line
dw =dat[ 8]; dw<<=8;
dw+=dat[ 9]; dw<<=8;
dw+=dat[10]; dw<<=8;
dw+=dat[11]; adre=dw;
if (adre+8>siz) adre=siz-8;
// adr of first BASIC line
dw =dat[38]; dw<<=8;
dw+=dat[39]; dw<<=8;
dw+=dat[40]; dw<<=8;
dw+=dat[41]; adrb=dw+12;
if (adrb>adre) adrb=adre;
// adr of variables table
dw =dat[46]; dw<<=8;
dw+=dat[47]; dw<<=8;
dw+=dat[48]; dw<<=8;
dw+=dat[49]; adrv=dw+12;
if (adrv>adrb) adrv=adrb;
// decode variables to text
txt+="[Variables]\r\n adr sz ix:\r\n";
for (l=0,adr=adrv;adr<adrb;adr++)
if (dat[adr]) // size
{
len=dat[adr]; adr++;
txt+=AnsiString().sprintf("%04X %02X %02X: ",adr-1,len,l);
for (lin="",i=0;(i<len)&&(adr<adrb);i++)
{
c=dat[adr]; adr++;
if (c<32) c='.';
lin+=char(c);
} adr--;
txt+=lin+"\r\n"; l++;
} txt+="\r\n";
// decode BASIC to text
if (_16bit) txt+="[Code 16bit]\r\n";
if (_32bit) txt+="[Code 32bit]\r\n";
txt+=" adr: lno code\r\n";
for (adrh=0,adr=0x000C,eof=false;!eof;) // process all sections
{
// section start line info
adr0=adr; // store starting line
tab=dat[adr+ 3]; // intendation
if (_16bit) // line number
{
dw= dat[adr+4]; dw<<=8;
dw+=dat[adr+5]; lno=dw;
}
if (_32bit)
{
dw =dat[adr+4]; dw<<=8;
dw+=dat[adr+5]; dw<<=8;
dw+=dat[adr+6]; dw<<=8;
dw+=dat[adr+7]; lno=dw;
}
sos=(lno>0); // section start (except first line (lno = 0)
dw =dat[adr+28]; dw<<=8; // offset to next line (normal line length)
dw+=dat[adr+29]; len=dw;
dw =dat[adr+14]; dw<<=8; // offset to next section (section length)
dw+=dat[adr+15];
if (dw>=0x8000) // last size iso negative
{
adr1=adr0-(dw^65535)-1;
eof=true;
}
else adr1=adr0+dw;
if (!sos)
{
adr+=len; // first line of code offset
adrh=adr;
}
if (sos)
sos=sos;
for (eos=false;!eos;) // process section lines
{
// inside section lin info
if (!sos)
{
len=dat[adr+ 1]; // line length
tab=dat[adr+ 3]; // intendation
if (_16bit) // line number
{
dw= dat[adr+4]; dw<<=8;
dw+=dat[adr+5]; lno=dw;
}
if (_32bit)
{
dw =dat[adr+4]; dw<<=8;
dw+=dat[adr+5]; dw<<=8;
dw+=dat[adr+6]; dw<<=8;
dw+=dat[adr+7]; lno=dw;
}
}
// ----------------------------------------------------------------------------------
// process line decoded line lno from dat[adr+0] to dat[adr+len] with intendation tab
// ----------------------------------------------------------------------------------
// line start
lin=AnsiString().sprintf("%04X: ",adr);
// line number
lin+=AnsiString().sprintf("%6u ",lno);
// intendation
if (tab<=32) for (i=0;i<tab;i++) lin+=" ";
// decode tokenized line to text
a0=adr; a1=adr+len;
if (_16bit) a0+=6;
if (_32bit) a0+=8;
ops=0; // no operands yet
cm=""; // no command yet
if (lno==22)
lno=lno;
for (_hex=0,_txt=0,_ok=1,_beg=1;a0<a1;)
{
cmd=dat+a0; // pointer just to simplify relative decoding for more BYTE tokens
c=dat[a0]; a0++; // actual processed BYTE
if (_ok) // no discrepancy yet?
{
if ((c==0x00)&&( _beg)) // Label:
{
a0++; lin+=AnsiString().sprintf("label%02X:",cmd[1]); continue;
}
if ((c==0x00)&&(!_beg)) // string operand
{
l=dat[a0]; a0++; // size
if (a0+l>a1) l=a1-a0;
for (op[ops]="\"",i=0;i<l;i++,a0++)
op[ops]+=char(dat[a0]);
op[ops]+="\"";
ops++; if (ops>=_ops) break; continue;
}
if ((c>=0x01)&&(c<0x30)&&(_beg)) // variable operand
{
op[ops]=AnsiString().sprintf("var%02X",c-1);
ops++; if (ops>=_ops) break; continue;
}
if ((c>=0x80)&&(c<0x8E)&&(_beg)) // small integer constant perand
{
op[ops]=BYTE(c-0x80);
ops++; if (ops>=_ops) break; continue;
}
_beg=0;
if (c==0x8E) // end of line [B3 8E]
{
lin+=" |8E|";
continue;
}
if ((c==0x90)&&(ops==2)) // op1 = op2
{
lin+=op[0]+"="+op[1]; ops=0; continue;
}
if (c==0xB3) // end of line [B3 8E]
{
if (cm!="") lin+=cm;
for (i=0;i<ops;i++) lin+=" "+op[i];
lin+=" |B3|";
ops=0;
cm="";
continue;
}
if (c==0xB5) // combination of operands
{
a0++; _beg=1;
continue;
}
if (c==0xB6) // ! comment
{
a0+=2; // skip [B6 8E 07]
l=dat[a0]; a0++; // size
if (a0+l>a1) l=a1-a0;
for (lin+="! ",i=0;i<l;i++,a0++)
lin+=char(dat[a0]);
continue;
}
if ((cmd[0]==0x30)||(cmd[0]==0x07)) // Cprint
if (cmd[1]==0xD4)
{
c=' ';
if (cmd[2]==0xB5) c='"'; // parentis
dw =cmd[4]; dw<<=8; // string size
dw+=cmd[5]; l=dw-1;
lin+="Cprint("; lin+=char(c);
a0+=5; if (a0+l>=a1) l=a1-a0-1;
for (i=0;i<=l;i++,a0++) lin+=char(dat[a0]);
lin+=char(c);
lin+=")";
continue;
}
if ((cmd[0]==0xBC)&&(cmd[1]==0x01)&&(cmd[2]==0x0B)||(cmd[2]==0x0F))
{
lin+="REM "; l=cmd[3];
a0+=3; if (a0+l>a1) l=a1-a0;
for (i=0;i<l;i++,a0++) lin+=char(dat[a0]);
continue;
}
if ((cmd[0]==0xBC)&&(cmd[1]==0x06)) // REAL variable declaration
{
lin+="REAL "+AnsiString().sprintf("var%02X",cmd[2]-1); a0+=2; continue;
}
if ((cmd[0]==0xBC)&&(cmd[1]==0x07)) // INTEGER variable declaration
{
lin+="INTEGER "+AnsiString().sprintf("var%02X",cmd[2]-1); a0+=2; continue;
}
if ((cmd[0]==0xBC)&&(cmd[1]==0x02)&&(cmd[2]==0x12)) // DATA
{
lin+="DATA "; l=cmd[3];
a0+=3; if (a0+l>a1) l=a1-a0;
for (i=0;i<l;i++,a0++) lin+=char(dat[a0]);
continue;
}
if (cmd[0]==0x09) // n dimensional array ???
{
lin+="(";
for (;a0+4<=a1;)
{
if (dat[a0]==0xA4) break; // end of variable declaration
dw =dat[a0]; a0++; dw<<=8;
dw+=dat[a0]; a0++; lin+=AnsiString().sprintf("%i:",dw);
dw =dat[a0]; a0++; dw<<=8;
dw+=dat[a0]; a0++; lin+=AnsiString().sprintf("%i,",dw);
}
lin[lin.Length()]=')'; continue;
}
if (cmd[0]==0xB9){ cm="END IF"; continue; }
if ((cmd[0]==0x9B)&&(cmd[1]==0xA5)){ a0+=1; cm="PRINT"; continue; }
if ((cmd[0]==0xBA)&&(cmd[1]==0x00)){ a0+=1; cm="RETURN"; continue; }
if ((cmd[0]==0xBA)&&(cmd[1]==0x01)){ a0+=1; cm="RETURN"; continue; }
if ((cmd[0]==0xBD)&&(cmd[1]==0x1C)){ a0+=1; cm="STOP"; continue; }
if ((cmd[0]==0xBD)&&(cmd[1]==0x05)){ a0+=1; cm="END"; continue; }
if ((cmd[0]==0xBD)&&(cmd[1]==0x06)){ a0+=1; cm="FNEND"; continue; }
if ((cmd[0]==0xBD)&&(cmd[1]==0x0D)){ a0+=1; cm="OFF ERROR"; continue; }
if ((cmd[0]==0xBD)&&(cmd[1]==0x17)){ a0+=1; cm="PAUSE"; continue; }
if ((cmd[0]==0xBD)&&(cmd[1]==0x23)){ a0+=1; cm="WAIT"; continue; }
if ((cmd[0]==0xBD)&&(cmd[1]==0x24)){ a0+=1; cm="BEEP"; continue; }
if ((cmd[0]==0x95)&&(cmd[1]==0xB3)&&(cmd[2]==0x8E)){ a0+=2; lin+="GOSUB"; continue; }
if ((cmd[0]==0xBB)&&(cmd[1]==0x03)&&(cmd[2]==0x0B)){ a0+=2; lin+="REPEAT"; continue; }
if ((cmd[0]==0xBB)&&(cmd[1]==0x03)&&(cmd[2]==0x15)){ a0+=2; lin+="END SELECT"; continue; }
if ((cmd[0]==0xBB)&&(cmd[1]==0x1F)&&(cmd[2]==0x22)){ a0+=2; lin+="CLEAR ERROR"; continue; }
if ((cmd[0]==0xBB)&&(cmd[1]==0x1F)&&(cmd[2]==0x40)){ a0+=2; lin+="CLEAR SCREEN"; continue; }
if ((cmd[0]==0xB8)&&(cmd[1]==0x8E)&&(cmd[2]==0x00)&&(cmd[3]==0x00)){ a0+=3+2; lin+="ELSE"; continue; }
if ((cmd[0]==0xBB)&&(cmd[1]==0x03)&&(cmd[2]==0x10)&&(cmd[3]==0x8E)&&(cmd[4]==0x00)&&(cmd[5]==0x00)){ a0+=5+2; lin+="CASE ELSE"; continue; }
}
_ok=0;
if ((c<32)||(c>127)) // non ASCII character
{
if ( _txt){ _txt=0; lin+="' "; }
if (!_hex){ _hex=1; lin+=" ["; } else lin+=" ";
lin+=AnsiString().sprintf("%02X",c);
}
else{ // ASCII character
if ( _hex){ _hex=0; lin+="] "; }
if (!_txt){ _txt=1; lin+=" '"; }
lin+=char(c);
}
}
if (_hex){ _hex=0; lin+="] "; }
if (_txt){ _txt=0; lin+="' "; }
if ((cm!="")||(ops)) // forced end of line [B3 8E]
{
if (cm!="") lin+=cm;
for (i=0;i<ops;i++) lin+=" "+op[i];
lin+=" |!!|";
}
// word wrap + hex debug
for (i=1;;) // proces wrapped lines
{
if (_enable_wordwrap) // cut code
{
if (i==1) s=lin.SubString(i,_sz_bas);
else s=lin.SubString(i,_sz_bas-_sz_lin);
}
else{
if (i==1) s=lin;
else s="";
}
l=s.Length(); if (l==0) break;
if (i!=1) s=slin+s; i+=l;
if (_enable_hexview)
{
while (s.Length()<_sz_bas) s+=" "; // align to end of basic column
// add hex view
for (s+=AnsiString().sprintf("| %04X:",adrh);(adrh<a1)&&((s.Length()<=_sz_hex-3)||(!_enable_wordwrap));adrh++) s+=AnsiString().sprintf(" %02X",dat[adrh]);
}
txt+=s+"\r\n";
}
// hex view (if not finished yet)
if (_enable_hexview) for (;adrh<a1;)
{
for (s=sbas+AnsiString().sprintf("| %04X:",adrh);(adrh<a1)&&((s.Length()<=_sz_hex-3)||(!_enable_wordwrap));adrh++) s+=AnsiString().sprintf(" %02X",dat[adrh]);
txt+=s+"\r\n";
}
// ----------------------------------------------------------------------------------
// ----------------------------------------------------------------------------------
// ----------------------------------------------------------------------------------
// next line offset
if (!sos)
{
adr+=len; // point out next line first byte position
c=dat[adr];
if (((c!=0x40)&&(c!=0x48)&&(c!=0xC0)&&(c!=0xC8))||(adr==adr1)) //check if this is 0x40/0xC0 or not!
{
eos=true; // if it is not or we are past the lines for this SUB/FN, done for this SUB/FN
adr=adr1; // quick jump to next start of SUB/FN
}
}
// secnd line in section offset
else{
sos=false;
adr+=len; // increment to next real line!
}
}
}
// remove zeros from txt
for (l=txt.Length(),i=1;i<=l;i++)
if (txt[i]==0) txt[i]='.';
delete[] dat;
return txt;
}
//---------------------------------------------------------------------------
I think this could get you started quite nicely. Simple commands has 4 BYTE codes where first BYTE also tells if the command use constant (0x80+constant
up to unknown range probably just 0..9 or similar) or variable operand (0x00+variable_index
). End of line usually ends with 0xB3 0x8E
.
Here few decoded lines outputted from the decoder (PS_EDI~1.PROG
):
3250: 385 END SELECT |B3| | 3250: 40 0A 5E 09 01 81 BB 03 15 B3
325A: 386 WAIT 5 |B3| | 325A: 40 0A 0A 09 01 82 85 BD 23 B3
3264: 387 [D4 B2 00 B3 8E] var09 |!!| | 3264: 40 0C 0A 09 01 83 0A D4 B2 00 B3 8E
3270: 388 RETURN 0 |B3| | 3270: 40 0A 0C 09 01 84 80 BA 01 B3
327A: 389 FNEND |B3| |8E| | 327A: 40 0A 0A 07 01 85 BD 06 B3 8E
and the same from ASCII reference file:
385 END SELECT
386 WAIT 5
387 Title_screen
388 RETURN 0
389 FNEND
LIST
command. Have you tried that, @Staffan?