x86 16-bit C compiler (Alex!!)

[voidstar tech]

I'm mentioning Alex because I think he wrote an article about all this back in 2004! :)


I've written a C game that targets the 6502 (PET/Apple][) and Z80 (TRS80) and fits within 32K.

I'm now trying to target the x86 16-bit (specifically the original IBM PC 5150), so I don't want any 80186, 80286, or 80386 instructions.

I've had success with Turbo C 2.01 in 86BOX. It wasn't exactly trivial and requires various hoops - but it did work. To summarize what I did:

create a 486 DOS image (86BOX) and installed MS-DOS 6.0, Turbo C 2.0, Turbo Assembler 1.01 (to a 20MB virtual HDD). The main.c example included with Turbo C has comments to describe basically exactly what I wanted to do, so I followed those instructions...

tcc -c -ms main
tasm c0 /D__TINY__ /D__NOFLOAT /t/mx;
tasm setargv /D__TINY__ /t/mx;
tlink c0 main setargv /c/m,main

exe2bin main.exe main.com
del main.exe

BUT, if you know MS-DOS -- DOS 6.0 didn't include EXE2BIN, so where did I get it? I copied it from the last version of DOS that did include EXE2BIN, which was MS-DOS 3.31. That was one of the hoops.

The next challenge was how to get the .COM file on a PC-DOS 1.0 image. That's a lot more hoops, because PC-DOS 1.0 only supports single sided 160KB disk (which WinImage doesn't support). Long story short, I just used PC-DOS 2.10 (which supports double sided disk, so I just copied my .COM file to a 360KB disk image). And voila, my TurboC built .COM runs. Yay!!

My goal is to show that, at least conceptually, my binary could be loaded via cassette on an original IBM PC - I don't want it to dependent on MS-DOS. In fact, I believe I should be able to skip BIOS calls, and just do direct OUT instructions (with some inline assembly). But let's not get distracted with all that just yet. First things first, I need to improve the workflow here...

TurboC means I'm limited to 8.3 filenames. My actual source code that I want to compile is 168KB (long story on why). I learned that TurboC has a limit of 64KB per .c file (a limitation of its editor). I can fix all that - I can rename all my files and #include's, and split the .c into smaller files. But my point is, the code-base as-is is compiling for PET, Apple2, TRS80 (using cc65 and z88dk C compilers) -- I'd like to also include x86 IBM PC into that same code-base (obviously I'm using #define macros where needed across the systems, for specific things like polling keyboard and drawing to the screen). Which is part of the point: to show we have "modern" development environments (runs under Win10) that can target these ancient platforms.


So with all those hoops, my test/debug cycle is really inefficient (compile in 86BOX TurboC DOS 6.0 image, use WinImage to export the binary to the PC-DOS 2.x image, use EXE2BIN {either before or after -- that's another reason to use PC-DOS 2.x, as PC-DOS 1.x didn't include EXE2BIN yet}). Plus also just having to re-arrange all the code (filenames and 64KB chunks).

NOTE1: I haven't explored if there is a way to network my 1981 8086 86BOX with my 1993 80486 86BOX together. That doesn't help the 8.3 filename pain.
NOTE2: I think all my code could fit on a single 360K disk. I'm not sure about ALSO fitting all the .obj and final linked program. But if so, then I could share the same floppy image between the "PC-DOS 2.10" machine and the "PC-DOS 6.0" machine that I'm doing the development on (just flip it back and forth). Again, still doesn't help the 8.3 filename pain.


So - I looked for a native Windows 10 compiler that could target 16-bit x86. WATCOM C did compile all my code, that was wonderful - but the .COM file it produced didn't actually work. When that failed, I went down the TurboC path just to verify that it would work -- and it does. I'm not 100% certain if WATCOM C 1.9 is producing 8086/8088 code or not (I'm just using its IDE, haven't scrutinized all the command line options yet).

I tried TurboC 3.X -- that at least adds mouse support and Search, making it a little easier to move around within the 86BOX emulator. But it still doesn't support long filenames (it basically old-school TurboC auto-packaged into an 86BOX).

TinyCC I don't think supports 16-bit x86.

I've tried DigitalMars - which seems like it should have worked (I used "-0 -mt" for 8088 and TINY .COM). DigitalMars includes an EXE2BIN, but it's a 16-bit build that won't run under 64-bit Windows. NOTE: I explored the public MS DOS 6.0 source code and noticed EXE2BIN.ASM source is there -- it is a format conversion tool with no real reason that it needs to built as a 16-bit application, so it seems to me someone should essentially be able to make a C version of the adjustments EXE2BIN is doing to an EXE, and make a 64-bit build of that). ANYWAY, I ran the MS-DOS version of EXE2BIN on the .EXE that DigitalMars produced - that ran and produced a .COM, but it didn't run (under DOS 3.X or DOS 2.X, or 6.X for that matter). So I'm not sure what went wrong there. Basically this is the same as the WATCOM C experience - they both produced a final .COM that just didn't run under the actual 8086 IBM PC. [ but TurboC did, so I know it's possible ]

I don't think C++Builder (i.e. anything past TurboC 3.X) will target 16-bit x86 either.


The next thing I'm going to try is to go to an older version of WATCOM C (1.6) and see if I get any luck there. I've only been exploring this for a day now -- my Apple2 and Z80 TRS80 ports only took a couple days. I'm surprised how much of a hassle real-mode/16-bit x86 support turning out to be (in C). (but I also get that there isn't much reason for it - but let me tell you what my inspiration is: the DOS port of Montezuma’s Revenge, which was a .COM program that could also have technically been loaded from a cassette tape - except I want my game compiled from the C code, not hand-assembled).


If anyone has any other suggestions - TurboC 2.0 is my fallback, but I'd really rather something that can work with my existing long filenames, target 8086/8088, TINY .COM target, and I'd like to do some PORT/OUT calls (inline assembly) and/or do interrupt calls (WATCOM 1.9 has IN/OUT calls in conio.h, and both of them have a version of int86 -- either in i86.h or dos.h; but I think it'd be nice to be able to just inline my own OUT calls for the very few things that my game does).

And if curious, the game (and the PET/Apple2/C64/TRS80 binaries) are at www.destinyhunter.org (and the source is also on github, if anyone else wants to take a crack on an x86 port).

Thanks!
SteveL

[JJ]

Post by voidstar tech
The next challenge was how to get the .COM file on a PC-DOS 1.0 image.
That's a lot more hoops, because PC-DOS 1.0 only supports single sided
160KB disk (which WinImage doesn't support). Long story short, I just
used PC-DOS 2.10 (which supports double sided disk, so I just copied my
.COM file to a 360KB disk image). And voila, my TurboC built .COM runs.
Yay!!

You can use HxCFloppyEmulator. The software itself is designed for HxC
floppy emulator hardware, but it can still run without it, and it has some
some useful floppy image tools including creating and populating a 160KB
SSSD floppy image from scratch.

Post by voidstar tech
I don't think C++Builder (i.e. anything past TurboC 3.X) will target 16-bit x86 either.

Borland C++ v5.x's BCC.EXE (not BCC32.exe) can produce 8086 code (at least
it claims to be). Since BCC.EXE does have an optional switch to produce
80186/80286 code. BCC32.EXE can only produce 80286 code at minimum (with a
switch to produce 80386 code). Both BCC.EXE and BCC32.EXE are DOS-PE32
hybrid programs. But TLINK.EXE is DOS only. While TLINK32.EXE is for Windows
target only.

[voidstar tech]

Post by voidstar tech
I'm now trying to target the x86 16-bit (specifically the original IBM PC 5150), so I don't want any 80186, 80286, or 80386 instructions.

WATCOM C 1.9 is working for me now. Not sure what I did differently, but it is producing a .COM file (16-bit DOS target) and it is running for me under PC-DOS 2.10.

I'm entering 40-column mode, and detecting keystrokes via int 16h.

My next challenge is I'd like to POKE directly to the screen at B800. I know this must be possible because I can do it from cassette ROM BASIC under PC-DOS 1.0 and 2.10. I do this as follows:
- remove A: drive disk, reboot the machine
- "boots" into cassette C1.00 ROM BASIC
- issue:
DEF SEG=&HB800
POKE 0,1 <-- draw symbol #1 at top left corner
POKE 1,14 <-- make that symbol yellow

So if BASIC can do this, it's in real-mode, and my .COM binary should be able to do this as well. Just I'm not exactly sure yet how to do the equivalent from WATCOM C.

I can define POKE as:
#define POKE(addr,val) (*(unsigned char*) (addr) = (val))

But how do I set the DS register?

[[e-mail]]

On Monday, June 7, 2021 at 7:46:54 PM UTC+10, ***@gmail.com wrote:

You are certainly doing something very interesting!

I'm not sure if you're using a C library, but bear in
mind that PDPCLIB supports 8086 and is public
domain.

#define POKE(addr,val) (*(unsigned char*) (addr) = (val))
But how do I set the DS register?

Try:

*(char far *)0xb8000000UL = 'X';

and see if that works for you.

BFN. Paul.

[wolfgang kern]

Post by voidstar tech

Post by voidstar tech
I'm now trying to target the x86 16-bit (specifically the original IBM PC 5150), so I don't want any 80186, 80286, or 80386 instructions.

WATCOM C 1.9 is working for me now. Not sure what I did differently, but it is producing a .COM file (16-bit DOS target) and it is running for me under PC-DOS 2.10.
I'm entering 40-column mode, and detecting keystrokes via int 16h.
- remove A: drive disk, reboot the machine
- "boots" into cassette C1.00 ROM BASIC
DEF SEG=&HB800
POKE 0,1 <-- draw symbol #1 at top left corner
POKE 1,14 <-- make that symbol yellow
So if BASIC can do this, it's in real-mode, and my .COM binary should be able to do this as well. Just I'm not exactly sure yet how to do the equivalent from WATCOM C.
#define POKE(addr,val) (*(unsigned char*) (addr) = (val))
But how do I set the DS register?

wouldn't all HLL references end up in the wild if you'd alter DS ?
IIRC x86-BASIC's def seg and peek/poke use ES.
__
wolfgang

[Rod Pemberton]

On Mon, 7 Jun 2021 02:46:52 -0700 (PDT)

Post by voidstar tech

Post by voidstar tech
I'm now trying to target the x86 16-bit (specifically the original
IBM PC 5150), so I don't want any 80186, 80286, or 80386
instructions.

WATCOM C 1.9 is working for me now. Not sure what I did
differently, but it is producing a .COM file (16-bit DOS target) and
it is running for me under PC-DOS 2.10.
I'm entering 40-column mode, and detecting keystrokes via int 16h.
My next challenge is I'd like to POKE directly to the screen at B800.
I know this must be possible because I can do it from cassette ROM
- remove A: drive disk, reboot the machine
- "boots" into cassette C1.00 ROM BASIC
DEF SEG=&HB800
POKE 0,1 <-- draw symbol #1 at top left corner
POKE 1,14 <-- make that symbol yellow
So if BASIC can do this, it's in real-mode, and my .COM binary should
be able to do this as well. Just I'm not exactly sure yet how to do
the equivalent from WATCOM C.
#define POKE(addr,val) (*(unsigned char*) (addr) = (val))
But how do I set the DS register?

Writing to the screen in C for OpenWatcom is different for PM (protected
mode) versus RM (real mode). PM usually works just like normal C code
for pointers, but RM (for OW) usually needs some helper macros and a far
declaration on the pointer.


For RM, (OpenWatcom)

#include <i86.h>
char __far *screen;

screen=MK_FP(0xB800,2);
*screen='B';


For PM, (OpenWatcom, DJGPP, etc)

char *screen;

screen=(char *)0xB8004;
*screen='C';


I extracted and converted that from some code of mine. So, it's
possible it's incorrect, but it doesn't appear to be to me. If so,
yell at me, and I'll test and fix it.

Obviously, the 2 with 0xB800 and 4 in 0xB80004 are the character
locations within the screen at 0xB800 (RM) or 0xB8000 (PM). Remember
that there is a color byte in between each char.


In addition to MK_FP(), FP_SEG(), FP_OFF(), OpenWatcom also has a
__segment keyword, a __based keyword, and a special :> operator to
use and construct segment based pointers.

[voidstar tech]

Post by Rod Pemberton
On Mon, 7 Jun 2021 02:46:52 -0700 (PDT)

Post by voidstar tech
DEF SEG=&HB800
POKE 0,1 <-- draw symbol #1 at top left corner
POKE 1,14 <-- make that symbol yellow

I'd like to disassemble the ROM BASIC (have the ROM image in a file) and get insight on exactly what DEF SEG is doing. My initial thought was it is changing the DS register, but think I am wrong about that. Recall BASIC is always interpreted, so the POKE command is probably doing a bit more than it appears (that BASIC might have its own pagefile or addresses to store down the "current segment" and the POKE does the address math when it's invoked?). Alternatively, I'd like a PC emulator that will let me monitor register and memory states on the fly - obviously that impacts the emulator performance, and may need a special build to communicate those states out in some protocol. But thus far, I'm not seeing how to do this with either 86BOX or PCem (heading to their forums to investigate further). But rather than disassemble of a ROM, I could just walk through what DEF SEG is doing to the system state, etc...

Post by Rod Pemberton
Not to put too fine a point on it, but 5150 was, perhaps, an
unfortunate choice of designator for the IBM PC.

That's funny, thanks Scott :)

Post by Rod Pemberton
You are certainly doing something very interesting!

It's been an interesting adventure for me, since I never knew about the IBM 5100 from 1975 - which I was surprised to learn it had a 64x16 screen like the Tandy TRS80 series (so I'm very curious if that's why Tandy chose that "weird" resolution, but I haven't found any actual evidence on that). However, at a cost of $20,000 (then), the IBM 5100 wasn't (in my opinion) yet a "personal computer" - that's all debatable, but for me personally this means: I'm not motivated to try to target an IBM 5100. i.e. I was on a quest to find the earliest piece of hardware that my game could run on (which I determined to be the PET 6502 1977) -- and technically, I think the IBM 5100 could do it (wild to think a 1975 machine could run this!), but it would be a lot of research to learn its not-off-the-shelf display hardware, no C compiler, and I doubt I'd ever get physical access to one to verify my binary actually works (whereas I've verified my binaries for all the other platforms, on physical machines).


Thank you all for the notes! I've gotten the game running in an IBM PC 5150 using WATCOM C. In case it matters, I'm actually using version 1.9 instead of the newer 2.0 (I actually never got WATCOM C 1.6 to run, so I found a 1.9 and tried that). The default "16-bit DOS" target does target the 8086 (verified the command line options it is passing to wcc). Parts of the manual (and command line options) seem to use the term SMALL instead of TINY - so I'm a little vague on if it's a 16/16 model. But all I can say is that the resulting .COM it is producing (no EXE2BIN hoop), that is running on an IBM PC 5150 with PC DOS 1.0.


Other notes in case of any possible future reference...

To clear the screen, I'm just switching to 40-column mode (which I want to run in anyway):
#include <i86.h> //< has REGS and int86
void go40() {
union REGS rg;
rg.x.ax = 0; /* Mode 0 is 40x25. Use mode 2 for 80x25. */
int86(0x10, &rg, &rg);
}
NOTE: of course, to be more polite, one should store down the current mode when the program is started and restore that mode when the program is exited.... I'm not that polite yet - once you start my game, you have to reboot to get out ;) I'm just learning how to swim here...


To read the keyboard, I'm using WATCOM C weird inline assembly: (result of 255 is "no key pressed" -- note this doesn't handle things like multiple key-presses at the same time, doesn't detect things like SHIFT, ALT, and other instructions are needed to do things like make cursor invisible and to speed up the repeat speed; this is just a basic "what's the last normal key that was pressed")

unsigned char READ_KEY();
#pragma aux READ_KEY = \
" mov ah, 01h" \
" int 16h" \
" jz _done" \
" xor ah, ah" \
" int 16h" \
" jmp _done2" \
"_done: " \
" mov al, 255" \
"_done2: " \
value [al] \
modify [al];


To write to the screen, I am doing this (in a header):

extern unsigned int screen_row_offset[25];
extern unsigned char far *screenout; // = MK_FP(0xB800, 0x0000);
#define WRITE_CHAR(x,y,ch) \
framebuffer[screenout[y]+((x)*2)] = ch

Then in the C file:

#include <i86.h> //< for MK_FP, which I'd like to study more and understand exactly what this is doing

unsigned char far *screenout = MK_FP(0xB800, 0x0000);

unsigned int screen_row_offset[25] =
{
0, // 0
80, // 1
160, // 2
240, // 3
...etc. up to 25 rows


NOTE: WATCOM C has MK_FP defined as
#define MK_FP(__s,__o) (((unsigned short)(__s)):>((void __near *)(__o)))
That ":>" is a custom syntax to their compiler, right?



Once you can monitor keys and draw stuff, you can make a game :) Kind of. The next ingredient is timing. The Apple ][ (out of the box) doesn't actually include a real-time clock - so timing on that platform (without requiring additional clock hardware) you just have to count cycles on the main-loop, which makes for a bit "sloppy" game (speed changes as more animation on the screen -- which you can try to adjust the cycle counting for, but it's never quite smooth). So I'm studying this next on the PC-side, what interrupts to access any kind of time counter [sans DOS] (the TRS80 model 3 has a real time clock, but it's "weird" 1-byte only and counts backwards -- ironically only the oldest PET comes out of the box with a decent clock for game-timing purposes). Meanwhile, my game is already playable on the IBM 5150 using the same Apple][ timing trick - I have to adjust the cycle counts still (and update characters), but it is certainly playable. Once I have a clock figured out, next will be sound. My notes with screenshots are here:

https://destinyhunter.org/ibm-5150-development-notes/


I've been reading the IBM Technical Reference manual that describes the ports, accessed by the OUT opcode. ROM BASIC itself has an "OUT" command to invoke that opcode (wild! and corresponding IN command). In the Commodore PET world, it was called "kernal" calls (all their manuals misspelled kernel -- at least they were consistent about it). The advantage of using these is that they exist in ROM, so they don't take up your own code space (with only 4K, 8K, or even 32K -- or maybe a 512 byte bootloader? -- code-space was a bigger deal). But the other advantage is, if the next system changes hardware (USB keyboards?), your ROM BIOS and software calls should still work -- so I get it, these INTs are like a "software API" realized "in hardware" (in a ROM that in turn invokes the IN/OUT opcodes to talk to ports). But if you have the code-space for it, you could do your own OUT calls (inline assembly) - as I assume "jumping" to an interrupt has a slight overhead (JMP/RET?). For the PET, you can clear the screen in 4-bytes by doing a system call to emulate pressing the HOME key -- but that system call translates that input and sends the appropriate signals to its video hardware. To clear the screen yourself is like 30-bytes of assembly, which only works with THAT video circuit (and much of that same 30-bytes is already in the system ROM, somewhere). It's interesting how one thinks of the problem differently when very resource limited - I've been critical of software development peers who are quite wasteful about resources, selfishly thinking the entire system is available to them. But no: Cores, RAM, network pipes all need to be shared - and that's the challenge (I think) of modern software -- organically scaling your algorithm or processing to a "slice" of the available hardware (which yes, there are "load balancing" solutions out there). Like say just compressing a file - do you want to go full bore where you can barely even move the mouse, or go casual and use a single core? If I'm an HR office worker paid by the hour, use a single core? ;) But if I'm doing crypto, yeah you're not gonna be able to move that mouse for awhile.

And that's the pain I feel for software managers -- if you "inherit" 5 million lines of code, how can you possibly know where the inefficiencies are to carry that code-base forward to the next set of customers and end-users? (I call this the "reading the Torah" problem {or "reading the LotR"} -- even if you sit down and READ the code, it takes a long time to read all that and understand all the characters and function interactions). And we're at that point -- original developers of outstanding base/core libraries are not just retiring, but passing away. We have a fantastic radar detection model, but the original authors are literally not available - and few people can decipher the original FORTRAN. And this is why DESIGN is so important - to communicate the algorithms and workflow/intent. But even UML failed as a way to sync design and implementation - as design is way more than just your class relationships. For this, I certainly get why Microsoft has to "reinvent" a new API every few years.... ;)

The other interesting thing I think is the IoT concept - in a future with an embedded processor everywhere, there will be an increase in demand for low power processors that actually don't need heatsinks or fans (as we already see in Tablet and smart phones-- though they have some sort of heatsinks). I'm thinking more medical, literal "embedded in the human body" type processors. Perhaps there could be a resurrection in 8 or 16-bit computing - so low powered, it runs off your blood flow? This is why maybe there is some increased interest in "real-mode" recently - "I don't want to run with an OS at all" because I'm going to be embedded in a toothbrush, etc.


Anyway - aside for a LITTLE bit of a learning curve, WATCOM C 1.9 has pulled through for me (16-bit real mode .COM binary). Cool, and kudos to those involved with that project!


-SteveL

[[e-mail]]

Post by voidstar tech
NOTE: WATCOM C has MK_FP defined as
#define MK_FP(__s,__o) (((unsigned short)(__s)):>((void __near *)(__o)))
That ":>" is a custom syntax to their compiler, right?

I was about to say that you must have a problem with
your editor, as that is not valid C code, but I checked
my own copy of Watcom, and you are correct. They
seem to have some sort of extension.

BFN. Paul.

[voidstar tech]

Post by ***@gmail.com

Post by voidstar tech
NOTE: WATCOM C has MK_FP defined as
#define MK_FP(__s,__o) (((unsigned short)(__s)):>((void __near *)(__o)))
That ":>" is a custom syntax to their compiler, right?

I was about to say that you must have a problem with
your editor, as that is not valid C code, but I checked
my own copy of Watcom, and you are correct. They
seem to have some sort of extension.
BFN. Paul.

Studied a little more.

Both DigitalMarsC and TurboC have this same macro defined in their version of dos.h (as opposed to i86.h). And they both essentially define it like this (for the 16-bit build):

#define MK_FP(seg,ofs) ((void far *) \
(((unsigned long)(seg) << 16) | (unsigned)(ofs)))

And I verified this version works in WATCOM C 1.9 as well -- only if you're in the TINY model, of course. So I don't need the fancy :> version. (but I assume :> is helping WATCOM C handle whatever changes are needed based on the memory model being used, TINY, LARGE, HUGE, etc -- in that regard, it's a little cleaner than relying on some #defines).


And I examined the WATCOM disassembled code: (with and without the "far")

(without -- wrong, will write to B852h in the current segment? or maybe 0052h... eitherway, wrong)
WRITE_CHAR(1,1,14);
0012 L$1:
0012 8B 1E 00 00 mov bx,word ptr _framebuffer
0016 C6 47 52 0E mov byte ptr 0x52[bx],0x0e

(with -- "FAR" keywords informs intent, use the "Load ES" {extra segment}, and add the "26" prefix of the MOV for segment override)
WRITE_CHAR(1,1,14);
0012 L$1:
0012 C4 1E 00 00 les bx,dword ptr _framebuffer <-- load ES (extra segment) with segment of framebuffer, and BX with the offset
0016 26 C6 47 52 0E mov byte ptr es:0x52[bx],0x0e <-- "52h" is the 1,1 offset in 40 column mode, Eh == 14 (color yellow)

Wild -- it's not just some selfish willy-nilly addition to the C compiler - the processor itself supports this FAR concept, so the compiler need some way to convey the idea of "no, use the segment registers and the register-prefix modifier".

Whew, well, I can appreciate why everyone is so glad those days are over :)

[Rod Pemberton]

On Tue, 8 Jun 2021 12:14:37 -0700 (PDT)

Post by voidstar tech
(result of 255 is "no key pressed" -- note this doesn't handle things
like multiple key-presses at the same time, doesn't detect things
like SHIFT, ALT, and other instructions are needed to do things like
make cursor invisible and to speed up the repeat speed; this is just
a basic "what's the last normal key that was pressed")
unsigned char READ_KEY();
#pragma aux READ_KEY = \
" mov ah, 01h" \
" int 16h" \
" jz _done" \
" xor ah, ah" \
" int 16h" \
" jmp _done2" \
"_done: " \
" mov al, 255" \
"_done2: " \
value [al] \
modify [al];

Is there something wrong with calling _bios_keybrd() which is supported
by more than a few DOS C compilers? E.g., OpenWatcom, DJGPP, Zortech,
Digital Mars, Symantec, Borland, Microsoft, ...

[voidstar tech]

Post by Rod Pemberton
On Tue, 8 Jun 2021 12:14:37 -0700 (PDT)
Is there something wrong with calling _bios_keybrd() which is supported
by more than a few DOS C compilers? E.g., OpenWatcom, DJGPP, Zortech,
Digital Mars, Symantec, Borland, Microsoft, ...
--

Nope, nothing wrong with that - since after all this time, it's highly unlikely any of those platforms would decide to change this particular API.

However, there are some benefits to my alternative: Code size and performance. My code started for the 1MHz 6502 - on a slow processor and being memory limited, every instruction and code-size byte mattered. www.destinyhunter.org is the project.
At 4Mhz and 64KB you start to have a lot more luxury, but I still wanted to understand WATCOMs inline assembly and use of INT16, so this mostly was just a personal exercise to practice that.


Still - to describe why one might NOT want to use this BIOS function call: The call to _bios_keybrd adds 64 bytes to the resulting .COM - and about 8 bytes of that is probably the call/jump and return overhead of the function call itself (the implementation is in a .lib and can't be inlined), My macro takes only 16 bytes and I only need to poll the keyboard in one place in my code, so inlining it means no overhead of the call/return instructions (if I start to poll the keyboard in multiple places, then obviously it ought to become a function and then consume about 24-bytes instead)... Anyway, with 48 bytes to spare, I can pack more content into my game, giving users more value for their time (yes, that's a bit hyperbole, but the point remains: cheap software may still have plenty of utility, but is generally very wasteful and will hog resources - this x86 system is inherently multi-taskable since additional TSRs could be defined, and it's just always good practice to be efficient with your code {except time is money, we can't spend years perfecting opcodes -- so it's always a compromise on development time vs prudent delivery time}). [ in my PET 6502 build -- the difference between having an extra 8-bytes available or not meant whether certain creatures could have more interesting animation effects ]

And then the performance side of things: Large Code also runs slower. Jump/return costs cycles. Those 64-bytes of bios_keybrd do way more than I need. The short type on the 16-bit platform is still 2-bytes, it is returning both an ASCII and scan code. If you dig deeper, the INT 16 call is actually doing that - the overhead in this BIOS call is just that it bothers to return that extra register value into another byte (which also costs cycles). So for that matter, it would be even more efficient to just invoke the OUT opcode myself if all I need is one code or the other - but then I risk not working with a system that has plugged in a different BIOS ROM and uses a different keyboard port address. Which is unlikely - since it wouldn't be a good IBM PC clone if it did, and such is the point of using BIOS calls at all: they abstract me from worrying about port addresses. But if I were building an arcade machine and trusted whoever was making my ROMs, we could squeeze down the code even further. Not that the x86 is really the "go-to" processor for arcade machines, but still - the BIOS was a useful thing to that "open system modularity" of the IBM PC, and the epic documentation thereof [ and that DOS could add interrupts in addition to the ROM BIOS interrupts ]



I'd have to do more measurements on what the cycle (time) cost of bios_keybrd is. But my game is simple, just has a few keyboard controls, and I want the main loop to be as fast/efficient as possible (because I have animation and colors to draw, I don't want a stutery or sluggish keyboard response). Obviously this is all old-school thinking, as modern coding is so abstracted from worrying about code-space size and op-code cycle counts - we think nothing of an EXE that is over 30mb, and since at 3+ ghz we can multi-task so seamlessly, we don't really worry if a zip-compression or transcoding takes 2 minutes vs 8 minutes (since we're occupied with catching up with news in another window). But, for crypto or gaming, the "off the shelf" standard libraries don't cut it - this is why there is sometimes "bad ports" of a game (or "horrible ports"), although these days there is a lot more consistency between platforms (as development studios have perfected their inhouse core libraries -- and those that didn't no longer exists)

I totally respect those who do practice the art of pure-assembly, or who can fluently weave back and forth between .asm and .c files. C can still get "good" performance, but you have to really pay attention and track what the code is getting assembled into. There are situations where even access to main memory becomes your bottleneck -- I've witnessed the difference in performance in a situation that was able to keep everything "on-chip" (within local registers), It's a difference that makes things like "8088 Domination" possible. Someday, I wonder if Software Engineering will become disciplined like Doctors -- wearing different uniforms to identify class of programming: you've got your Surgeons in White (assemblers), Database Programmers in Blue (ODBC, SQL), Application Developers in Green, Web Developers in Brown. No, I don't think that will ever actually happen - but I do think we are evolving different "categories" of programmers. Systems, Library, and Application developers all kind of "think differently" and have different skill sets. Assembly requires a more steady and focused hand, like a surgeon, etc. Application Developers have to continuously chase the hot-new SDK/API of the decade, etc. I've also noticed some people aren't very good at the "0 to 1" task: the initial creation of an initial piece of software, the Version 1.0 pioneers. However, those people might be handy at maintaining or evolving a piece of a software, but given a blank canvas int main() they are a bit lost on how to get started. I've wondered if these "maintainers" are a sort of "Programming Nurse." But software still lacks the discipline of "Software Blueprints" to express design in a meaningful and enduring way - UML tried, but design is way more than just class relationships.

Sorry to ramble on - I'm just curious if IoT and ideas of literally embedded-in-your-body chips (cyborg implants) might lead to a renaissance in interest back to actual efficient software. i.e. will the trend change where "throwing more hardware into the solution" reverses? For example, why can't my toothbrush connect to WiFi and give a cheerful ring when stocks are going good that day -- and let the vibration itself provide enough energy to do so. Let's see .NET or Java do that ;)

v*

[wolfgang kern]

On 11.06.2021 03:01, voidstar tech wrote:

[...about avoiding INT16 BIOS calls...]

I use the direct approach with port 0x60/0x64 access,
this seems to be the shortest/fastest option.

BUT this needs additional code for scan-code to ASCII decoder,
routines for timer-IRQ (delays are required) and keyboard-IRQ.

as long you don't have all this you better go with BIOS INT16.
__
wolfgang

[voidstar tech]

Post by wolfgang kern
[...about avoiding INT16 BIOS calls...]
as long you don't have all this you better go with BIOS INT16.

wolfgang - I started down that path, enough to understand what all needed to be done. In a real-time game application, I do need a bit more control and responsiveness of the keys. With the INT16 calls, there is a delay between transitioning between aim and movement (I think the 250ms delay minimum) -- my Apple][ build suffers from this same thing. I'm keeping with an INT16 call for now, to move on and complete my x86 port. I have symbols and colors all sorted out. The next issue (for me) is audio, the PC speaker. There doesn't appear to be versatile INT calls for freq/duration audio - so the specific challenge in determining a timing policy, to keep the audio consistent whether the user is running at 4MHz or 16MHz. (obviously Adlib or SB support would be better - but I'm porting to the "out of the box" experience of these platforms, their "as-is" configuration when bought from the stores on Day 1).


www.destinyhunter.org

v*

[voidstar tech]

Post by Rod Pemberton
On Tue, 8 Jun 2021 12:14:37 -0700 (PDT)
Is there something wrong with calling _bios_keybrd() which is supported
by more than a few DOS C compilers? E.g., OpenWatcom, DJGPP, Zortech,
Digital Mars, Symantec, Borland, Microsoft, ...

Sorry for the earlier lecture, as I'm sure everyone here is familiar with code/size/cycles issues.

I should have simply said:  I'm ported to more than just DOS (e.g. cc65 and z88dk for the 6502 and z-80 processors don't have this interface, nor an equivalent BIOS).   So my "core" keyboard function standardizes on a single byte return, with "255" indicating that no key was pressed.     And certainly I could have wrapped bios_keybrd for the DOS build of the program - but as my own exercise, I'm preferring to have my core functions using #define's and inline assembly so I can better account for what is going on in terms of cycles (and just better understanding these systems).

v*

[James Harris]

On 10/06/2021 17:52, Rod Pemberton wrote:

...

Post by Rod Pemberton
Is there something wrong with calling _bios_keybrd() which is supported
by more than a few DOS C compilers? E.g., OpenWatcom, DJGPP, Zortech,
Digital Mars, Symantec, Borland, Microsoft, ...

Speaking of something wrong ... there's something very wrong with having
a function which is supported only by certain compilers.

What you say sounds like the old 1980s mess where different compiler
companies offered their own facilities which were meant to be an
improved superset of C but all it did was end up with lots of programs
which were tied to specific compilers.

Worse, because people wanted to use different compilers we ended up with
programs where the logic was interrupted with ifdefs so the programs
could adapt to different compiler supersets.

There was never a need for any of that. Any C compiler ought to be able
to emit code to call any legitimate function - including functions to
interface with a PC keyboard.

Rant over!

[voidstar tech]

Post by James Harris
What you say sounds like the old 1980s mess where different compiler
companies offered their own facilities which were meant to be an
improved superset of C but all it did was end up with lots of programs
which were tied to specific compilers.

Ah, but that's not quite fair. Different situations have different needs.

For example, I couldn't use stdio because printf was "too much" for my needs - it linked in over 2K of code, since it had a lot more capability than I needed. I just needed to print some small integers (2-4 bytes), no exotic padding and no floating points. So I have my own functions that do so directly to the screen (that's the other "baggage" that stdio brings in, it can re-direct the "i/o" to other streams - like a line printer or file). Or as my other example: one library might have a keyboard function that returns both a scan code and ASCII code - I'm in a tight loop that is timing critical, I'll just take the scan code (and don't want any overhead of indexing into an ASCII cross-reference table or populating another byte with that ASCII code). It's a catch-22: I can't expect a library to account for every need and situation -- because if they try to do so, they now become "bloated" (i.e. passing 10 arguments with a bunch of knobs and dials, that itself has a performance and stack cost).

Plus in the early 1980s, there was no wide-spread public internet, so "coordinating" libraries was difficult - it's 1987 and Borland has a release ready, they can't wait 10+ years to perfect it and make sure the entire industry is onboard with what they came up with. Although, I suppose any addition or alternations they did could have been more clearly marked as a separate "Borland Lib SDK" instead of mingled in with "standard C" headers.

My only complaint is that these "extra" libraries were "closed source", with just .h and .lib, so you couldn't verify the implementation or adjust it. However -- I'm now more sympathetic about that: disk space was very limited back then. Sure, vendors wanted to keep their "secret sauce" of how they provided certain functions -- but also just as a practical matter, they couldn't afford to include many extra disks for the relatively few people who would have appreciated that library source. And again, it wasn't like they could host their libsrc at their website -- as websites didn't exist in the 80s (perhaps some did offer a BBS to download a src, or would provide if you just called and found the right person to ask, likely for a price).

And when you start dealing with lots of systems, you realize the "C standard library" is not all that standard - there are subtle differences across versions of the Microsoft compiler, or when transition between Windows and Linux or MacOS, or consider even between Win32 and Win64 transitions (there were subtle differences in float handling on 64-bit, as internal portions of your processing might get "up converted' to the newer 128-bit float registers, despite being a 32-bit float type). Any change in platform requires new rounds of regression testing.

I suppose this is why I'm critical of the use of Python for certain projects. We have young engineers who will just use Python as part of an integrated solution -- yes, it can reformat some content in a .txt file and solve certain problems very quickly. But you have less "control" over it - to programmatically invoke those functions, or maybe to exercise certain things in parallel. And the result is subject to changes in behavior in the next release (I've found differences between Python 2.7 and 3.X, and eventually those that maintain OS and system images will drop 2.X support). Python is like a pre-compiled "all that extra API stuff that your compiler platform couldn't provide" :) I know, not exactly. But it is the same effect of bringing in "JoeBlows C Library" - it might do exactly what you need at the moment, but can also lead to "Vendor Lock". But yet, even for in-house demos or making release schedules, you have to depend on some existing library eventually.

It's just the "yucky" thing about C or C++ -- those "platform.h" files to try to "normalize" cross-platform and cross-compiler aspects of your project. It has to be done, but yep it's a pain.

v*

[James Harris]

Post by voidstar tech

Post by James Harris
What you say sounds like the old 1980s mess where different compiler
companies offered their own facilities which were meant to be an
improved superset of C but all it did was end up with lots of programs
which were tied to specific compilers.

Ah, but that's not quite fair. Different situations have different needs.
For example, I couldn't use stdio because printf was "too much" for my needs - it linked in over 2K of code, since it had a lot more capability than I needed. I just needed to print some small integers (2-4 bytes), no exotic padding and no floating points. So I have my own functions that do so directly to the screen (that's the other "baggage" that stdio brings in, it can re-direct the "i/o" to other streams - like a line printer or file). Or as my other example: one library might have a keyboard function that returns both a scan code and ASCII code - I'm in a tight loop that is timing critical, I'll just take the scan code (and don't want any overhead of indexing into an ASCII cross-reference table or populating another byte with that ASCII code). It's a catch-22: I can't expect a library to account for every need and situation -- because if they try to do so, they now become "bloated" (i.e. passing 10 arguments with a bunch of knobs and dials, that itself has a performance and stack cost).

I agree with you but why not just omit stdio.h from the compile and its
corresponding library (libc?) from the link and, instead, compile and
link to your own IO code?

Post by voidstar tech
Plus in the early 1980s, there was no wide-spread public internet, so "coordinating" libraries was difficult -

Yes, that's a very good point.

Post by voidstar tech
it's 1987 and Borland has a release ready, they can't wait 10+ years to perfect it and make sure the entire industry is onboard with what they came up with. Although, I suppose any addition or alternations they did could have been more clearly marked as a separate "Borland Lib SDK" instead of mingled in with "standard C" headers.

Indeed. I recognise that for at least some of the compilers Rod
mentioned it was a natural response to commercial pressures and the
desire to get ahead of the competition that led to the non-standard
supersets of C to which I objected ... but it still led to an awful mess
of code which was tied to and dependent on certain compilers if we as
programmers went along with it which, understandably, we did at the time.

Lock-in to their products was what compiler vendors would have wanted,
of course, but it was a bad situation for programmers. As you suggested,
below (now snipped), useful libraries could have been distributed
separately but there was little or no commercial incentive to do so.

...

I agree with what you said about Python, too. There's a point at which
programming in a certain language morphs from using the language itself
to using libraries that someone else has written "because it's more cost
effective to use pre-tested and pre-debugged code". The problem is that
programmers then have the added burden of learning many different
libraries or frameworks rather than just having to learn the language.

My own view on that is that a language should have a standard library
which provides the majority of data structures which a programmer might
want, and a syntax in which data flows are always evident so that a
programmer can see the basics of what a statement will do even if he
doesn't know about a particular function being called. But this is more
a topic for comp.lang.misc than a.o.d so I'll not go in to it here.
Suffice to say a language and a standard library can do a lot to reduce
the learning curv

[[e-mail]]

Post by James Harris

Post by voidstar tech
Plus in the early 1980s, there was no wide-spread public internet, so "coordinating" libraries was difficult -

Yes, that's a very good point.

Well, people managed to organize ASCII. And there was
an ANSI C draft well before 1990. Why couldn't they
organize MSDOS APIs too?

Regardless, here is my offering, which is designed not
just for 16-bit MSDOS, but also 32-bit and 64-bit MSDOS:

https://sourceforge.net/p/pdos/gitcode/ci/master/tree/src/pos.c

Actually, it can be an alternative to Posix and the Win32
API and the OS/2 API. Programmers can go via this instead
of their native API.

Post by James Harris

Post by voidstar tech
it's 1987 and Borland has a release ready, they can't wait
10+ years to perfect it and make sure the entire industry is
onboard with what they came up with.

Yes, but we can, belatedly.

BFN. Paul.

[James Harris]

Post by ***@gmail.com

Post by James Harris

Post by voidstar tech
Plus in the early 1980s, there was no wide-spread public internet, so "coordinating" libraries was difficult -

Yes, that's a very good point.

Well, people managed to organize ASCII. And there was
an ANSI C draft well before 1990. Why couldn't they
organize MSDOS APIs too?

ASCII is a standard and could be published on a single page in a
magazine; it didn't need distribution. By contrast, libraries would have
needed to have been distributed to users on floppy disk.

Further, the commercial pressures would have been for a character set to
be standardised but for compiler packages to be distinct.

Post by ***@gmail.com
Regardless, here is my offering, which is designed not
https://sourceforge.net/p/pdos/gitcode/ci/master/tree/src/pos.c

It looks like very neat code.

Post by ***@gmail.com
Actually, it can be an alternative to Posix and the Win32
API and the OS/2 API. Programmers can go via this instead
of their native API.

As long as it can run on their machine. See below.

Post by ***@gmail.com

Post by James Harris

Post by voidstar tech
it's 1987 and Borland has a release ready, they can't wait
10+ years to perfect it and make sure the entire industry is
onboard with what they came up with.

Yes, but we can, belatedly.

Ah, but can we? AIUI you don't want to use emulation but do want to
standardise on real mode and a BIOS. (Forgive me if I've got your goals
wrong.) But I think I saw in another thread people saying that newer
machines come without a BIOS and boot into protected mode or long mode
so what you want will become harder and harder to provide universally,
won't it, rather than easier?

[[e-mail]]

Post by James Harris
ASCII is a standard and could be published on a single page in a
magazine; it didn't need distribution. By contrast, libraries would have
needed to have been distributed to users on floppy disk.

Well BBSes existed. After that, floppies are fine. This is
for developers, after all. Developers are keen enough to
find a way.

Post by James Harris
Further, the commercial pressures would have been for a character set to
be standardised but for compiler packages to be distinct.

Ok, so there is a flaw in the capitalist system that requires
someone who understands the problem (ie you) to step in.

Governments already understand the problem with monopolies,
so Bell got broken up for example.

There is room for intervention in capitalism.

Post by James Harris

Post by ***@gmail.com
Regardless, here is my offering, which is designed not
https://sourceforge.net/p/pdos/gitcode/ci/master/tree/src/pos.c

It looks like very neat code.

Thankyou. And like I said, it's just a suggestion. I'm
open to alternatives. Not alternatives like some
copyright API though. I saw that drama unfold a
short time ago. 2021 I think the year was.

Post by James Harris

Post by ***@gmail.com
Actually, it can be an alternative to Posix and the Win32
API and the OS/2 API. Programmers can go via this instead
of their native API.

As long as it can run on their machine. See below.

It's just an API. Of course it will. It can be defined in
terms of both Posix and Win32.

Post by James Harris

Post by ***@gmail.com

Post by voidstar tech
it's 1987 and Borland has a release ready, they can't wait
10+ years to perfect it and make sure the entire industry is
onboard with what they came up with.

Yes, but we can, belatedly.

Ah, but can we? AIUI you don't want to use emulation but do want to
standardise on real mode and a BIOS. (Forgive me if I've got your goals
wrong.)

No, that's not quite correct. First of all I'm going to start
bombing (via NATO airstrikes, not personally, I'll be in
bed watching the war unfold on http://news.bbc.co.uk)
Taiwanese manufacturers who don't provide CSM
support.

In my experience, dum-dum bullets really drive the point home.

And then as a separate exercise I'm going to try zapping
firmware to get CSM back.

And then, as a last resort, I'm going to pollute my beautiful
(imagine if you had seen that in 1990, when C90 came out)
pdos.img which you can find at pdos.org with a bleah
EFI/BOOT/BOOTX64.EFI
which will provide a CSM for those without CSM and who
are worried that something might break if they zap their
firmware.

They can add that on themselves, I'm not going to pollute
my own software.

It's bad enough seeing my beautiful directory listing being
polluted with a "System Information" directory whenever
someone sends me a screenshot. Trust me, that's not me
who did that.

Post by James Harris
But I think I saw in another thread people saying that newer
machines come without a BIOS and boot into protected mode or long mode
so what you want will become harder and harder to provide universally,
won't it, rather than easier?

Don't worry too much about PDOS. That's my war with Taiwan,
and I'm good friends with Jens, so I don't like their chances.

Pos* is independent of that, and will allow you to write
programs that work on Windows, although I've never
proven that for sure because it hasn't been my priority
to get the Pos* functions to call Win32 equivalents.
Although I should at least do that for the functions like
PosMakeDir() that I used in my version of "unzip" which
you can find in custom.zip at http://pdos.org

But I have other priorities before I do that.

BFN. Paul.

[[e-mail]]

Post by ***@gmail.com
Pos* is independent of that, and will allow you to write
programs that work on Windows, although I've never

And note that 32-bit Windows executables, at least
basic ones, are universal. Every PC that had an 80386
or better can run them, regardless of which operating
system you are using (almost, anyway), although you
will sometimes need to install extra software (like
Wine) to do it. MSDOS/Freedos has HX. PDOS/386
is native. And in all cases, your Win32 executables
will run directly on the CPU.

BFN. Paul.

[[e-mail]]

Post by ***@gmail.com
It's bad enough seeing my beautiful directory listing being
polluted with a "System Information" directory whenever
someone sends me a screenshot. Trust me, that's not me
who did that.

http://www.ratman.biz/archive/young_ones/summerholiday.html

NEIL: [grimacing, threatening an imaginary opponent] I didn't get them
at Tesco's, OK?

[Scott Lurndal]

I'm mentioning Alex because I think he wrote an article about all this back=
in 2004! :)
I've written a C game that targets the 6502 (PET/Apple][) and Z80 (TRS80) a=
nd fits within 32K.
I'm now trying to target the x86 16-bit (specifically the original IBM PC 5=
150), so I don't want any 80186, 80286, or 80386 instructions.

Not to put too fine a point on it, but 5150 was, perhaps, an
unfortunate choice of designator for the IBM PC.

https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=WIC&sectionNum=5150