VOGONS


First post, by RetroMaster137

User metadata
Rank Newbie
Rank
Newbie

As always, I hope it's posted in the right section, pardon me in case it's not.

Introduction
Hello. On my spare time I have worked on games and other misc software (i.e. PoCs) for old/"embedded" systems in MC68000's supervisor mode, both in C and Assembly (sadly not x86, I don't even know its non-extended instruction set just yet), and lately on my own custom engine. Mentally accostumed to big-endian, my work usually involved demoscene-like effects on very limited GPUs; by this point I wanted to work in VFX and maybe just try making some plugin for After Effects in the future or whatever, but the industry's fate is quite uncertain.
Instead, I made the leap to high-level or "modernity", worked some stuff in C# (including Unity-based games), a PHP game server with poor security, tinkering and modding bits of a leaked Unreal Engine 1 game's source code (C++ and UnrealScript for the most part), a bit of javascript...
This time I'd like to learn the low-level of a PC; I get most of my info from OSDev's wiki, but its forums don't seem to be too active anymore and I can't post either because my ISP's IP range is in some public blacklist (I'm under NAT, and I swear it's not my fault! 😜). I'm no hardware expert.

My PC's Specifications (once again, but just in case)

  • Motherboard: ASUS A55BM-K
    • Chipset: AMD A55 FCH (Hudson D2)
    • BIOS: AMI BIOS (A55BM-K BIOS v2002)
      • Microcode version: 6001119 (2012-07-13)
  • APU: AMD A8-6600K (Quad @4.2Ghz)
    • iGPU: Radeon HD 8570d
  • dGPU: Aisurix RX 580 8GB 2048sp
    • VBIOS: RX 580's original ATOMBIOS, v015.050.002.001.000000
  • RAM: 2x8 (=16) GB @1833Mhz

PC design
Unlike a system with a defined setup and specific documentation, PC is so variable it's much more tedious and difficult in comparison.
Unlike the old systems I'm accostumed to, the CPU has multiple execution modes and in different bit extensions, each giving different privileges and address maps. A PC in general seems to have really powerful address mapping system(s), for the general address map and the RAM itself, correct?
Due to design and compatibility matters, the initial state is 16bit real mode (just like the original 8086), BIOS changes it for its own usage, but returns to real mode in the hand-off to boot device/OS, correct? Then any other mode is an extension (i.e. 64bit is called "long mode"), and you can either back to real mode (if got the privilege to do so) or enter virtual 8086 mode should it be needed (the latter causes many faults/interrupts during runtime, for a supervisor program to take care and decide how to deal with them). Do those advanced virtualization settings from BIOS work by enabling a full-fledged environment for v8086 mode, in turn allowing it to switch into a "virtual long mode" if the code asks for it?

Vectors (Interrupt ReQuest (IRQ) or Interrupt Vector Table (IVT))
Where is it? (EDIT: At the very beginning of the map) Is it possible to edit the pointers and/or remap the table's address to another?
Just took a quick read to this, and I wonder, is the BIOS ROM (or part of it) truly visible from the address map, or something else makes a copy of it into RAM? Does the CPU begin execution at a specific address (0xF0000 ?) not specified by IVT? And finally, how does PC deal with larger BIOS, such as the latest v2002 update for this mobo, which is 8192kb?

Microcode
Because processor designs change over time, "microcode" itself is an x86(+extensions) emulator, which translates all instructions with its registers into whatever set the chip "internally" uses, correct? In this case, how does the BIOS even manage to boot and load a microcode software/version to use at all? Is a default microcode incorported into the CPU itself?, or, do BIOS devs need to be aware of such "internal" instruction set so microcode contained within BIOS could be loaded before actually executing x86?

BIOS / UEFI
The first full-blown program that gets mapped (or loaded through RAM?) into address map, whose visible execution consists of integrity checks, a "general configuration menu", formerly included BASIC (lol), boot into a device chosen or preset by the user, and so on. It also manages the Option ROMs or OpROMs used by/for attached devices, for networking and such.
The BIOS itself could also subdivide itself into different, if "virtual" OpROMs as well, such as iGPU's VBIOS, chipset init and setup code, or others for misc. purposes, correct? However, weeks ago I read on something related to coreboot, that some retail BIOSes COULD keep their VBIOS display tables blank (i.e. VESA modes), only to fill them from "somewhere else" (in theory, I guess that means the tables are editable at runtime?).
I have a hard time conceptually understanding UEFI. Is it like, a standarized way of making a "general configuration menu" + API? as in, is UEFI a BIOS OpROM or submodule of some form, possibly compiled into some specific bytecode whose interpreter is located within the BIOS itself?
For this AMI & ASUS scenario, which part is which? Does AMI make the "basic" BIOS part & API, which in turn hands-off to a "UEFI" part & API made by ASUS?

AMD AGESA
What exactly is this? Chipset/overall initialization OpROM/submodule, AMD-specific?

AMD VBIOS
For all I know and understand, this is called ATOMBIOS here, and in order to use anything from the GPU, you either (A) use the old API, virtualized ports, and areas of the address map (i.e. VBE or such), or, (B) you use the VBIOS' API, correct? In order to actually make use of the ATOMBIOS functions, they must be ran through an interpreter for its own bytecode format. But then how does Windows Basic Display Driver, MS-DOS by itself, or VBE even work? Does the BIOS feature its own ATOMBIOS interpreter, used to virtualize the "A" way? If I'm not mistaken, there's no true text modes anymore, so the BIOS has to perform text-to-graphic conversion anyway; the program unknowingly writes to a backbuffer, while BIOS transforms and copies into the true graphical frontbuffer.

Planning a TSR
With all this information, as a learning exercise I'd like to try arranging or making a TSR, such that it runs under VBE 3.0 (or 2.0, if it happens to not be too different?) at some "fail-safe" setting such as 640x480px or 1024x768px in 16 or 32 bit color mode, modifies the display table and/or makes the system return a different table with more options, and traps/remaps whatever necessary in order to be able to produce a scaled or arranged output for the true display mode the TSR is set to run at, or fallback if settings are the same. The end goal is a software-based compatibility layer.
I have thought on getting the source code of a well-functioning TSR, such as SBEMU if it's not trouble (with all due credits of course), and give a try at writing this planned TSR on top of it.
Is this possible and doable?

Thanks for your time.

Reply 1 of 3, by jakethompson1

User metadata
Rank Oldbie
Rank
Oldbie

As far as I remember those virtualization settings in your BIOS just mask away certain processor features before boot if you either need to hide them from an incompatible OS, or don't want them used for some security reason. You would have to reset the CPU to re-enable them, i.e. the OS can't turn them back on; that's the idea. Hardware assisted virtualization of protected mode on x86 came in fairly late (like 2005) so it isn't really relevant to the rest of this discussion as nothing from before then depends on it.

In real mode (which is what you care about for a TSR) the interrupt vectors are indeed at 0:0 and can't be moved. Note that interrupt handling is vastly different than that in protected mode, but for TSR development, you can ignore that. It's part of the DOS extender/VMM's job to get you back to real mode.

The CPU resets in a special setup with CS=F000 IP=FFF0 in the registers but with the descriptors in a state such that the physical address actually presented is FFFFFFF0. You can read about it on page 248 here http://bitsavers.trailing-edge.com/components … Manual_1990.pdf Part of that is so that you can force those reads to occur from ROM whereas FFFF0 can be remapped to RAM.

I'm not familiar with such late BIOS, but I can tell you that 128KB ones can boot in a state where the first 64KB is at E0000 and the rest is at F0000, and the E part is unmapped before booting. It's possible newer BIOSes also use such bank-switching of 64KB portions into E0000 or maybe they use an entirely different approach. Something you haven't mentioned is System Management Mode which could be using a different memory map without the need to maintain IBM PC compatibility and perhaps the BIOS there is mapped into a different and much larger memory window.

The system really does begin booting with code executing directly from ROM at F000:FFF0, however, everything from about 1995 onward uses a compressed BIOS ROM, so it doesn't remain in that state for long. The first part of the BIOS is just a stub that tweaks PCI configuration registers to configure the chipset to make F0000 write to RAM but read from ROM, then decompresses the relevant portions into RAM, then tweaks registers again to reconfigure F0000 to read from RAM and be write protected. This is "shadowing" and becomes mandatory around the time of PCI because the BIOS is too big to fit otherwise.

The CPU powers up already capable of executing x86 code. I don't know whether the microcode updates can even replace 100% of the microcode or if Intel is abusing the definition of "microcode" somewhat. I believe the FDIV bug was part of the reason for the runtime-upgradeable microcode. I think there were some CPUs that were so buggy they couldn't even fully boot without the microcode being upgraded, and that's part of the reason the BIOS has to be stuffed with microcode for every x86 CPU that motherboard supports. I do not know if it would be possible to try to do a modern BIOS without any microcode updates, leaving that to the OS.

I don't know anything about UEFI at this deep level. It is a complete rewrite though, not just another bolt on to BIOS. In fact many 2010s systems with a BIOS are actually a Compatibility Support Module on top of UEFI, not the reverse. One unfortunate thing about UEFI is that Microsoftisms like FAT32 file systems and PE-format executables are now permanently baked into the PC architecture.

I'm not sure about the VBE part of your TSR, but it sounds like what you want to do is hook Int 10h. When called, you check the state of the registers to see if it's an Int 10h call you understand, and if not, pass it to the old Int 10h handler that was in place when your TSR loaded. If it is one you understand, you handle it in your TSR. That's all pretty standard TSR programming. When called, you'll re-enable interrupts, and will have to remember that CS: points to your TSR while DS: points somewhere undefined, and your stack is SS: in whatever program is running at the time, and could be arbitrarily small so that's a gotcha.

Reply 2 of 3, by bakemono

User metadata
Rank Oldbie
Rank
Oldbie
RetroMaster137 wrote on 2024-09-11, 22:47:

Vectors (Interrupt ReQuest (IRQ) or Interrupt Vector Table (IVT))
Where is it? (EDIT: At the very beginning of the map) Is it possible to edit the pointers and/or remap the table's address to another?
Just took a quick read to this, and I wonder, is the BIOS ROM (or part of it) truly visible from the address map, or something else makes a copy of it into RAM? Does the CPU begin execution at a specific address (0xF0000 ?) not specified by IVT? And finally, how does PC deal with larger BIOS, such as the latest v2002 update for this mobo, which is 8192kb?

That 8MB of firmware is contained in a SPI flash ROM chip, which is incredibly slow relative to modern CPUs. Early in the power-on sequence, pieces of it get copied to CPU cache (since DRAM can't be accessed until the memory controller is initialized) and later to RAM.

Microcode
Because processor designs change over time, "microcode" itself is an x86(+extensions) emulator, which translates all instructions with its registers into whatever set the chip "internally" uses, correct? In this case, how does the BIOS even manage to boot and load a microcode software/version to use at all? Is a default microcode incorported into the CPU itself?, or, do BIOS devs need to be aware of such "internal" instruction set so microcode contained within BIOS could be loaded before actually executing x86?

Most of the x86 instructions get decoded directly into so called micro-ops. Only certainly operations, which are too complex or obscure for direct translation, are handled by microcode. CPUs ship with microcode in an internal ROM, and if it turns out to be buggy then a patch can be applied at runtime by the BIOS or OS.

BIOS / UEFI
However, weeks ago I read on something related to coreboot, that some retail BIOSes COULD keep their VBIOS display tables blank (i.e. VESA modes), only to fill them from "somewhere else" (in theory, I guess that means the tables are editable at runtime?).

I imagine that would be similar to ACPI tables, which are generated during POST and stashed in RAM somewhere. For instance, if I boot to DOS and check the pointer to the ACPI table it's at 0xDC840000.

I have a hard time conceptually understanding UEFI. Is it like, a standarized way of making a "general configuration menu" + API? as in, is UEFI a BIOS OpROM or submodule of some form, possibly compiled into some specific bytecode whose interpreter is located within the BIOS itself?

I think basically, UEFI is a new generation of firmware, which supplants BIOS. Whereas 'CSM' fills the role of the traditional BIOS, for compatibliity purposes. The main difference between the old BIOS and UEFI (besides massive bloat) is that UEFI has a standardized execution environment, like a mini-OS, to run programs.

AMD AGESA
What exactly is this? Chipset/overall initialization OpROM/submodule, AMD-specific?

It is AMD specific. It originally did CPU identification and some other things like filling in data tables used for power management. Probably serves more functions these days.

AMD VBIOS
For all I know and understand, this is called ATOMBIOS here, and in order to use anything from the GPU, you either (A) use the old API, virtualized ports, and areas of the address map (i.e. VBE or such), or, (B) you use the VBIOS' API, correct? In order to actually make use of the ATOMBIOS functions, they must be ran through an interpreter for its own bytecode format. But then how does Windows Basic Display Driver, MS-DOS by itself, or VBE even work? Does the BIOS feature its own ATOMBIOS interpreter, used to virtualize the "A" way? If I'm not mistaken, there's no true text modes anymore, so the BIOS has to perform text-to-graphic conversion anyway; the program unknowingly writes to a backbuffer, while BIOS transforms and copies into the true graphical frontbuffer.

Don't know anything about VBIOS or ATOMBIOS, but the VGA BIOS and VBE APIs have always depended on the video card's firmware which accesses the hardware directly.

Planning a TSR With all this information, as a learning exercise I'd like to try arranging or making a TSR, such that it runs un […]
Show full quote

Planning a TSR
With all this information, as a learning exercise I'd like to try arranging or making a TSR, such that it runs under VBE 3.0 (or 2.0, if it happens to not be too different?) at some "fail-safe" setting such as 640x480px or 1024x768px in 16 or 32 bit color mode, modifies the display table and/or makes the system return a different table with more options, and traps/remaps whatever necessary in order to be able to produce a scaled or arranged output for the true display mode the TSR is set to run at, or fallback if settings are the same. The end goal is a software-based compatibility layer.
I have thought on getting the source code of a well-functioning TSR, such as SBEMU if it's not trouble (with all due credits of course), and give a try at writing this planned TSR on top of it.
Is this possible and doable?

You could implement your own VBE layer for applications that use VBE. For those that use VGA registers/memory, then you'd have to depend on the EMM386/SBEMU type of strategy, of intercepting I/O and whatnot.

GBAJAM 2024 submission on itch: https://90soft90.itch.io/wreckage

Reply 3 of 3, by jtchip

User metadata
Rank Member
Rank
Member

I guess you've decided to implement the video TSR for "modern" AMD Northern Islands graphics yourself. I guess you should follow @jakethompson1's suggestion of writing a TSR that hooks on to int 10h. I only have limited experience with writing DOS TSRs, having written similar TSRs that hook on to interrupts back when DOS 6 and Win 3.1 were current (and haven't done any DOS programming since). I have no experience with protected mode DOS programming and one of the things you'll probably have to implement is support for protected mode since most of the interesting software that use SVGA modes like 640x480x8b would be in protected mode.

The Northern Islands GPUs should still have 8bpp modes since AMD's app note mentions that they do have a VBIOS that supports such modes in VBE but don't supply it by default. So I think this should be limited to just reimplementing parts of the VBE layer and not go to the extent of trapping port IO.

I suggest starting with a simple TSR that hooks on to int 10h and just calls through to the existing handler. Once that's done, intercept function 01h (return VBE mode information) and add some 8bpp modes. Then intercept function 02h (set VBE mode) but when a call tries to set one of the added 8bpp modes, set a 16bpp mode instead at the same resolution. At this point, users of that mode should display something but it will be half as tall, with 2 rows combined into 1, and have the wrong colours.

Then change the modesetting for the 8bpp modes based on some other method, probably see the Linux KMS (Kernel ModeSetting) code though I'm not sure if that actually supports paletted modes, you might have to look up the datasheets for that. Maybe disassemble the int 10h handler from the VBIOS to see what it does for your card. You'll also have to implement function 08h/09h to get/set palette.

Hopefully someone more familiar with VBE can help in case what I posted was nonsense 😀