clb wrote on 2021-07-23, 19:59:

Hello,

I am trying to wrap my head around the mechanism that Boot ROMs work, in particular around booting and BIOS. After watching videos on the topic and reading XT-IDE and other BIOS ROM product pages, there are still some things I'm a bit uncertain about. Actually, quite a few things.

1) If I understand correctly, the general machinery is that the motherboard BIOS decides at boot which real mode addresses should map to onboard RAM, and which addresses don't? The addresses that do not map to RAM will generate loads and stores on the ISA bus? Then each ISA card snoops the bus for these accesses and when they see one, they go "hey, I have a ROM on board that is configured to live in that address range, I'll respond to that memory request". The mechanism is a bit uncoordinated/"dumb": only manual jumper configuration ensures that multiple ISA cards won't overlap memory addresses for their ROMs.

As I understand it the upper 384K (A000-F000) can never map to onboard RAM, there is either an ISA card listening at those addresses or not. The only exception to that (and the only piece it seems like your description is missing), is Shadow RAM, where special support in the chipset maps some RAM at some of these addresses, puts a copy of the ROM into that RAM, and then write-protects it. The only reason for this is that RAM is faster than ROM. Some chipsets have special support to emulate an EMS card in hardware using onboard RAM, but that's an aside.

clb wrote on 2021-07-23, 19:59:

2) Are the colloquial terms "BIOS ROMs" (besides the main BIOS ROM onboard), "Boot ROMs" and "Option ROMs" all referring to the same thing - an EEPROM on an ISA card providing a memory block in some address range?

Yes, I think Option ROM is most correct though. The only oddball case you're missing here is ROM BASIC.

clb wrote on 2021-07-23, 19:59:

3) How does the BIOS know which real mode addresses (in UMB) should get routed over to RAM, and which real mode addresses will contain BIOS ROMs? E.g. if I put in a network card with an 32k EEPROM socket to my 386, how will DOS know not to allocate precious RAM over that memory area, but assign that RAM over to Extended memory instead? Or other way around, if I don't have any ROMs in e.g. address B4000-B7FFF, how does DOS know to map memory there for an UMB block? It seems to me that there would need to be some kind of discovery protocol in play for the system BIOS to locate all the ROMs?

The BIOS will not put normal (read-write) RAM there. EMM386 takes care of creating UMBs by using features of the 386 processor to map virtual accesses to that 384KB region to either the same physical addresses (if there is video RAM/option ROM there), or to a different physical location in extended memory if the UMB is "empty". Usually BIOSes will shadow the system BIOS (64K) and video BIOS (32K or maybe 64K?) . Some chipsets have a "Memory Remapping" feature to, in hardware, move the "lost" RAM at A000/B000/D000/E000 to the end of extended memory like you describe. That feature seems to have disappeared with PCI chipsets. That's also when BIOSes started getting compressed so maybe the "lost" RAM is used by the BIOS at runtime in some kind of bank switching technique?

clb wrote on 2021-07-23, 19:59:

4) XTIDE Universal BIOS (https://www.xtideuniversalbios.org/) enables booting from large hard drives. Does XTIDE Universal BIOS somehow replace the motherboard system BIOS altogether (F0000h upwards)? Or does it live in a separate ROM memory block than the main motherboard BIOS?

It has to live somewhere else. At boot time the system BIOS will scan these possible-option-ROM locations for a 55AA signature similar to what a boot sector has. If it is there, the BIOS passes control to it, expecting it to pass it back afterward. The XT-IDE BIOS would hook hard drive related software interrupt vectors like a chain. When an interrupt comes in, it either handles it itself, or if it's unwanted (like floppy-related) passes it along to the original interrupt vector, so in this case floppy access would still go through the system BIOS. A SCSI BIOS would work the same way.

clb wrote on 2021-07-23, 19:59:

5) How does XTIDE hook into the boot process? Is it some kind of a general feature/protocol of all PCs that the system BIOS asks each ROM if they want to take over booting? Or do the ROMs somehow provide hard disk information to the BIOS?

Yes, this is what the 55AA signature is for, so the BIOS can scan for them.

clb wrote on 2021-07-23, 19:59:

6) Finally, if I want to dump a ROM, is it identical to do it via the DEBUG command ( http://mess.redump.net/dumping/dump_bios_using_debug ) (after using e.g. MSD or some other tool to locate the ROM address and size in memory) vs an
EEPROM reader?

For XT-IDE, I think that is fine. It may be harder to dump a complete BIOS this way, because an uncompressed portion of the BIOS sits at F000 at runtime, but what's actually on the ROM chip is compressed so you would either need the EEPROM reader or some other way of reading the chip that is not just dumping RAM (which is what I guess awdflash etc. use?)

Thanks for the great answer! I think I got a much better picture of this now!

jakethompson1 wrote on 2021-07-23, 20:27:

The BIOS will not put normal (read-write) RAM there. EMM386 takes care of creating UMBs by using features of the 386 processor to map virtual accesses to that 384KB region to either the same physical addresses (if there is video RAM/option ROM there), or to a different physical location in extended memory if the UMB is "empty".

Oh I see. So this mapping between ROMs and RAM in upper memory requires virtual memory addressing(?), and so having any upper memory at all must then require that the CPU is in the Virtual 86 mode? That was introduced with 386, so does that mean that 286s and 8088/8086s cannot get any upper memory at all, even if one installs a RAM board like this? https://www.lo-tech.co.uk/product/1mb-ram-board-pcb/

jakethompson1 wrote on 2021-07-23, 20:27:

It has to live somewhere else. At boot time the system BIOS will scan these possible-option-ROM locations for a 55AA signature similar to what a boot sector has.

Gotcha - so I suppose there exists some kind of "master list" of recognized Option ROM locations, like B000h, B800h, C000h, C800h etc. that the BIOS will poke on the ISA bus to discover Option ROMs? (since surely the BIOS would not scan through all memory addresses)

jakethompson1 wrote on 2021-07-23, 20:27:

If it is there, the BIOS passes control to it, expecting it to pass it back afterward. The XT-IDE BIOS would hook hard drive related software interrupt vectors like a chain. When an interrupt comes in, it either handles it itself, or if it's unwanted (like floppy-related) passes it along to the original interrupt vector, so in this case floppy access would still go through the system BIOS. A SCSI BIOS would work the same way.

Thanks, this was the missing bit. So BIOS calls into the ROMs at boot, but after the BIOS has already set up its own interrupt vector tables (and all other initialization?), so that whatever the ROMs do, they'll have a guaranted late init order in order to be able to override interrupts.

jakethompson1 wrote on 2021-07-23, 20:27:

clb wrote on 2021-07-23, 19:59:

6) Finally, if I want to dump a ROM, is it identical to do it via the DEBUG command ( http://mess.redump.net/dumping/dump_bios_using_debug ) (after using e.g. MSD or some other tool to locate the ROM address and size in memory) vs an
EEPROM reader?

For XT-IDE, I think that is fine. It may be harder to dump a complete BIOS this way, because an uncompressed portion of the BIOS sits at F000 at runtime, but what's actually on the ROM chip is compressed so you would either need the EEPROM reader or some other way of reading the chip that is not just dumping RAM (which is what I guess awdflash etc. use?)

Check - I am looking to preserve some rare old hardware BIOSes, so maybe I should just order an EEPROM reader to make sure I get the data out intact.

clb wrote on 2021-07-23, 20:53:

Thanks for the great answer! I think I got a much better picture of this now!

jakethompson1 wrote on 2021-07-23, 20:27:

The BIOS will not put normal (read-write) RAM there. EMM386 takes care of creating UMBs by using features of the 386 processor to map virtual accesses to that 384KB region to either the same physical addresses (if there is video RAM/option ROM there), or to a different physical location in extended memory if the UMB is "empty".

Oh I see. So this mapping between ROMs and RAM in upper memory requires virtual memory addressing(?), and so having any upper memory at all must then require that the CPU is in the Virtual 86 mode? That was introduced with 386, so does that mean that 286s and 8088/8086s cannot get any upper memory at all, even if one installs a RAM board like this? https://www.lo-tech.co.uk/product/1mb-ram-board-pcb/

I'm not super familiar with actually using UMBs (loadhigh/devicehigh) on pre-386 systems. EMM386 is a universal solution. For 286 systems, I believe UMB providers are chipset-specific. With that card, I think you would have to do careful setting of the switches to not overlap with your video RAM and (if present) video BIOS, and XT-IDE BIOS, so the correct settings will depend on whether you have CGA/MDA/EGA. And using that upper memory wouldn't be as simple as loadhigh/devicehigh, I think. The documentation for it https://www.lo-tech.co.uk/wiki/Lo-tech_1MB_RAM_Board suggests that "TSRs, disk cache and print spoolers" can use the upper memory, but I guess those would have to either be specifically written, or someone could write a small "stub" EMS driver that allows the use of the memory on the card but won't do any bank switching if the card isn't capable of that.

clb wrote on 2021-07-23, 20:53:

jakethompson1 wrote on 2021-07-23, 20:27:

The BIOS will not put normal (read-write) RAM there. EMM386 takes care of creating UMBs by using features of the 386 processor to map virtual accesses to that 384KB region to either the same physical addresses (if there is video RAM/option ROM there), or to a different physical location in extended memory if the UMB is "empty".

Oh I see. So this mapping between ROMs and RAM in upper memory requires virtual memory addressing(?), and so having any upper memory at all must then require that the CPU is in the Virtual 86 mode? That was introduced with 386, so does that mean that 286s and 8088/8086s cannot get any upper memory at all, even if one installs a RAM board like this? https://www.lo-tech.co.uk/product/1mb-ram-board-pcb/

That 1MB board can provide UMBs if you set the switches correctly. You need a software solution like UMB_DRVR.SYS to tell DOS about that memory, though. This is a workable solution for PC/XT computers, but 8-bit ISA memory is very slow compared to AT onboard memory, so 8-bit memory cards make little sense in AT-class computers.

clb wrote on 2021-07-23, 20:53:

jakethompson1 wrote on 2021-07-23, 20:27:

It has to live somewhere else. At boot time the system BIOS will scan these possible-option-ROM locations for a 55AA signature similar to what a boot sector has.

Gotcha - so I suppose there exists some kind of "master list" of recognized Option ROM locations, like B000h, B800h, C000h, C800h etc. that the BIOS will poke on the ISA bus to discover Option ROMs? (since surely the BIOS would not scan through all memory addresses)

The master list is all multiples of 2K (iirc) between C000 and DFFF. Possibly, some PC/XT bioses that occupy 64K or less also scan the Exxx range, but this is non-standard.

clb wrote on 2021-07-23, 20:53:

jakethompson1 wrote on 2021-07-23, 20:27:

If it is there, the BIOS passes control to it, expecting it to pass it back afterward. The XT-IDE BIOS would hook hard drive related software interrupt vectors like a chain. When an interrupt comes in, it either handles it itself, or if it's unwanted (like floppy-related) passes it along to the original interrupt vector, so in this case floppy access would still go through the system BIOS. A SCSI BIOS would work the same way.

Thanks, this was the missing bit. So BIOS calls into the ROMs at boot, but after the BIOS has already set up its own interrupt vector tables (and all other initialization?), so that whatever the ROMs do, they'll have a guaranted late init order in order to be able to override interrupts.

Exactly. Usually, the order is like this

• Perform some basic test of mainboard components (processor, RTC, DMA controller, interrupt controller)
• Set up the memory controller (if required - old boards have jumpers for memory configuration, and don't need BIOS initialization)
• Set up BIOS interrupt vectors to point to the mainboard BIOS services
• Call graphics BIOS (scan C000-C680), if any
• Perform memory test
• Call other BIOS extensions (scan C800-DF80), in order of ascending addresses
• Call INT19 (default system BIOS handler tries to boot A:, and C: if A: fails to boot; INT19 is commonly overwritten by network boot ROMs)

mkarcher wrote on 2021-07-23, 21:07:

clb wrote on 2021-07-23, 20:53:

jakethompson1 wrote on 2021-07-23, 20:27:

The BIOS will not put normal (read-write) RAM there. EMM386 takes care of creating UMBs by using features of the 386 processor to map virtual accesses to that 384KB region to either the same physical addresses (if there is video RAM/option ROM there), or to a different physical location in extended memory if the UMB is "empty".

Oh I see. So this mapping between ROMs and RAM in upper memory requires virtual memory addressing(?), and so having any upper memory at all must then require that the CPU is in the Virtual 86 mode? That was introduced with 386, so does that mean that 286s and 8088/8086s cannot get any upper memory at all, even if one installs a RAM board like this? https://www.lo-tech.co.uk/product/1mb-ram-board-pcb/

That 1MB board can provide UMBs if you set the switches correctly. You need a software solution like UMB_DRVR.SYS to tell DOS about that memory, though. This is a workable solution for PC/XT computers, but 8-bit ISA memory is very slow compared to AT onboard memory, so 8-bit memory cards make little sense in AT-class computers.

Nice, I hoped you would join in 😁
Given that the card puts memory at whatever locations you set the switches for and it is done in hardware, I assume UMB_DRVR.SYS would have little to nothing to do, right? Besides tell DOS there is or is not memory at a given UMB location.
As I was thinking earlier, would it not also be possible to write a "stub" EMS driver for the card that makes a 64K page frame available but refuses all other EMS functionality (since the card has no ability to, in hardware, remap memory addresses)

jakethompson1 wrote on 2021-07-23, 20:27:

clb wrote on 2021-07-23, 19:59:

2) Are the colloquial terms "BIOS ROMs" (besides the main BIOS ROM onboard), "Boot ROMs" and "Option ROMs" all referring to the same thing - an EEPROM on an ISA card providing a memory block in some address range?

Yes, I think Option ROM is most correct though. The only oddball case you're missing here is ROM BASIC.

Option ROM is indeed the generic term for BIOS extension ROMs installed on option cards (hence the name). The term "boot ROM" is usually applied only to option ROMs that provide additional ways of booting the machine, especially network boot. I never heard the term "boot ROM" applied to video card ROMs, and seldomly to SCSI or MFM hard disk controllers. Some BIOS setups that allow you to configure shadowing call only ROMs in the range C800-DFFF "Option ROMs", but ROMs in the C000-C7FF range "video ROM". This is technically correct, but you must not imply that this means that video ROMs are not option ROMs - it's just that this range of option ROMs is reserved for one special kind of option ROMs, specifically video ROMs.

jakethompson1 wrote on 2021-07-23, 20:27:

[...]

Very comprehensive overview. Thanks!

The BIOS will not put normal (read-write) RAM there. EMM386 takes care of creating UMBs by using features of the 386 processor to map virtual accesses to that 384KB region to either the same physical addresses (if there is video RAM/option ROM there), or to a different physical location in extended memory if the UMB is "empty".

Perhaps it would be useful to add what EMM386 is emulating: the original Expanded Memory cards that physically did just that: put RAM chips at option ROM addresses with paging hardware in between, crucually not setting it to read-only, so that programs that were EMS-aware could use this extra memory and select pages as desired.

EMM386 used i386 functions to map those pages to somewhere higher up in (extended) RAM memory space.

jakethompson1 wrote on 2021-07-23, 21:11:

As I was thinking earlier, would it not also be possible to write a "stub" EMS driver for the card that makes a 64K page frame available but refuses all other EMS functionality (since the card has no ability to, in hardware, remap memory addresses)

Even if a EMS 3.2 compliant EMS driver only provides 64K of physical pages, it needs to be able to remap those pages, i.e. swap the order of the four 16K blocks. It must not refuse the page mapping calls. A stub driver might work with some specific applications that just allocate 64K and map them in default order and never change the mapping, but is in no way EMS compliant and a lot of EMS-compatible applications may trip over this stub driver.

Now you all have me thinking. What if, either using an EMS card, or a "hardwired" card like the Lo-tech linked in this thread, you were to load one of the valid option ROM addresses with something that either is or looks like a valid option ROM and then rebooted? Would the BIOS be tricked into executing it?

Did EMS cards try to avoid this? Like upon receiving the ISA reset signal would the card either unmap all the memory or clear it?

mkarcher wrote on 2021-07-23, 21:18:

jakethompson1 wrote on 2021-07-23, 21:11:

As I was thinking earlier, would it not also be possible to write a "stub" EMS driver for the card that makes a 64K page frame available but refuses all other EMS functionality (since the card has no ability to, in hardware, remap memory addresses)

Even if a EMS 3.2 compliant EMS driver only provides 64K of physical pages, it needs to be able to remap those pages, i.e. swap the order of the four 16K blocks. It must not refuse the page mapping calls. A stub driver might work with some specific applications that just allocate 64K and map them in default order and never change the mapping, but is in no way EMS compliant and a lot of EMS-compatible applications may trip over this stub driver.

Ah. So I guess you could simulate this in software by actually exchanging the memory contents between those blocks word-by-word, but that would be slow (albeit faster than the drivers that simulate EMS by swapping out contents to disk)

dionb wrote on 2021-07-23, 21:17:

Perhaps it would be useful to add what EMM386 is emulating: the original Expanded Memory cards that physically did just that: put RAM chips at option ROM addresses with paging hardware in between, crucually not setting it to read-only, so that programs that were EMS-aware could use this extra memory and select pages as desired.

EMM386 used i386 functions to map those pages to somewhere higher up in (extended) RAM memory space.

Actually, EMM386 can perform two functions: It can emulate EMS which is remappable in a 64K page frame of 4 16K pages. This memory is managed using the INT 67 API. On the other hand, it can map further memory which is (by conventional means) not remappable and behaves like standard RAM at high addresses (this is called UMBs). DOS learns about UMBs using the XMS API. You can have EMM386 perform both functions, but if you don't need EMS, you can disable the EMS part using "NOEMS" on the command line, so the range that would be used for the page frame can be used to provide 64K more UMB.

If I remember correctly, EMS does not provide UMBs by default, unless you either disable EMS using "NOEMS" or explicitly request UMBs as additional feature to EMS emulation using "RAM".

jakethompson1 wrote on 2021-07-23, 21:19:

Now you all have me thinking. What if, either using an EMS card, or a "hardwired" card like the Lo-tech linked in this thread, you were to load one of the valid option ROM addresses with something that either is or looks like a valid option ROM and then rebooted? Would the BIOS be tricked into executing it?

Did EMS cards try to avoid this? Like upon receiving the ISA reset signal would the card either unmap all the memory or clear it?

EMS cards are unable to produce write cycles to their RAM themselves, so they won't clear the EMS memory. On the other hand, most EMS cards are able to unmap the memory (i.e. not respond to access at all), and I guess most EMS cards actually did unmap their four pages on ISA reset. The possibility of completely unmapping a page was used by some EMS card vendors so that you could plug two EMS cards that share the same page frame address, but have different I/O ports for mapping. The driver would ensure that only one of the two cards has a mapping for a page enabled at a time, so you can double the amount of EMS that is mappable to the same page fram by just adding a second EMS card.

There was a de-facto standard how EMS cards map their memory which had 7-bit page numbers (and one enable bit). This allows 128 pages of 16K, i.e. 2MB on a single EMS card. Some later EMS cards with 4MB or 8MB actually behaved like 2 or 4 of those 2MB cards.

jakethompson1 wrote on 2021-07-23, 21:22:

Ah. So I guess you could simulate this in software by actually exchanging the memory contents between those blocks word-by-word, but that would be slow (albeit faster than the drivers that simulate EMS by swapping out contents to disk)

Even this gets difficult. The EMS standard is explicitly written in a way that allows emulation like disk swapping (or XMS swapping), but the EMS standard still allows a single page to be mapped multiple times. So it is perfectly valid for an EMS application to map the first physical page into all four slots (called "logical pages") of the page frame, and expect to read the same contents from all four logical pages. This means the contents of the three other pages needs to be swapped somewhere else, like a buffer in conventional memory. So your EMS driver needs to allocate 48K swap space somewhere, and the net win is limited to 16K (64K EMS - 48K swap space).

The effect of writes to EMS pages that are mapped multiple times is explicitly unspecified: If I write to one copy of a page, the other copies might change at the same time (as with hardware EMS), or be unaffected (as with software emulation by swapping). I don't think it is clearly defined what changes stick if you modify a page that is mapped multiple times, but just reading multiply mapped pages is perfectly fine.

clb wrote on 2021-07-23, 19:59:

Hello,
...

Maybe the notes at the bottom of this page can be helpful http://minuszerodegrees.net/rom/rom.htm

Thanks all for the interesting discussion! This quite exhaustively clarified a lot of the details I was missing.