I've just fixed a bug in the 8042 PS/2 controller, where the 0xDD and 0xDF commands had their meanings switched(0xDD disabled the A20 line, while 0xDF enabled it, which is supposed to be reversed).
Now the first protected mode(PIQ flush) JMP tries to load the descriptor from it's correct address in physical memory(segment 0x110, offset 0x09BB as the JMP(opcode 0xEA) implies).
The descriptor address resolves to address 0x00110110. It passes through the A20 line unmodified(which is corrected now).
It then resolves into the modified memory bank of physical memory B0110 by the memory controller itself(which applies the A0000-FFFFF bank to be removed in physical memory, removing the memory of that area, effectively moving the 1MB+ memory back to the A0000 address(thus moving the UMA to the end of physical memory). So the memory remapping works without problems.
The descriptor that's loaded contains only 0x00 bytes, thus causes a #NP(Not Present) fault(return point 1170:09B6). Thus interrupt 0xB is started.
The 0xB interrupt handler resolves to address 0x120058 in linear memory(IDT is at 0x120000).
The interrupt handler is also an empty handler(filled with 0x00 bytes). Thus another #NP is faulted, which resolves to a Double fault interrupt.
The Double fault handler also isn't present, which causes the CPU to triple fault, resetting the CPU and rebooting the machine.
Anyone knows why the GDT/IDT aren't pointing to valid tables? Why are they empty?
Is the #NP being faulted when the present bit isn't set correct?
Edit: I've just logged the entire 1MB+ memory accesses when runnning WIN.COM(of Windows 3.0) on the Compaq Deskpro 386 machine emulation. The only 0x0012XXXX addresses popping up in the log are the reads from RAM just before the triple fault(the invalid 00120058-"5F and 00120040-"47, all containing 0x00 data). The GDT entry that's loaded into CS(selector 0x0110) is at 0x110110-0x110117, which also contains 0x00 values.
The entire log of 0x100000+ memory accesses booting Windows 3.0:
https://www.dropbox.com/s/gdbxztoumfrt1ks/deb … 5_0939.zip?dl=0
So that means a problem in HIMEM.SYS moving data to high memory?