Hmmm... If an IRET returns, does it inhibit interrupts like STI does?
The new 85C496 motherboard emulation has an issue with the IRQ14/15 in level-triggered mode(from the ELCR registers), while the hard drive is in legacy mode, thus being in edge-triggered mode(leaving the IRQ line high when triggering it).
So it generates an infinite interrupt storm in this case, retriggering after each IRET instruction.

Edit: OK. So now I've modified the ELCR behaviour to apply to the PCI IRQ (PIRQ) lines only, the legacy lines being unaffected by it (they have their ELCR bits forced to 0, effectively).
So now the hard drives are detected correctly on the 85C496 motherboard emulation, but won't be booted from somehow? PCI IRQs should work properly as well, provided the ELCR is setup correctly by the OS.

The CD-ROM drives also aren't detected on the 85C496 chipset for some reason?

Just fixed the PCI IDE hard drive emulation (when not emulating a PIIX device) to report the correct vendor and device ID. The IDs for them were swapped (the vendor ID was reporting the device ID and vice versa) 😖

Took a running of HWDIAG in Windows 95 to figure that one out :p

Edit: Just figured out another bug. It was using the BAR1 for the primary channel PCI bus mastering DMA and BAR3 for the secondary channel PCI bus mastering DMA, instead of using BAR4 for both.
Edit: Also found a bug in the usage of the secondary channel alternate status port, where it was checking against PCI IDE bus mastering being available instead of ignoring it.
Edit: Windows 95 still doesn't use PCI bus mastering DMA or native drivers (error 10 in device manager)?

OK... PC Doctor 2004's Disk 0 "Controller" test fails on some "Controller data loopback"?
Edit: Perhaps because the disk is an empty (formatted, but not partitioned) to test with (boot sector is cleared)?

Just improved the ATA execute device/drive diagnostic to have improved results:
- The master/slave bit is ignored for the command itself (allows normal execution, always on the master, if present).
- Both master and slave always report 0x01 in the error register.
- Only the active drive gets it's signature set (Either CD-ROM, ATA-1 or not present).
- The master gives an IRQ after 205 timing units. The slave (only if present, otherwise it gives no IRQ) 5 timing units earlier.

OK. Just restored the APIC command register to just have the interrupt requests accepted if the IRR bit is already set. Making it wait on other CPUs to acnowledge an IRR when sending it to itself causes Windows NT 4.0 to hang up on itself, because it's trying to send an IRR that's already pending to itself.
Having fixed that makes it continue to boot again.

Then, fixing the ATAPI identify packet device command to make it leave the last DMA mode when it was in one fixes that command as well. This was already done for non-ATAPI commands(which always reset DMA) and the ATAPI packet command (which reloads the setting for the ATAPI commands when the command is received, to be used once the packet data bytes have been transmitted and the ATAPI commands start giving/getting results).

Just, besides some DMA priority checks (the command register bit 0 being ignored if the channel was last used) being fixed, the APIC is also improved.
When the APIC command register is sent, the IRR already being set causes the Local APIC to accept it. If it's from the IO APIC, it isn't accepted if that happens(keeping it pending until the APIC can accept it).

Just removed the whole IO APIC double bookkeeping of IRR requests when triggered by the mode that's currently loaded(either rising edge or being level high). Said triggering when the IRQ is masked (bit 16 set) won't cause it to be triggered later when unmasked. So the trigger mode for level has the same effect(the live line will be updated anyways and re-detected after EOI, while edge triggered won't set the IRR again (unless the line is lowered first) and level-mode will set the IRR again when the level is still high).
The bookkeeping of the parallel IRQ lines is still done by the generic PIC handling (which reduces all IRQ lines to a basic went-high and went-low handling, which the PIC parses directly and the APIC simply registers and will handle it at it's own pace (depending on the modes and what the CPU does with it (e.g. EOI making it check the lines again, as will the local APIC executing a vector))).

OK. With the latest changes, somehow Windows NT 4.0 only has the CMOS time interrupt left? The PIT IRQ is disabled, as is the APIC timer? It's displaying "..." on the start of the second row of text, below the OS version.
So the only timing source it still has left is the CMOS, which runs at up to 65KHz (ticking around each 2ms)?

At least the HDD/CD-ROM seems to be working afaik (their IRQ lines are low and commands not pending anymore, so they probably should've finished correctly).
Doesn't Windows usually require an IRQ for timing? Why would it only have IRQ8 enabled(together with IRQ1(keyboard), IRQ12(PS/2 mouse), IRQ13(unimplemented and reserved for the FPU error IRQ), IRQ14(UniPCemu primary channel HDDs) and IRQ15(UniPCemu secondary channel CD-ROMs))?

Hmmm... Do ATAPI drives allow writing to CBR if the drive is busy?

Windows NT 4.0 now hangs with the latest commits, waiting for 801473A0 to clear it's bit 0, where said byte is set to 01h. That happens at 80003CC4(FS=0030 located at FFDFF000).

It has printed the "..." on the boot, but didn't get any further than that?

Edit: Hmmmm... Just restored the APIC IRR not accepting interrupts if they're already set (according to the Intel 64 and IA-32 Architectures Software Developer's Manual Vol 3A at 10.8.2 and 10.8.4, the IRR will only load 1 IRQ. Once it's loaded, it won't accept any more until it's loaded into the ISR, at which point it can get accepted again (so 2 can be 'queued' in total: 1 in the IRR (pending to fire) and 1 in the ISR(fired)).
The main issue here is that the command register code is waiting for it to send a Command Register to set an IRR bit on it's own core, which already has the IRR bit set and pending, but the interrupt won't fire for some reason?
That happens during NT 4.0 workstation's boot?

Edit: Just implemented the ICR register with proper (as documented) send (nobody accepted) and receive (nobody received) errors, along with proper invalid vector errors (on both APICs that send and receive it).
Edit: OK. That fixed the SMP issue itself, now still the "..." issue remains?
Edit: Hmmm... Just improved the nIEN and INTRQ on the device. Now nIEN is device-specific and INTRQ is only raised (whether the drive is selected or not) when nIEN isn't set to 1. The "..." in Windows NT 4.0 became 6 dots instead ("......")?

Just fixed some issues with the APIC lowest priority mode's focus processors and arbitration IDs to be calculated better (they were using a simple MAX(x,MAX(y,z)) instead of MAX((x&y),z) formula.

OK. Looking at why Windows NT 4.0 hangs, it seems to be in an eternal spinlock:
ECX=801473A0
top: TEST [ECX],00000001h
JZ continueloc (80003CBA)
jmp top (80003CC4)

Disassembling 80003CBA with https://defuse.ca/online-x86-assembler.htm results in the following:
0(=80003CBA): f0 0f ba 29 00 lock bts DWORD PTR [ecx],0x0
5: 72 03 jb 0xa
7: c3 ret
8: 8b c0 mov eax,eax
a: f7 01 01 00 00 00 test DWORD PTR [ecx],0x1
10: 74 ee je 0x0
12: eb f6 jmp 0xa

Looking at it again, the A label is actually the currently executing loop.

So that loop waits for address ECX to clear and be set by itself, otherwise jumping to 0xA, which it currently is hanging itself infinitely, because the address never clears?
Perhaps an issue with the BTS instruction somehow?

Just looked at the code running. I've even modified the IDE code to be what it was before (when it was booting into the GUI at least). It still enters the invalid deadlocked loop mentioned above.

So perhaps there's some weird kind of issue with the APIC and it's handling of things somehow?

That hex string seems to match halmps.dll (along with a couple of other dlls) and more specifically the function KfAcquireSpinLock().
So yes, that does point to APIC as you suspected. Some info here:
https://geoffchappell.com/studies/windows/km/ … index.htm?tx=14

It seems "mp" in halmps.dll refers to "multi-processor". Was your test setup with multiple cpus?
(Idk, maybe this same code is run with uniprocessor setups too...that just caught my eye)

mr.cat wrote on 2022-02-28, 00:37:

That hex string seems to match halmps.dll (along with a couple of other dlls) and more specifically the function KfAcquireSpinLo […]
Show full quote

That hex string seems to match halmps.dll (along with a couple of other dlls) and more specifically the function KfAcquireSpinLock().
So yes, that does point to APIC as you suspected. Some info here:
https://geoffchappell.com/studies/windows/km/ … index.htm?tx=14

It seems "mp" in halmps.dll refers to "multi-processor". Was your test setup with multiple cpus?
(Idk, maybe this same code is run with uniprocessor setups too...that just caught my eye)

Yup. halmps.dll should be correct. MPS 1.4 is enabled in the BIOS settings(i440fx Pentium II with APIC). I can see the 8259A PIC is masked and IO APIC enabled.
I also see that it won't use any timing source other than the CMOS timer(IRQ8)? IRQ0 (PIT) which is on IRQ2 on the IO APIC is masked. The Local APIC timer (LVT) is also masked. The initial count is setup though, so it's counting down at 333MHz speeds(divided by the Divide configuration register using binary shifts).

So the main question is: what does that lock pointed to by ECX belong to? What kind of data does it lock? Afaik, multiprocessor locks(2 CPUs emulated) should be working(although with 'equal chance' locks using the c++ random function with modulo 2(numemulatedcpus) to determine which CPU wins if done at the same instruction(IPS clocking mode(current)) or cycle(cycle-accurate mode), see emu/core/emucore.c).

Although I didn't verify the locks using software.
Edit: Hmmmm... Single CPU configuration boots into the GUI without said hanging, although the 8042-style CPU reset is still performed by the OS.

So does that mean there's a (multicore) LOCK prefix contention bug in there?

The DMA controller and PCI IDE bus mastering won't take the bus when it's taken by anyone else or a bus lock is active.
The CPUs on the other hand wait for each other only when a bus lock is active during a BIU memory access. Those stall the CPU that can't get the lock, to retry taking the bus when the lock is released on the other CPU(s). The main bus locking algorithm (when trying to take it) is in emucore.c, just outside of the CPU processing loop(which ticks all CPUs in parallel for 1 cycle or instruction(during IPS mode). One of the CPUs will randomly obtain the lock, unless only 1 CPU is requesting it(which then always obtain it).

Managed to fix the multiprocessor issue on NT workstation 4.0. There was a missing check in the BIU for new and busy requests to check non-LOCK prefixed instructions against a different CPU locking the bus (it's supposed to wait for it to be released first).

Having fixed that, NT 4.0 boots the same (until the 8042 CPU reset in graphical mode) on both multiprocessor and uniprocessor systems (on the multiprocessor kernel). Still having the MPS 1.4 set in the BIOS.

OK. Just ran PC check's MP Symmetry test. It passed, so the multiprocessing part at least seems to work?

OK. Windows NT 4.o doesn't produce any logs, so probably the only indication of what's going on is by comparing to Bochs, where the same disk image properly boots? (first boot for second phase of GUI setup)

Have you tried the so called "checked" (debug) builds? They're supposed to have some internal testing so maybe worth a try.
Also, is there a /debug switch for boot.ini in NT4?

mr.cat wrote on 2022-03-02, 23:32:

Have you tried the so called "checked" (debug) builds? They're supposed to have some internal testing so maybe worth a try.
Also, is there a /debug switch for boot.ini in NT4?

Not yet. I'll try the comparison to Bochs first, then (if no obvious results are found), I'll try those builds. Although it requires a full reinstall in UniPCemu, which will take some time (it's slow).