Just have been thinking... Since I've already checked all instructions executing manually, perhaps the issue here isn't seen looking at the instructions execute, but instead only when looking at the different x86 mechanics that surround the instructions themselves(e.g. interrupts, IRET, task switching, prefetching, fault handling and protected mode segment checks and algorithms)?

Tracking errors upwards, I see strange loads of ES, backtracking it leading to a #NP fault at 00e7:000042e3, with SS=28Fh, with the following stack frame for the interrupt:

Somewhere between there and the #GP error location, this issue lies?

The value being loaded into ES is 0x4BF there?

OK. The start of the #NP handler seems the same as for the #GP(x) fault handler, until it reaches 0020:00000c17, where it pushes 10Bh on the stack instead of 10Dh.

Edit: OK. Eventually, the #NP handler's subfunction(which has 10Bh on the top of it's stack(above IP) executes a STI instruction and executes a JMP 00003C69 immediately after that, which jumps there and immediately at that point executes a Sound Blaster interrupt(due to hardware interrupts becoming active again at that point, after the STI delaying interrupts by 1 instruction).

So at 0020:00003C69 it starts the normal handling of a Sound Blaster IRQ5 handler.
Edit: OK... SS0 is FFFFh there, so it's an interrupt from kernel mode.
Edit: So it reaches "0020:000013fb CF iret", where it starts the IRQ5 handler in segment 417h, then returns at the #GP fault handler for terminating the interrupt at "0020:00000c7a 55 push bp".

SS0 is F74 there.

Edit: OK. It reaches 0020:00003f21 after recognizing the return from the IRQ5 handler.

Edit: It finds the origin of the IRQ5 handler to start in kernel mode at 0020:00003f3d, so it reaches 0020:00003f4f, then reaches 0020:000013a1.
It eventually reaches 0020:000013fb CF iret, where it returns to the interrupted kernel-mode handler(back to the #NP handler, before handling the IRQ5 handler).
So now it's back at the #NP handler, executing "0020:00003c69 50 push ax".
Edit: It finds it's from user mode(segment E7), so jumps to 0020:00003c75.
OK. So the #NP handler loads the #NP user-mode handler from the protected mode IVT-like structure in the TSS at address TSS+34h. This is located at E7:9576.
It finds a non-zero segment(which is E7), so jumps to 0020:00003c85.
It pushes a lot of things on the stack, then calls 0020:00003632 at 0020:00003d03.
It then reaches 0020:00003653 because it's parameter is 1...
It has no interrupts now anymore, thus reaching 0020:00003660 to start handling the interrupt handler in user-mode.
OK... It copies a pretty big block of memory to the user-mode stack(much bigger than the ) at "0020:000036d0 F3 A5 rep movsw", 24h words(thus 48h bytes) to be exact.
So it then loads the interrupt handler address into CX::DX.
After having written the interrupt handler data to the stack, it jumps to "0020:00001386 FA cli".
It finds the erroring task in user mode(E7h), thus reaching 0020:00001390.
It restores the original interrupted task's SS0 into the TSS(which was FF8).
It eventually reaches 0020:000013fb to execute an IRET to the user-mode handler for the handling of the #NP fault in user-mode.

So it starts executing the user-mode #NP handler at E7:9576 with FLAGS=3246h SS:SP=5B:FB8.

But when said handler starts executing, immediately an interrupt seems to occur(SP=FEEh)? So immediately after when returning to the #NP user-mode handler, it triggers a new IRQ5 handler?

Edit: Having handled said interrupt and having returned to the interrupted #NP user-mode handler at "00e7:00009576 50 push ax" it finally starts executing...

OK. The #NP handler reaches "00e7:00001c96 2E FF 1E C7 1B callf word cs:[1bc7]", which calls B3:0034 as a function. That probably(very likely, as all the B3 segment seems to do is execute HLT instruction with some special parameters following it) faults back to the #GP fault handler to terminate the interrupt handling and return to the interrupted procedure in user-mode(in this case segment E7).

So the #GP fault handler once again starts executing, at 0020:00000c7a.
So the base of the #NP fault handler was FF8h, it detects that the difference was 0xE, thus having pushed the error code(and qualifying it to a return from the #NP interrupt handler).
It then starts up the normal 10Dh handler(the end-of-interrupt handler as you can call it) and calls 0020:0000131c.

It handles the fault handler as normal, setting SS0 to FFFFh for handling the post-STI-kernel privileged interrupt handler from kernel mode.

And indeed, it reaches "0020:00000cae FB sti" for returning from the interrupt handler.
Then checks against segment B3 and immediately after that triggers a IRQ5 interrupt(what a surprise!).
ESP=FAEh there. SS0=FFFFh there.

It then reaches 0020:00000c7a 55 push bp with SP=F68h to handle the return from the IRQ5 handler.
Said return reaches "0020:00000cb4 jnz 00000cb9".
So that's probably the continuation of the #NP handler?
So it reaches 0020:00003e80.
Said handler has 01h at the DI(34h)+1 location.
Edit: OK. That's different from the value of 02h that was there on the other handler locations (when terminating an interrupt handler). Interesting...
Edit: So it reaches 0020:00003e94 in that case. Then jumps to 0020:00003f17.
It loads the stack pointer(it seems) from 5b(the user stack):f8c(which was the stack pointer at the time the exception handler escaped itself to kernel mode).

Then loads that into AX(1C9Bh).
Then loads that to FEE(the interrupted user-mode instruction IP that was faulting on loading ES).
The same for the CS value that was just past that one(2 bytes after the IP value), which is E7h.
Then the same for the FLAGS of the interrupt handler(which is 3202h).
It then adds 6 to address FF4h(which is the SP of the interrupted E7 function). So it discards 3 words from there(F8Ch becoming F92h).

OK. So then it calls the address in the kernel segment of the user's stack DI+6, which is 3F1Eh. So that's probably the handling of the actual #NP interrupt handler, but at the kernel side?
Edit: So it clears the carry flag on the user's FLAGS register.
It then uses some value on the stack(0006h), then loads that into AX.
It checks AH against 12h, which it's lower. Thus it jumps to 0020:0000163b.
That sets BL to AH(00h), multiplies it by 2 for an index. Thus index 0. Then it loads SI from 15D6h using said index as an index into said table. Thus loading value 3302h.
Then compares AL against said index(at 0020:3302), reading 0xE.
It finds it's less than that, thus jumping to 0020:00001650.
Then it loads AL into BL, multiplies by 2 and uses that as an index into the earlier using SI(3302h). Thus index 0xC into table at 0020:3302.
So it calls 0020:0000165A from 0020:00001656.

It loads earlier saved user-calculated 4BFh segment(which was the #NP's error code modified as it's user segment selector with added privilege level of user mode) into BX at 0020:0000342d...
So that's the kernel part of the handler executing!

It then calls 0020:00003322. That pushes AX, tests the Local/Global bit of the error code(it's set). It finds it set, thus reaching 0020:00003329.
It executes a LAR AX(6),BX(4BF) at 0020:00003329.

So it finds a Access Rights value of 72h. AX becomes 7200h and the zero flag becomes set.

It finds the zero flag set, clears the carry flag and checks the result(72h) against being non-zero.
It then jumps to 0020:00003337.

It then returns from said function, to "0020:00003433 73 06 jnc 0000343b".

So it then finds the carry flag cleared, jumping to "0020:0000343b 53 push bx" because it was successfull.
It then pushes the selector(4BFh) and calls 0020:000008a1 from 0020:0000343c.

Then sets up a new stack frame at SP=BP=FAAh using an ENTER 0,0 instruction.
Then pushes the parameter(4BFh) again and calls 0020:00000804 at 0020:000008a8.
Then sets up another stack frame at SP=BP=FA4h and loads the selector into AX.
It checks against it being in the LDT(it is), thus reaching 0020:00000810.

It then saves the LDT(168h) into DX and adds 8(what a surprise, it's going to access the LDT's raw data!)...
It then jumps to 0020:0000081b.
It masks the selector of the faulting segment with fff8h, thus getting the selector field only for the LDT selector to use(=LDT+4b8h).

Then executes LEAVE and RET to return to the calling instruction and stack, reaching 0020:000008ab.

It then discards the parameter from the stack, then pushes DX(the LDTR's data selector).
It also pushes AX(the faulting selector).
It then calls 0020:00000785 at 0020:000008b0.
Then sets up a stack frame using ENTER 0,0 below FA4h, so becoming SP=BP=FA2h.
It saves ES and DI on the stack, then loads the pointer to 170:4B8(the LDT's entry of the faulting #NP exception) into ES:DI using LES.
Thus ES:DI points to the descriptor of the segment that faulted when reaching 0020:0000078e.
It loads it's base address low field(C66h) into AX. Then base address high into DL(54h). Then base high byte into DH(54h).
So the address stored into the base address field of the descriptor is 54540C66h.

Then it pops DI(34h). Then ES is popped(B3h). Leave and RET is executed to return to the calling function.
It then discards 4 bytes on the stack, then LEAVE and RET again, reaching 0020:0000343f.
Then 2 more bytes are discarded on the stack and it reaches 0020:00003442.
It saves the high base to SS:FCEh. And low base to SS:FCAh. So SS:FCAh contains the dword value that is the 32-bit base of the faulting selector.

Then it returns twice, reaching 0020:00003f1e. Which jumps to 0020:00001386.

OK. So that reaches 0020:00001390 for the return to user mode to happen(SP=BP=FB4h) to handle it.
That reaches 0020:0000139d, which destroys the kernel stack SS0(thus returning it to FF8h). Then restores DS(98h):63E to it's old BP(FB4h).
Then it reaches 0020:000013b2 for the simulated POPAD using seperate instructions instead.

It then reaches the return to user mode using IRET after the normal stack breakdown after an interrupt:

So it returns to E7:00001C9B, EFLAGS=3202, SS:SP=5b:F92...
OK. So SP returns to FF8 with that IRET.

Then another IRQ5 happens(what a surprise!).

After that one is finished and cleared up, it reaches "00e7:00001c9b C3 ret". SS:SP=5b:F92.
OK. DX::CX contains the address of the #NP fault hander's descriptor, at 00e7:000092cb.

Then it returns and moves, using the stack, to BP-2 and BP-4, then from there to AX(low) and DX(high).
Then it pops CX from the stack, which contains the selector(4BFh).

Then something seemingly weird: it ands AX with 0? That happens at 00e7:000095a8(opcodes 25 00 00).
And then DX with FFFFh.
Then it compares DX with 5454h at 00e7:000095ae!
It matches, so it goes through to 00e7:000095b4.
It then ORs AX with itself and executes a conditional jump to E7:9622 it it's not zero(essentially (AX&0)!=0), which it cannot ever be(logically), since it was ANDed with 0.

Then it loads AX from the stack, loading E7(the selector), from SS:FC0h.
Then DX from BP+1A(the return IP)? That's 42E3h.
Then it moves AX to DS:[7052](CS) and DX to DS:[7050]. So DS(EFh):7050 contains a far pointer to the interrupt return?

Then it loads [BP+22] into AX, which is 28Fh.
And [BP+20] into DX, which is 36D2h.
It then loads those into DS:7058(AX=28F) and DS:7056(DX=36D2h).
Then it loads [BP+1E] to AX and saves it to DS(EFh):7054(AX=3246h).
Then it loads [BP+18] to AX and saves it to DS(EFh):704e(AX=4BCh).
Then it loads ES:BX from [BP-04], which is 5b:FBC.
It substracts 2 from ES:[BX+08], so from 5b:FC4.
Then it loads ES:BX from SS:[BP+20], so SS:FC4, so it loads 28F:36D0 into ES:BX.
Then loads [BP+1E] to AX again(SS:FC2h=3246h).
Then it saves AX(=3246h) to ES:BX(28F:36D0).
It then loads ES:BX from SS:[BP-04], which is from 5b:FA0, being 5b:FBC.
It then substracts 4 from said location, 36D0h becoming 36CCh.
It then loads ES:BX from SS:[BP+20], so 5b:FC4.
It then loads AX from SS:[BP+1C](which reads 5b:FC0=E7h), DX from BP+1A(which reads 5b:FBE=42E3h).
It then moxes AX(=E7h) to ES:[BX+02], which is 28F:36CEh. And it moves DX(=42E3h) to ES:[BX], which is 28F:36CC.

Ah, something interesting: wasn't 36CCh one of those invalid values that was being used somehow by the kernel and causing the crash?

So far reached 00e7:00009607.

It then loads ES:BX from 28F:FA0(SS:[BP-04]). That's 5b:FBCh.
Then it clears bit 8 of ES:[BX+06], thus 5b:FC2h. So from 3246h being unchanged.
Then it moves 1883h right before that, so probably some stored CS field(at 00e7:00009610) at 5b:FC0h.
And 951Bh right before that, at ES:[BX+04], so 4b:FBEh.
Then it moves CS right over the 1883h value, overwriting it with E7h? That's some very weird behaviour? (Writing something, then immediately overwriting it with something else?)
Then it jumps to 00e7:0000963a.

It gets weirder: it jumps to 00e7:0000963a, then moves BP into SP(preparing for a stack pop probably, returning to FA4h), then POPs ES from the stack(popping 19Fh). Then DS(EFh), DI(390Bh), SI, BP, BX, BX again(?), DX, CX, AX, ending with a RETF to B3:000C.

OK. So that seems to result in another #GP(0) fault it seems? Seeing as SP=FECh, it's lowered by 0xC, thus having an error code pushed? So that's a normal interrupt return being handled there, at 0020:00000c7a, directly after the RETF.

I've been thinking... Perhaps there's an issue somewhere within the plain protected-mode behaviour(instead of instruction behaviour)?

Most of it is in https://bitbucket.org/superfury/unipcemu/src/ … pu/protection.c
So that's the segmentWritten(...) function for segment writes, CPU_MMU_checklimit(...) for instruction parameter validation and faulting(from the Execution Unit(forReading bit 4 set for BIU itself checking a memory access), checkSpecial(Port)Rights for privileged instructions, then there's port rights and IR map lookups and finally CPU_ProtectedModeInterrupt(...) for interrupts in protected mode.

All stack behaviour is in https://bitbucket.org/superfury/unipcemu/src/ … cpu/cpu_stack.c

Perhaps even some weird modr/m behaviour(although the test386.asm testsuite verifies it as valid)?
https://bitbucket.org/superfury/unipcemu/src/ … emu/cpu/modrm.c

Anyone can see anything wrong with those?

OK. Looking at the point it's handling the IRQ5 from within kernel mode, I see it eventually using the SS:SP from the originally interrupted code(which is invalid) instead of actually using the interrupted stack of the kernel that's being interrupted?

So the kernel has interrupted(the #NP fault handler) the user with an invalid stack(let's call this task #0), but the kernel itself is interrupted while it's handling something(let's call that task #1), causing the interrupt handler within the kernel's interrupted state (itself essentially being task #2 nested within task #1) to use the SS:SP from the task #0 instead of the more correct task #1 one?

And of course, the kernel doesn't like itself being faulting, thus aborts the whole thing as a fatal error.

Now then, why is the kernel trying to use the #NP fault handler's IRET frame(task #0) instead of the IRQ5 fault handler's IRET frame(task #1, which is obviously SHOULD use)?

OK. So 0020:0000362d (mov di,word ss:[bp+40]; with BP=FB4) loads from address FF4h? But that's the SP value of the interrupt handler of not the current interrupt handler, but instead the handler of the original stack that was interrupted by task #1. It should instead use the kernel's stack in that case, as the original interrupted stack was invalid?

Why would it use the stack of the interrupt that the kernel's interrupt handler was handling a fault of(which is invalid), instead of the valid stack that the kernel has(and build on top of that)?

The main issue is at some fault triggering to 0020:00000c7a , with SP becomes 0xFEE and pushing an invalid SP value on the stack(SS=5b, SP=36CCh in that one)?

Edit: The last #NP fault was for 4DCh as the error code?

OK. Trying it again makes it stuck at 4B4 instead?

Edit: And again(with very slow stepping through using a debugger), it instead has the last ESP value of 28F:36D2 instead?

So it seems kind of inconsistent on it's errors too (except the error location/code, which is 0020:36D0 ErrCode 0000)?

OK. I don't see any weird kind of ESP values being stored on the stack when going from user mode to kernel mode during exceptions?

So it's either with normal interrupts that cause the invalid (E)SP to be written somehow. Or it's actually being overwritten somewhere in the kernel/nested user part somehow?