On POP, the increment happens before (E)SP is assigned. So if 0x(ffff)fffc is what's on the stack, (E)SP will be 0x(ffff)fffc.

You're using PUSH SP though, which adds complications. On 8088/8086 the value of SP that's written will be after it's decremented, so 0xfffa will end up on the stack and that's what will be in SP at the end of the sequence. On 80186 and later the written value is before the decrement, so 0xfffc will end up on the stack and then in SP following the POP.

Just realized something: imagine the following running(with a 32-bit stack attribute size):

1MOV ESP,FFFFFFFC
2MOV [SP],FFFC
3POP SP

That would cause ESP to become an invalid FFFC instead of FFFFFFFC? The overflow from the add is still applied, but the low 16 bits gets mixed in? Thus invalidly clearing the high 16 bits?

No, because you've written to [SP] rather than [ESP]. So you've corrupted [0xFFFC] and the POP will then read whatever was at [0xFFFFFFFC].

If your code was :

1MOV ESP,FFFFFFFC
2MOV [ESP],FFFC
3POP SP

then ESP will indeed be 0x0000fffc.

Whoops, my bad at my code example. You're right. So, if you don't watch 9ut mixing 16/32-bit code, you can set the high word of ESP corrupted(cleared in this case of 0verflow) due to the 32-bit code mixed with 16-bit code and pop sp combined with that case, even though that won't happen when using 32-bit offsets. So that hidden addition mechanic WILL cause ESP to get unexpected results popping such an value(at popping sp at offset FFFC). Even though it's an edge case, it might be good to have it emulated correcly(before I would just read the word and write SP in that case, which would cause incorrect ESP when ESP=FFFFFFFC, 32-bit stack and [ESP]=FFFC. You would logically expect ESP=FFFFFFFC after the pop, but it will become ESP=0000FFFC instead. Good to know. Don't know if the others, like vladstamate(capex86), have it implemented in that way already?

When you have "POP SP" for a 32bit stack segment descriptor, I think you get different behavior whether or not you have 66h 67h preceding your "POP SP". I think with address mode override and register override top 16bits of ESP will not change, right? Or does it?

Address size override has no effect on that? SP/ESP is determined by SS descriptor size(big bit) only?

http://www.logix.cz/michal/doc/i386/chp17-01.htm

The only way the address size operand has effect with push/pop is with the r/m data pointing to memory, e.g. pop [cs:0001] vs pop [cs:00000001] (opcode 8F). The stack address size is set by the B-bit in the SS-descriptor in both cases.

superfury wrote:

Address size override has no effect on that? SP/ESP is determined by SS descriptor size(big bit) only?

Correct. SP/ESP is indeed entirely determined by SS, and prefixes have no effect.