It's an address (so mod==3 is invalid) within a page that will have its TLB entries flushed. It can't fault because it doesn't access memory.

What happens when mod==3 in V86 mode? Will it throw a #UD(because of the MOD) or a #GP(0) because of CPL?

The instruction is invalid regardless in V86 mode, #GP(0). The description in Intel's manual is pretty straightforward.

Looking at Dosbox, all other opcodes seem to have higher priority on the CPL check than the #UD check, but in the case of INVLPG it's reversed?

So does it throw #UD or #GP when in V86 mode with MOD=3? Which one is checked first?

In the end, #UD and #GP are two totally different exception handlers. The Virtual 8086 monitor might handle them differently.

Looking at Dosbox and Bochs, it seems that in Bochs the #UD is generated AFTER the CPL(thus lower priority), but Dosbox seems to first check the MOD, then CPL after that, which is the reversed order?

Edit: I've just modified it to handle #GP(0) over #UD. So V86 mode will always throw #GP(0), while real/privileged(CPL=0) protected mode will throw #UD when invalid. So privilege having higher priority than #UD.
Would that be correct behaviour(matching Bochs' behaviour)?

Edit: OK. The last I see is a verify sectors command sent to the INT13h handler, after which it starts single-stepping through the code/BIOS, because the trap flag was set?

So perhaps there's some kind of unknown issue with the INT13h function 04h handling?

Just have been thinking: does INVLPG use the offset as the linear address, or does it translate the offset of the segment:offset pair to a linear address using the segment descriptor cache first? What if the segment descriptor cache entry(e.g. DS's descriptor cache) used isn't present?

Or what happens when you execute INVLPG on a non-zero based descriptor(e.g. DS.base=0x10000, INVLPG DS:[0])? Does it invalidate page for linear address 0 or the page for linear address 0x10000?

Edit: OK. So if I have to believe Bochs' behaviour, the offset is combined with the descriptor to obtain a linear address(like I thought), then the resulting linear address is used to evict said page from the TLB. So the latter case will happen: the page at 0x10000 will be invalidated.