MMIO is unconditionally disabled when setting a video mode (lines 764-768 in current int10_modes.cpp). If that has an effect in this case, could try only clearing bit 4 on the ext_mem_ctrl register if the new mode has pstart==0xA0000, so that MMIO will be left in place when there is no conflict.

Thank you for the excellent suggestion and pseudocode. I see those lines in int10_modes and will test further.

Edit: opening the dos box does goes through that section of code inside the function INT10_SetVideoMode. Removing the specific lines to disable mmio hasn't yet prevented the previous issue, but it seems important to keep track of those lines. I also tried the change in conjunction with a few changes to vga_memory.

Just noted that TheGreatCodeholio already had debugged this issue:
https://github.com/joncampbell123/dosbox-x/co … 4a31d77263074b7

Quote: "Windows 95 OSR2 relies on purposely causing a GP fault (touching an execute-only code segment with zero length) to start a DOS VM."

There are other supporting commits to this one, but it indicates that it is an OSR2 specific issue (presumably not present in retail Win95) and on real hardware the GP fault is necessary. However, it's not clear to me why this event only occurs with active video acceleration and clearing of the mmio region; it does suggest that the GP fault is caused only where the mmio is accessed by the starting dos box.

On further testing, I think the GP fault is related to the mmio issue. Under the dynamic core and Win95 OSR2 at 256 colors, along with reversion of mmio code for case3, the S3 PCI driver allows the opening of the dos box window. However, the S3 Trio 32/64 driver leads to missing GUI elements in this exact same case -- very likely an issue related to the original way of resetting of the mmio at the "A0" location.

If setting color depth at 16 bit and reverting the commit 3974, then both drivers (1998 versions) open the dos box without issue.

However, the "segment limit emulation" (normal core) from dosbox-x corrects the issue for the S3 Trio 32/64 driver at 256 color depth, and this suggests that the GP fault and mmio issue are related. This emulation also allows commit 3974 to work without issue, too; although the dynamic core is not yet compatible with the segment limit emulation nor the commit 3974 in this example, and so reversion of this commit and use of the S3 PCI driver may be preferred for additional compatibility with the Win95 GUI at 256 color depth.

r3974 also causes lots of "corrupt MCB chain" crashes with machine=pcjr. That's because DOSBox puts a memory control block at A000:0000 with the PCjr machine type, and that memory is disabled by r3974. DOSBox should put the MCB at 9FFF rather A000 like it does in the other machine types (except Tandy).

1--- dos_memory.cpp	2015-01-29 12:10:03 +0000
2+++ dos_memory.cpp.new	2016-04-19 06:11:57 +0000
3@@ -443,7 +443,7 @@
4 		/* memory from 128k to 640k is available */
5 		mcb_devicedummy.SetPt((Bit16u)0x2000);
6 		mcb_devicedummy.SetPSPSeg(MCB_FREE);
7-		mcb_devicedummy.SetSize(0x9FFF - 0x2000);
8+		mcb_devicedummy.SetSize(0x9FFE - 0x2000);
9 		mcb_devicedummy.SetType(0x5a);
10 
11 		/* exclude PCJr graphics region */

r3974 does not *cause* the MCB to be wrong, and it's only wrong in the umb=true case.

Related:
Regression: r3974 caused Corncob 3D to fail on launch
https://sourceforge.net/p/dosbox/bugs/443/

The actual cause of the problem with Corncob 3D is fixed in r3988. Thanks to r3974, such problems no longer go unnoticed.