It's not a bug but a feature of the dynamic core to avoid pagefault recursion which is part of the
original design of the normal core when dosbox was transitioned to support paging.

Maybe check the svn logs, think Antara should be mentioned there but don't remember.

Is betrayal in Antara even be playable in normal core?

Well it'll crash or so iirc.

WD - Thanks for the hint about page faults.

It looks like Betrayal in Antara has a double page fault which doesn't seem to be supported by the normal core.

I started looking at implementing double faults, I saw a table of conditions which can lead to a double fault (interrupt 8) at http://www.logix.cz/michal/doc/i386/chp09-08.htm. I verified that a 0 is pushed onto the stack as the error code when creating the exception. Based on when it would previously crash it appears that my double fault routine gets executed around the right time, but so far no luck. I bet I messed up the eip or perhaps there is more going on that I don't understand.

There are no "double pagefaults", what you're seeing is that there is a pagefault, it's handled by win3x but does not
return to the code where it was started so depending on the situation a "hanging" pagefault may or may not
create problems.

double/triple faults are something completely different, they occur if for example the ivt/idt is bad/intentionally wrong
and a fault happens, the fault can not be executed causing a second one etc.

WD - Thanks for the explanation of double faults, the more I read about them the more it seemed like it couldn't be the case since the Intel specs I was reading said that after a double fault the current task state would be bad.

I didn't know that a page fault could be handled but not return back to the same instruction. That would explain a lot.

It's actually quite simple to have it not return, a pagefault is nothing more than a pmode interrupt
so it pushes stuff on the stack, sets some special information and calls the handler. The handler routine
can do anything it likes, examining the code that caused the pagefault, then paging in memory,
then resuming execution at the faulting instruction (or leave out this point and do something else).

I replaced the page fault handler / loop with this:

1paging.cr2=lin_addr;
2CPU.CPU_Exception(CPU.EXCEPTION_PF,faultcode);
3throw new PageFaultException();

Sorry this is Java code, I didn't know how to achieve the same thing as easily with c++ (my c++ is gettiing rusty 😊) , plus I wasn't sure if there was a good reason (portability, etc) that dosbox just didn't jump to the top of the running loop.

I catch this java exception in the normal loop

1                   try {
2                        ret=CPU.cpudecoder.call();
3                        if (ret<0) return 1;
4                        if (ret>0) {
5                            /*Bitu*/int blah=Callback.CallBack_Handlers[ret].call();
6                            if (blah!=0) return blah;
7                        }
8                    } catch (Paging.PageFaultException e) {
9                    }

In the dosbox code cpu.mpl is stored then set cpu.mpl = 3, and when the page fault returns it restores it. I didn't have a good way to do it so I just left this out. Does anyone know if this will have a bad side effect? I assume it was there for a reason.

Along with h-a-l-9000's paging patch that I ported to Java (For Testers: CGA/VGA Video BIOS separation and Paging patch), this makes it so that Win98 boots all the way and explorer doesn't crash with my Java port. "Betrayal in Antara" also gets a lot farther, at least it opens many more files as seen in the debug log, before it crashes the Java code. So perhaps this fix addressed one issue and I'm on to another.

It would be interesting to come up with a c patch that can do this same thing, perhaps Windows and "Betrayal in Antara" could run with the Dosbox normal core.

Exception handling like that is usually very slow (because normal codeflow assumes exceptions are, um, exceptionally).
I don't know in how far this would affect java though.
The main reason back then to use the current logic is that not only the cores may trigger pagefaults, but the callback
code as well so your code wouldn't attribute for that (if left alone like that).

The cpu.mpl is used to have better pagefault codes since some memory accesses that are done in the internal code
belong to the system even though the current mode is user-code. You should try to retain these but not too many
bad things will happen if you kick out the whole mpl logic at all (minus effects on win9x systems).

WD - Thanks for the hint about the callbacks, I probably would have wasted a lot of time debugging that the first time it bit me.

I'm messing around with Dosbox Megabuild 6 source since it has the paging patches and Windows 98 runs just fine with the dynamic core. I'm still trying to get a normal core to run Win98. I'm getting a lot closer. It can boot up and run Diablo without any errors. But some things, like IE5 cause crashes.

I modified the code so that page faults will pop back to the top of the main cpu loop instead of running another instance of the loop in place. The only other changes I made was to the normal core string functions so that they update edi/esi/ecx immediately instead of using local variables so that page faults would work correctly with them.

I was wondering if someone had some pointers as to what I might have over looked and what else might be different between the normal and dynamic cores.

1EXPLORER caused an invalid page fault in
2module OLE32.DLL at 0167:7ffa9fd5.
3Registers:
4EAX=0007620 CS=0167 EIP=7ffa9fd5
5EFLGS=00000287
6EBX=00c2f9e4 SS=016f ESP=00c2f8a0
7EBP=00c2f8c5
8ECX=00000010 DS=016f ESI=9000c2f8 FS=239f
9EDX=00000010 ES=016f EDI=00c2f9e4 GS=0000
10Bytes at CS:EIP:
11f3 a6 0f 85 77 06 00 00 8b 7d 08 33 c0 ab ab ab

1static void PAGING_NewPageFault(PhysPt lin_addr, Bitu page_addr, 
2								bool prepare_only, Bitu faultcode) {
3	paging.cr2=lin_addr;
4	//LOG_MSG("FAULT q%d, code %x",  pf_queue.used, faultcode);
5	//PrintPageInfo("FA+",lin_addr,faultcode, prepare_only);
6
7	if (prepare_only) {
8		cpu.exception.which = EXCEPTION_PF;
9		cpu.exception.error = faultcode;
10	} else {
11		if (in_callback==0) {
12			FillFlags();
13			CPU_Exception(EXCEPTION_PF,faultcode);
14			longjmp(top_of_loop, 1);
15		}
16		// Save the state of the cpu cores
17		LazyFlags old_lflags;
18		memcpy(&old_lflags,&lflags,sizeof(LazyFlags));
19		CPU_Decoder * old_cpudecoder;
20		old_cpudecoder=cpudecoder;
21		cpudecoder=&PageFaultCore;
22		if (pf_queue.used >= PF_QUEUESIZE) E_Exit("PF queue overrun.");
23		PF_Entry * entry=&pf_queue.entries[pf_queue.used++];
24		entry->cs=SegValue(cs);
25		entry->eip=reg_eip;
26		entry->page_addr=page_addr;
27		entry->mpl=cpu.mpl;
28		cpu.mpl=3;
29		CPU_Exception(EXCEPTION_PF,faultcode);
30#if C_DEBUG
31	//	DEBUG_EnableDebugger();
32#endif
33		DOSBOX_RunMachine();
34		pf_queue.used--;
35		LOG(LOG_PAGING,LOG_NORMAL)("Left PageFault for %x queue %d",lin_addr,pf_queue.used);
36		memcpy(&lflags,&old_lflags,sizeof(LazyFlags));
37		cpudecoder=old_cpudecoder;
38		//LOG_MSG("FAULT exit");
39	}
40}

1jmp_buf top_of_loop;
2
3void DOSBOX_RunMachinePF(void){
4	Bitu ret;
5	do {
6		ret=(*loop)();
7	} while (!ret);
8}
9
10void DOSBOX_RunMachine(void){
11	Bitu ret;
12	setjmp(top_of_loop);
13	do {
14		ret=(*loop)();
15	} while (!ret);
16}
17
18static Bitu Normal_Loop(void) {
19	Bits ret;
20	while (1) {
21		if (PIC_RunQueue()) {
22			ret=(*cpudecoder)();
23			if (GCC_UNLIKELY(ret<0)) return 1;
24			if (ret>0) {
25				in_callback++;
26				Bitu blah=(*CallBack_Handlers[ret])();
27				in_callback--;
28				if (GCC_UNLIKELY(blah)) return blah;
29			}

Partial extract of string changes

1static void DoString(STRING_OP type) {
2	if (core.prefixes & PREFIX_ADDR)
3		DoString32(type);
4	else
5		DoString16(type);
6}
7
8static void DoString16(STRING_OP type) {
9	PhysPt  si_base,di_base;
10	Bitu	count,count_left;
11	Bits	add_index;
12	
13	si_base=BaseDS;
14	di_base=SegBase(es);
15	count=reg_cx;
16	if (!TEST_PREFIX_REP) {
17		count=1;
18	} else {
19		CPU_Cycles++;
20		/* Calculate amount of ops to do before cycles run out */
21		if ((count>(Bitu)CPU_Cycles) && (type<R_SCASB)) {
22			count_left=count-CPU_Cycles;
23			count=CPU_Cycles;
24			CPU_Cycles=0;
25			LOADIP;		//RESET IP to the start
26		} else {
27			/* Won't interrupt scas and cmps instruction since they can interrupt themselves */
28			if ((count<=1) && (CPU_Cycles<=1)) CPU_Cycles--;
29			else if (type<R_SCASB) CPU_Cycles-=count;
30			count_left=0;
31		}
32	}
33	add_index=cpu.direction;
34	if (count) switch (type) {
35	case R_OUTSB:
36		for (;count>0;count--) {
37			IO_WriteB(reg_dx,LoadMb(si_base+reg_si));
38			reg_si+=add_index;
39			if (TEST_PREFIX_REP) reg_cx--;
40		}
41		break;
42	case R_OUTSW:
43		add_index<<=1;
44		for (;count>0;count--) {
45			IO_WriteW(reg_dx,LoadMw(si_base+reg_si));
46			reg_si+=add_index;
47			if (TEST_PREFIX_REP) reg_cx--;
48		}
49		break;

I modified the code so that page faults will pop back to the top of the main cpu loop instead of running another instance of the loop in place.

I think that won't work in the general case since the normal core wasn't written with atomic instruction behaviour in mind,
so if you just jump out on pagefaults some registers may already be modified so the state you're leaving the core in
is messed up (that's why the current pagefault handler tries to return to the intercepted instruction).

WD: Thank you for putting up with my questions. I too thought that one of the instructions wasn't reentrant after the page fault. But with your 2nd opinion I redoubled my efforts. I logged all the instructions that generated page faults while Win98 ran and there was only 59 different ops so I focused on each of them. Turns our I missed jumps.

1#define JumpCond32_d(COND) {					\
2	SAVEIP;											\
3	if (COND) reg_eip+=Fetchds();				\
4	reg_eip+=4;										\
5	continue;										\
6}

Obviously eip shouldn't be saved until after the fetch.

Now I no longer get the crashes around IE 5. In fact Win98 SE seems pretty stable running on my java port of the normal core.

😀