Pagefault handling in the normal core

Here you can discuss the development of patches.

Pagefault handling in the normal core

Postby truth_deleted » 2013-8-27 @ 06:15

The normal core was designed for pagefault recursion, a feature incompatible with a Windows9x guest in DOSBox. However, the dynamic core avoids this pagefault recursion and therefore 9x guests do not generate errors. A couple of years ago, danoon created a "patch" to Hal's Megabuild6 (MB6) to avoid pagefault recursion in the normal core; this was also implemented in his jDOSBox build. There are two strengths of this way of handling pagefaults within the normal core.

The first is that the normal core is portable across computer architectures, whereas the dynamic core is specific to an architecture (such as x86). Having a normal core with Win9x friendly pagefault handling would avoid the necessity of programming a new dynamic core for non-x86 architectures. This would allow an easy port of DOSBox with 9x compatibility.

The other reason is for debugging: [danoon coded an ISA-based IDE controller (based on qemu) for jDOSBox (normal core). If an attempt is made to port this to DOSBox, then it would be ideal to test it under the same normal core (given any chance that it is incompatible with the dynamic core). There are reports that the qemu IDE controller has undergone "bit rot", so it would be wise to port the Java version instead of the qemu version; yet use the qemu for code where it matches to danoon's Java code base.]

I attached an incomplete patch to disable pagefault recursion. This is based on danoon's Java code and also this Vogon thread: http://www.vogons.org/viewtopic.php?t=29243. This patch would be applied against DOSBox 0.74/SVN+Hal's MB6 paging code.

I commented out the "longjmp" function and the "jmp_buf top_of_loop" statement. These are posted in that Vogon thread but don't seem to appear in the Java code. Also, I have not yet located the CMPW, CMPB, CMPD functions in the Java code. All these bits are examples of unfinished parts of this patch. I also would like to verify the ported parts appear correct. Danoon has performed the herculean task of providing the 9x friendly pagefault code - it should be much easier to port it and it will future-proof the use of DOSBox with 9x.
You do not have the required permissions to view the files attached to this post.
truth_deleted
 

Re: Pagefault handling in the normal core

Postby Dominus » 2013-8-27 @ 07:01

Moved to the patches forum (users are not allowed to create new topics there to keep that forum somewhat clean).

Thanks for looking into this. I am just wondering whether this should be a core of its own, maybe W9xcore :) to keep the normal core clean. But that is something the devs need to decide at some point (I'm all for limited Windows 95 compatibility (limited as in almost supportless :))).
User avatar
Dominus
DOSBox Moderator
 
Posts: 7644
Joined: 2002-10-03 @ 09:54
Location: Ludwigsburg

Re: Pagefault handling in the normal core

Postby danoon » 2013-8-27 @ 16:05

setjmp / longjmp is not allowed in Java, instead Java uses try catch. In PAGING_NewPageFault I throw an exception in Java so that it will exit the current instruction and start execution of the page fault handler. In c, you might need to use setjmp/longjmp to implement that functionality.

After the page fault is handled, in most cases it will resume executing the instruction that caused the page fault. This means that every instruction needs to be re-entrant

Take this code for example

CASE_W(0x60) /* PUSHA */
{
Bit16u old_sp=reg_sp;
Push_16(reg_ax);Push_16(reg_cx);Push_16(reg_dx);Push_16(reg_bx);
Push_16(old_sp);Push_16(reg_bp);Push_16(reg_si);Push_16(reg_di);
}

This instruction is obviously not reentrant. The first push modifies ESP then if the second push causes a page fault you would be in trouble when this instruction gets called again.

Here is another one off the top of my head

CASE_W(0xc4) /* LES */
{
GetRMrw;
if (rm >= 0xc0) goto illegal_opcode;
GetEAa;
if (CPU_SetSegGeneral(es,LoadMw(eaa+2))) RUNEXCEPTION();
*rmrw=LoadMw(eaa);
break;
}

Notice how the LoadMw(eaa) comes after the CPU_SetSegGeneral, this is not reentrant because the cpu state was changed before a read that could cause a pagefault.

Sorry I don't have a list of all the instructions I changed to make reentrant, it was an iterative process.

For Windows 95, I didn't see any problems with the existing page fault code, but Windows 98 would crash in weird places, like just viewing the favorites menu or opening a file in media player. In those places Windows 98 would not handle the page fault and thus the instruction that causes the page fault would never resume which causes the dosbox normal core problems because it expects all page faults to be handled. I'm not sure how the dynamic core works, I kind of assumed it called PAGING_NewPageFault at some point.
danoon
Member
 
Posts: 132
Joined: 2011-1-04 @ 19:12

Re: Pagefault handling in the normal core

Postby truth_deleted » 2013-8-28 @ 04:50

danoon: "setjmp / longjmp is not allowed in Java, instead Java uses try catch. In PAGING_NewPageFault I throw an exception in Java so that it will exit the current instruction and start execution of the page fault handler. In c, you might need to use setjmp/longjmp to implement that functionality."

Thank you for the helpful description of your pagefault code. I attached an updated patch which implements the setjmp/longjmp to exit PAGING_NewPageFault and enter DOSBOX_RunMachine; this appears to match your 2011 post showing the MB6 patch. I believe I correctly defined the variable so the jump can occur across source files.

danoon: "After the page fault is handled, in most cases it will resume executing the instruction that caused the page fault. This means that every instruction needs to be re-entrant."

I appreciate the examples on making the instructions reentrant. I believe the port I made of your "string.h" code is correct, but I now realize that changes would be required of the prefix header files, too. Your code certainly provides a very helpful reference to find the non-reentrant instructions. However, I was expecting to have Win95b or Win98 boot without changing the prefix header files (both Windows versions hang before reaching the graphical boot screen).

danoon: "For Windows 95, I didn't see any problems with the existing page fault code, but Windows 98 would crash in weird places.... In those places Windows 98 would not handle the page fault and thus the instruction that causes the page fault would never resume which causes the dosbox normal core problems because it expects all page faults to be handled."

The other issue is I'm using Win95b (OSR2) and Win98. Win95b may have a paging scheme which is similar to Win98 and hence my dismay at finding out that both do not boot using normal core.

[Edit: I verified that Win95a does boot with normal core, unlike Win95b, but directx games will not work. However, they do work upon changing to dynamic core.]

I also have to test whether Win95 runs in jDOSBox without paging falling back to (16-bit?) compatibility mode. It does in DOSBox and have wondered whether this was related to lack of an IDE controller or lack of your paging patch or that all versions fallback to this mode. I was concerned for IDE controller development and whether the paging is an impediment to its functioning in 9x.

Dominus: "Thanks for looking into this. I am just wondering whether this should be a core of its own, maybe W9xcore."

I appreciate the support! I hope I can find the errors in the ported code.
You do not have the required permissions to view the files attached to this post.
truth_deleted
 

Re: Pagefault handling in the normal core

Postby crazyc » 2013-8-30 @ 14:25

[quote="danoon"]Sorry I don't have a list of all the instructions I changed to make reentrant, it was an iterative process.[/quote]
One that was a bit hard to find while I was trying to get OS/2 running in MESS was POP Ev (0x8f). If the SIB byte references ESP then the ea is calculated based on the post-pop value but ESP has to be reset if the write faults.
crazyc
Member
 
Posts: 100
Joined: 2013-2-02 @ 16:17

Re: Pagefault handling in the normal core

Postby truth_deleted » 2013-8-31 @ 05:16

The attached patch enables the use of Windows 9x in DOSBox under the normal core. One directx game already works - Jedi Knight (9x); also, Internet Explorer works (95). There are page fault errors in some other directx games, but I will try to fix these errors according to danoon's post and code. Thank you, danoon!!

Please test the attached patch against DOSBox 0.74/SVN+Hal's paging patch. This compilation should allow non-x86 platforms to run Win9x (given available SDL drivers).
You do not have the required permissions to view the files attached to this post.
truth_deleted
 

Re: Pagefault handling in the normal core

Postby truth_deleted » 2013-9-01 @ 05:25

The attached patch extends 9x paging to the support.h file. This was enabled by using danoon's Java code as a guide and his Vogons post on this issue. The Win95 shell is working well and better than the Win98 shell in normal core mode. For the few I tested, most directx games appear to work well; the exception is Unreal Tournament which generates a page fault error prior to starting. This error (and any others which appear) should be fixable by editing CPU instructions in header files under the core_normal directory, and ensuring that they are reentrant as described by danoon above (i.e., string.h, prefix_0f.h). As he also pointed out, this is an iterative process; so, one strategy is to debug for the instruction which causes the page fault error and then make that instruction reentrant. This appears to be the only path to a fully 9x friendly normal core. Even if this patch is not extended further, it appears to provide compatibility for most directx games under the normal core (preferably in Win95 because the shell is stable).
You do not have the required permissions to view the files attached to this post.
truth_deleted
 

Re: Pagefault handling in the normal core

Postby truth_deleted » 2013-9-02 @ 04:48

Attached patch which extends 9x paging to instructions in prefix_none.h and those changed correspond to danoon's post above. I am fairly confident that the PUSHA instruction is now reentrant. However, the changes to LES and LDS are not fully verified; the same for "JumpCond32_d" which was changed previously. I made these changes to document danoon's examples and to provide a reference for implementing future reentrant instructions in the normal core.

Directx games are nearly as stable under the normal core as they are under the dynamic core. Unreal Tournament is still generating a page fault and I assume this is fixable by the method above. For future bugs, it would be best to find the offending instruction by debugging and then use danoon's code as a reference to recode the instruction so it is reentrant. Because the programming language is different, this is not a simple process and I used a slightly different approach for PUSHA.
You do not have the required permissions to view the files attached to this post.
truth_deleted
 

Re: Pagefault handling in the normal core

Postby truth_deleted » 2013-9-02 @ 09:10

Modified string.h source file to correspond with danoon's post and jDOSBox; did not port jDOSBox code where it led to instability in Win95. I am unable to verify the effects of my recent code changes, but this patch version appears to work best and supports 16-bit colors.

Try Win95/640x480/16-bit colors; core=normal. Directx games appear fairly stable from limited testing; 2d games, such as Starcraft, show decent framerates under normal core (core2duo system). Quake 1 (3dfx) has nearly playable framerates. It *seems* like the 1996-97 directx games do not show the pagefault errors as seen in some later releases.

[Edit: added fix to patch so no error upon closing DOSBox.]
You do not have the required permissions to view the files attached to this post.
truth_deleted
 

Re: Pagefault handling in the normal core

Postby truth_deleted » 2013-9-02 @ 16:50

Modified the prefix_66.h source file so the PUSHAD instruction is reentrant. Attached the patch which also seems to improve the shell and in-game menu responsiveness of Win9x games; some run at 40% of the dynamic core speed. I haven't yet tested the effect of compiler optimizations on the normal core speed, but it is already showing impressive in-game framerates. I haven't yet confirmed whether the "normal core paging" patch has a similar effect on DOS games.

The patch is sufficient for starting many directx5/6/3dfx games. These games should be further tested for stability under the normal core. I've only seen one certain page fault in Win95 so far: UT99; I am hesitant to debug the code for one game and it is possible that the error is not fixable in the page fault handling code.
You do not have the required permissions to view the files attached to this post.
truth_deleted
 

Re: Pagefault handling in the normal core

Postby truth_deleted » 2013-9-05 @ 06:45

Updated patch with new (unverified) reentrant instructions in prefix_0f.h source file. More important, I removed modifications to the string.h file; these previous changes did not fully mirror danoon's code and caused glitches in Win98. The attached works well in Win95 and is somewhat compatible with Win98, although 95 is highly preferred under the normal core (as danoon posted earlier).
You do not have the required permissions to view the files attached to this post.
truth_deleted
 

Re: Pagefault handling in the normal core

Postby yahma » 2013-9-30 @ 19:21

Patch seems to break something in dynamic (dynrec) core. Prior to patch, win98 worked (enough to boot and use for a while) in dynamic core. After patch, always getting pagefault exception on boot of win98 in dynamic core. Reverting the patch allows win98 to work again.
yahma
Newbie
 
Posts: 15
Joined: 2012-2-20 @ 20:16

Re: Pagefault handling in the normal core

Postby truth_deleted » 2013-10-01 @ 01:21

If testing (or further developing) this patch, then dosbox can be built with the patch and then used with the normal core only. For general use of dosbox with an architecture supported by a dynamic core, then this patch will not be necessary since the dynamic core already supports a Win9x guest.
truth_deleted
 


Return to DOSBox Patches

Who is online

Users browsing this forum: No registered users and 1 guest