How can I place breakpoints on 32-bit code of DOS4GW bound executable file?

General information and assistance with DOSBox.

How can I place breakpoints on 32-bit code of DOS4GW bound executable file?

Postby dodamn » 2010-5-08 @ 10:34

I want to explore the internal of a certain DOS4GW bound executable file. So I want to place breakpoints on 32-bit code of DOS4GW bound EXE. But I can't find how to do it on DOSBox debugger.

As you know, DOS4GW bound EXE file consists of two parts: a 16-bit DOS stub that executes dos4gw.exe, and 32-bit LE(Linear Executable) that is loaded by the dos4gw.exe loader. Of course the LE part contains the application code!

It is possible and easy to place breakpoints on 16-bit DOS stub. But I don't know how to place breakpoints on 32-bit stub.


To try to solve this problem, I tried to use DOS32A instead of DOS4GW. As you know, DOS32A is complete alternative of DOS4GW. I expected that DOS32A helps me to how to do it. Because DOS32A is open source, so I can see DOS32A internal.

I used DOS32A ver.7.35. I made the EXE file, which I want to explore, to use DOS32A instead of DOS4GW. After then, I executed the EXE file on DOSBox debug mode using command 'debug <the EXE file>'.
On 'Code Overview' of DOSBox debug window, DOS32A code is shown. It is exactly same as dos32.asm file which is included in DOS32A source code. I was happy! So I clicked 'F10' key to run step-by-step. But at line no.222 of dos32.asm file(you can see assembly code 'call far ptr pm32_init' on dos32.asm file at line 222.), I clicked 'F10' key. After then, the status of DOSBox debugger is changed from 'debug mode' to 'Running mode'. And the 32-bit stub code was executed. So I never clicked 'F10' key again.

The following is some code of dos32.asm of DOS32A ver.7.35.

<code>
mov es,_membase
mov bx,_version
mov dx,offs critical_handler
call far ptr pm32_init ; enter Protected Mode
jc report_error

cli
mov _sel_cs,cs ; save PM selectors
</code>

Whan debugger's cursor is on 'call far ptr pm32_init', I clicked 'F10' key. After then, I expetecd that the cursor was on 'jc report_error'. But the cursor remained on 'call far ptr pm32_init'. And the status of debugger changed to 'Running' automatically. In other words, the 32-bit code was executed.

Is there any way to place breakpoints on the 32-bit code?
Please help me.

Sorry for my ugly english.
dodamn
Newbie
 
Posts: 6
Joined: 2010-5-08 @ 10:06

Re: How can I place breakpoints on 32-bit code of DOS4GW bound executable file?

Postby ripsaw8080 » 2010-5-08 @ 10:45

core=normal
User avatar
ripsaw8080
DOSBox Author
 
Posts: 3378
Joined: 2006-4-25 @ 23:24

Re: How can I place breakpoints on 32-bit code of DOS4GW bound executable file?

Postby dodamn » 2010-5-08 @ 10:49

I changed 'core=normal' on my config. And I tried again. But the problem remains.

I use DOSBox ver.7.3 on Ubuntu 9.10 64-bit.
dodamn
Newbie
 
Posts: 6
Joined: 2010-5-08 @ 10:06

Re: How can I place breakpoints on 32-bit code of DOS4GW bound executable file?

Postby h-a-l-9000 » 2010-5-08 @ 10:54

F10 = step over, F11 = step into?
1+1=10
h-a-l-9000
DOSBox Author
 
Posts: 4179
Joined: 2005-2-23 @ 00:14

Re: How can I place breakpoints on 32-bit code of DOS4GW bound executable file?

Postby ripsaw8080 » 2010-5-08 @ 10:56

core=auto switches to dynamic core when running in protected mode, and the dynamic core is what causes breakpoints to be missed, because the recompiled code executes in uninterruptable blocks. You can also change the source code to use blocks of a single instruction.

You may have another problem, but it's a fact that dynamic core causes breakpoints and single-stepping to not work properly.
User avatar
ripsaw8080
DOSBox Author
 
Posts: 3378
Joined: 2006-4-25 @ 23:24

Re: How can I place breakpoints on 32-bit code of DOS4GW bound executable file?

Postby dodamn » 2010-5-08 @ 11:10

Thank you ripsaw8080.

So.. Do I have to change the source code of DOSBox? If I have to, what part of code have to be changed?
dodamn
Newbie
 
Posts: 6
Joined: 2010-5-08 @ 10:06

Re: How can I place breakpoints on 32-bit code of DOS4GW bound executable file?

Postby ripsaw8080 » 2010-5-08 @ 11:15

If you need to debug while in dynamic core for some reason, find this line in core_dyn_x86.cpp:
Code: Select all
block=CreateCacheBlock(chandler,ip_point,32);
and change the 32 to a 1.

However, you may not need to go through this exercise. I have traced through extender stuff before, and just because a call is executed does NOT mean it returns. Sometimes a call is just a way they push a return address on the stack to figure out who did the calling.

Edit: oops, you said Ubuntu 64-bit, so you'll probably have to change the line in core_dynrec.cpp
User avatar
ripsaw8080
DOSBox Author
 
Posts: 3378
Joined: 2006-4-25 @ 23:24

Re: How can I place breakpoints on 32-bit code of DOS4GW bound executable file?

Postby dodamn » 2010-5-08 @ 12:29

I'm using 'core=normal', not 'core=auto'. So I think that the source code modification is not needed. Right?
dodamn
Newbie
 
Posts: 6
Joined: 2010-5-08 @ 10:06

Re: How can I place breakpoints on 32-bit code of DOS4GW bound executable file?

Postby ripsaw8080 » 2010-5-08 @ 12:35

Correct, not needed if using core=normal. You can also try setting an explicit breakpoint on the line after the call by moving the highlight there and pressing F9, but I suspect you've simply got a case of a call that doesn't return, at least not to the instruction after the call.
User avatar
ripsaw8080
DOSBox Author
 
Posts: 3378
Joined: 2006-4-25 @ 23:24

Re: How can I place breakpoints on 32-bit code of DOS4GW bound executable file?

Postby dodamn » 2010-5-08 @ 12:55

I think you're right. The function call does not return. So I'm sad.
Is there no way to poke into the 32-bit code of DOS4GW(or DOS32A) bound executable??

But I believe that there are other way to poke into the 32-bit code.
OK, think about it....
Assume that I have a .exe file which is bounded with dos4gw. And I want to poke that file. I mean that I want to poke the 32-bit code(program main routine), not 16-bit code(dos4gw related stub).
How can I poke it? As you know, it is simple. Just follow the steps.
step 1. Run DOSBox.
step 2. Run command 'debug <a .exe file>'.
step 3. Place a breakpoint on the entry point of the 32-bit code of the .exe file.
step 4. Click 'F5' key.

I expect that DOSBox debugger stops the program running. And the cursor of DOSBox debugger is placed on the entry point of the 32-bit code.

But there is a probelm. I do NOT know the address of the 32-bit code of the .exe file. So I can't place a breakpoint on it.

How can I know it?
dodamn
Newbie
 
Posts: 6
Joined: 2010-5-08 @ 10:06

Re: How can I place breakpoints on 32-bit code of DOS4GW bound executable file?

Postby h-a-l-9000 » 2010-5-08 @ 13:05

F11 = step into!!!!
1+1=10
h-a-l-9000
DOSBox Author
 
Posts: 4179
Joined: 2005-2-23 @ 00:14

Re: How can I place breakpoints on 32-bit code of DOS4GW bound executable file?

Postby ripsaw8080 » 2010-5-08 @ 13:06

Switching to protected mode does some things like changing the CR0 register and and finally executing an IRET, so maybe trace into the call and keep single stepping until you land on the instruction right after the IRET. You'll know that the switch to PM has taken place because the debugger indicates Real/PM16/PM32 mode. Tedious, but it should work.
User avatar
ripsaw8080
DOSBox Author
 
Posts: 3378
Joined: 2006-4-25 @ 23:24

Re: How can I place breakpoints on 32-bit code of DOS4GW bound executable file?

Postby dodamn » 2010-5-08 @ 13:35

Thank you.

I finally got the entry point of the 32-bit stub!
I manually found it. Just clicked 'F10', 'F11', 'F5' and 'F9'. Yes it is really tedious. But I'm happy! Haha!
dodamn
Newbie
 
Posts: 6
Joined: 2010-5-08 @ 10:06


Return to DOSBox General

Who is online

Users browsing this forum: mperelman and 1 guest