jez wrote on 2025-09-14, 18:45:

IDA 5 can give you a decent call graph of a real mode DOS program, and you can do analysis of later Windows stuff. But am I right in thinking that extended mode DOS programs are virtually impossible?

Huh, while I don't have the capabilities of all IDA versions and editions at hand, I am very confident that at least commercial IDA in the 5.x range was perfectly capable of loading the protected mode part of DOS4GW software (IIRC that's typicall "LX", a "linear executable") just fine.

mkarcher wrote on 2025-09-14, 19:01:

jez wrote on 2025-09-14, 18:45:

IDA 5 can give you a decent call graph of a real mode DOS program, and you can do analysis of later Windows stuff. But am I right in thinking that extended mode DOS programs are virtually impossible?

Huh, while I don't have the capabilities of all IDA versions and editions at hand, I am very confident that at least commercial IDA in the 5.x range was perfectly capable of loading the protected mode part of DOS4GW software (IIRC that's typicall "LX", a "linear executable") just fine.

I just took a look at a DOS/4GW executable (from Wacky Wheels) and it seems more subtle than this. [3ch] points outside the range of the file and IDA's efd (which is freeware and understands a lot of these formats) doesn't identify it as LE. However, DOS32A is open source and seemingly can run the file, and its source suggests it uses LE, possibly compressed. DOS32A is all assembly and in taking a 10 minute look I haven't been able to pinpoint how the code identifies which executable format it's parsing (it supports multiple, not just LE).

ignore or remove the stub. just decode the LE format.

the biggets problem your probably going to run into is that you will have to work out the calling convention that was used (and it could be multiple if using third party libraries), and yes it can still have real mode code that it calls back into (some audio libraries had 16bit trampolines for IRQs)

mkarcher wrote on 2025-09-14, 19:01:

jez wrote on 2025-09-14, 18:45:

IDA 5 can give you a decent call graph of a real mode DOS program, and you can do analysis of later Windows stuff. But am I right in thinking that extended mode DOS programs are virtually impossible?

Huh, while I don't have the capabilities of all IDA versions and editions at hand, I am very confident that at least commercial IDA in the 5.x range was perfectly capable of loading the protected mode part of DOS4GW software (IIRC that's typicall "LX", a "linear executable") just fine.

Not sure what's going on with mine then because it just seems to get lost at the bit where the far call (presumably to DOS4GW's loaded code) happens.

In fact, that makes sense. If DOS4GW's code is loaded into memory at runtime, IDA doesn't have it in the file being analyzed so how can it come up with anything meaningful?

jez wrote on 2025-09-14, 19:23:

mkarcher wrote on 2025-09-14, 19:01:

jez wrote on 2025-09-14, 18:45:

IDA 5 can give you a decent call graph of a real mode DOS program, and you can do analysis of later Windows stuff. But am I right in thinking that extended mode DOS programs are virtually impossible?

Huh, while I don't have the capabilities of all IDA versions and editions at hand, I am very confident that at least commercial IDA in the 5.x range was perfectly capable of loading the protected mode part of DOS4GW software (IIRC that's typicall "LX", a "linear executable") just fine.

Not sure what's going on with mine then because it just seems to get lost at the bit where the far call (presumably to DOS4GW's loaded code) happens.

In fact, that makes sense. If DOS4GW's code is loaded into memory at runtime, IDA doesn't have it in the file being analyzed so how can it come up with anything meaningful?

Because the DOS-extended portion is encoded as a Linear Executable (LE) a standard executable format.
It's not as simple as the stub+LE portion concatenated together (as a PE executable is--it's a DOS program saying "This program cannot be run in DOS mode" plus the Windows executable, and at file offset 3C, a DWORD pointing to the start of the PE), but in any case, I was able to use SB /U from sb.exe here (https://github.com/amindlost/dos32a/tree/main/binw) and unbind the stub/extract the LE out of the DOS4GW-extended program I was using for testing.

This produced an LE file, which IDA's extensive file dumper (efd) can understand. Note that IDA free won't be able to understand LE files.

mkarcher wrote on 2025-09-14, 19:01:

This produced an LE file, which IDA's extensive file dumper (efd) can understand. Note that IDA free won't be able to understand LE files.

You sure about that? IDA 5's open dialog seems to have an option for MZ/LE files.

I see; for retrocomputing stuff I've been using IDA Free 7.0 linux version which has everything but PE support removed. Maybe it will work in 5, but I remember trying it on VxDs (another case of LE files) and couldn't get it to work. Ghidra doesn't do well on those either unfortunately.

Well I diid manage to unbind the program and open it in IDA 5, though it recognized almost no subs and most of the code is just displayed as literal bytes.

jakethompson1 wrote on 2025-09-14, 20:19:

I see; for retrocomputing stuff I've been using IDA Free 7.0 linux version which has everything but PE support removed. Maybe it will work in 5, but I remember trying it on VxDs (another case of LE files) and couldn't get it to work. Ghidra doesn't do well on those either unfortunately.

Could you tell me what you were suggesting when you mentioned using IDA's efd on an LE file? Do you mean you need to use a recent paid version of IDA and you should get a decent disassembly graph view?

I've heard good things about IDA 4.

What I actually found worked best was to pause the game whilst it was running in the DOSBox-X debugger, and dump enough of the CS segment (maybe 2.5MB) into a bin file, then open that with IDA 5. Some bytes seem to get changed at runtime from the "unbound EXE", but the memory dump is analyzed quite successfully by IDA 5. As a bonus, various memory locations are filled with meaningful values that help with interpretation.

IDA added support for DOS/4G in version 6.0

However, I just stripped out the stub using the SUNSYS Bind Utility and used IDA (v4.9?) Freeware, this worked out well enough for my purposes.

Interesting... I thought I'd read that they dropped support for DOS format files after version 5.

Ah well, this memory dump method works OK anyway. I found that the program code being executed by the DOSBox-X debugger was subtly different from the original unbound exe, DOS/4GW patching up various bytes presumably (sometimes even changing instructions so that IDA couldn't interpret them), so it needed to be a memory dump.

jez wrote on Yesterday, 11:36:

Interesting... I thought I'd read that they dropped support for DOS format files after version 5.

You must not confuse the free edition and the commercial edition. The free edition switched over from supporting Windows and DOS executables to supporting Windows and Linux executables with 7.0 (the next free edition after 5.0), but the commercial edition still supports DOS executables. The quoted changelog applies to the commercial edition.