First post, by jspenguin0
This is a sample exploit showing a flaw in Dosbox's program system. In order for Dosbox to have built-in programs like COMMAND.COM, MEM.COM, etc., it sets up a handler with the callback system (usually #4), and then creates a virtual file on the system containing the callback instruction (FE 38). When the program callback handler is called, it loads a pointer from the program in memory at CS:0x113. Then it calls this pointer to obtain a PROGRAM class object, which it then runs.
However, if a malicious program knew the host location of a portion of emulated memory, it could write its own code to the location, and then call the program callback with the known location, causing the emulator to run the supplied code as a native program.
The most stable section of memory is video ram, the address of which can be obtained by running a debugger and finding the address of the variable vgapages.
This exploit should not be considered serious, for the following reasons:
1. Dosbox is open-source. Simply recompiling it with a different set of options should change the address of video memory.
2. The attack would have to be specific to a single version of Dosbox.
3. Dosbox is cross-platform. An attack would have to be specific to one platform.
Attached is a sample exploit for Dosbox 0.62 on Linux.
