VOGONS


First post, by evo

User metadata
Rank Newbie
Rank
Newbie

The current behavior of DOSBox when encountering an invalid opcode is to exit with an error message.
The expected behavior on a real CPU is to raise exception 6 with the (E)IP on the stack pointing to the unrecognized instruction.
Now, I've come across a program which actually uses this to create a new 'instruction' with opcode FF FF, which I found to appear quite a few times in said program.
If I change DOSBox to properly raise the exception, the game works fine as it should.
Even though in most cases the encounter of an invalid opcode indicates that something went wrong, we cannot assume this a priori.
We could check whether a program sets its own handler for int 6 and only exit with an error when it's still the default handler. Or maybe just printing a warning is ok too.

Reply 1 of 10, by wd

User metadata
Rank DOSBox Author
Rank
DOSBox Author

> The current behavior of DOSBox when encountering an invalid opcode is
> to exit with an error message.

No, those cases which are used by games/programs raise an int6.
Most others are, as you said, considered to be the result of
executing rubbish, thus exit dosbox to avoid damage.

> 'instruction' with opcode FF FF,

What program is it?

Reply 2 of 10, by evo

User metadata
Rank Newbie
Rank
Newbie
wd wrote:
> The current behavior of DOSBox when encountering an invalid opcode is > to exit with an error message. […]
Show full quote

> The current behavior of DOSBox when encountering an invalid opcode is
> to exit with an error message.

No, those cases which are used by games/programs raise an int6.
Most others are, as you said, considered to be the result of
executing rubbish, thus exit dosbox to avoid damage.

I see, but the possibility that there is a game/program that is not considered is always greater than zero. I mean, there's tons of not widely spread programs and privately written code which might make use of it in any way. I still prefer the idea of checking for a modified int 06 handler, a debug build can still spit out warnings. I know that extenders like DOS4/GW install their own exception handlers but they usually dump out informative error messages.

wd wrote:

> 'instruction' with opcode FF FF,

What program is it?

DIF-2

Reply 3 of 10, by wd

User metadata
Rank DOSBox Author
Rank
DOSBox Author

> that there is a game/program that is not considered is always greater than zero.

I'm quite sure about that as well, but it's a lot better imo to really
check each case if it really forces an int6 (win3.x does that for
example to install some of its exception handlers). And some of
those invalid opcodes have pretty odd behaviour in v86 mode or
pmode for example.

Reply 4 of 10, by evo

User metadata
Rank Newbie
Rank
Newbie
wd wrote:
> that there is a game/program that is not considered is always greater than zero. […]
Show full quote

> that there is a game/program that is not considered is always greater than zero.

I'm quite sure about that as well, but it's a lot better imo to really
check each case if it really forces an int6 (win3.x does that for
example to install some of its exception handlers). And some of
those invalid opcodes have pretty odd behaviour in v86 mode or
pmode for example.

Are you refering to some undocumented opcodes here? (http://www.x86.org/secrets/intelsecrets.htm)

The point is, stuff like this can safe us some headaches, i mean, I spend quite some time until I realized that execution of this FF FF was intentional. Ok, it wasn't a complete waste for me as it helped me understanding DOSBox, but not everyone is familiar with machine level programming and track down 'bugs' like this.
Why not write a simple program and check on a real CPU whether specific invalid opcodes raise an exception or not?

Reply 5 of 10, by wd

User metadata
Rank DOSBox Author
Rank
DOSBox Author

Undocumented opcodes are present, besides some 286-only ones.

> until I realized that execution of this FF FF was intentional.

That's exactly what should be done (and already was done for
some antidebugging stuff): trying to understand if it IS intentional
or just hits some bug in dosbox. If we'd throw an int6 always
for unhandled stuff, it'd be a lot harder to debug crashing games
(they just hang somewhere or do evil things at wost).
But the case of the 0xff 0xff for example should have been rather
easy to detect, as dosbox E_Exits with an error message, so you
know already the (last) point of failure. The code around that gives
hints about what could be wrong in most situations (wrongly
decrypted code should be noticable e.g.)

> Why not write a simple program and check on a real CPU whether
> specific invalid opcodes raise an exception or not?

I personally don't see any point in this, as forced invalid opcodes
are a really rare thing (only found in a few anti debugging tools
and some other things). And it'll add some code which is never
needed at all.

> DIF-2

The game or is it some other program?

Reply 6 of 10, by evo

User metadata
Rank Newbie
Rank
Newbie
wd wrote:

Undocumented opcodes are present, besides some 286-only ones.

You guys are really doing great work 😀

wd wrote:
That's exactly what should be done (and already was done for some antidebugging stuff): trying to understand if it IS intentiona […]
Show full quote

That's exactly what should be done (and already was done for
some antidebugging stuff): trying to understand if it IS intentional
or just hits some bug in dosbox. If we'd throw an int6 always
for unhandled stuff, it'd be a lot harder to debug crashing games
(they just hang somewhere or do evil things at wost).
But the case of the 0xff 0xff for example should have been rather
easy to detect, as dosbox E_Exits with an error message, so you
know already the (last) point of failure. The code around that gives
hints about what could be wrong in most situations (wrongly
decrypted code should be noticable e.g.)

> Why not write a simple program and check on a real CPU whether
> specific invalid opcodes raise an exception or not?

I personally don't see any point in this, as forced invalid opcodes
are a really rare thing (only found in a few anti debugging tools
and some other things). And it'll add some code which is never
needed at all.

I understand your reasoning, and personally I can live with it now that I know about it. Are there more things that cause incompatibilities and that are good to know (besides all the error handling of segmentation/task switching stuff). Maybe it's a good idea to mention this in the docs, if it is already, sorry for not RTFM 😉

wd wrote:

> DIF-2

The game or is it some other program?

the game

Reply 7 of 10, by wd

User metadata
Rank DOSBox Author
Rank
DOSBox Author

> Are there more things that cause incompatibilities and that are good to know

Yes, for example dosbox does not check segment limits, only does some
small attempt to check for segment/page privileges (suppose that's what
you mean by error handling of segmentation). All those things would
result in a huge slow down, so they'll never be added. Luckily it only
affects OS software (yet bears no real problem there) and two or three
tricky demos/games.
Task switching is fine btw. 😀

Reply 8 of 10, by franpa

User metadata
Rank Oldbie
Rank
Oldbie

state as clearly and precise as humanly possible what games and/or programs that require this change in DOSbox.

AMD Ryzen 3700X | ASUS Crosshair Hero VIII (WiFi) | 16GB DDR4 3600MHz RAM | MSI Geforce 1070Ti 8GB | Windows 10 Pro x64.

my website

Reply 9 of 10, by evo

User metadata
Rank Newbie
Rank
Newbie

> All those things would result in a huge slow down, so they'll never be added.

I fully agree there, can't think of any efficient way either besides using virtualization. Paging slows down things already enough.
Is paging support actually fully compatible? That's the part I'm currently about to figure out.
Also A20 gate switching seems only to affect the HMA part of the memory and is done using the paging mechanism. There are conflicts when paging is used and A20 switching behaviour is crucial, but that doesn't affect too many real world programs. I'm wondering because I have come across strange behaviour when interrupt handlers are called that reside in the HMA and occassionally find only garbage code there. Funnily, this only seems to happen when gate switching is done via keyboard controler. When port 92h is used for switching things seems much more stable (btw. the respective flag for port 92h gate switching support is not set by the bios. Is there any reason for that?)

Reply 10 of 10, by wd

User metadata
Rank DOSBox Author
Rank
DOSBox Author

> state as clearly and precise as humanly possible what games and/or
> programs that require this change in DOSbox.

I remember segment limits to be used by a pacman clone and some
demo (which was supposed to be a game but was never finished).

> Is paging support actually fully compatible? That's the part I'm currently
> about to figure out.

The main problem about paging is the way it works: as it was added later
on, it starts a new dosbox core (full core in all cases) to let the handler run,
and then continues the faulting code wherever it was (that can be inside an
opcode, or somewhere in a callback which can be quite nasty). This assumes
that the page fault handler actually returns, which is not the case in win98+
for example.
The dynamic core has been changed in that point for memory access through
opcodes because it can hide the emulated registers (reg_eax etc.) for quite
a while, so the pagefault handler gets old ones (several games didn't work
in the past under the dyncore because of this, like c&c, but are fine now).

> I'm wondering because I have come across strange behaviour when
> interrupt handlers are called that reside in the HMA and occassionally
> find only garbage code there.

Hm sounds a bit like emm386 😉
I don't remember any problems with that, you might want to be sure they
don't use the vcpi code of dosbox, which might have serious problems
with that.
In the debugger you can print out the current pagetable by the command
"paging" just in case you didn't notice it. The a20 handling is done at a
quite low level, and the paging on a level above that (as real hardware
does it as well).
Are they using v86 mode in that program?

> Funnily, this only seems to happen when gate switching is done via
> keyboard controler.

Could also be that they're using some unimplemented keyboard codes,
so something "hangs" in the kb control regs when they're trying to
switch a20 on/off.
Otherwise i think the code for the actual switching is identical.

> btw. the respective flag for port 92h gate switching support is not set by
> the bios. Is there any reason for that?)

Don't know, is it used by the program? Might have to check a few extenders
if they query it.