for unpacking old dos executables

what executables - maybe someone know already how to unpack them

no idea (cup386 don't seem to like EMM386) - but have you tried UNP from http://unp.bencastricum.nl/ - well known, works under dosbox and maybe able to unpack you exe

Yes, UNP does decompress executables, however, I was just curious as to why CUP386 crashes under dosbox. I tried turning off XMS, emms and umb, but CUP386 still falls over.

normal core?

Nope, looks like that option won't work either.

I don't even get any errors when trying to use the program.
In fact, merely typing "cup386" without any switches simply freezes the dosbox session and I had to ctrl-F9 to shut it down.

cup386.com is not for DOSBox. DON'T use it under DOSBox. Use it under pure DOS.

cup386 itself should runs under real mode, then it will enter 386 protected mode and starts unpacking.
DOSBox can not offer real mode.

That's why it freezes.

All DOS programs start in real mode. If DOSBox didn't support real mode it wouldn't be able to run anything.

jmarsh wrote on 2020-07-22, 04:00:

All DOS programs start in real mode. If DOSBox didn't support real mode it wouldn't be able to run anything.

It's not real "real mode", but v86 mode.

Privileged instructions / Sensitive instructions can run under real "real mode".

DOSBox doesn't support them.

DOSBox executes all DOS programs in real mode unless they explicitly enable V86 mode/protected mode. It's an emulator, it doesn't try to execute the privileged instructions on the host system.
How do you think it runs on non-x86 systems like arm or powerpc if it's using V86 mode?

jmarsh wrote on 2020-07-22, 06:23:

DOSBox executes all DOS programs in real mode unless they explicitly enable V86 mode/protected mode. It's an emulator, it doesn't try to execute the privileged instructions on the host system.
How do you think it runs on non-x86 systems like arm or powerpc if it's using V86 mode?

Excuse me, but do you really know what I am talking about?

A guy asks why cup386 doesn't run in DOSBox and I am trying to give the answer.

The point is cup386 is written before more than twenty years ago. It runs only on x86 system and uses x86 privileged instructions.

So I think my words are based on x86.

Yeah I know, because I wrote two emulation cores for DOSBox - one for x86_64, and another for PowerPC. So trying to tell me it can't run real mode programs is a waste of time.

Every DOS program starts in real mode.
When they switch to protected mode, they use privileged instructions to thunk back and forth to real mode to communicate with DOS and hardware.
How do you think any of that works for all the games DOSBox runs if DOSBox doesn't support it?

The reason that the program is not supported by DOSBox is that DOSBox is for running DOS games, not DOS programs. So there are a number of DOS calls not emulated by DOSBox because they are not used by DOS games.

whocrazy wrote on 2019-08-18, 08:45:

Hi. I'm trying to run a program cup386.com for unpacking old dos executables, but when I run it under dosbox, it freezes. Is the […]
Show full quote

Hi.
I'm trying to run a program cup386.com for unpacking old dos executables, but when I run it under dosbox, it freezes.
Is there a fix for this?
Thanks.

Yes, you can use cputype=386_prefetch

Hangs during unpacking is a mix of unimplemented DOS call and anti-debugging traps.
You will hit those anti-debugging traps in real DOS too, btw.

Dosbox works fine for most unpacking problems, I rarely go into real DOS at all.

Akuma

CUP386 calls DOS to execute the packed program and then monitors the unpacking process, so problems can arise from the packed program rather than the unpacker. I suppose that's not unusual for unpacker programs, but it's worth noting that what is being unpacked affects the result. Lightweight test: with an LZEXE-packed executable, I find that the /1 option seems to work okay in DOSBox 0.74 and SVN with default settings, but cputype=386_prefetch appears to be needed for the /3 option.

firstrose wrote on 2020-07-22, 10:35:

A guy asks why cup386 doesn't run in DOSBox and I am trying to give the answer.

It is better not to give any wrong answers if you do not know the stuffs. BTW, jmarsh does know his stuffs and I reaffirmed that your answer of "DOSBox cannot execute privileged instructions for x86 real-mode" is non-sense.

CUP386 belongs to the class of programs that heavily manipulates platform quirks and undocumented CPU/DOS behaviors similar to SoftICE for DOS. It is a universal unpacker that employs techniques of code tracing (with built-in debugger) to find the unpacked code/data of any unknown EXE packers. It is not uncommon if emulation may have failed to implement the behaviors for such class of programs to function properly.

dosbox has a lot of trouble with a lot of these old tools of this kind, softice, cup386, a lot of cryptors (gardian angel). sometimes normal core works ok. normal works better than dynamic for these. some old stuff you need prefetch, some dont. you need to experiment, but dont expect miracles getting a lot of these old packer/unpacker/scrambler tools running.

It usually helps a lot if you unpack them first 😉 and its faster too.