I have encountered DEP problems in XP64 and Vista, but not with DOSBox. I can't recall if I have used dynamic core in either, though. I'll have to give it a try.

http://windowsvistablog.com/blogs/windowsvist … onvenience.aspx

Note that you can turn on DEP for all programs and services if you want. This is clearly a more secure state, but it could create some application compatibility issues. I certainly recommend that businesses test to see if they can use DEP for all programs and services. In some cases it might be possible; in others it won't be (yet). There’s that tradeoff again!

Internet Explorer was a particularly difficult case because we certainly wanted IE to benefit from the protection afforded by DEP. But prior to the Windows Vista release there were compatibility issues with several well known third-party IE add-ons, so by default we could not enable IE to run with DE. It turns out that there are two pieces of good news here. The first is that it is possible for dynamically generated ("jitted") code to be DEP-compatible -- it just takes a few lines of new code (and an upgrade to the new code). We expect most third parties to update their add-ons to support this. The second piece of good news is that Adobe, whose Acrobat and Flash Player add-ins were previously incompatible with DEP, has updated their software to be compatible with DEP. (Be sure to get these updates.)

So although it is not the default today, you can turn on DEP for IE for the additional protection. Michael Howard wrote a great blog post on how to enable DEP in Internet Explorer 7 on Windows Vista.

Have to try this on my work machine. Looks like DEP is only turned on for essential Windows Programs and Services by default and nothing else.

Nevertheless, you are right, malloc()ed memory is not guaranteed to work for things that the dynamic core does. On Linux (and other POSIX systems), you should mmap() with the right flags set. On windows it probably works differently. I bet that dosbox dynamic doesn't work on hardened linux kernels as well.

it won't compile there because of ebx I recall
DOSBox uses mprotect. This works at least for kernels that are aware of not running code from datapages (64 kernels ? and intel macs)

1#if (C_HAVE_MPROTECT)
2		if(mprotect(cache_code_link_blocks,sizeof(cache_code_link_blocks),PROT_WRITE|PROT_READ|PROT_EXEC))
3			LOG_MSG("Setting excute permission on cache code link blocks has failed");
4#endif

Don't know when mprotect is available.
Under windows maybe VirtualAlloc&co could be used.

mprotect _should_ work on all posixish platforms. In the case of hardened kernels, it doesn't always do, however. There are configurations where mprotect is not allowed to retroactively set PROT_EXEC. That one is only allowed in mmap-calls for these kernels.

Actually, I don't think this is much of a problem, as these patches are usually used in server environments. But it may still be useful to know if anyone ever rewrites that code.