high memory and extended memory are not directly accessible which would be a reason if the corresponding procedures were actually called by these benchmarks, but i think they are not; only explanation i see is that there is some code that is being run on interrupt?

Because of these discrepancies, I will need to run nearly a hundred 386 MB and CPU configurations all over again. What is the consensus on how DOS benchmarks should be run: with or without himem loaded? If with himem, then DOS = HIGH or not?

I wouldn't stress, as long as all your results are done the same way. You cannot control how someone else configures the system, so I wouldn't stress about this aspect. There are a lot of other factors that can influence the results in a machine someone uses to compare with your results.

For benchmarking I use menu option 8 in my DOS Starter Pack which uses:

DOS=HIGH,UMB FILES=30 BUFFERS=30 LASTDRIVE=H DEVICE=C:\DOS\HIMEM.SYS /TESTMEM:OFF […]
Show full quote

DOS=HIGH,UMB
FILES=30
BUFFERS=30
LASTDRIVE=H
DEVICE=C:\DOS\HIMEM.SYS /TESTMEM:OFF

It is not that simple - there are discrepancies from within my own tests. The reason for this is because I am testing several 386 chipsets with different Cyrix chips. Each have different L1 enable lines. To speed up booting, I would often hit F5 to bypass the startup files, then alter the cyrix L1 line in the autoexec.bat. Sometimes I would reboot, sometimes I would run the cyrix line from the command prompt and run the tests (before booting - before loading himem). So you see, I have no way of knowing which system and which test I bypassed the startup files, and which I did not. I cannot even compare my own results to my own results. They all need to be re-verified for consistency.

From your answer, it sounds like you are running DOS HIGH with HIMEM. That is usually what I have stuck to and will likely follow suit, unless someone else could make a claim to the contrary.

Oh I see, yes that is annoying.

I wonder if this issues scales with CPU speed, or if it gets insignificant.

I always run DOS=HIGH and with HIMEM.SYS. Why would you not want to, since you free up much more conventional memory for real mode DOS games and older DOS based applications?

Interestingly, what the OP described, I did experience with and without EMM386.EXE loaded.

It is infuriating. I was trying to find the fastest "386" CPU and motherboard configuration to setup in a case. I thought I was coming to a conclusion until I noticed this. What else can be irritating is that some boundaries of RAM are set as non-cacheable. If the program is running right in that small window, the results are much lower, so I normally set the whole range as cacheable when testing in DOS, even though for stability in Windows 3.11, the non-cacheable regions must be set.

Perhaps this is something related to the Cyrix based CPU's only?
A quick test with an AMD or Intel 386 CPU should perhaps confirm this?

jesolo wrote:

Perhaps this is something related to the Cyrix based CPU's only?
A quick test with an AMD or Intel 386 CPU should perhaps confirm this?

I'll have a quick look on a MMX 233 to see if there is any difference.

This is on a Pentium MMX 233.

3DBench 1.0c
Dos=HIGH, HIMEM loaded 182.3
REM DOS=HIGH, No HIMEM loaded 182.3

Chris's 3D Bench:
Dos=HIGH, HIMEM loaded 164.3
REM DOS=HIGH, No HIMEM loaded 164.4 <= HUGE lead

Doom:
Dos=HIGH, HIMEM loaded 686 ticks
REM DOS=HIGH, No HIMEM loaded 685 <= Lower is faster, HUGE lead

Check Landmark v2.

The issue may only reveal itself with these slower 386 CPUs, or perhaps only with clock-doubled CPUs on certain 386 chipsets? I haven't really narrowed it down yet, but I know the issue manifests itself when using a SXL2-50 on a VIA 481/495 chipsetted board.

I always do my benchmarks with F5 pressed - no devices/tsr loaded. It's pretty common that devices/tsr slow down the system a bit and if the benchmark doesn't require himem or anything else I don't use it. Of course it's more obvious on a slower system.

Yes, EMM386 is the devil. I've always had a DOS boot menu option to load without EMM386, or even without himem.sys.
Some software runs better without it. Some software doesn't even run at all when such stuff is loaded (eg early DOS extenders).

Out of curiosity, did you folks try different versions of himem.sys yet ? I know of at least three different main types of himem.sys.
a) the "80s" version, which switched between real/protected-mode (v2.00, link)
b) himem.sys from DOS 5/6, which has two code paths for 286/+386 (~v.2.77-3.10, supported loadall on 286, unreal mode on 386)
c) the version which comes with DOS 7.x and is said to allocate memory in reverse (v.3.95?, +386-only, might be interesting in respect of caches, cacheable area (some mobos could only cache up to 64MB), etc., link)

1 vote for pressing F5 to benchmark
3 votes for HIMEM and DOS HIGH

I do not load EMM386.
No, I have not tried earlier or later versions of HIMEM. I have always used HIMEM with DOS 6.22.

Ah, okay, it was just an idea. I also experienced what Scali said on my 386 (EMM386 really slowed down my machine).
I don't know much about all the different 486 models, but perhaps it shares the peculiarities of early intel models (no enhanced V86, etc.) ?
Or since this is a Cyrix chip, maybe it has other limitations (cache beeing flushed during DMA or other specific memory operations)..

That's a pretty statistically insignificant change, I wouldn't worry at all.

I never run EMM386 on my 386 DOS/Win3.11 setup. I use Enhanced 386 in Win 3.11, though.

It could have something to do with the Ti/Cyrix's L1 cache flushing scheme, which can vary by motherboard. I will attempt to isolate the problem when I get back to testing these. I've been swamped with other things recently.

SRQ: What is statistically insignificant? And ALU result from 108 to 211? That is almost 100%.

Could it be the A20 gate activating during the benchmark? HIMEM controls it when loaded and allows DOS=HIGH. 386 boards can be old enough to rely on the slow A20 gate on the keyboard controller.