very nice work friend.

Congratulations - this is amazing stuff!

Hey trixter, while reading sources may be fun it is rather time consuming. Can you give a brief overview here or in your blog about the ideas/focusing behind the specific benchmark routines. You reference Dosbox as comparison metric and you know the cycle count of 1 per instruction is very different from real world CPUs.
Prepare for some critical review 😉.

If I load QEMM9 with Stealth on my 386 the MemEA goes down from 100 to 74 while Opcodes goes up from 66 to 71.
Opcodes seem to make sense, why does MemEA goes down?

I would also like to be able to choose a system myself in the live comparison for quick testing. (like with qemm / without qemm) So I don't have to save a result for each setting.

I would also like to see that it remembers the previous content of the input fields for the result entry, so that I do not have to enter every time most of the same data for the same system with slightly changed settings.

And some kind of quicksearch, find as you type or NC style with ALT+name for the result browser would be fine.

UIDF8DE6985,"homebuilt 386 system",91,163,66,224,100,66,AMD Am386DX,40,"R(C)1985-1991,American Megatrends Inc.,All Rights Reserved.,1346 Oakbrook Dr.,#120,GA-30093,USA.(404)-263-8181. (07/07/91, rev. 0)",19910707,F8DE,VGA,"VGA, Tseng ET 4000 with HiColor RAMDAC, DRAM, VESA, 1024kb Video","Am386DX-40/i387-33 ETEQ386-8 32MB/ET4000AX/GUSPnP/SB16+WB1/ELNK3","Michael Huth"

UIDF8DE8143,"homebuilt 386 Turbo Off",29,715,277,398,434,289,AMD Am386DX,10,"R(C)1985-1991,American Megatrends Inc.,All Rights Reserved.,1346 Oakbrook Dr.,#120,GA-30093,USA.(404)-263-8181. (07/07/91, rev. 0)",19910707,F8DE,VGA,"VGA, Tseng ET 4000 with HiColor RAMDAC, DRAM, VESA, 1024kb Video","Am386DX-40/i387/ETEQ386-8/32MB/GUSPnP/SB16+WB1/ELNK3","Michael Huth"

Do you need samples from 486DX2-50, 66, 80, 386SX-40?

elianda wrote:

Can you give a brief overview here or in your blog about the ideas/focusing behind the specific benchmark routines. You reference Dosbox as comparison metric and you know the cycle count of 1 per instruction is very different from real world CPUs.

The benchmark wasn't created specifically for DOSBox, but more for every emulator. This is why individual timings of code segments are recorded into the database, so that emulator authors who are striving for cycle-exactness have a reference. As to why there are different metrics, they were divided into various profiles that target what varies the most from system to system and CPU to CPU (memory speed, effective address calculation, etc.) Only the realtime benchmarking, where it compares the current system to what is in the database, was created with DOSBox specifically in mind (because I've always wanted some realtime feedback when I dial DOSBox cycles up and down in realtime).

TOPBENCH is a system benchmark, not a CPU benchmark (although 4 of the five tests are CPU-focused). The five metrics are:

Opcode Exercise: As the name implies, every opcode that doesn't have variable exexecution time is exercised. https://github.com/MobyGamer/TOPBENCH/blob/ma … er/_OPCODES.INC lists all of the opcodes that are NOT exercised and why.

3DGame opcode mix: An instruction mix with the same distribution as some 16-bit era 3D games. I highly recommend looking at https://github.com/MobyGamer/TOPBENCH/blob/ma … er/_3DGAMES.INC to learn how this was created and the rationale behind it.

Memory exercise: System RAM block read and write speeds are measured.

Video RAM write speed: Writes to Video adapter RAM are measured.

Effective address calculation: This varied from CPU to CPU and is one of the more commonly-overlooked areas in getting an emulator cycle-exact, so I felt it necessary to dedicate a section to this.

Prepare for some critical review 😀

That's fine, I'm ready for it. 😀 Just keep in mind that TOPBENCH was intentionally written targeting 16-bit real-mode code, so it is not the correct choice for some research, such as the recent "best 486" and "best 686" mega-research projects I've seen here (which were both mighty impressive!). TOPBENCH doesn't touch a single 32-bit register or operation, and that's deliberate, because if it did, it couldn't be used as a level comparison for 286s and lower.

elianda wrote:

If I load QEMM9 with Stealth on my 386 the MemEA goes down from 100 to 74 while Opcodes goes up from 66 to 71.
Opcodes seem to make sense, why does MemEA goes down?

I honestly don't know, because I would have expected both to go down since V86 management adds overhead. The benchmark is supposed to run with a clean boot, so I'll add a V86 check to the next release.

I would also like to be able to choose a system myself in the live comparison for quick testing. (like with qemm / without qemm) So I don't have to save a result for each setting.

Interesting; I'll add that to the to-do list to see how I can implement it. Also, try /i on the command-line which spits out the Score quickly and then exits, if you were looking for a tweak-test-tweak-test-etc. cycle. The Score is the synthetic number meant to quickly and directly compare systems (meaning, the system with the higher Score is faster).

I would also like to see that it remembers the previous content of the input fields for the result entry, so that I do not have to enter every time most of the same data for the same system with slightly changed settings.

Because I don't accept multiple entries per system (see below), I can't think of any field that would need to persist between entries other than Submitter. It's trivial to persist that in the same session so I'll add that to the to-do list.

And some kind of quicksearch, find as you type or NC style with ALT+name for the result browser would be fine.

ALT+letter should definitely be possible; I'll add that to the to-do list. I'll have it jump to the first system when in sorted-by-alpha mode, and jump to the next system that matches in sorted-by-speed mode.

As for your sample results, if you could post those exactly as they appear in DATABASE.INI (it's a text file), that would help me import them into the master database. I don't accept artifically slow results (like systems with turbo off) or results from emulators (must be real hardware), but otherwise I am definitely interested in more systems for the database 😀 Just because I don't accept them doesn't mean I discourage them though; by all means, experiment.

RacoonRider wrote:

Do you need samples from 486DX2-50, 66, 80, 386SX-40?

Yes, especially the -50, -80, and 386SX-40 (I thought they only made DX-40s)! You can either email your resulting DATABASE.INI file to trixter - at - oldskool.org, or PM it to me, or post it here.

Oh wow trixter! Sure you've heard it all too many times before but 8088 corruption is fracking awesome. And I've enjoyed reading oldskool.org too 😀

Never get tired of hearing it, thanks 😀

New version available for testing, with some changes as suggested by eliandia:

+ Recompiled stub so it has the new measurement wrapper

+ Added v86 check (and command-line to override)

+ Enhanced /i so that it also spits out usec timings

+ Added [letter] to jump to next system that starts with that letter in the browsing interface

+ Add command-line option to work with a different database file.

+ Added some small "special effects" if working on a color monitor (with command-line option to override)

Snag it at http://dosbenchmark.wordpress.com/downloads/

Nice!
Although it is disturbing to see my M919@150MHz on the same level as a generic DX2-66.
😳

Any other cacheless M919 owners want to check it out?
My score was 170.

MobyGamer wrote:

As for your sample results, if you could post those exactly as they appear in DATABASE.INI (it's a text file), that would help me import them into the master database. I don't accept artifically slow results (like systems with turbo off) or results from emulators (must be real hardware), but otherwise I am definitely interested in more systems for the database 😀 Just because I don't accept them doesn't mean I discourage them though; by all means, experiment.

ok here we go:

;AT clone 386 home built by Michael Huth
[UIDF8DE26E1]
MemoryTest=162
OpcodeTest=68
VidramTest=222
MemEATest=81
3DGameTest=64
Score=92
CPU=AMD Am386DX
CPUspeed=40 MHz
BIOSinfo=R(C)1985-1991,American Megatrends Inc.,All Rights Reserved.,1346 Oakbrook Dr.,#120,GA-30093,USA.(404)-263-8181. (07/07/91, rev. 0)
BIOSdate=19910707
BIOSCRC16=F8DE
VideoSystem=VGA
VideoAdapter=VGA, Tseng ET 4000 with HiColor RAMDAC, DRAM, VESA, 1024kb Video
Machine=AT clone 386 home built by Michael Huth
Description=386DX40/i387/32MB/GUSPnP/SB16+WB1/ELNK3
Submitter=Michael Huth

Why are the timings shown in the real time benchmark so different from the timings you get when a system is added to the database?

Some more results:

;PS/2 Model 80 386-16
[UID2709A18EA]
MemoryTest=537
OpcodeTest=174
VidramTest=1245
MemEATest=191
3DGameTest=173
Score=22
CPU=Intel 80386DX
CPUspeed=16 MHz
BIOSinfo= COPR. IBM 1981, 1987 (03/30/87, rev. 0)
BIOSdate=19870330
BIOSCRC16=2709
VideoSystem=VGA
VideoAdapter=VGA, unknown Chipset, 256kb Video Memory (BIOS)
Machine=PS/2 Model 80 386-16
Description=8580-071 BR55-7568723 /8514A/ELNK3/TC5046/MemExp6MB/Dual16550Serial
Submitter=Michael Huth

;AT clone / home built K6
[UID77C3D4FA1]
MemoryTest=10
OpcodeTest=6
VidramTest=119
MemEATest=4
3DGameTest=4
Score=374
CPU=AMD K6
CPUspeed=233 MHz
BIOSinfo=war,Copyright (C) 1984-98, Award Software, Inc. (04/30/98, rev. 0)
BIOSdate=19980430
BIOSCRC16=77C3
VideoSystem=VGA
VideoAdapter=VGA, unknown Chipset, VESA, 256kb Video Memory (BIOS)
Machine=AT clone / home built K6
Description=K6-233/GA586HX/3dfxBanshee/EWS64XL/GUS/SCB55/2940UW/PX-40TS
Submitter=Michael Huth

;AT clone home built P55C
[UIDCB37DC92B]
MemoryTest=25
OpcodeTest=8
VidramTest=82
MemEATest=4
3DGameTest=5
Score=403
CPU=Intel Pentium with MMX
CPUspeed=166 MHz
BIOSinfo=war,Copyright (C) 1984-98, Award Software, Inc. (10/21/98, rev. 0)
BIOSdate=19981021
BIOSCRC16=CB37
VideoSystem=VGA
VideoAdapter=VGA, VESA, 256kb Video Memory (BIOS)
Machine=AT clone home built P55C
Description=P166MMX/GA586HX/Riva128/MaxiSound64HS/GUS/DB60XG
Submitter=Michael Huth

Here are the results (for 486 - same hardware, different bus speed):

;386SX-40 basic build
[UIDEC3DA3D02]
MemoryTest=182
OpcodeTest=82
VidramTest=238
MemEATest=88
3DGameTest=71
Score=76
CPU=AMD Am386SX
CPUspeed=40 MHz
BIOSinfo=MR BIOS (r) Copyright (c) 1992, Microid Research Inc. (11/24/92, rev.0)
BIOSdate=19921124
BIOSCRC16=EC3D
VideoSystem=VGA
VideoAdapter=VGA, Trident TR8900C, VESA, 1024Kb Video Memory (BIOS)
Machine=386SX-40 basic build
Description=386SX-40, cacheless, 16Mb SIMM, generic noname mobo
Submitter=RacoonRider avermakov[at]list.ru

;486DX2-50 with VLB
[UID26053BB3D]
MemoryTest=119
OpcodeTest=48
VidramTest=161
MemEATest=60
3DGameTest=41
Score=146
CPU=Intel i486DX2
CPUspeed=50 MHz
BIOSinfo=(C)1985-1993,American Megatrends Inc.,All Rights Reserved,6145F Northbelt Parkway,GA-30071,USA.(404)-263-8181. (12/15/93, rev. 0)
BIOSdate=19931215
BIOSCRC16=2605
VideoSystem=VGA
VideoAdapter=VGA, Cirrus CL-GD5428, VESA, 256kb Video Memory (BIOS)
Machine=486DX-50 with VLB
Description=486DX2-80, 256Kb cache, 64Mb FPM, VESA local bus mobo
Submitter=RacoonRider avermakov[at]list.ru

;486DX2-80 with VLB
[UID260531125]
MemoryTest=75
OpcodeTest=30
VidramTest=124
MemEATest=37
3DGameTest=25
Score=210
CPU=Intel i486DX2
CPUspeed=80 MHz
BIOSinfo=(C)1985-1993,American Megatrends Inc.,All Rights Reserved,6145F Northbelt Parkway,GA-30071,USA.(404)-263-8181. (12/15/93, rev. 0)
BIOSdate=19931215
BIOSCRC16=2605
VideoSystem=VGA
VideoAdapter=VGA, Cirrus CL-GD5428, VESA, 256kb Video Memory (BIOS)
Machine=486DX-80 with VLB
Description=486DX2-80, 256Kb cache, 64Mb FPM, VESA local bus mobo
Submitter=RacoonRider avermakov[at]list.ru

And thank you for a great job, very cool software!

;P55C 233 at 266Mhz
[UID3AA2CDC43]
MemoryTest=15
OpcodeTest=6
VidramTest=110
MemEATest=2
3DGameTest=4
Score=368
CPU=Intel Pentium with MMX
CPUspeed=266 MHz
BIOSinfo=war,Copyright (C) 1984-95, Award Software, Inc. (07/08/98, rev. 0)
BIOSdate=19980708
BIOSCRC16=3AA2
VideoSystem=VGA
VideoAdapter=Matrox Mystique 220 2Mb + 3dfx Voodoo Graphics
Machine=P55C 233 at 266Mhz
Description=i430HX, 512Kb cache, 64Mb EDO, Mystique220, Voodoo
Submitter=RacoonRider avermakov[at]list.ru

It's curious that 166MMX scored more than 266MMX because of Riva TNT 😀

elianda wrote:

MobyGamer wrote:

Why are the timings shown in the real time benchmark so different from the timings you get when a system is added to the database?

Can you clarify what "so different" looks like? If the Score is off by 1, or the microsecond timings are off by single digits, that is normal interrupt noise. The benchmark deliberately does not disable interrupts or try to get the CPU/video/DRAM refresh/etc. in lockstep because 1. that's only safe for short timing durations, and 2. I didn't want to benchmark the system in a special case, only the system running in its "natural" state.

If the timings are WAY off, that shouldn't be happening because the code is the same, so shoot me an example.

RacoonRider wrote:

It's curious that 166MMX scored more than 266MMX because of Riva TNT 😀

Indeed, which is why video write speed is part of the benchmark 😀 It is a system benchmark, not just a CPU benchmark.

A similar interesting result is the IBM XT-286 vs. the original IBM AT. Both are 6MHz 286s, but because the XT-286 uses 0-wait-state RAM, it is 1.6x faster. In fact, the IBM XT-286 has nearly the same effective throughput as the IBM PS/2 55SX which is a 16MHz 386sx system 😀

Thanks to everyone for the results, I'll add them to the database and put out another update. Keep those results and suggestions (and critcisms, I have a relatively thick skin these days) coming!

Ah, so as it turns out the Mach64 was the reason for the abysmal score...

1;PC-Chips M919
2[UID252A16E447]
3MemoryTest=50
4OpcodeTest=35
5VidramTest=78
6MemEATest=55
73DGameTest=25
8Score=292
9CPU=AMD Enhanced Am486DX4
10CPUspeed=150 MHz
11BIOSinfo=(C)1985-1994,American Megatrends Inc.,All Rights Reserved,6145F Northbelt Parkway,GA-30071,USA.(404)-263-8181. (10/10/94, rev. 0)
12BIOSdate=19941010
13BIOSCRC16=252A
14VideoSystem=VGA
15VideoAdapter=VGA, S3 86c801/5 D, VESA, 1024kb Video Memory, 1024kb Video Memo
16Machine=PC-Chips M919
17Description=No L2 Cache
18Submitter=