kixs wrote:

The other option is to give score 0 to non complete benchmarks and the formula for calculation of AVG should stay the same for all systems = (PCPBench+Doom+Quake)/3 in this case the non complete benchmarks wouldn't artificially increase their AVG score.

So far I think this is the fairest option. We can add a footnote to such systems name of these machines and explain the situation. We have a few 386 with FPU and they would stand out a little bit that way which isn't too bad as a few games do indeed benefit from it.

I don't think that applying one formula to all systems is the right way to go to get some information from the collected data.
To get still a relation between the systems a large overlap range can be used.
For the low speed systems the quake score is not important, even if there are two? dos games that can use a fpu. FPUs started to get really used in the Pentium era.
So for the early systems I would use 3dbench, pcplayer and doom until mid range P3 where the 3dbench score goes over 1K.
I also would not take the equal weighted average but some better choice, like 20%, 40%, 40% f.e.

For the faster systems 3dbench score evaluation has to be dropped and quake to be included as some fpu performance indicator. The question is then only how strong the fpu should be weighted. So for pcplayer 30%, Doom 30%, Quake 40% it would be 60:40 for Integer vs. fpu. This can be calculated for systems where the scores are available, so you have two scores starting from the 386 with FPU to the mid range P3s, where this score starts to make sense with early pentium class performance.

For extended analysis some relation between CPU frequency and performance would also be nice, but this may have to be some interactive chart with zoom capability. I could also imagine some functionality where you move the mouse to a certain point additional information as mainboard and graphics card get displayed.

I agree. But if you want to have a "universal" FPS score, then the same benchmarks should be used with the same formula.

I also see one problem. This benchmarks are not CPU, VGA or motherboard tests. But actually a system benchmark as a whole. So I would also like to see raw synthetic CPU (Drystones, MIPS), FPU (Whetstones), VGA data to eventually get the correlation between different subsystems. But I don't know of a "universal" benchmarks to be useful on the old and newer systems (ESSI, TopBench for CPU, FPU? VSPEED for VGA?).

Update to sorting the entries

After long and careful consideration the entries will, for the time being, sorted by PCPBENCH score as all machines, apart from 286 have them. 286 will be at the bottom of the pile and the 3DBENCH2 score will determine which 286 is faster 😀

I don't exactly agree that just sorting by PCPBench score is correct. I mean, take the AMD X5-P75 vs the DX4, here according the PCPBench score there are no difference, but the other scores are higher for the AMD CPU (which they should).

I think having a weighted system to get an average score is the way to go.

That score could be off, might have to do it again 😊

No solution is perfect and it's hard to please everyone. For the time being this is the sorting method, when we come up with something better we can always change it. What about a cumulative score?

PS: Check out the page "interactive chart".

Could you combine for the names in the interactive chart the CPU, Mainboard and Graphics card ?
The Y scaling doesn't look so well due to the huge peak at the end. So either you leave a linear scale and just cut the Core2 / i systems that they appear graphically at some maximum or change the Y scale to log.

Edit: One more thing - why do you use the same Y axis for all scores. It would be more convenient to give each a separate Y axis, maybe put two to the left and one to the right.

the interactive chart is actually quite cool - I like it!

elianda wrote:

Could you combine for the names in the interactive chart the CPU, Mainboard and Graphics card ?
The Y scaling doesn't look so well due to the huge peak at the end. So either you leave a linear scale and just cut the Core2 / i systems that they appear graphically at some maximum or change the Y scale to log.

Edit: One more thing - why do you use the same Y axis for all scores. It would be more convenient to give each a separate Y axis, maybe put two to the left and one to the right.

Ok I added a few more versions.

One with log, one with 2 Y axis, one without the "top dogs".

I think it looks great.

Perhaps adding separate charts like the 1.0 one for all tests would be a good idea.
It would be nice if both CPU and Video Card was listed in those 1.0 style charts.
That way it is easy to get a more detailed overview of the tests and hardware even of 3dbench2.

I tried a bit on the sheet with the log scale chart, combined some tooltip output, but now the tooltip labels disappeared and it shows just the data points in the tooltips.
Don't know how to get the Labels back. And it seems google docs is in this aspect rather limited. Reminds me somehow of programs from the early 90ies, where you have just 3 options. (And in google docs these 3 options seem to be always connected with some automatic 'adjustments'.)

A couple things -
- I see you've discovered you can't really change your test suite after you've created it and run it 100 times. If you change it again, there's certainly people who (because they're off doing something else) won't run it again. 😀
- I would've hired one of the master x86 hackers around here to see if they could fix the 3dbench score calculation problem by patching the EXE.
- Have you tried TOPBENCH? http://dosbenchmark.wordpress.com/
- Mau1wurf1977, "Jannuary" is spelled with one "N" - "January". That's been annoying me since I saw it first in one of your charts. 😀

New benchmarks it not a problem for me at this time.
At least all Video Cards tested with the 440BX system is easy to test again in very little time.
But I will not dig out other systems from closets and storage until we have settled on a "final" test suite.

Guys we are NOT adding more benchmarks. The ship has sailed so to speak...

In regards to CPU benchmarks like Sysinfo, Landmark and others. I find that they scale poorly (Sysinfo craps out quickly on faster CPUs) and / or have inconsistent results, especially with non-Intel CPUs. I think most of the benchmarks we have in the suite scale really well (I mean come on, from a 386 to an i7). 3Dbench 2 is unfortunate, however it is a very popular and precise benchmark with slower machines. Once we have more data it won't be hard to find a cut-off point, which at the moment seems to be at a P3-800, definitely not a slouch.

Also the majority of people coming to VOGONS are interesting in playing old games not AutoCad or Eagle.

We are making good progress, hopefully reaching 200 entries soon.

@ Stiletto Will fix January typo when I get to it 😀 New charts out by Monday.

@Elinda and Racoonrider You have "unstable" for Quake. Any chance you can fix this? Only 3 entries have this. Might just interpolate the results.

@ Elinda Did your scores on the AMD K6 improve after using SetK6D? I'm back to using an Intel MMX in my time-machine but this interests me.

@all Please avoid using , instead of . for decimal points

Please put OC results in the overclocked page. OC is overclocking of CPU clock or FSB (chipset overclocking).

@ Vulturius76 Please double check your CPU clocks and multiplier. The scores should be higher. Also did Intel have a 486DX2-80?

Mau1wurf1977 wrote:

I'm chasing VESAMTRR: http://www.mdgx.com/umb.htm#VMT

There is a link to a project. The files are all there but it seems they need to be compiled, which is beyond my skills 😒

Nope. As MDGX.com says, it's included in HXGUI.
http://www.japheth.de/HX.html#hxdl

Attached VESAMTRR v1.3 as included with HXGUI v2.17. While it is included, it appears Japheth no longer supports the utility. (It is in a folder called UNSUPP in hxgui217.zip.)

@Elinda and Racoonrider You have "unstable" for Quake. Any chance you can fix this? Only 3 entries have this. Might just interpolate the results.

@ Elinda Did your scores on the AMD K6 improve after using SetK6D? I'm back to using an Intel MMX in my time-machine but this interests me.

The system does run Quake in normal play mode stable, but the timedemo mode crashes after a few frames. So I could just leave this cell empty if you like as I can't get a score.

My K6-233 does not support MTRRs. I think the K6-2 300 MHz was the first with the "Chompers eXTended" core that first supports MTRRs.

Things have been slowing down a bit, but results keep coming in 😀

I have added some more results, a UMB 486 and IBM 6x86 as well as that OPTI chipset 386 I received today.

I will add a socket or platform category (386, 486) and use this to have more charts. Starting to bring out some detail from all the data.

elianda wrote:

@Elinda and Racoonrider You have "unstable" for Quake. Any chance you can fix this? Only 3 entries have this. Might just interpolate the results.

The system does run Quake in normal play mode stable, but the timedemo mode crashes after a few frames. So I could just leave this cell empty if you like as I can't get a score.

I have the exact same problem, crash after a few seconds. If I run 33MHz FSB, RAM starts to fail, I start getting error messages in POST from time to time. Perhaps I'll try it with different chips, probably smaller amont.

I once had an issue (overclocked K6) that the game would crash after a short moment. I tried like 20 times and then it would run through 😀

Seems quake is a good stability tester.

Not sure if I should ask this here or somewhere else. I have been fixing up a recent thrift store find and plan to add my results to this amazing project. The problem is I am not sure how to list my CPU as I have conflicting information. The computer is a Compaq Presario 2256, which is listed a few places online as having an AMD K6 300. The BIOS also says this as does the writing on the chip itself.

The confusion is NSSI and the CPU info from this bench identify it as a K6-2 300. Windows identifies it as a K6-3D 300. I looked up the family, model, and stepping number that NSSI reports (580) and it does infact come up as a K6-3D, which is supposedly an early name for the K6-2. Out of curiosity I did install the 3D Now patch for Quake 2 and it did raise my FPS, which again leads me to believe that it is in fact a K6-2. The only reason I doubt this is that the chip itself does not say K6-2 or K6-3D, just plain ole K6.

Truth be told I'm not sure how much it really matters, I just want to be accurate as possible when I post my results.