arncht wrote on 2025-05-03, 05:54:

CkRtech wrote on 2025-05-03, 02:49:

Do you guys have a consensus on hdd benchmarking apps for DOS? It seems like the needle has pointed in several directions over the years.

Re: DOS Zip - are those benchmarks timed manually, or is that a function of some sort of benchmarking software? Only ever used pkzip/unzip.

I saw the Descent zip, and that made me think of “real world” benchmarking as well. (Timing the press of Enter on executing Descent.exe and reaching Parallax/Title screen)

Of course not... that’s what synthetic benchmarks are for.

As Vogons is a community of people that share benchmark results in attempt to find the best way to pair hardware with BIOS, firmware, settings, and so forth, it is beneficial to have some short list of commonly accepted benchmarking methods. This seems to be much more common with benchmarking CPUs and VGA cards than it does HDD. However, it is also possible I haven't hit up enough threads to see what people run.

I also feel that real world benchmarking is always more useful than synthetic benchmarks, but I also recognize that organizing something like that for something as niche as benching old drives with their interfaces is perhaps a waste of time.

But hey - using dos zip was a great selection, imo. It seemed to provide some real results. You have me wanting to perform similar testing.

CkRtech wrote on 2025-05-03, 21:04:

As Vogons is a community of people that share benchmark results in attempt to find the best way to pair hardware with BIOS, firm […]
Show full quote

arncht wrote on 2025-05-03, 05:54:

CkRtech wrote on 2025-05-03, 02:49:

Do you guys have a consensus on hdd benchmarking apps for DOS? It seems like the needle has pointed in several directions over the years.

Re: DOS Zip - are those benchmarks timed manually, or is that a function of some sort of benchmarking software? Only ever used pkzip/unzip.

I saw the Descent zip, and that made me think of “real world” benchmarking as well. (Timing the press of Enter on executing Descent.exe and reaching Parallax/Title screen)

Of course not... that’s what synthetic benchmarks are for.

As Vogons is a community of people that share benchmark results in attempt to find the best way to pair hardware with BIOS, firmware, settings, and so forth, it is beneficial to have some short list of commonly accepted benchmarking methods. This seems to be much more common with benchmarking CPUs and VGA cards than it does HDD. However, it is also possible I haven't hit up enough threads to see what people run.

I also feel that real world benchmarking is always more useful than synthetic benchmarks, but I also recognize that organizing something like that for something as niche as benching old drives with their interfaces is perhaps a waste of time.

But hey - using dos zip was a great selection, imo. It seemed to provide some real results. You have me wanting to perform similar testing.

HDD benchmarks are always significantly more cumbersome than CPU or VGA tests. Ideally, they should be run on an empty drive — otherwise it’s very difficult to guarantee that the HDD performs the same mechanical movements across runs. Unless the testing is done very precisely, low-level benchmarks (e.g. sector-based reads) tend to yield more reliable results. Their downside, however, is that they can’t account for cache behavior, ignore OS-level effects, and write tests can be destructive.

That said, if you look at the results from cacheless controllers, they almost exactly mirror what I got in my Core3 test. My guess is that the more tests you run, the closer the averages will converge to that baseline.

Interestingly, I reinstalled my silver AC2700 yesterday (not an empty drive either), and on the section I tested, it completed the same tests almost twice as fast as the AC2540 — despite low-level measurements showing barely any difference between the two. And this difference is noticeable in real use. It suggests that on this drive, the cache controller likely contributes very little, and apart from small files, it may not even be worth enabling.

So ultimately, a lot depends on how and under what conditions the measurements are made. My impression is that these cache controllers were typically beneficial in the context of early 90s hardware — around 1993–1994. Taken out of that context and tested with more modern setups, they often lose their purpose or effectiveness.

arncht wrote on 2025-05-04, 05:42:

HDD benchmarks are always significantly more cumbersome than CPU or VGA tests. Ideally, they should be run on an empty drive — o […]
Show full quote

HDD benchmarks are always significantly more cumbersome than CPU or VGA tests. Ideally, they should be run on an empty drive — otherwise it’s very difficult to guarantee that the HDD performs the same mechanical movements across runs. Unless the testing is done very precisely, low-level benchmarks (e.g. sector-based reads) tend to yield more reliable results. Their downside, however, is that they can’t account for cache behavior, ignore OS-level effects, and write tests can be destructive.

That said, if you look at the results from cacheless controllers, they almost exactly mirror what I got in my Core3 test. My guess is that the more tests you run, the closer the averages will converge to that baseline.

Interestingly, I reinstalled my silver AC2700 yesterday (not an empty drive either), and on the section I tested, it completed the same tests almost twice as fast as the AC2540 — despite low-level measurements showing barely any difference between the two. And this difference is noticeable in real use. It suggests that on this drive, the cache controller likely contributes very little, and apart from small files, it may not even be worth enabling.

So ultimately, a lot depends on how and under what conditions the measurements are made. My impression is that these cache controllers were typically beneficial in the context of early 90s hardware — around 1993–1994. Taken out of that context and tested with more modern setups, they often lose their purpose or effectiveness.

I agree benchmarking the VLB caching controllers is tough. A 16MB cache on the controller distorts most DOS synthetic benchmarks. So does the memory mapped vs PIO transfers. Compressing large files is a good choice in tests, since that was something I/O intensive that you actually have to wait for under DOS and it is fairly repeatable. Another test I like is Windows startup times. It is I/O intensive, and if Windows starts faster, the system feels faster. There's a little more variance in that measurement though, from start up to start up and from system to system.

I am curious about what do you mean by "more modern setups". Are you talking faster CPU, more RAM ? A VLB controller that can do PIO4 ? Solid state storage? Or are you taking about a PCI system with an IDE controller that can do DMA ?

arncht wrote on 2025-05-03, 05:32:

douglar wrote on 2025-05-02, 19:44:

[*]Even though the WD Caviar 2540 is only PIO3, it should support multi-sector transfers. While most PATA solid state solutions report support for multi-sector transfer commands, very few actually support M.S.T. values > 1. M.S.T. should give the Caviar drive an edge on tasks that do sequential reads & writes, like file compression. Less chatter and less latency between sectors. Do you have it enabled in the BIOS or Drivers?
[/list]

Multiword transfers are enabled in the BIOS, but I never use DOS drivers — even though some later models could support DMA transfers, which would significantly boost performance.

Not sure if "Multiword transfers" is what I was talking about here. "Multi-sector transfers" is sometimes it is called "IDE Block Mode" or "Read/Write Multiple". So it isn't multiple words, it's multiple sectors. Causes the IDE device to stream multiple sectors to the host on a single request, reducing bus chatter, improving latency, which can be noticeable if your disk is able to saturate your PIO bandwidth. Int 13h isn't going to do that unless you enable it in the BIOS or you load the DOS device driver. It really only helps spinning disks though, since few solid state devices support M.S.T. > 1.

If you don't want to use the DOS drivers with the non-caching controllers, it might be worth while checking out Xtide Universal Bios with the 386l build when using Promise or QDI controllers. Every so often I would stumble across a combination where X.U.B. would out perform expectations quite a bit. As always, there was a lot of variation, depending on what storage device was used.

QDI controllers: Re: List of VLB IDE Controllers
Promise Controllers: Re: List of VLB IDE Controllers

douglar wrote on 2025-05-04, 14:08:

arncht wrote on 2025-05-04, 05:42:

HDD benchmarks are always significantly more cumbersome than CPU or VGA tests. Ideally, they should be run on an empty drive — o […]
Show full quote

HDD benchmarks are always significantly more cumbersome than CPU or VGA tests. Ideally, they should be run on an empty drive — otherwise it’s very difficult to guarantee that the HDD performs the same mechanical movements across runs. Unless the testing is done very precisely, low-level benchmarks (e.g. sector-based reads) tend to yield more reliable results. Their downside, however, is that they can’t account for cache behavior, ignore OS-level effects, and write tests can be destructive.

That said, if you look at the results from cacheless controllers, they almost exactly mirror what I got in my Core3 test. My guess is that the more tests you run, the closer the averages will converge to that baseline.

Interestingly, I reinstalled my silver AC2700 yesterday (not an empty drive either), and on the section I tested, it completed the same tests almost twice as fast as the AC2540 — despite low-level measurements showing barely any difference between the two. And this difference is noticeable in real use. It suggests that on this drive, the cache controller likely contributes very little, and apart from small files, it may not even be worth enabling.

So ultimately, a lot depends on how and under what conditions the measurements are made. My impression is that these cache controllers were typically beneficial in the context of early 90s hardware — around 1993–1994. Taken out of that context and tested with more modern setups, they often lose their purpose or effectiveness.

I agree benchmarking the VLB caching controllers is tough. A 16MB cache on the controller distorts most DOS synthetic benchmarks. So does the memory mapped vs PIO transfers. Compressing large files is a good choice in tests, since that was something I/O intensive that you actually have to wait for under DOS and it is fairly repeatable. Another test I like is Windows startup times. It is I/O intensive, and if Windows starts faster, the system feels faster. There's a little more variance in that measurement though, from start up to start up and from system to system.

I am curious about what do you mean by "more modern setups". Are you talking faster CPU, more RAM ? A VLB controller that can do PIO4 ? Solid state storage? Or are you taking about a PCI system with an IDE controller that can do DMA ?

For example, a more modern HDD, or maybe an SD card, or even newer controllers.

douglar wrote on 2025-05-04, 14:35:

Not sure if "Multiword transfers" is what I was talking about here. "Multi-sector transfers" is sometimes it is called "IDE Blo […]
Show full quote

arncht wrote on 2025-05-03, 05:32:

douglar wrote on 2025-05-02, 19:44:

[*]Even though the WD Caviar 2540 is only PIO3, it should support multi-sector transfers. While most PATA solid state solutions report support for multi-sector transfer commands, very few actually support M.S.T. values > 1. M.S.T. should give the Caviar drive an edge on tasks that do sequential reads & writes, like file compression. Less chatter and less latency between sectors. Do you have it enabled in the BIOS or Drivers?
[/list]

Multiword transfers are enabled in the BIOS, but I never use DOS drivers — even though some later models could support DMA transfers, which would significantly boost performance.

Not sure if "Multiword transfers" is what I was talking about here. "Multi-sector transfers" is sometimes it is called "IDE Block Mode" or "Read/Write Multiple". So it isn't multiple words, it's multiple sectors. Causes the IDE device to stream multiple sectors to the host on a single request, reducing bus chatter, improving latency, which can be noticeable if your disk is able to saturate your PIO bandwidth. Int 13h isn't going to do that unless you enable it in the BIOS or you load the DOS device driver. It really only helps spinning disks though, since few solid state devices support M.S.T. > 1.

If you don't want to use the DOS drivers with the non-caching controllers, it might be worth while checking out Xtide Universal Bios with the 386l build when using Promise or QDI controllers. Every so often I would stumble across a combination where X.U.B. would out perform expectations quite a bit. As always, there was a lot of variation, depending on what storage device was used.

QDI controllers: Re: List of VLB IDE Controllers
Promise Controllers: Re: List of VLB IDE Controllers

I meant IDE Block Mode, I just wrote it incorrectly.