I've been experimenting with a lot of different video cards to decide what I want to use in this build. My options are limited by having only PCI and PCIe, and I want to straddle the point where I have some reasonable 3D, but also good DOS/9x compatibility.
I have tried the following PCI cards:
ATI Rage XL
ATI Rage 128
ATI Radeon 7000
ATI Radeon 7200
NVIDIA GeForce 2 MX
NVIDIA GeForce 4 MX 420
NVIDIA GeForce FX 5200
3dfx Voodoo 4 4500
And the following PCIe cards:
ATI Radeon X300
NVIDIA GeForce PCX 5300
NVIDIA Quadro FX 1300
I have had nothing but problems with all of the NVIDIA cards in some form or another; even after extensive testing with drivers as far back as 7.7.6, all the way to 82.69. Often, I could find some combination that worked well enough to solved one problem, but then another would spring up elsewhere. My opinion is that, for Windows 9x, NVIDIA cards on AGP are excellent. But for PCI, they are pretty terrible. My PowerVR PCX2 had ghosting problems on all of these NVIDIA cards with Tomb Raider, and the Direct3D HAL was always broken. I did not have the same ghosting problem with MechWarrior 2, however.
I was always happy with the compatibility and performance of the ATI cards; even the Rage XL, despite its flaws. I've been a fan of the Rage 128 in AGP form for a while now, and it's still a solid pick on PCI. The Radeon 7000 series were "OK", but there wasn't as much of an improvement over the Rage 128 as expected. Either is a strong pick for Windows 9x, and the correct choice will depend on the software you want to run. For example, the Radeons have much better OpenGL support, but you may lose compatibility with newer divers.
The 3dfx Voodoo 4 is a very solid pick, which great legacy compatibility and support for Glide. I was disappointed in the overall performance compared to some of the other offerings; Direct3D is not as fast and OpenGL support is not as complete. A Voodoo 5 PCI might have offered a better experience. I think that Voodoo cards are best paired with lower spec processors, where 3dfx GPUs are often much stronger than the competition for this particular era.
PCIe support for Windows 9x was very spotty. NVIDIA drivers were absolutely terrible, and when things did work, performance was pitiful compared to the same card in Windows XP. Some games like Doom 3 could not render without some kind of glitchiness or corruption on any driver. The ATI Radeon X300 however, was an absolute joy to work with. Compatibly was great and performance was excellent.
I had a lot of fun trying out all of these different PCI cards on Windows 98 to see how they would perform when competely unrestricted by the CPU. My favorites are the Rage 128 for compatibly and overall features, and the Radeon X300 for sheer speed. A Voodoo 1 or 2 can also be paired for Glide, and I think that this provides a wider range of compatible games than a Voodoo 4. Unless you have an AGP slot, just avoid NVIDIA entirely. An NVIDIA Riva TNT2 PCI card may be an alternate to the ATI Rage 128, but I don't have one to test with.
So far I'm leaning towards a Rage 128 PCI for Windows 98 and a Radeon X800 PCIe for Windows XP. My choice of video card for Windows 98 must also have good compatibility with DOS, so there is still further testing to do.
I would like to revisit compatibility relative performance figures in the future with a Riva TNT2, Radeon 9250, and a Voodoo 5. I am particularly interested in the TNT2, but every showing by NVIDIA so far has been a great disappointment. I didn't even bother with a Riva 128, since its drivers are so terrible. I see no reason to ever choose a Riva 128 over a Rage 128 for Windows.
Games that I've been using for testing have included:
Duke Nukem 3D (DOS)
Tomb Raider 1 (DOS)
Quake 1 (DOS/Windows OpenGL)
MechWarrior 2 (for Windows)
Pirates of the Caribbean
The Elder Scrolls 3: Morrowind
Final Fantasy 7
Final Fantasy 8
X-COM UFO Defense
I don't know why, but X-COM UFO Defense seems to be a particularly difficult test to pass. I think it's invoking a very strange video mode or refresh rate that is not well supported.
Some surprises with this motherboard included none of my ESS sound cards (Solo, Allegro, Maestro) working with VxD drivers (WDM worked fine), OpenGL failing to work on my GeForce FX 5200 PCI in certain PCI slots, and severe incompatibility (blue screens/hard locks galore) with Yamaha YMF 724 and 744 PCI sound cards with VxD drivers (WDM drivers worked fine). I also experienced lockups with a MediaForte FM801. Luckily, I had no problems with my Aureal Vortex 3 and a Creative Audigy 2 ZS.
I also decided to remove the Promise Ultra133 TX2 ATA controller. While it provides excellent performance in DOS and Windows 9x, I plan to have this machine dual boot, and the performance in Windows XP is terrible. I have instead opted to use the onboard SATA in IDE compatibility mode, and this performs very well in both operating systems. I can still use the onboard IDE controller for a bootable CD-ROM and Redbook analog audio.