Also the Radeons before the R300 chips have very flaky DVI. With the 7500 and 8500, I can't even get an image outside of Windows with my 1920x1200 monitor. Just a corrupted image during boot for a moment sometimes. Once the ATI Windows driver loads things seem ok.
In the NVidia camp this was ironed out by GeForce 3 (except some cheaper FX 5200 cards).
To elaborate on ATI drivers...
They lack support for some old D3D features like table fog and palletized 8-bit textures. These are "sort of" 3dfx related features that were part of D3D back then. Other vendors may have had native support or they did some emulation/translation. If they are missing it breaks some old D3D 3-5 games. Shadows of the Empire for example won't have any fog. With the original R100 Radeon there was a registry hack to unofficially enable fog table. ATI didn't care to officially support it. NVidia supported fog table until GeForce 7 and palettized textures to GeForce FX.
The OpenGL problems are another issue. ATI did not have quality OpenGL back then and developers just developed for NVIDIA who had the most marketshare and better drivers. So you have stuff like Bioware OpenGL games using NVidia extensions and just being poorly tested with ATI cards. KOTOR and KOTOR2, and Neverwinter Nights will give you some examples. Missing effects and crap performance. KOTOR and KOTOR2 each work best with one specific Catalyst release each. Catalyst 4.2 for KOTOR. I think KOTOR2 likes Catalyst 4.11. These get you all effects IIRC.
Quake based games usually run fine though. I'm sure ATI made sure those worked well because reviews always benchmarked them.