I think root of the problem is trying to play on GTX 260 with 8-16x AA. This is causing your high memory usage. This GPU is good for 4x AA. For 16x AA you probably want GTX 580. In period correct reviews they tended to use 4x AA.

There are 4 versions of GTX 260, the best one is core 216 rev 2 (GT200B), those often come OCed with small difference to stock GTX 275.

I tested last night Quake 4 to see how GTX 285 can cope with high levels of AA. The pictures below tell the whole story. Without AA around 600-700 Mb of VRAM is utilized. With AA 2X there is practically no difference; at 4X AA there is a very small increase in VRAM usage; with AA 8X it jumps to 700-800 Mb of VRAM used up. There is a huge increase the moment I select 16X AA to 900-1000 Mb and sometimes even more; the moment GTX 285 starts to use system ram, even with the game installed on a SSD, it starts to stutter - very rare, but it does: the game hangs for a fraction of a second and then continues and so on. Because this would impacts the results and also because GTX 285 can not maintain a 60 FPS locked experience at these settings so we would get a video card limitation, I decided that the maximum AA level for all games is going to be 8X! Only games based on IDTECH 4 engine have the option for 16X AA so games like Doom 3 will be tested at only AA 8X.
Settings: 1280*1024, ultra quality, advanced settings all ON, AA 16X ,VSYNC off:

Did you test 1600x1200 with 8x AA? Or just 1280*1024? More pixels has priority over very high AA.

Just 1024*768 and 1280*1024. I will test at 1600*1200 when I put back my only 21 inch monitor but I do not think VRAM usage will go over 1024Mb even in Quake 4 or Company of heroes - another game that loves ram as it takes around 2.2gb in a skirmish game on a large map.
I tried GTX 260 and it will not fit on the AX8 because several capacitors are blocking the card from being fully inserted into the PCI-express slot. I also tried radeon 4890 from Sapphire, 9800 GTX+ from Galaxy, GTX 460 that I borrowed from a friend and they simply do not fit... I will try tonight Radeon 5870 from Powercolor to see if I can install the card.

Finally found 2 cards that fit! Gigabyte GTX 460 fits just fine but requires drivers from the 2XX series and Powercolor Radeon 5870 also fits and can run using catalyst 10.2, the last good XP driver from ATI.
Now GTX 460 should be performing lower than GTX 285 - which is actually bottlenecked by the Athlon 64 X2 4800 - while Radeon 5870 should perform better.
What should I use? GTX 460 which requires a newer driver with unknown or questionable compatibility and performs worse than previous card or the Radeon 5870 which perform better, has a driver that I know is good but requires a full Windows XP reinstall as I do not want any remains from nvidia drivers to influence the results?

You should be able to remove unused devices using DevManView and then uninstall the driver. It shouldn't be necessary to reinstall Windows XP.

From what you have Radeon 5870 is clearly the best choice. GTX 460 is more for s754. I know that 2XX series drivers have performance problems with GTX 4xx and Need for Speed Hot Pursuit 2. One of the last Windows XP drivers released by NVidia are needed for these cards.

Radeon 5870 seems to be about 25% faster than GTX 285.

It got reviewed here https://www.techpowerup.com/review/ati-radeon-hd-5870/ , although with a much faster CPU.

There is DDU version that works on WinXP ("18" is official claim, but last "17" version works better for me) - just FYI.

agent_x007 wrote on 2026-09-05, 07:47:

There is DDU version that works on WinXP ("18" is official claim, but last "17" version works better for me) - just FYI.

Thank you, I did try it!
Despite radeon 5870 physically fits into the pci-express 16X slot it is not actually compatible with AX8/K8T890. At startup I get 1 long beep and 3 short beeps and the debug display shows the postcode 25. The card is good as it is installed on this moment in my daily driver that has an Abit IX38QUAD GT board. It has nothing to do with the PCI-express version - radeon has 2.0 - as GTS 250 works just fine. I thought about replacing the caps but I would have to find some really small ones!
On the other hand GTX 460 works fine as long as I install an appropriate driver version. I started with 260.89.

More incompatibilities uncovered between AX8 and different video cards. As I do not have any other board based on VIA K8T890 chipset I do not know if the fault belongs to Abit or to the VIA chipset!
I have several radeon 4890 and some are the 1 GB version but 2 of them are the Saphire vapor X 2gb version. As long as I use a model with 1 gb the system starts just fine, the moment I put a 2gb version it hangs on startup with post code 25!
The system also hangs with a 1792mb GTX 260 despite working just fine with a 896mb GTX 260. It is clear that this particular board does not support video cards with more than 1gb of VRAM and radeons after the HD4000 series!
I happened to stumble upon a radeon 5770 and the board also gets stuck with that card.
Because the highest clocked Athlon you can put is FX-60 a radeon 4890 1GB is more than enough to eliminate the GPU bottleneck.

Let's put aside the video card problems and focus on the BIOS. It is very similar to the one on AV8. It has a few differences and some uncommon options that I will explain in a short manner.

Despite having 4 1GB dimms correctly "seen" by the BIOS Windows XP recognizes only 3GB and nothing more! Even if you lose 1gb is more than enough for any XP era game.

The monitoring sensors on this board are the same as on AV8; the DRAM contains all memory related settings - the burst length of 8 means more data is written or read with each cycle and that means more performance; LDT is actually hypertransport which was named lightning data transport early on and is 1000 MHz on socket 939.
Please note an automatic overclocking of the board by 2%; I have to set it manually to 200Mhz in the BIOS.

The V-link is the proprietary connection between the northbridge and the southbridge on VIA boards:
Mode 0 means an interconnect speed of 266MB/s;
Mode 1, 2 and 3 means a speed of 533MB/s;
Mode 4 means a speed of 1066MB/s and is the speed on AX8. K8T890 is the only VIA chipset with this speed, the AGP only K8T800 pro has 530MB/s.

And here is the system summary. As you can see GTX 460 runs without a problem on this board. I am still testing different drivers to see if any benchmark/game crashes or has some oddities .