Depends on whether you need to drive a digital display (DVI-HDMI converters are cheap).

Having a DVI out is not a requirement.
I use analog outputs with these kinds of retro PCs. I just try to balance the components of such a set.

I'd personally go for the FX5200, even with 64-bit bus. From what I've read, because these have DX9 support, they can run VooDoo emulation through nGlide... which might be something handy for certain late Win9x games that support it.

I'm actually in a similar boat and have a PII 400 MHz PC that I'm thinking to pair either with that (an FX5200... though will see if 64-bit vs. 128-bit makes a difference, as I have both), a Radeon 9200 SE, or a Riva TNT2 (Asus v3800). I've had the 9200 in there before and know that, despite being a weak card, it's not really hampering the performance too much (at least in the games I like to play) as the CPU itself also isn't that capable either. So I think it's a good balance for that system, despite being not-so-era-appropriate. I'm leaning towards the FX5200 for the re-built of that system, simply because I'm curious to see how nGlide will work with a few old games (namely, Need For Speed High Stakes.)

Back to your rig... and depending on the games you want to play, see if you can find a GeForce 4 MX 440 or 460 instead. Those would be a good match for that PIII and most DX7 and older games. In fact, these two cards have better DX7 performance than the FX5200.

This PC build is one of my few currently active PC assembly projects.
I haven't played games since yearly 2000's. Now, I want to revisit some old familiar titles.
I'm planning to load titles like DOOM, Quake, Blood 2, Unreal, Wolfenstein, Hexen II, Heretic, Diablo, Caesars III, AoEII, Starcraft, C&C, Toca Touring Cars, Colin McRae Rally.

You should consider Geforce FX 5600 and Geforce 4 MX 440.

I've played Colin McRae Rally 2 on my 9200 SE, and it ran pretty well, so CMR1 should be no sweat. CM3 is where it's too much for it (20-30 FPS @ 800x600 resolution.) A buddy of mine back in those days had an FX5200, and we always liked to compare whose PC would be worse that the other. In the end, though, I think his FX5200 came slightly on top, but ever so slightly. I don't remember if he had a 64-bit or 128-bit FX5200, but I think the former. In any case, both were pretty poor performers by the onset of the mid-2000's games. Anything up to 2001 or so was OK. Beyond that, the performance tanked hard.

On my 9200 SE, I played a lot of the Need For Speed games. Everything up to Porsche Unleashed / 2000 would run at 60 FPS @ 1024x768. Beyond that, NFS Hot Pursuit 2 was... not great (around 30ish FPS average.) Also played a lot of Half-Life Deathmatch and Counter-Strike 1.5/1.6, both of which ran at more or less stable 60/72 FPS @ 1024x768.

This is good info!
From my perspective, I want to achieve a balance and minimize the bottleneck between the CPU and the video card.
If a game demands better performance, so be it. I can load it on another PC with better specs. I have lined up a few other options - a PIII 1000/133/S370/Ti4200; a P4 3.2/S478/Ti4800SE; an Athlon 64/S754/FX5600.

If I can play up to CMR2 on this PC, that all I'm hoping for. I'm not planning to push anything newer on it.

Colin McRae Rally 2.0 runs fine on my PIII and FX5600. But then I installed Colin McRae Rally 2005 on my Athlon 64 and GeForce 7600GT...

Thanks! I got a crippled 5600XT earlier this week. I'll test it to see how it goes in this PC.

BTW I'm not going to dual boot my P4 and Athlon 64. I'll have just Windows 98 on them. I have a couple of Core 2 Quad CPUs mated with GTX 750 ti and Quadro 2000 for XP and games from the mid 2000s.