I was using a GeForce 4 Ti4200 for a couple of years now, and have recently switched to a GeForce FX 5900XT, so I thought I'd share some of my experiences.
The FX card provides slightly better image quality over VGA. This is most noticeable when playing older games which run at a fixed resolution of 640x480 on a 19" LCD monitor. In contrast, when using DVI and playing newer games which can run at the monitor's native resolution, both cards deliver the same image quality.
There are some issues with texel alignment on the FX card which don't occur on the GeForce 4. This results in corrupted text and other minor graphical glitches in certain games.
Using Anti Aliasing and Anisotropic Filtering has less of an impact on frame rate when using the FX card. Obviously, it is much faster than the GeForce 4, but even taking that into account, it seems to handle AA and AF in a more optimized way.
Some games like Blood 2 and Shogo don't run well with 45.23 drivers on the FX card, and need a newer driver version to display the menu screens correctly. There are no such issues with the GeForce 4 card when using the same 45.23 driver version.
TL;DR
You get slightly lower compatibility in exchange for better image quality and higher performance when upgrading from a GeForce 4 to an FX card.