OK it is definitely the card, took an ATI Rage Pro and stuck it into the machine and the problem is gone.
So what could it be? Bad capacitors? Is it worth repairing? It's just a Vanta after all (I may even have another one lying around).

I have similar smears on a Matrox Millenium, though only at the top resolution of 1600x1200.

A general consensus is that the DAC is dying or just degraded severely enough, so high amount of sharp transitions like on the contrast B/W text makes it unable to keep up the steady levels, and it dips below the reference values. It gets the most ugly when a checkerboard pattern is shown, like when win95/98 shutdown dialog is shown, and background is covered with this 1-pix checkerboard pattern. The DAC is on the die itself, so there's nothing you can do really.

vstrakh wrote on 2025-05-14, 08:29:

The DAC is on the die itself, so there's nothing you can do really.

Could always try a voltmod, or a downclock, or both?

vstrakh wrote on 2025-05-14, 08:29:

A general consensus is that the DAC is dying or just degraded severely enough, so high amount of sharp transitions like on the contrast B/W text makes it unable to keep up the steady levels, and it dips below the reference values. It gets the most ugly when a checkerboard pattern is shown, like when win95/98 shutdown dialog is shown, and background is covered with this 1-pix checkerboard pattern. The DAC is on the die itself, so there's nothing you can do really.

Good info! I'll consider it a goner.
As luck may have it, as I mentioned, I found my other Vanta, and it's a 32MB model so a little bit better. It did act weird on first boot (gave horrible patterns) but now seems to be working fine.
If that fails I have other options of course. The joy of retro computing, I have an arsenal of video cards 😂