I just tried all of the different video modes in Descent. The behavior is indeed exclusive to stock 320x200.
The readme hints at 320x240 using "ModeX page flipping to reduce "shearing"", whatever is meant by the latter (and it also hilariously calls the 320x100 mode "a bit" blocky, but that is a different story 🤣). Anyway, on Wikipedia it says this about Mode X:
Higher resolutions: up to 360x480 in 256 colours is possible
Double buffering and triple buffering for flicker free animation
Sm […]
Show full quote
Higher resolutions: up to 360x480 in 256 colours is possible
Double buffering and triple buffering for flicker free animation
Smooth hardware scrolling of the video display window
Graphics stored in 'off-screen' VRAM can quickly be moved around in VRAM using the VGA latches
Planar mode allows up to 4 adjoining pixels to be modified in one byte write operation, which is ideal for solid filling of objects such as polygons, rectangles, lines, etc.
Screen splitting, where one part of the display is taken from one area of memory and the other from a different area, which is ideal for status displays in games that utilise smooth hardware scrolling
Square pixels are possible where the height/width ratio matches the monitor's, as they usually do in the 320x240 mode.
I'm guessing the 640x400 and 640x480 VESA modes have similar functionality. To me this really hints at some issue with lack of double/triple buffering in plain VGA mode, however the question is why this is not apparent on other cards...
I also really wonder why this is only apparent in Descent. I just tried Dune 2 as a random other game running in 320x200 and there is no "tearing" at all there. Does it use 13h?