Ok, got an answer over at the OSSC forums. It seems there are advanced timing settings that one can use, but more importantly the latest firmware (0.81) already has a preset for VGA 400 line modes, which should fix this issue.

Did you ever figure it out? I am experiencing the same problem.

You have to create a new timing profile for the resolution at hand, and it won't hurt to have the latest firmware/gateware version either.

If you end up creating a profile yourself, here are the most common timing geometry parameters for the 80x25 text mode:

I wish we could go back and find the "clever" engineer that decided to have two modes with the exact same total line count and refresh but different pixel clocks and kick him in the dick

maxtherabbit wrote on 2025-01-16, 14:11:

I wish we could go back and find the "clever" engineer that decided to have two modes with the exact same total line count and refresh but different pixel clocks and kick him in the dick

There is a good separation 25MHz vs 28MHz between these two modes, so a device like OSSC with its ~10x faster 200 MHz Altera FPGA would definitely be able to have the Nyquist bandwidth many times over to automatically distinguish between the two.

It is still needing heuristics, you have to specifically go and try to see a difference in the active video and hopefully it has a segment where that difference can be reliably made...
But if the sync pulse was say 1 pixel shorter or longer, it would have made everyone's life a whole lot easier. One simple counter and that's all it takes to know what goes on...