Solution:

Buy a 1600x1200 or 19200 x 1200 monitor.

It's the only way to get square pixels for 320 x 200 games.

That game however seems to have a higher resolution. If it's 640 x 480 then 1280 x 960 would be integer scaled, but such a monitor doesn't exist.

Opengl is softer which masks the pixel issue a bit.

The game is Heroes of Might and Magic, and it uses a 640x480 resolution. With output=surface you should get black borders all around, but it's the ONLY way you can get perfect pixels with a 640x480 resolution on your screen because you'd need at least 960 vertical pixels to scale 2x vertically. For example, if you have a 1920x1080 monitor, you could use a "scaler=normal2x forced" setting and you'd get a centered 1280x960 image with perfect 2x pixels.

Yes the game (Heroes of Might and Magic) is in 640x480.
My temporary solution for clear graphics is to launch dosbox in 1280x960 (double of 640x480) in windowed mode, but the window is greater than my screen.

I guess I'm gonna play in blurred mode.

Thanks for your answer.

If you have a Nvidia card, create a custom resolution of 1280 x 960 and set GPU scaling to "do not scale". Then play around with DOSBox to make it work.

I don't have a Nvidia card and I can't create custom resolutions.
I learned a bit more about scaling pixel sized images.
It just makes sense that I couldn't scale it between 1:1 and 2:1 without distording pixels.
Thank you for your answers !

If you have an AMD/ATI card you can get into the Catalyst Control Center and check Properties (Digital Flat Panel) and enable GPU scaling with Use centered timings so you get a centered and unstretched 1280x960 on your screen. Annoyingly though this is greyed out if you're at your monitor's native res, so you'd have to change your resolution first 😒