I'm kind of having trouble with this as well.
Let's take for instance the game StarGunner.
I have a 1080p 27" monitor, and I launch my games through DBGL. Using just the options it provides I usually end up with a stretched out 16:9 image which fills the entire screen.
What I want to get is a 1080p scaled output image (with scanlines) that pillarboxes to the correct resolution.
The best I've been able to get is an image that doesn't quite scale up all the way. My GPU is set to scale by aspect in Nvidia control panel, and this is the config file dosbox references for the game (I added the fullresolution and aspect lines) scan2x or scan3x will only get it so big. Interestingly I can't start the game in fullscreen either or it just silent crashes to desktop. Seems to have something to do with the Apogee logo screen I think. It'll work fine if you manually switch it past that (or even in the middle of the animation)
So basically I want it to scale as large as possible, while keeping the aspect ratio correct.
I suspect this is what the OP wants too.