Thanks to the mod for splitting this out.
Ok, so firstly - Hexen and Doom/BOOM are not equal in how the renderer works, so while one could draw similarities, I would say this is an apples/oranges situation, and one could as much compare, say, Duke 3D (why not?) and Doom. While Hexen adapts quite easily to changes in screen buffer, I am not sure the same can be achieved easily with Doom/BOOM. Well, maybe it is, but with MBF at least that would be coding effort and not a trifle.
E.g. all Bernewfie does is really just scale the player weapon sprites and locate the spot to draw HUD elements, the rest is handled by the original Hexen code (well, there's also column renderer code imported from MBF which has been slightly extended, and automap drawing code but that does not change the situation much). Now, what happens in hires mode in MBF (and by extension in Tartar) is that screen buffer is increased by x2 horizontally and x2 vertically, and then once the view of the level through the player's "eyes" has been rendered, every pixel in HUD, menus , tally screens, intermissions and automap is simply drawn 4 times (i.e. scaled x2x2). With 1280x900 renderer buffer which is output to 1280x1024 VESA resolution in Tartar every such pixel is drawn 16 times (i.e. scaled x4x4). Of course there are Doom engine ports for DOS out there that support resolutions like 800x600, etc (e.g. Doom Legacy) - but this is just not the case for MBF.
So if you would like to run MBF (or Tartar) in an emulator, I would first suggest determining if you are after original Doom/BOOM picture (320x200) - which makes a lot of sense for retro gaming in my opinion - or slightly more elaborate picture (640x400) (or in case with Tartar an even more elaborate picture of 1280x900 renderer). That will be the base picture for the emulator software to scale to your output window or display screen size.
Secondly, there's the aspect ratio question. While I am (ahem) neutral about that topic, knowledgeable people have proven that original artistic vision for Doom agrees with 4:3 screen aspect ratio much better than with 16:10 - DoomWiki has a dissertation quality article on this. Since you are running with the emulator, I would suggest letting your output device do the scaling from the original 16:10 picture Doom renderer produces (similarly to how a CRT display would do). This is where shaders and scaling in RetroArch step in. While scaling output of the game to you window or monitor size, RetroArch will apply the aspect ratio you select and the shader-based solution will do the scale up much-much better in my opinion than DOSBox core built-in scalers. I'd leave scaler at 1x in the core (or switch to a core that does not try any of that on its own - e.g. *Pure) and avoid using built in aspect ratio correction in favor of allowing the GPU do that!
Finally, you mention missing pixels, but there are no missing pixels in MBF! 😀 All the pixels of the 640x400 screen buffer are there and switching to 640x480 just adds 80 rows of black pixels (40 on top and 40 at the bottom). Otherwise the picture will be exactly the same. If you are after eliminating black areas on the screen switching MBF to 640x480 will make things even worse but you still have, well, a few options:
1 - scale to full output window size/full screen ignoring aspect ratio; this is achieved with selecting Full as Aspect Ratio
2 - scale proportionally to 4:3 aspect ratio but show _fewer_ pixels of the original picture on the screen; this can be done with Custom aspect ratio
for example in my case of 1280x1024 display, I can have X position of -42, Y position of 0, Width of 1365 and Height of 1024
this fills my screen completely with properly scaled image with no black pixels (attached), but not all of the original picture is visible
3 - let it go and let them black pixels be; after all they don't really get in the way of sawing the demons
4 - switch to a source port which supports natively the screen resolution or output window size you are after.
DoomWiki and Youtube (e.g. Dwars's channel) has a lot of information on this, I am not even going to name the few 😀
GA-G41M-Combo G41/ICH7 - Core 2 Quad Q9550 - DDR3 1033 - Radeon RX570 - YMF744 (Cobra) - X3MB (Buran)
Beetle/M/i815+ICH2 - Celeron 566Mhz - Opti 924 (Typhoon Media)
