Let's work with Lords of the Realm 1 because now I have several folders of DOSBox installs testing things. I have opened another ticket with GOG for support and if something comes of that I'll update on that end.
I backed up what GOG installed, which appears to be mostly the original game files + a DOSBox .74-2 installation folder. Nothing too fancy and I looked again to make sure no mouse.exe or any mouse altering program already existed in these directories. Nope.
I did some thinking and confirmed again by comparing in the game window vs. in windows, carefully moving the mouse with the same motion, the Y direction is 50% of the expected motion. If you move from the bottom left corner on a diagonal, you'll end up at the middle right edge of the game window when you feel as though you should be top right.
I downloaded DOSBox .74-3 and setup another environment for DOSBox fresh. I ran the reset options command within. I renamed all the DOSBox config files in the main game's directory and dragged LORDS.exe over to the newly installed DOSBox's directory. I noted settings as it loads are default (3000 cycles instead of max) and the application says the newer versionn. It also won't launch the other DOSBox without those configs setup, so I'm confident it's running both the new version and a default config file. Unfortunately, I am still getting the same mouse behaviour.
I took a look at cutemouse and extracted that within the game directory and mounted the game directory. There are no instructions, but I went on the assumption you run one of the \bin files (presumably the English one), and it said PS/2 driver installed. I ran this with the freshly installed DOSBox .74-3 with default config. Ran the game after running that through the DOSBox command prompt and the mouse appears to have changed sensitivity (more sensitive) but still behaves the same. I think here I am missing a command/config to slow down X or increase Y.
I had a weird hunch that maybe this has to do with unusual registry settings regarding the mouse. I checked the mouse thresholds in HKEY_CURRENT_USER\Control Panel\Mouse . Learned one is related to 'precision' mouse control. Tried it both on and off and searched through those registry keys for anything unusual as well and I do not see it. I'm just beginning to think this is an interaction of how mouse movement is being translated and it's not specific to any game.
I had the thought to try Wing Commander (also setup via GOG) which 'seems' to track better, but the mouse's sensitivity in the game itself (it's set to 100 in the config) is so tiny in the menus that it is difficult to tell if it's behaviour is correct or not. The game is largely setup for a gamepad / joystick I feel and that was a poor example.
I'm now curious to attempt to run this LOTR1 on another machine when I get another chance to work on this to see if the X/Y tracking is likewise off on that to isolate that it's not a condition being imposed by this operating system's relationship to DOSBox.