Hi all! Sorry for the long text in my weak English (I'm almost sure that I use English prepositions and articles incorrectly).

I have developed and made an experimental prototype of a serial optical mouse specifically designed for use with DOS. The purpose of this development was to practically test its usability and, in general, to determine the right to exist of such a conceptual solution. Since this development is purely experimental in nature and contains an idea whose usefulness may be questioned, I decided to put it in a separate topic. My previous solutions can be found here - Homemade Serial Optical Mouse? It's Simple! .

A few words about the reasons that made me create such a mouse. For some of the devices I develop for my retro computers, I also have to develop simple programs that control these devices. The most convenient way to develop and debug programs is directly on the target retro computer in the DOS environment. For these purposes, I use a dual-monitor system (VGA and monochrome). In this case, using one or more DOS sessions of Windows, I run the development environment on one monitor (usually monochrome), and on the other monitor I display help information or texts with program examples or something else useful during the development process. There is a constant need to move back and forth through the text. I am so used to using mouse wheel scrolling in Windows that I was always irritated by the lack of such an option in DOS. Therefore, my initial goal was to be able to implement mouse wheel scrolling emulation in DOS.
One of the options is to use an IntelliMouse compatible mouse together with the CuteMouse driver and write a resident utility that emulates pressing the UP and DOWN keys when rotating the wheel. However, this will not work in DOS sessions of Windows, since in this case Windows uses its own mouse drivers. For this reason, I decided to develop a hardware solution that is independent of drivers and the OS used. During the development, I also added support for some popular old DOS games. Many DOS games access the keyboard controller directly, so using a resident utility to map buttons and a mouse wheel to keyboard keys becomes impossible, again a hardware solution is needed.
In my solution, support for DOS games is quite limited due to the complexity of my hardware implementation method. Adding something more is prevented by the lack of space on the mouse PCB. This limitation can be overcome by using a microcontroller, but this may be the next possible step, at this stage it was easier and faster for me to make a simplified hardware solution.

The experimental prototype I developed is a hybrid device. That is, a combination of two independent devices - the mouse itself and a keyboard simulator in the mouse body. The mouse is an ordinary serial optical two-button MS Mouse. The keyboard simulator is based on a controller board from a PC keyboard. The mouse is connected to both the COM port and the keyboard port. Since the keyboard port cannot be directly connected to two devices at the same time, and using a computer without a keyboard is practically pointless, it is necessary to use Keyboard Port Tripler ( Simple DIY Keyboard Port Tripler ) developed by me earlier for similar purposes.
A similar solution is possible for a mouse with any interface. I chose a mouse with a serial RS232 interface as the most universal, in order to cover the possibility of using it on any computer except XT. If you use a keyboard key controller with support for switching between AT and XT modes, the solution will be completely universal. I only have one keyboard with such controller and I didn't want to destroy it.
Due to the fact that the mouse I developed does not require any drivers other than standard system ones, it can be used not only with real retro computers, but also with PC emulators, the main condition is that the computer has a PS/2 keyboard port and a COM port. For many not very new computers, this condition is met.

Mouse block diagram

The mouse has three additional side buttons for the thumb. Two buttons can be used both in DOS and in games. These buttons imitate pressing ALT and CTRL. In the game settings, you can assign some actions to these keys, such as running or squatting or something else. When working in DOS and Windows, I found it convenient to use them for combinations such as CTRL-C, CTRL-V, ALT-F4, ALT-ENTER and some others.
The third button is the game mode button, used to support DOS games. By pressing and holding it, you can select four modes by rotating the wheel towards you. The selection occurs in a ring, i.e. after the fourth, the first again. Each mode is accompanied by yellow and red LED indication. The modes are reset to default DOS mode by rotating the wheel away from you while holding down the game mode button, allowing you to quickly return to DOS mode when you finish a game. Additionally, when you press and hold the game mode button, all game modes are paused and the normal mouse action (without scrolling) is temporarily activated. This can be useful in the game menu and in some other cases.
The first mode is the DOS/Game1 mode, i.e. a regular two-button mouse, but with the ability to scroll with the mouse wheel. This is the default mode and is active immediately after loading the mouse driver. It is important to note that the keyboard simulator starts working immediately after turning on the computer, even before the mouse driver is loaded. Scrolling in DOS applications such as various viewers (browsers) is implemented by simulating pressing the UP and DOWN keys when rotating the wheel. The default scrolling mode is not convenient for text editors, so an alternative scrolling mode such as PGUP and PGDOWN is provided. Switching between scrolling modes is done by pressing the wheel button. The alternative mode is indicated by a green LED. Scrolling with the wheel also works in Windows, although not as fully as native Windows scrolling, but in many cases it is sufficient.
In the first mode, but when used in games, you can switch to a sub-mode of the default scrolling mode, in which HOME/END are used instead of UP/DOWN. This sub-mode frees up the arrow keys on the keyboard for independent use in the game. The ability to use the alternative scrolling mode is preserved in the same form. Switching between the sub-mode and the default scrolling mode is done by pressing the wheel button while holding down the game mode button. The sub-mode is indicated by a blue LED.
In all other game modes, HOME/END are always used instead of UP/DOWN, regardless of whether the sub-mode or the default scrolling mode is selected. The ability to switch to the alternative scrolling mode is retained.
The second mode (Game2) locks the mouse movement along the Y axis. This mode is indicated by a yellow LED. The mode is intended for DOOM and games based on its engine.
The third mode (Game3) permanently holds the SHIFT key. This can be used in some games to enable permanent running if the game does not have a toggle key or for something else. This mode is indicated by a red LED.
The fourth mode (Game4) is a combination of the second and third. This mode is specially designed for Wolf3D. If you use SHIFT to enable side step, then left-right steps will be performed with the arrow keys on the keyboard and the game controls will be similar to control in modern games. This mode is indicated by yellow and red LEDs lit simultaneously.
The mouse wheel also finds its uses in games. It can be used, for example, to look up and down in games like HEXEN or ROTT. Additionally, some games, such as ROTT, allow you to use the alternative scrolling mode to change weapons or, for example, to change inventory in HEXEN.
Since the mouse is also intended for games, I implemented the ability to select an interface speed increased to 4800 baud using a jumper on the mouse board, which increases the number of reports per second from 40 to 160. Of course, in this case it is necessary to use modified mouse drivers or a COM port accelerator. Modified drivers can be found here - Re: USB to Serial Mouse using the Pi Pico! .

Internal mouse design

The OM02 optical sensor and mouse body are used from the A4tech SWOP-35 PS/2 mouse.
The keyboard key controller marked as PCB-00046-22E is from some PS/2 keyboard.
The Truedox RTS781 controller is taken from the DINOS (FCC ID: HQA2VTEAO-1) serial ball mouse. I could not find a datasheet for this controller, so the way to connect it was obtained by reverse engineering the mouse circuit. The purpose of some pins remained unclear.

Electrical diagram

Since I couldn't find a thin enough six-wire cable, I had to use a USB mouse cable and a transistor switch inside the DB9 connector housing to transmit the RTS signal along with the +5V supply. This is a forced solution and it is not mandatory. In order to reduce the number of components in the electrical circuit and save space on the PCB, I used a simplified method of connecting the serial mouse controller to the COM port. This method does not meet the requirements of the RS232 standard, but it works with IBM PC-compatible computers. Since the mouse is designed specifically for such computers, this is not a serious drawback.
Those who wish can use the part of the diagram that relates to the mouse to independently make the simplest version of a serial optical mouse powered, for example, from the keyboard port. In this case, you need to remove the jumper J2, short-circuit the diode SD1 and instead of the resistor R6, install a zener diode for a voltage of 4.3V with the anode to ground. You do not need to use the transistor switch, but instead you will only need a four-wire cable. Also, resistor R3 can be shorted.
Those who would like to independently develop and make a gamepad for old DOS games can use the Key Control Unit diagram as an example of connecting a keyboard key controller to get the ability to replace the mapping of the gamepad buttons to the keyboard keys on the fly. It should be noted that different controllers have different correspondence between the pin numbers and keys, and you will need to determine it yourself.

The electrical circuit and design of the mouse look quite cumbersome, but this is an inevitable consequence of the hardware implementation from ready-made units. I am not sure that anyone will dare to repeat it. Therefore, the optimal solution would be to use one of the currently available optical mouse sensors and a programmable microcontroller. Any five-button mouse can be used as the basis for the design. In this way, it is possible to develop a gaming mouse specifically designed for old DOS games. When using a microcontroller, it is possible to implement a completely software-redefinable assignment of buttons, including the main mouse buttons. You can get a mouse in the IntelliMouse standard for Windows and a two-button mouse with scrolling for DOS. You can make it so that the rotation of the wheel will sequentially cause pressing of keys 1, 2, 3, etc., which will allow you to change weapons in DOOM. You can implement multiple keystrokes or even sequential pressing of different keys when pressing a mouse button. You can use double-clicks of the side buttons and pressing the side button simultaneously with the main mouse button. Mouse movements can be interpreted as key presses, which may allow the mouse to be used in programs and games that do not support it. You can come up with a lot more. It is enough to implement only a small set of basic functions, and leave their specific content to the users. To do this, users need to be given a corresponding tool ¬– a utility for creating, saving and loading a mouse game profile into the mouse. Apparently, nothing prevents using the COM port to transfer packets with profile settings from the computer to the mouse.
Developing a microcontroller version of the mouse exceeds my current capabilities and will require the involvement of a specialist familiar with microcontroller programming, as well as investment in the project. However, anyone interested in creating such a mouse can develop it themselves. I would be glad if my idea is interesting and useful to someone.

To find out the relevance of a possible future development, I want to ask the Vogons members: Do you need such a mouse? If yes, then:
Q1. Do you need scrolling with the wheel in DOS?
Q2. Do you need support for DOS games? The opinion of retro gamers is especially important.
Q3. What functionality would you like from the mouse?
Q4. Is there support for such mouse functionality in popular PC emulators?
Q5. What demand do you think it could have among retro gamers? What demand among ordinary retro computer enthusiasts?
Q6. Considering that with the future production of the mouse in small series, and also because of the need to supply a Keyboard Port Tripler with it, the mouse cannot be cheap. What price would you be willing to pay for such a mouse?

However, there is an alternative and possibly cheaper way to get similar mouse functionality. This is the way to create a special version of the converter, similar to the one developed by rasteri in his HIDman project ( HIDman - USB to PS/2 converter (Open Source) ). In my opinion, it will be enough to modify the HIDman code, perhaps change the electrical circuit a little, and develop a utility for managing mouse game profiles. Since the characteristics of USB mice from different manufacturers can differ significantly, it may be necessary to have tools for their software unification. To quickly switch the assignment of mouse buttons and wheel functions, you can use the assignable keys on the system keyboard, and use the LEDs on the keyboard in blinking mode for indication. Such development is beyond my interests and can be carried out by interested persons.
If you dream a little, it would be nice if you could add the ability to connect a USB gamepad to such a converter. This would allow any combination of gamepad buttons to be mapped to any combination of keyboard keys, making it possible to use gamepads in old DOS games.

Thank you for your attention to my project. Please, if it is not too much trouble, express your opinion about it. Critical comments and possible suggestions for its improvement are especially welcome.

Best regards.

Darmok.