I've read code of Adlipt adnd SoftMPU, both ues the same approach and need EMM386 4.46+ to work. I use the same approach too, that's what I've learned from them.
The basic idea of IO virtualization in EMM386 is set IO Privilege Level (IOPL). For code execution, normal code with ring3 cannot execute ring0 code directly, that will cause general protection (#GP) fault. Similar rules applies to IOPL.
When there's no IO virtualization, the IOPL for a port is 3, which is accessible by normal program. When virtualization enabled, EMM386 will set the IOPL of the port to 0, which will cause *exception* when a IOPL3 program read or write the port. Then, in the *exception handler*, EMM386 will go to execute the virtualization handler that was installed. I've heard that NT's NTVDM uses the same approach to allow 16bit program to access devices. Maybe it's the same way for 9x?

If a program's IOPL is already 0, that means it have the access to all the port, and that will invalidate the virtualization. Guess Doom is running in IOPL0 and no IO virtualization happens. If, however, I can manage to hack doom's IOPL without patching the binary executable, things might wok. I really don't want to patch a binary file.

Found interesting stuff here: Help wanted: developing a Sound Blaster emulator for both older and modern hardare
and here: Re: EMM386 NOEMS Vs EMM386

VCPI provides a ring0 protected interface, most DPMI hosts use it.
If I can hook off VCPI and provide a ring3 DPMI, that might force doom into ring3 protected mode through DPMI. That's probably what win9x does. But... it is impractical 🤣

IIRC, the PCI YMF744 DOS drivers enable FM support even without EMM386, and it works with all games I've played on it, including Doom, Duke3D & Mortal Kombat. It only requires EMM386 to enable DMA (the DSDMA.EXE driver) for digital sounds, and that too works successfully with the above mentioned titles.

I'm just mentioning this to show that it is possible to achieve this port (& DMA) rerouting for protected-mode games. Though I can't help with the how, sorry 😅

Yamaha and ESS PCI products have a direct hardware 388 and 2x0 port support. So the FM is in hardware, not in software.

wbahnassi wrote on 2022-08-08, 00:43:

IIRC, the PCI YMF744 DOS drivers enable FM support even without EMM386, and it works with all games I've played on it, including Doom, Duke3D & Mortal Kombat. It only requires EMM386 to enable DMA (the DSDMA.EXE driver) for digital sounds, and that too works successfully with the above mentioned titles.

I'm just mentioning this to show that it is possible to achieve this port (& DMA) rerouting for protected-mode games. Though I can't help with the how, sorry 😅

If you have PC-PCI/SB-Link, the Yamaha cards work just like an ISA card and DSDMA is not needed.

The slightly less compatible mode is DDMA which also doesn't require DSDMA.

DSDMA is only for chipsets that don't at least support DDMA but you can use it on the chipsets that support DDMA IF you want to intentionally gimp compatibility.

crazii wrote on 2022-08-08, 00:10:

VCPI provides a ring0 protected interface, most DPMI hosts use it.
If I can hook off VCPI and provide a ring3 DPMI, that might force doom into ring3 protected mode through DPMI. That's probably what win9x does. But... it is impractical 🤣

Take a look at the DOSBox code. It supports passthrough to real OPL hardware and it also implements an OPL3 emulator so you might find something useful in there.

crazii wrote on 2022-08-08, 00:10:

VCPI provides a ring0 protected interface, most DPMI hosts use it.
If I can hook off VCPI and provide a ring3 DPMI, that might force doom into ring3 protected mode through DPMI. That's probably what win9x does. But... it is impractical 🤣

Actually, it's not that impractical and most likely the best way to go. Take an open source DPMI server (like CWSDPMI or HXDPMI), and add port trapping to it, then run the games with that DPMI server installed. You could also go one step further, and take an open-source DOS4GW replacement (like DOS32A), and add hardware virtualization into it. This is how DOS32AWE achieves NMI forwarding. The drawback on replacing DOS4GW instead of just providing a DPMI server is that some games rely on extensions or quirks of DOS4GW that are not compatible with DOS32A. Furthermore, it only works for games that use a DOS4GW-like DOS extender and executable format. Providing your own DPMI and using the built-in DOS extender of the games seems like the better way forward.

This has been something of a holy grail for me as well, these last few years. 😅

You might find this thread on GitHub particularly interesting.

Basically, the advice of some skilled retro DOS developers further down that thread comes down to the same thing that mkarcher recommended here above: taking an existing open-source DPMI host and implementing port-trapping in there.

Another approach, at least for more modern systems, is to go even lower than ring 0 and let a hardware-assisted hypervisor take care of it. That's the approach javispedro1 took with his VMusic project, which is an extension pack for VirtualBox that adds support for a number of retro sound devices, including OPL3.

My latest DOS retro programming hobby work takes yet another route: I'm trying to write an AIL/32 protected mode driver for more modern PCI-era sound devices, based on the open-sourced drivers by John Miles. I know, only a limited number of DOS games support that particular driver standard, but it would help me gain some more experience with low-level retro DOS sound device programming.

I'd definitely like to pick up work on an emulation-enabled HDMI host again, though. Would you like to team up? 🙂 I only have limited time, mostly in some of my weekends, to work on stuff like this, though.

Cyberdyne wrote on 2022-08-08, 01:49:

Yamaha and ESS PCI products have a direct hardware 388 and 2x0 port support. So the FM is in hardware, not in software.

This doesn't work on more modern PCI Express motherboards, though, even with a PCI-to-PCIe adapter, because the lack of support for "subtractive decode" prevents PCI cards from claiming any legacy I/O port addresses (anything below 1000h). On such systems, you'd still have to implement some kind of port-trapping to reroute port access from 388h to whatever higher I/O address was allocated to the card. Or alternatively, you'd have to patch each game to work specifically with the I/O address allocated to your ESS card.

Also, there is the practical matter of not everyone having access to those relatively few PCI sound cards that have native OPL3 compatibility.

cyclone3d wrote on 2022-08-08, 03:10:

crazii wrote on 2022-08-08, 00:10:

VCPI provides a ring0 protected interface, most DPMI hosts use it.
If I can hook off VCPI and provide a ring3 DPMI, that might force doom into ring3 protected mode through DPMI. That's probably what win9x does. But... it is impractical 🤣

Take a look at the DOSBox code. It supports passthrough to real OPL hardware and it also implements an OPL3 emulator so you might find something useful in there.

OK, actually the port IO part was learned from DOSBox and AIL2.0, to handle the OPL3 registers. DOSBox probably doesn't help much on the protected mode thing, it's totally emulated and can do what ever it want.

mkarcher wrote on 2022-08-08, 07:19:

crazii wrote on 2022-08-08, 00:10:

VCPI provides a ring0 protected interface, most DPMI hosts use it.
If I can hook off VCPI and provide a ring3 DPMI, that might force doom into ring3 protected mode through DPMI. That's probably what win9x does. But... it is impractical 🤣

Actually, it's not that impractical and most likely the best way to go. Take an open source DPMI server (like CWSDPMI or HXDPMI), and add port trapping to it, then run the games with that DPMI server installed. You could also go one step further, and take an open-source DOS4GW replacement (like DOS32A), and add hardware virtualization into it. This is how DOS32AWE achieves NMI forwarding. The drawback on replacing DOS4GW instead of just providing a DPMI server is that some games rely on extensions or quirks of DOS4GW that are not compatible with DOS32A. Furthermore, it only works for games that use a DOS4GW-like DOS extender and executable format. Providing your own DPMI and using the built-in DOS extender of the games seems like the better way forward.

Hmmm, Interesting. It's probably the most doable way, but I still see it as impractical, in the driver's view. 🤣 It involves too much external code. But I think I'm gonna try it. I think the protected mode IO trap is an option besides EMM386 for the sound driver, not essential. I don't want to add too much explicit dependency for the driver. (Although the modified DPMI host will be included in a binary bundle eventually)
So the proper way to do this might be:
1.add port trap support to exist open source DPMI , as an extension to the DPMI 1.0 spec
2.driver could try to check the extension, and use it, but won't fail without it, still we have real/v86 mode working. 😁

digger wrote on 2022-08-08, 21:22:

You might find this thread on GitHub particularly interesting.

This might be good that those things come all in one. Actually the RetroWave OPL3 is a USB Serial(CDC) device and in the beginning I was thinking to write a .SYS driver to support it as a serial device for DOS. But now I'm writing a USB stack for DOS, TSR mode, that utilizes DPMI, in a very tricky way (or maybe DPMI not really needed). It also enables the potential ability to support more devices, such as USB audio/midi stuff. But it involves too much work and currently the host controller drivers only implements USB1.1(OHCI), device class driver only implements CDC.

digger wrote on 2022-08-08, 21:22:

I'd definitely like to pick up work on an emulation-enabled HDMI host again, though. Would you like to team up? 🙂 I only have limited time, mostly in some of my weekends, to work on stuff like this, though.

I've used CWSDPMI and HDPMI in a VirtualBox VM, it seems that HDPMI is much faster and reside in himem with low memory footprint,. CWSDPMI is good(at least for me) in that it's written in C, more readable. 🤣
Teamup is good, I have little spare time too, mostly weekends, or some weekday evening.

crazii wrote on 2022-08-09, 06:28:

I've used CWSDPMI and HDPMI in a VirtualBox VM, it seems that HDPMI is much faster and reside in himem with low memory footprint,. CWSDPMI is good(at least for me) in that it's written in C, more readable. 🤣

Although I agree that something written in C would be much easier to comprehend, HDPMI was the one that stsp and japheth (Baron-von-Riedesel on GitHub) recommended in the other thread. And they have expressed an openness to cooperate w.r.t. extending it with a port trapping API, so we could count on their help too if we were to pick this option to hack on.

Teamup is good, I have little spare time too, mostly weekends, or some weekday evening.

Great! 🙂 Do you have a GitHub account too? Perhaps you can join that thread and chime in?

The last I remember doing, based on the info I got there, was to try to get HDPMI to build, as a starting point for this extension work. I couldn't (cross-)build it in Linux, but I created a project for a Docker image to allow DOS builds in GitHub Actions, so builds can be automated in a CI/CD pipeline, even if the builds have to be done in DOS, using DOS toolchains.

I'd still prefer to be able to cross-assemble the project from a Linux development environment, though. But let's not spend too much time on getting that going.

digger wrote on 2022-08-09, 07:49:

The last I remember doing, based on the info I got there, was to try to get HDPMI to build, as a starting point for this extension work. I couldn't (cross-)build it in Linux, but I created a project for a Docker image to allow DOS builds in GitHub Actions, so builds can be automated in a CI/CD pipeline, even if the builds have to be done in DOS, using DOS toolchains.

While I prefer a more "retro" way: coding in a VirtualBox VM, I have a DOS VHD configured in config.sys with 3 option: borland C++(3.1); open watcom; DJGPP. actually my code support the 3 compilers but unfortunately Vbox will stop when I use the watcom build, invoking EMM386 IO trap calls through DPMI. I tried CauseWay instead of DOS/4GW, the VM won't stop but will freeze when I ran a game after TSR installed. 🤣
Currently I'm working on the Borland C build, it's only working in pure realmode without EMM386. I'm trying to make it working in v86, and load it to himem.
Coding directly in vbox using Borland C or RHIDE is frustrating so actually use them when I need test, build & run. I use vscode out of the vmbox while doing massive coding, and copy the code to vm to build.
The VHD is convenient that it can be directly mount in windows (dunno if it is so in Linux) as a hard drive. It's less convenient than a shared folder but it's OK. I haven't succeeded setup a shared folder in DOS guest yet. Maybe its impossible and need a guest addition which doesn't exist.

I had a glance at the HDPMI's source and the JWasm tool chain, maybe will try it in the VirtualBox VM first. Actually I don't like VBox or docker, most of those VM things need kernel driver support and I got trust issue with them. 😁 PCem/86box/DOSBox is much better running in user mode.
BTW my github account is the same name as in here.

I've been using QEMU VMs for my retro DOS programming projects lately. It's easy to mount a host folder as a virtual ("VVFAT") drive in QEMU. There are also other ways to copy code and artifacts over to the VMs. I like to work from the command-line (and with automated shell scripts!) a lot, so this workflow works well for me, and I've found QEMU to be more flexible than VirtualBox in many ways. It also supports remote debugging, dumping memory regions (such as the video memory), etc.

The Docker image I created actually uses QEMU under the hood, but without hardware-assisted emulation, since it's not necessarily available in the standard GitHub CI/CD test runners. (Although I haven't really tested that yet.)

You might want to give QEMU a try, if you are still looking for a better alternative. But whatever works for you, of course. 🙂

digger wrote on 2022-08-09, 13:50:

I've been using QEMU VMs for my retro DOS programming projects lately. It's easy to mount a host folder as a virtual ("VVFAT") drive in QEMU. There are also other ways to copy code and artifacts over to the VMs. I like to work from the command-line (and with automated shell scripts!) a lot, so this workflow works well for me, and I've found QEMU to be more flexible than VirtualBox in many ways. It also supports remote debugging, dumping memory regions (such as the video memory), etc.

The Docker image I created actually uses QEMU under the hood, but without hardware-assisted emulation, since it's not necessarily available in the standard GitHub CI/CD test runners. (Although I haven't really tested that yet.)

You might want to give QEMU a try, if you are still looking for a better alternative. But whatever works for you, of course. 🙂

Cool, I'm now stilling working on the BC build, I'll try QEMU when that is done. 😁

digger wrote on 2022-08-09, 13:50:

You might want to give QEMU a try, if you are still looking for a better alternative. But whatever works for you, of course. 🙂

I tried QEMU today, the virtual fat of QEMU is very convenient, but I found the key press is frequently missing or doubled in windows host, simply typing "dir" will get "dr", "cls" will get "clss". I then I tried it on native linux, it's much better but sometimes the problem still happens. The arrow key press in a dos GUI program is always doubled. Here's a same problem I found in the forum: Qemu On Windows - DOS machine - Keyboard arrows problem (double press detected in some Filemanagers)
Guess more time needed to tune the configs to make best of if.

The BC3.1 build is done. It will enter 16bit protected mode directly from real mode or via VCPI, very much like the way of a DPMI host, but much more simplified. VCPI is provided by EMM386 so it is natural to use VCPI and port trap together. This will eliminate the dependency of DPMI.
After that I realized that the VCPI clients has a different kernel/memory space from its host (EMM386), only sharing 1M~4M memory with its host, and totally different privilege/interrupt setup and such. That's why EMM386 cannot trap a DPMI client even if the DPMI client's server is a VCPI client of EMM386 (complicated sh*t 🤣). they are in a parallel space.

For the IO trap of a DPMI host, IMHO ideally it would provide 3 interfaces:
1. compatible with EMM386's call, support of real mode handlers, that will make real mode program which's already trapping EMM386 very much easy to install a DPMI trap handler. but that might be difficult to be fully compatible in internal details, i.e. the SoftMPU will hack EMM386. if SoftMPU is to be supported easily in its own way, the DPMI host should make the internal details similar to EMM386.
2.for its own DPMI client to install IO trap.
3.for protected program which is not its client (very much like my BC build driver).
Although the internal implementation might be the same for those three.