crazii wrote on 2023-02-03, 08:50:

I read some comments in HDPMI source, saying those callbacks are local because DOS/4GW may not restore them on exit. I was trying to call IRQ5/7 manually to simulate the sound card IRQ, and only reading from IVT works.

vectors of IRQs are indeed handled a bit tricky. Please also see hdpmi.txt, cmdline option -i.

To be fully DPMI compatible, it's best to use DPMI functions 0x200/0x201 to get/set IRQ vectors. Else - if the IVT is read/written by accressing page 0 directly - it might be necessary to install HDPMI with option -i ( or set HDPMI environment variable HDPMI=1 ).

Baron von Riedesel wrote on 2023-02-03, 11:08:

crazii wrote on 2023-02-03, 08:50:

I read some comments in HDPMI source, saying those callbacks are local because DOS/4GW may not restore them on exit. I was trying to call IRQ5/7 manually to simulate the sound card IRQ, and only reading from IVT works.

vectors of IRQs are indeed handled a bit tricky. Please also see hdpmi.txt, cmdline option -i.

To be fully DPMI compatible, it's best to use DPMI functions 0x200/0x201 to get/set IRQ vectors. Else - if the IVT is read/written by accressing page 0 directly - it might be necessary to install HDPMI with option -i ( or set HDPMI environment variable HDPMI=1 ).

OK, I saw the -i option and code which trigger page fault when access page 0, tested it some days ago but seem not working? I'll try it again.
back to the popf question, I just read the POPF instruction reference and it seems only VM mode can trigger an exception. If such bug exists in doom/its DOS extender, so how does win9x handles it? I remember doom works good in win98se, does win98se provided a DPMI host with IOPL3?

crazii wrote on 2023-02-03, 11:14:

back to the popf question, I just read the POPF instruction reference and it seems only VM mode can trigger an exception. If such bug exists in doom/its DOS extender, so how does win9x handles it? I remember doom works good in win98se, does win98se provided a DPMI host with IOPL3?

AFAIR win98se always runs clients with IOPL0. However, I guesse it runs clients with bit CR4.PVi = 1 ( if it's a Pentium at least ). Since DOOM was so popular, it might even have applied some tricky patch to avoid the hang. Would surely be very interesting to investigate ...

Baron von Riedesel wrote on 2023-02-03, 11:27:

crazii wrote on 2023-02-03, 11:14:

back to the popf question, I just read the POPF instruction reference and it seems only VM mode can trigger an exception. If such bug exists in doom/its DOS extender, so how does win9x handles it? I remember doom works good in win98se, does win98se provided a DPMI host with IOPL3?

AFAIR win98se always runs clients with IOPL0. However, I guesse it runs clients with bit CR4.PVi = 1 ( if it's a Pentium at least ). Since DOOM was so popular, it might even have applied some tricky patch to avoid the hang. Would surely be very interesting to investigate ...

Very interesting, I'd like to know how win9x trick this one too. I actually tried enable PVI in cr4 in the STI/CLI handling of HDPMI, but not working.
BTW I just now tested HDPMI with -i option, still no luck when using dpmi api or "int 0F", the real mode game's IRQ 7 handler didn't get called. It's no a PM game, but a pure real mode game. I was calling it in a IRQ routine too. should be IRQ 10.

crazii wrote on 2023-02-03, 02:00:

rasz_pl wrote on 2023-02-02, 22:30:

crazii wrote on 2023-02-02, 13:57:

EMM386 doesn't support trapping port below 100h, which makes it unable to virtualize DMA & IRQ.

Wouldnt it be easier to patch EMM386 instead of writing new code? Its not like Microsoft is releasing new DOS making it troublesome to keep up with versions.

Ahh, patch off the port range limitations? I didn't thing of that, and I never patched a DOS binary. Is it easy, or can you do it?

EMM386 already has facilities to trap all the ports anyway (and uses them for Keyboard Data/cmd ports 60/64h for A20 changes, all the DMA channels for Virtual DMA Services (VDS), whole PIC), so the limitation is clearly arbitrary/artificial. Underpants Gnomes tell me sadly int 2f ax=4a15 routines werent part of DOS source leak (5.0?) so cant just look it up.

rasz_pl wrote on 2023-02-03, 11:38:

crazii wrote on 2023-02-03, 02:00:

rasz_pl wrote on 2023-02-02, 22:30:

Wouldnt it be easier to patch EMM386 instead of writing new code? Its not like Microsoft is releasing new DOS making it troublesome to keep up with versions.

Ahh, patch off the port range limitations? I didn't thing of that, and I never patched a DOS binary. Is it easy, or can you do it?

EMM386 already has facilities to trap all the ports anyway (and uses them for Keyboard Data/cmd ports 60/64h for A20 changes, all the DMA channels for Virtual DMA Services (VDS), whole PIC), so the limitation is clearly arbitrary/artificial. Underpants Gnomes tell me sadly int 2f ax=4a15 routines werent part of DOS source leak (5.0?) so cant just look it up.

OK, a pity. another thing really unfriendly to a DPMI client is that EMM386 calls the port trap handler function in ring0, it is not easy to recover the DPMI context, last time I built a new GDT and and load it in the handler, and disable paging to make a DPMI TSR to work. QEMM calls the trap handler in real mode, so I can easily utilize the RMCB of the DPMI host.

crazii wrote on 2023-02-03, 11:36:

BTW I just now tested HDPMI with -i option, still no luck when using dpmi api or "int 0F", the real mode game's IRQ 7 handler didn't get called. It's no a PM game, but a pure real mode game. I was calling it in a IRQ routine too. should be IRQ 10.

I think that's already a bit too off-topic here. Why don't you open an "issue" in github, supplying a test case? ( Preferably in ASM or C )

Baron von Riedesel wrote on 2023-02-03, 11:50:

crazii wrote on 2023-02-03, 11:36:

BTW I just now tested HDPMI with -i option, still no luck when using dpmi api or "int 0F", the real mode game's IRQ 7 handler didn't get called. It's no a PM game, but a pure real mode game. I was calling it in a IRQ routine too. should be IRQ 10.

I think that's already a bit too off-topic here. Why don't you open an "issue" in github, supplying a test case? ( Preferably in ASM or C )

Yeah, fair enough. It might not be easy to make a simple test case here, but I'll try it later.

crazii wrote on 2023-02-03, 02:05:

Cool, it's worth a try. But I remember reading some comments in VSB that it only works with QEMM, doesn't that means other emms is not capable, or VSB didn't bother to support other emms?

My guess would be that Andrew Zabolotny noticed the >=100h limitation in EMM386, and decided to just focus on just supporting QEMM's own QPI API instead. I do remember QEMM being a very popular EMM manager. I also used it myself, back in the day. I never used 386MAX, and it seems it was a lot less well-known, even then. Also, 386MAX supposedly didn't gain support for the INT 2fh I/O port trapping API until version 7.0.2, which perhaps wasn't released until after Andrew wrote VSB and TEMU.

We could email Andrew to ask this. I've emailed him before and he seems friendly and approachable. 🙂

rasz_pl wrote on 2023-02-03, 11:38:

EMM386 already has facilities to trap all the ports anyway (and uses them for Keyboard Data/cmd ports 60/64h for A20 changes, all the DMA channels for Virtual DMA Services (VDS), whole PIC), so the limitation is clearly arbitrary/artificial. Underpants Gnomes tell me sadly int 2f ax=4a15 routines werent part of DOS source leak (5.0?) so cant just look it up.

Unlike with Microsoft EMM386, the 386MAX sources are available these days, so I dug through it a little, and I found some promising clues in an include file called IOTRAP.INC:

1@IOTRAP_API equ  4A15h		; API signature

...And further down in that file:

1@IOTRAP_MINIO equ 0		; All I/O ports trapped must be >= this value

We really should try to figure out how to build this code, preferably with Open Watcom! 🙂

@crazii In the meantime, you could just try using the existing 386MAX binary distribution that you can find online, as long as you use 7.0.2 or higher.

This also seems to confirm that other existing drivers/emulators that leverage the EMM386 "I/O Trapping Extensions Spec", such as ADLiPT and SoftMPU, should work with later versions of 386MAX as well.

Baron von Riedesel wrote on 2023-02-03, 11:27:

crazii wrote on 2023-02-03, 11:14:

back to the popf question, I just read the POPF instruction reference and it seems only VM mode can trigger an exception. If such bug exists in doom/its DOS extender, so how does win9x handles it? I remember doom works good in win98se, does win98se provided a DPMI host with IOPL3?

AFAIR win98se always runs clients with IOPL0. However, I guesse it runs clients with bit CR4.PVi = 1 ( if it's a Pentium at least ). Since DOOM was so popular, it might even have applied some tricky patch to avoid the hang. Would surely be very interesting to investigate ...

I tried hacking PVI in HDPMI's CLI/STI handling, and enable/disable VIF, just without check the CPUID. but at least doom and duke3d works. 😁

digger wrote on 2023-02-03, 12:14:

My guess would be that Andrew Zabolotny noticed the >=100h limitation in EMM386, and decided to just focus on just supporting QE […]
Show full quote

crazii wrote on 2023-02-03, 02:05:

Cool, it's worth a try. But I remember reading some comments in VSB that it only works with QEMM, doesn't that means other emms is not capable, or VSB didn't bother to support other emms?

My guess would be that Andrew Zabolotny noticed the >=100h limitation in EMM386, and decided to just focus on just supporting QEMM's own QPI API instead. I do remember QEMM being a very popular EMM manager. I also used it myself, back in the day. I never used 386MAX, and it seems it was a lot less well-known, even then. Also, 386MAX supposedly didn't gain support for the INT 2fh I/O port trapping API until version 7.0.2, which perhaps wasn't released until after Andrew wrote VSB and TEMU.

We could email Andrew to ask this. I've emailed him before and he seems friendly and approachable. 🙂

rasz_pl wrote on 2023-02-03, 11:38:

EMM386 already has facilities to trap all the ports anyway (and uses them for Keyboard Data/cmd ports 60/64h for A20 changes, all the DMA channels for Virtual DMA Services (VDS), whole PIC), so the limitation is clearly arbitrary/artificial. Underpants Gnomes tell me sadly int 2f ax=4a15 routines werent part of DOS source leak (5.0?) so cant just look it up.

Unlike with Microsoft EMM386, the 386MAX sources are available these days, so I dug through it a little, and I found some promising clues in an include file called IOTRAP.INC:

Copy code to clipboard

1@IOTRAP_API equ  4A15h		; API signature

...And further down in that file:

Copy code to clipboard

1@IOTRAP_MINIO equ 0		; All I/O ports trapped must be >= this value

We really should try to figure out how to build this code, preferably with Open Watcom! 🙂

@crazii In the meantime, you could just try using the existing 386MAX binary distribution that you can find online, as long as you use 7.0.2 or higher.

This also seems to confirm that other existing drivers/emulators that leverage the EMM386 "I/O Trapping Extensions Spec", such as ADLiPT and SoftMPU, should work with later versions of 386MAX as well.

For now I think I'm sticking to QEMM to make the emulation stable first. 😁

BTW I've read the source of SoftMPU, it hacks EMM386 kernel's GDT to add a protected mode data segment/selector to use. I don't think that will work with other EMMs.

crazii wrote on 2023-02-03, 13:04:

I tried hacking PVI in HDPMI's CLI/STI handling, and enable/disable VIF, just without check the CPUID. but at least doom and duke3d works. 😁

Good! However, this is pretty hackish, because without further adjustments the client will then receive interrupts although it had "disabled" them. Probably you could as well replace the CLI "emulation" by a NOP to get the same result - it might work and it might as well cause a crash.

crazii wrote on 2023-02-03, 06:41:

Nah, I was too simply describing the method, what HDPMI actually does is modify the EFLAGS on the faulted stack, the flags will be updated when performing execution transition from DPMI kernel(ring0) to client (IRET).

That is exactly my point - when the program's execution resumes (at the instruction after the STI opcode), interrupts will be enabled and any pending interrupt will trigger an exception before the following instruction is executed. That's not how STI would function if it wasn't intercepted.

A specific example would be STI followed immediately by CLI, which should not allow any interrupts to be serviced.

jmarsh wrote on 2023-02-03, 14:50:

crazii wrote on 2023-02-03, 06:41:

Nah, I was too simply describing the method, what HDPMI actually does is modify the EFLAGS on the faulted stack, the flags will be updated when performing execution transition from DPMI kernel(ring0) to client (IRET).

That is exactly my point - when the program's execution resumes (at the instruction after the STI opcode), interrupts will be enabled and any pending interrupt will trigger an exception before the following instruction is executed. That's not how STI would function if it wasn't intercepted.

A specific example would be STI followed immediately by CLI, which should not allow any interrupts to be serviced.

OK, I didn't know that. Is there a interrupt window in between two instructions? I thought in that case interrupt can happen after STI is executed and IP point to the following STI. Do you have any references?

It's detailed in intel's x86 manuals for the STI instruction.

If IF = 0, maskable hardware interrupts remain inhibited on the instruction boundary following an execution of STI. (The delayed effect of this instruction is provided to allow interrupts to be enabled just before returning from a procedure or subroutine. For instance, if an STI instruction is followed by an RET instruction, the RET instruction is allowed to execute before external interrupts are recognized. No interrupts can be recognized if an execution of CLI immediately follow such an execution of STI.) The inhibition ends after delivery of another event (e.g., exception) or the execution of the next instruction.

jmarsh wrote on 2023-02-04, 03:37:

It's detailed in intel's x86 manuals for the STI instruction.

If IF = 0, maskable hardware interrupts remain inhibited on the instruction boundary following an execution of STI. (The delayed effect of this instruction is provided to allow interrupts to be enabled just before returning from a procedure or subroutine. For instance, if an STI instruction is followed by an RET instruction, the RET instruction is allowed to execute before external interrupts are recognized. No interrupts can be recognized if an execution of CLI immediately follow such an execution of STI.) The inhibition ends after delivery of another event (e.g., exception) or the execution of the next instruction.

Cool, that's useful information.

I've come to a bottleneck, some games just won't work. doom still won't with SFX enabled. kindda frustrating.🤣
I dont' have a big collection of games, so here's a test build.
Does anyone want test it?

SBEMU is a TSR with Sound blaster and OPL3 emulation for AC97 sound cards.

Supported boards/sound cards:
Intel ICH (only tested with an ICH4 machine, ICH1 tested in virtualbox)
VIA VT82C686 and VT8233 (only VT82C686 tested)
Intel High Definition Audio (tested in virtualbox only)

Requirement: HDPMI32i, contained in the zip; QEMM (optional, but no real mode support without QEMM); HIMEM.sys.

Tested working games:
real mode: prince of persia 1 & 2, skyroads
protected mode: jungle book, lionking, duke3d, warcraft2.
Notes: Some DOS/4GW games with Miles sound needs QEMM, it seems miles sound is called in real mode.
Only 8bit mono is tested, for duke3d's setup, set the right mode.

Setup & run:
run "hdpmi32i -r" and then sbemu.exe
For QEMM, modern system might freeze on default setup. in such case, boot safemode and modify config.sys, remove the "RAM" parameter of the "DEVICE=C:\QEMM\QEMM386.SYS RAM" line,
if you want disable EMS, add "FRAME=NONE" parameter

You may encounter:
sound cracks, pops, suddenly mute, low speed, system error, freeze, reboot.

crazii wrote on 2023-02-04, 05:47:

I dont' have a big collection of games, so here's a test build.
Does anyone want test it?

Well, I did a test, DOOM2, on a Ryzen 3600 with onboard HDA.
I was aware that there's a hang with SFX currently, so I enabled "Music" in setup only ( tried both "AdLib" and "SoundBlaster" ) - the game starts and runs, but no music.

What should be mentioned is that DOS4GW has a problem if too much extended memory is offered - so HDPMI32i has to be installed with the -x cmdline option.
As far as HDA is concerned: I tried lineout rear panel and headphone front panel. HDA tells if a "device" is attached, but the software may ignore that info - perhaps you might consider to add an option to let the user select the output device?

Before running hdpmi32i, I ran another sound app in DOS to ensure that there's no "mixer" problem, that is, volume set to 0 or something similiar

Baron von Riedesel wrote on 2023-02-04, 08:47:

Well, I did a test, DOOM2, on a Ryzen 3600 with onboard HDA. I was aware that there's a hang with SFX currently, so I enabled […]
Show full quote

crazii wrote on 2023-02-04, 05:47:

I dont' have a big collection of games, so here's a test build.
Does anyone want test it?

Well, I did a test, DOOM2, on a Ryzen 3600 with onboard HDA.
I was aware that there's a hang with SFX currently, so I enabled "Music" in setup only ( tried both "AdLib" and "SoundBlaster" ) - the game starts and runs, but no music.

What should be mentioned is that DOS4GW has a problem if too much extended memory is offered - so HDPMI32i has to be installed with the -x cmdline option.
As far as HDA is concerned: I tried lineout rear panel and headphone front panel. HDA tells if a "device" is attached, but the software may ignore that info - perhaps you might consider to add an option to let the user select the output device?

Before running hdpmi32i, I ran another sound app in DOS to ensure that there's no "mixer" problem, that is, volume set to 0 or something similiar

What does SBEMU output? does it said found HDA sound card?
I can try adding options for HDA but it may fail, I just port the code from MPXPlay and not clear about how do to that. I think I need a real machine to debug, they're quite different from the virtualbox emulation.

crazii wrote on 2023-02-04, 10:09:

What does SBEMU output? does it said found HDA sound card?

Yes, forgot to mention that.

crazii wrote on 2023-02-04, 10:09:

I can try adding options for HDA but it may fail, I just port the code from MPXPlay and not clear about how do to that. I think I need a real machine to debug, they're quite different from the virtualbox emulation.

That's true. Btw. what software synthesizer is included in your SBEMU binary?