Baron von Riedesel wrote on 2023-02-11, 16:05:

Hello,

there might be another problem, this time related to emulated interrupts. While that works for protected-mode emulation, it seems that no interrupts are generated in V86-mode.
I attached a simple test prog that should display strings "playing part 0" or "playing part 1". It does so on real machines and in DosBox, but not in SBEMU. I don't have the newest version of SBEMU available, though, so you might have fixed that in the meantime?

I had no idea, I will test it in virtual box.
EDIT: tested in virtualbox with QEMM & JEMM, both works, there's infinite loop. I checked the log, IRQ fired and get an ACK. It MIGHT have problem on real machine, I'll test later.

BTW I was debugging doom with VirtualBox debugger. I find Doom will enable interrupt temporarily for a short time (maybe twice) in int08 handler, I don't think its OK, probably the code runs fast so another int08 doesn't happen, but when with SBEMU's port trapping, it delays so another int08 happened?

One evidence is that if I disable SBEMU's log (text output through COM port) in virtualbox, doom runs fine (didn't work in real machine).

I'll try to add a INIRQ flag so that STI is ignored by HDPMI32i.

EDIT: added some log to HDPMI, and confirmed that int08 are nested 7 times, and then transfer stack overflow. all the 7 handlers are doing DMA/SB io. I think the emulation speed is a big problem.
also tried add INIRQ, will freeze, 08~0F, 70~77 all added, I will try only 08.

crazii wrote on 2023-02-12, 10:17:

EDIT: added some log to HDPMI, and confirmed that int08 are nested 7 times, and then transfer stack overflow. all the 7 handlers are doing DMA/SB io. I think the emulation speed is a big problem.

Yes, that' strange. Usually an STI inside an interrupt routine is no problem because no EOI has been sent to the PIC yet. However, I noticed that SBEMU also enables the HDA controller interrupt. How often is that IRQ triggered? Because, if there are 2 different IRQs frequently triggered and both sending "unspecific" EOIs to the PIC - that might indeed be a problem.

Baron von Riedesel wrote on 2023-02-12, 14:45:

crazii wrote on 2023-02-12, 10:17:

EDIT: added some log to HDPMI, and confirmed that int08 are nested 7 times, and then transfer stack overflow. all the 7 handlers are doing DMA/SB io. I think the emulation speed is a big problem.

Yes, that' strange. Usually an STI inside an interrupt routine is no problem because no EOI has been sent to the PIC yet. However, I noticed that SBEMU also enables the HDA controller interrupt. How often is that IRQ triggered? Because, if there are 2 different IRQs frequently triggered and both sending "unspecific" EOIs to the PIC - that might indeed be a problem.

probably 5ms. there's 128 samples per interrupt, sample rate is 22050. sound card irq is 11 in virtualbox. I disabled the irq handler and leaves only port trapping. so it could be irrelevant.

I added INIRQ for int08, and skip call another int08 if INIRQ is set. still cause problems. and it seems the int08 of doom is extremely long, that's probably why it enable interrupts, and some thing might be wrong with the detection. as I previously stated, the 0FAh command, it doesn't finish.

crazii wrote on 2023-02-12, 10:17:

BTW I was debugging doom with VirtualBox debugger. I find Doom will enable interrupt temporarily for a short time (maybe twice) in int08 handler, I don't think its OK, probably the code runs fast so another int08

Interesting - so you'll soon become a debug expert.

I vaguely remember that one of the first things the Rational DOS extender does after switching to protected-mode is to change ALL protected-mode interrupts and exceptions to its own handlers. So I wonder how ( and when ) SBEMU's int08 handler is called. Could you verify that your int08 code had a chance to run? Maybe your code didn't run at all, and, since you ( or rather the MPXPLAY code ) modified the PIT timer frequency, the DOOM code stumbled upon its own feet.

Baron von Riedesel wrote on 2023-02-12, 16:52:

crazii wrote on 2023-02-12, 10:17:

BTW I was debugging doom with VirtualBox debugger. I find Doom will enable interrupt temporarily for a short time (maybe twice) in int08 handler, I don't think its OK, probably the code runs fast so another int08

Interesting - so you'll soon become a debug expert.

I vaguely remember that one of the first things the Rational DOS extender does after switching to protected-mode is to change ALL protected-mode interrupts and exceptions to its own handlers. So I wonder how ( and when ) SBEMU's int08 handler is called. Could you verify that your int08 code had a chance to run? Maybe your code didn't run at all, and, since you ( or rather the MPXPLAY code ) modified the PIT timer frequency, the DOOM code stumbled upon its own feet.

SBEMU doesn't have int08, it uses the irq of the sound card, triggered by the real sound card (irq 10 in virtualbox). I also modified MPXPlay so that it doesn't use int08 ( a flag defined by MPXPlay).
the 5ms interval is calculated that 22050 samples per second, divided by the buffer size (128 samples). If the sound card needs to keep up to 22.05kHz, it will trigger the irq at a approximately constant interval. BTW I disabled the sound card irq to test doom, only with port trapping. that means there's no interrupt happening in SBEMU, only SB/DMA io port emulation.

As I previously tested, the protected mode in08 is called with DOS/4GW, doesn't DOS/4GW chain the int calls? it should be chained (calls the old handler), or something will be definitely wrong.

crazii wrote on 2023-02-13, 07:27:

SBEMU doesn't have int08, it uses the irq of the sound card, triggered by the real sound card (irq 10 in virtualbox). I also modified MPXPlay so that it doesn't use int08 ( a flag defined by MPXPlay).

That's very good, I feared that Int 08 was still used and cannot be easily replaced by the true sound controller's IRQ.

As I previously tested, the protected mode in08 is called with DOS/4GW, doesn't DOS/4GW chain the int calls? it should be chained (calls the old handler), or something will be definitely wrong.

I just verified with a PL0 debugger that DOS4GW does indeed route the sound IRQ to SBEMU - no problem here. I even single-stepped thru SBEMU's IRQ code - there's no STI, and EOIs sent to the PIC are exactly as they should be.

Hello,

I made some progress.

First, I adjusted the sbemu source to the current hdpmi32i's trap API. So your variant isn't needed anymore.

Second, I implemented another approach to fix the doom "transfer stack overflow" problem. The idea is to simple hide the true soundcard ISR from the DOS4GW extender - so since it cannot hook into that ISR, the "transfer stack overflow" cannot appear in this context anymore.

The good news: I tested a few other dos4gw games ( Raptor, Shadowcaster ): sound works now for them.
The bad news: the DOOM freeze is not affected by this fix - so DOOM SFX still doesn't work.

And a sidenote: I have some technical problems with linking djgpp object modules with jwasm object modules - they aren't really compatible, forcing me to implement ugly workarounds. That's an old problem, still not fixed in the djgpp linker LD.

Did I miss something? Wasn't digital sound in DOOM fixed already?

Bondi wrote on 2023-02-14, 19:06:

Did I miss something? Wasn't digital sound in DOOM fixed already?

NO, not yet, it's just working on your machine, but not on mine, neither for Baron von Riedesel.

Baron von Riedesel wrote on 2023-02-14, 17:36:

First, I adjusted the sbemu source to the current hdpmi32i's trap API. So your variant isn't needed anymore.

I don't get this, what kind of variant? Can you help to clarify?

Baron von Riedesel wrote on 2023-02-14, 17:36:

Second, I implemented another approach to fix the doom "transfer stack overflow" problem. The idea is to simple hide the true soundcard ISR from the DOS4GW extender - so since it cannot hook into that ISR, the "transfer stack overflow" cannot appear in this context anymore.

You mean the real sound card's ISR? not IRQ5 or 7? I tried to add an INIRQ flag for int08, and skip the sound card ISR when already in INT08, the stack overflow gone but doom still freeze, some similar result?
I also found that the nested IRQ in HPDMI (when LPMS is in use) that doesn't handle IF. I tried added it like the _RTINT_ way, jump to ring0 first and set the original EFLAGS, but it only works in VirtualBox, on real machine the LPMS overflows.

Baron von Riedesel wrote on 2023-02-12, 14:45:

Yes, that' strange. Usually an STI inside an interrupt routine is no problem because no EOI has been sent to the PIC yet.

As I debugged, doom send EOI and then do other things. It seems a common technique: http://www.techhelpmanual.com/254-int_1ch__us … _interrupt.html

Most TSR popup programs prefer to intercept the lower-level INT 08H vector, call the original vector, then perform the timer-based operation after BIOS has finished its housekeeping chores.

crazii wrote on 2023-02-14, 23:34:

Baron von Riedesel wrote on 2023-02-14, 17:36:

First, I adjusted the sbemu source to the current hdpmi32i's trap API. So your variant isn't needed anymore.

I don't get this, what kind of variant? Can you help to clarify?

Yes. You cloned hdpmi while the port trapping API was still W.I.P. ( both design and implementation ). It had to be changed later, because not all possible cases were covered by the first design.

Bondi wrote on 2023-02-14, 19:06:

Did I miss something? Wasn't digital sound in DOOM fixed already?

If it works for you, it would be interesting to find out why. For me, all the "DOOM engine" games ( doom, doom2, heretic, hexen ) freeze on startup ( both on Intel and AMD machines with integrated HDA sound ). Please tell us the binary sizes and dates that work on your machine!

EDIT: thinking about it, it might be that on your machine the PVI bit in CR4 is activated as default.

I just tried and activated it on my machine and ... DOOM works now with SFX sound!!! Since the "transfer stack overflow" error is no longer happening, no further obstacles for a "real DOOM feeling" on modern machines.

Ok, the PVI bit doesn't exist on pre-Pentium machines, but those machines aren't the target for SBEMU, so no problem.

EDIT: attached the binaries for your own tests

Baron von Riedesel wrote on 2023-02-15, 05:23:

If it works for you, it would be interesting to find out why. For me, all the "DOOM engine" games ( doom, doom2, heretic, hexen […]
Show full quote

Bondi wrote on 2023-02-14, 19:06:

Did I miss something? Wasn't digital sound in DOOM fixed already?

If it works for you, it would be interesting to find out why. For me, all the "DOOM engine" games ( doom, doom2, heretic, hexen ) freeze on startup ( both on Intel and AMD machines with integrated HDA sound ). Please tell us the binary sizes and dates that work on your machine!

EDIT: thinking about it, it might be that on your machine the PVI bit in CR4 is activated as default.

I just tried and activated it on my machine and ... DOOM works now with SFX sound!!! Since the "transfer stack overflow" error is no longer happening, no further obstacles for a "real DOOM feeling" on modern machines.

Ok, the PVI bit doesn't exist on pre-Pentium machines, but those machines aren't the target for SBEMU, so no problem.

EDIT: attached the binaries for your own tests

I tested and it works on my machine!
but, I wonder why? enabling PVI only make CLI/STI nearly as NOP, and a little bit faster without entering ring0 exception handling. Is there any other difference? Am I understanding correctly? or didn't you do something extra, like handling VIF?
BTW I also tested the SETPVI binary with sbemu logs on, there's still a internal stack overflow in VirtualBox. the logs in VirtualBox will trigger a lot of IRQ4 and mess up the doom IRQ0.

crazii wrote on 2023-02-15, 07:30:

but, I wonder why?

Honestly, I also wonder why a bit 😀.

enabling PVI only make CLI/STI nearly as NOP, and a little bit faster without entering ring0 exception handling. Is there any other difference?

Not much, AFAIU.

or didn't you do something extra, like handling VIF?

No. Actually, what is hdpmi32 supposed to do if the client has disabled the (virtual) IF? It has no protected-mode drivers available that can handle the interrupts, and they cannot be reliably postphoned either.

BTW I also tested the SETPVI binary with sbemu logs on, there's still a internal stack overflow in VirtualBox. the logs in VirtualBox will trigger a lot of IRQ4 and mess up the doom IRQ0.

In its current state, SBEMU might just work "by chance", there's still the need to do more tests on real machines before the software will enter the "beta" stage.As for me, I avoid testing it on VBox or Qemu, these are not the target platforms for SBEMU.

I think it is working not by "chance". Thus you guarantee IRQs triggered while in VM86 will not interrupt the VM86 mode and will be postponed while CPU enters PM via a program interrupt or exception that do not include the PIC (hardware interrupt) and won't pass via DOS4GW's VM86=>PM IRQ callback. Here is a quote from DOS32A 's reference:

When you install a protected mode IRQ handler, the DOS Extender will automatically install a special real mode IRQ callback, which will send the IRQ occurred in the real mode to a protected mode interrupt handler. Unlike DOS/4GW which supports this technique only for the first eight IRQs (IRQs 0-7 = INTs 08-0Fh), DOS/32 Advanced will install real mode IRQ callbacks for all 16 hardware interrupts (IRQ 0-7, IRQ 8-15 = INT 08-0Fh, INT 70-77h).

The real mode call back referred above must be related to the VM86 mode too plus the overhead of several V86<=>PM switches some of them probably passing through VCPI in your EMM case? You are well aware that during these transitions interrupts are disabled/enabled additionally several times by the EMM?

I may be completely wrong in my guesses here though. Should look at the DOS32A sources again after over a year to figure out the chain of events....

I can now confirm that DOOM 1 and DOOM 2 work perfectly on my Pentium M laptop with FM sound and SFX in plain dos (HP NX7000 - i855PM, Radeon Mobility 9000/9200, AC97 AD1981B(L)).
Duke 3D also works but there is no volume control for SFX (switching SFX off works).

Baron von Riedesel wrote on 2023-02-15, 05:14:

crazii wrote on 2023-02-14, 23:34:

Baron von Riedesel wrote on 2023-02-14, 17:36:

First, I adjusted the sbemu source to the current hdpmi32i's trap API. So your variant isn't needed anymore.

I don't get this, what kind of variant? Can you help to clarify?

Yes. You cloned hdpmi while the port trapping API was still W.I.P. ( both design and implementation ). It had to be changed later, because not all possible cases were covered by the first design.

you mean the INS/OUTS string instructions?

georgel wrote on 2023-02-15, 08:48:

I think it is working not by "chance". Thus you guarantee IRQs triggered while in VM86 will not interrupt the VM86 mode and will […]
Show full quote

I think it is working not by "chance". Thus you guarantee IRQs triggered while in VM86 will not interrupt the VM86 mode and will be postponed while CPU enters PM via a program interrupt or exception that do not include the PIC (hardware interrupt) and won't pass via DOS4GW's VM86=>PM IRQ callback. Here is a quote from DOS32A 's reference:

When you install a protected mode IRQ handler, the DOS Extender will automatically install a special real mode IRQ callback, which will send the IRQ occurred in the real mode to a protected mode interrupt handler. Unlike DOS/4GW which supports this technique only for the first eight IRQs (IRQs 0-7 = INTs 08-0Fh), DOS/32 Advanced will install real mode IRQ callbacks for all 16 hardware interrupts (IRQ 0-7, IRQ 8-15 = INT 08-0Fh, INT 70-77h).

The real mode call back referred above must be related to the VM86 mode too plus the overhead of several V86<=>PM switches some of them probably passing through VCPI in your EMM case? You are well aware that during these transitions interrupts are disabled/enabled additionally several times by the EMM?

I may be completely wrong in my guesses here though. Should look at the DOS32A sources again after over a year to figure out the chain of events....

AFAIK PVI is only effective in protected mode, it's called Protected mode Virtual Interrupt anyway. the similar one VME is working in VM86. they are supported if CPUID has "Virtual 8086 mode extensions" feature.