georgel wrote on 2023-02-16, 10:31:

If these sound cards can adjust their resources to ISA sound card addresses without leaving/relying after initialization on memory resident utilities then they are fine for the proposed test. Just make sure after their init there are no TSRs left in DOS memory or EMM/XMM handles allocated.

It can be tested, I'll confirm it later. I think it will work ISA cards, since real IO could be faster than trapped emulation. for PCI cards I it need test.

Baron von Riedesel wrote on 2023-02-16, 11:03:

crazii wrote on 2023-02-16, 09:38:

Incompatible how? Is there any workaround?
I believe it is not the reason causing doom freeze, at least not the only reason. because I've tested without SBEMU's IRQ handling - a version that did nothing but emulate the port io. and doom still freeze, or there's stack overflow. You can test it too, just change PM_ISR = TRUE and comment out the next line of PIC_UmaskIRQ, then there should be no interrupts handling in DJGPP, yet doom still freeze.

The "incompatibility" I meant is not related to the "freeze", but to the "transfer stack overflow". It's caused by a PM-RM-PM-RM-PM-...-switch cascade, apparently because djgpp's and dos4gw's assumptions about some details of IRQ routing differ. At least that's my understanding and that's why I implemented some asm code to hide the HW sound IRQ from dos4gw.

The "freeze" is definitely a dos4gw bug. Might be interesting to try DOOM in a Win31 DOS box on a real machine. Win9x in enhanced mode was released before any PVI flag was invented, so according to that theory DOOM in a Win31 DOS box should also "freeze"...

OK, I believe djgpp doesn't do anything extra, it is not even a dos extender itself, nor it have one; all its __dpmi_xx APIs are all simple wrapper of INT31h, nothing more. only its go32 API have extra codes.
I get a transfer stack overflow on int 31h when I disabled interrupt handling from djgpp(SBEMU).
and after trapping CLI codes, DOOM2 works on my machine, DOOM is still the same.

I don't I have a pre-pentium platform without PVI, or I will test it to see.

Baron von Riedesel wrote on 2023-02-16, 11:03:

The "freeze" is definitely a dos4gw bug. Might be interesting to try DOOM in a Win31 DOS box on a real machine. Win9x in enhanced mode was released before any PVI flag was invented, so according to that theory DOOM in a Win31 DOS box should also "freeze"...

I doubt it. The DOS extended DOOM worked under win98 dos box: https://www.gamers.org/dhs/helpdocs/windoom.html

crazii wrote on 2023-02-16, 11:30:

I don't I have a pre-pentium platform without PVI, or I will test it to see.

You don't need one, just don't enable the PVI. As a reminder that you can set the CPUID return values in virtual box or vmware.

georgel wrote on 2023-02-16, 11:38:

Baron von Riedesel wrote on 2023-02-16, 11:03:

The "freeze" is definitely a dos4gw bug. Might be interesting to try DOOM in a Win31 DOS box on a real machine. Win9x in enhanced mode was released before any PVI flag was invented, so according to that theory DOOM in a Win31 DOS box should also "freeze"...

I doubt it. The DOS extended DOOM worked under win98 dos box: https://www.gamers.org/dhs/helpdocs/windoom.html

I was definitely talking about Win31 - the Win9x was obviously a typo ...

Baron von Riedesel wrote on 2023-02-16, 11:53:

georgel wrote on 2023-02-16, 11:38:

Baron von Riedesel wrote on 2023-02-16, 11:03:

The "freeze" is definitely a dos4gw bug. Might be interesting to try DOOM in a Win31 DOS box on a real machine. Win9x in enhanced mode was released before any PVI flag was invented, so according to that theory DOOM in a Win31 DOS box should also "freeze"...

I doubt it. The DOS extended DOOM worked under win98 dos box: https://www.gamers.org/dhs/helpdocs/windoom.html

I was definitely talking about Win31 - the Win9x was obviously a typo ...

But the link is about Win 3.1.

crazii wrote on 2023-02-16, 11:22:

georgel wrote on 2023-02-16, 10:31:

If these sound cards can adjust their resources to ISA sound card addresses without leaving/relying after initialization on memory resident utilities then they are fine for the proposed test. Just make sure after their init there are no TSRs left in DOS memory or EMM/XMM handles allocated.

It can be tested, I'll confirm it later. I think it will work ISA cards, since real IO could be faster than trapped emulation. for PCI cards I it need test.

I tested on my SONY VAIO PCG SR17K with YMF724, load jemm in config.sys, hidpmi32i in autoexec, and then setupds. after setupds, I cannot switch to drive D:, system freezes while disk LED on. then I removed them, things are good again.
also tested on IBM 240, with ESS Solo-1, this time the doom freeze just like the AC97 machines. then I tested modified HDPMI32i that I was working on, still freeze in "setting up heads up display" like the AC97 machines. test some real mode games all working fine.

crazii wrote on 2023-02-16, 12:22:

crazii wrote on 2023-02-16, 11:22:

georgel wrote on 2023-02-16, 10:31:

If these sound cards can adjust their resources to ISA sound card addresses without leaving/relying after initialization on memory resident utilities then they are fine for the proposed test. Just make sure after their init there are no TSRs left in DOS memory or EMM/XMM handles allocated.

It can be tested, I'll confirm it later. I think it will work ISA cards, since real IO could be faster than trapped emulation. for PCI cards I it need test.

I tested on my SONY VAIO PCG SR17K with YMF724, load jemm in config.sys, hidpmi32i in autoexec, and then setupds. after setupds, I cannot switch to drive D:, system freezes while disk LED on. then I removed them, things are good again.
also tested on IBM 240, with ESS Solo-1, this time the doom freeze just like the AC97 machines. then I tested modified HDPMI32i that I was working on, still freeze in "setting up heads up display" like the AC97 machines. test some real mode games all working fine.

Try doom with dos32a then. Are you sure the ess solo-1 is not using any resident software to emulate SB? I have such card and can test it too after a day or two.

georgel wrote on 2023-02-16, 12:28:

crazii wrote on 2023-02-16, 12:22:

crazii wrote on 2023-02-16, 11:22:

It can be tested, I'll confirm it later. I think it will work ISA cards, since real IO could be faster than trapped emulation. for PCI cards I it need test.

I tested on my SONY VAIO PCG SR17K with YMF724, load jemm in config.sys, hidpmi32i in autoexec, and then setupds. after setupds, I cannot switch to drive D:, system freezes while disk LED on. then I removed them, things are good again.
also tested on IBM 240, with ESS Solo-1, this time the doom freeze just like the AC97 machines. then I tested modified HDPMI32i that I was working on, still freeze in "setting up heads up display" like the AC97 machines. test some real mode games all working fine.

Try doom with dos32a then. Are you sure the ess solo-1 is not using any resident software to emulate SB? I have such card and can test it too after a day or two.

Yes ESS Solo-1 has a TSR. it also has a SYS in the config.sys. You can test with PIC & ISA if you have both. I still believe ISA's are ok. PCI on the other hand may have emulation so the situation could be the same.

zyzzle wrote on 2023-02-16, 05:24:

This seems like a dream project. Most of my modern systems (laptops) have Intel IHD / HDA audio, but I have a couple of desktops with AC'97 codec and I'll try them. Should I stick to DOS4GW (most of which I've stubbed over to DOS/32a, which claims to be 100% compatible and improve upon the original DOS4GW 1.97 / 2.01 versions) or should I focus on CWSDPMI games? I'll start with Doom, Doom2, Duke3d.

You can test both DOS4GW/DOS32A and CWSDPMI, Quake which uses CWSDPMI works fine. I'm testing the games with original set, that means DOS4GW is used, but you may compare DOS32A with DOS/4GW if you can.
BTW Quake won't load CWSDPMI if it detects one existing DPMI service, so with the SBEMU & HPDMI installed, Quake will just use the DPMI service that HDPMI provided. Any other CWSDPMI games will be the same.

zyzzle wrote on 2023-02-16, 05:24:

However, your emulator TSR appears to be generic enough that it doesn't need special compiled binaries of the games, it just needs to be loaded first as a wrapper in memory?

Yes it uses sound card drivers from MPXPlay and emulate the SB, the target is to run games without any rebuilds or modification.

georgel wrote on 2023-02-16, 11:59:

But the link is about Win 3.1.

Ok. The suggestions in this "document" are a bit "flaky", and there's also the "recommendation" that SFX has to be turned off ...

I just tried in qemu, DOS 6.22 with Win3.1. DOOM setup with music and SFX for SB, works well in pure DOS, but in a DOS box - freeze.
Repeating this procedure in a qemu VM with Win95sr2, DOOM runs without freeze and SFX is ok.

crazii wrote on 2023-02-15, 18:34:

rasz_pl wrote on 2023-02-15, 17:28:

Still doesnt affect POPF. Now Im lost. Whats the point of PVI? How does it help with silently ignored POPF? It looks like with PVI hypervisor can keep receiving interrupts thus doesnt hang, but you still dont know when to start sending them down to client/application?

PVI just skip a #GP exception and set VIF directly, in that case it doesn't handle POPF, only make STI/CLI a little faster, maybe fast enough to skip some problem I guess.

how would speed help here? If PVI still ignores POPF (no VIF change) then dont we still end up with a situation:
-Doom stores flags and disables interrupts
-PVI clears VIF instead of IF letting hypervisor keep receiving interrupts
-Doom tries to restore flags, but popfd is silently ignored and both IF and VIF arent changed
-Doom doesnt receive any interrupts until some other part of the code executes STI?

what happens to all the interrupts being received but not passed to Doom? is that the source of stack overflow - too many buffered interrupts?

How can one easily reproduce the doom "freezing" in relation to the game (I am not much of a gamer)? When you debug a "frozen" VM what is the CPU executing then?

Baron von Riedesel wrote on 2023-02-16, 11:03:

t's caused by a PM-RM-PM-RM-PM-...-switch cascade

Is it RM or V86?

rasz_pl wrote on 2023-02-16, 13:41:

-Doom doesnt receive any interrupts until some other part of the code executes STI?

NO, doom will always receive interrupts. because HDPMI doesn't do anything on VIF, once it gets IRQ, it will route it to doom.

Baron von Riedesel wrote on 2023-02-16, 04:03:

- pressing Ctrl-C/Ctrl-Break causes the TSR to "exit" ( which usually results in a DOS MCB chain corruption ).
- if the emulated sound IRQ and the real sound HW IRQ are identical, the emulation doesn't work. This case can probably be handled properly, but for now it might be sufficient if SBEMU displays an error and exits.

The Ctrl+Break is fixed. The other one could be dealt with later.

crazii wrote on 2023-02-18, 00:24:

The Ctrl+Break is fixed. The other one could be dealt with later.

Very good - using _djgpp_exception_toggle() is by far the best thing to do, since it fully disables SBEMU's protected-mode keyboard handler.

I'm still struggling with the doom freeze problem, here's some observations during this weekend:

1.my workaournd works on my Pentium M 855M AD1981 laptop, doom works fine.

2.for my NEC P3M VT82C686 laptop, it freezes on "HU_Init: setting up heads up display". after left it running for hours, it actually runs to the next phase, setting up status bar. I think it doesn't "freeze" now, it turns out to be running extremely slow. The reason might be, in order to break the recursion, I send EOI to the PIC and exit. but without sending ACK to the sound card (HDPMI doesn't know that and shouldn't do that directly), that will cause the IRQ signal to keeping at raising state. which will cause the cpu freeze doing the interrupt.

I'll try adding a IRQ ACK routine registration as HDPMI vendor entry API, like a backdoor to hack the problem, still not sure whether it works.

The above approach works, DOOM now is working without SETPVI. Here the binary for testing:

What I did for HDPMI:
1. add trapping method described as the win9x way: set TF if IF set on PUSHF. plus PUSHF(d) + POP (e)AX detection for doom. I think a opcode table might be added to trap all popping GPRs. may be done later.
2. add a IRQ mask when calling interrupt, detect the DOS4GW recursion when IRQ mask is already set. when re-entrance happens, stop the calling chain and call the IRQ ACK routine
3.add vendor entry API to register an IRQ ACK.
I'm not sure whether it works for other PCs yet.

some improvements made for SBEMU:
1.fixed noise (notable in prince of persia2 voice and quake, and test cases provided by Baron von Riedesel)
2.added sound volume setting support. tested working in Duke3D & Doom.

Excellent progress you continue to make, crazii! 🚀

I do have one concern, though: you are using sources from the Mpxplay project for the backend drivers. But those sources have not been released under an OSI-approved license, making the terms of use a bit unclear.

These comments at the top of the source files are particularly concerning:

1//*  Please contact with the author (with me) if you want to use           *
2//*  or modify this source.                                                *

Also, this could potentially complicate things like eventually having the SBEMU project admitted into the FreeDOS distribution. (I'm not sure if that's your ambition, but I think that would be very useful for the retro DOS community!)

At the very least, to "shield off" your own hard work, is there a sufficiently decoupled abstraction layer between your own code and the Mpxplay drivers, so that those driver sources could be swapped out for something else with relatively little effort later if need be? As a bonus, that would make is easier to add additional backend drivers later as well.

Possible open-source replacements you could consider would be the Apogee Sound System and JUDAS. (The FastDoom project has also adopted these drivers.)

But if I may also make a somewhat more ambitious suggestion: dust off the protected mode AIL/32 drivers, which John Miles released as open source years ago. These are modular drivers in the form of DLLs, and I've already been wanting to try implementing AIL/32 drivers for more modern sound devices.

We could make SBEMU more modular by allowing the user to select an AIL/32 driver DLL for SBEMU to load and route the audio to. That way, other people (such as hopefully myself) could assist you by focusing on the backend drivers, while you focus on getting the actual emulation code right.

Just an idea. 🙂

That, or maybe we should just port OSS to DOS. 😁

But even if you don't like these suggestions, I would still recommend we either try to convince the Mpxplay author to release their drivers under a proper open source license, or otherwise look for an alternative driver model/framework to use on the hardware backend.

crazii wrote on 2023-02-19, 11:30:

The above approach works, DOOM now is working without SETPVI. Here the binary for testing: SBEMU.zip […]
Show full quote

The above approach works, DOOM now is working without SETPVI. Here the binary for testing:
SBEMU.zip

What I did for HDPMI:
1. add trapping method described as the win9x way: set TF if IF set on PUSHF. plus PUSHF(d) + POP (e)AX detection for doom. I think a opcode table might be added to trap all popping GPRs. may be done later.
2. add a IRQ mask when calling interrupt, detect the DOS4GW recursion when IRQ mask is already set. when re-entrance happens, stop the calling chain and call the IRQ ACK routine
3.add vendor entry API to register an IRQ ACK.
I'm not sure whether it works for other PCs yet.

some improvements made for SBEMU:
1.fixed noise (notable in prince of persia2 voice and quake, and test cases provided by Baron von Riedesel)
2.added sound volume setting support. tested working in Duke3D & Doom.

Tried Heretic and it works prfectly now 👍 The previous version had the background noise. DOOM 1/2 works fine as before 😀

For some reason I get "runtime error (0025) cannot initialize" running Chasm the rift. It runs fine without sbemu/HDPMI32i, with no sound obviously.
I get the same (0025) error for Jack Jazz Rabbit and Tyrian. However, they don't run without the sbemu/HDPMI32i too, but with a different error (Error 200, due to the fast cpu iirc). Nevertheless it may work on other target computers, so probably worth checking.

EDIT: So i patched the JJR and Tyrian and they now run (without sound drivers), and with "runtime error (0025) cannot initialize" with sbemu/hdpmi.

Bondi wrote on 2023-02-19, 15:58:

For some reason I get "runtime error (0025) cannot initialize" running Chasm the rift. It runs fine without sbemu/HDPMI32i, with no sound obviously.
I get the same (0025) error for Jack Jazz Rabbit and Tyrian. However, they don't run without the sbemu/HDPMI32i too, but with a different error (Error 200, due to the fast cpu iirc). Nevertheless it may work on other target computers, so probably worth checking.

EDIT: So i patched the JJR and Tyrian and they now run (without sound drivers), and with "runtime error (0025) cannot initialize" with sbemu/hdpmi.

IIRC Tyrian is a 16-bit protected-mode program - it hasn't been mentioned as of yet, but these are not compatible with SBEMU. The problem is that 16- and 32-bit clients cannot run in the very same VM.
That's a pity, because I would have loved to hear the Tyrian music with SBEMU...
To overcome this restriction, one would have to implement SBEMU as a 16-bit protected-mode TSR - a true "challenging" task, IMO. Or implement it as a "jemm loadable module" - that's possible, but is also pretty "advanced".