Gmlb256 wrote on 2021-09-29, 13:07:

Tested WOLFTEST.EXE with both EMM386 and QEMM. With QEMM it still crashes at exit without the NOXMS parameter, EMM386 works fine as long that not many TSR are loaded (especially with AWEUTIL loaded).

VERY strange. How about this EXE? (This is essentially a build of the plain Wolf3D source without any wolfdosmpu stuff at all. I need to isolate whether it's wolfdosmpu or my Borland C++ environment.)

Also, are you using the Activision commercial version EXE?

ericvids wrote on 2021-09-29, 15:30:

VERY strange. How about this EXE? (This is essentially a build of the plain Wolf3D source without any wolfdosmpu stuff at all. I need to isolate whether it's wolfdosmpu or my Borland C++ environment.)

Downloaded that EXE and got no strange issues with expanded memory manager.

ericvids wrote on 2021-09-29, 15:30:

Also, are you using the Activision commercial version EXE?

Yes, I kind of mentioned it when talking about the EXE compression you did. The files required by the Activision version are very similar to the compiled executable with GOODTIMES defined.

Gmlb256 wrote on 2021-09-29, 15:41:

Downloaded that EXE and got no strange issues with expanded memory manager.

I basically confirmed at this point that it was my own module that's causing the problem, but I still don't know how.

Gmlb256 wrote on 2021-09-29, 15:41:

Yes, I kind of mentioned it when talking about the EXE compression you did. The files required by the Activision version are very similar to the compiled executable with GOODTIMES defined.

Sorry, that was so many posts ago -- I was just confirming because I also noticed that id's sources was part-Activision, part-GT (it displayed the GOODTIMES exit screen but it had the WOLFHACK stuff only present in Activision).

Anyway, here's Yet Another Build. I now isolate my own buffer space before id's memory manager allocates everything else to itself (hence main memory is reported as 64k less on the signon screen). I'm running out of ideas, so if this still crashes QEMM, I really don't what else I can potentially tap. 🙁

ericvids wrote on 2021-09-29, 16:26:

Anyway, here's Yet Another Build. I now isolate my own buffer space before id's memory manager allocates everything else to itself (hence main memory is reported as 64k less on the signon screen). I'm running out of ideas, so if this still crashes QEMM, I really don't what else I can potentially tap. 🙁

That new EXE no longer brings any problems with expanded memory manager, so we can confirm that it's around the memory allocation routine used for the MPU-401. However due to additional use of conventional memory, I had to go with a clean configuration for it to start (just with EMM386 or QEMM).

Could we try with 32KB of buffer for the MPU-401?

That's essentially the buffer for the MIDI file -- it has to be read from _somewhere_ while playing, and I don't think I want to go through the trouble of streaming it from disk. But id's memory manager apparently insists on relocating it to XMS from time to time, even if I locked it.

I can reduce the size of this buffer to the longest MIDI file at runtime (ULTIMATE is the longest song, with 21406 bytes... id's not kidding there 😜). Would that be acceptable?

ericvids wrote on 2021-09-29, 16:58:

I can reduce the size of this buffer to the longest MIDI file at runtime (ULTIMATE is the longest song, with 21406 bytes... id's not kidding there 😜). Would that be acceptable?

Yes, 32KB is even enough to fit that music. 😁

On a side note memptr is defined as void _seg *, where _seg is a 16-bit variable used by Borland.

Gmlb256 wrote on 2021-09-29, 17:24:

Yes, 32KB is even enough to fit that music. 😁

I know, but I don't want to suddenly break when some user puts in something bigger. Conversely, if they decided to do away with (or simplify) a particular MID so that it runs on their system with less memory, they can do that.

Of course that's assuming they convert their .MIDs in the first place. (For this mod, I only load files with a tick rate of 350 TPQN (or 700Hz), and I don't process BPM tempo changes.)

Just some fun trivia: ULTIMATE.MID's 21406 bytes sum up to 1337 paragraphs (16-byte chunks). 🤣 Okay there's one more paragraph allocated, I ignored the modulo...

Gmlb256 wrote on 2021-09-29, 17:24:

On a side note memptr is defined as void _seg *, where _seg is a 16-bit variable used by Borland.

Yeah, I know. 😉 In case you're wondering, I had to change the mpuBuffer to a far * in the current code (it was a _seg * when I was still using id's mem routines).

Anyways, how's this .EXE? Also, if you can check whether it works WITHOUT disabling XMS/EMS on AWEUTIL, that'd be awesome.

ericvids wrote on 2021-09-29, 18:32:

Anyways, how's this .EXE? Also, if you can check whether it works WITHOUT disabling XMS/EMS on AWEUTIL, that'd be awesome.

More efficient with memory than the previous one. It even works without having to disable XMS/EMS on AWEUTIL. 😀

I'd recommend a lot of free conventional memory if one intends to use AWEUTIL (about 615-620KB without it loaded) for MIDI emulation though.

Gmlb256 wrote on 2021-09-29, 19:16:

I'd recommend a lot of free conventional memory if one intends to use AWEUTIL (about 615-620KB without it loaded) for MIDI emulation though.

Is it possible to use AWEUTIL itself in upper memory using LOADHIGH? Or is the recommendation more like "load everything else that you need in upper memory if you want to use AWEUTIL"?

ericvids wrote on 2021-09-29, 19:31:

Is it possible to use AWEUTIL itself in upper memory using LOADHIGH? Or is the recommendation more like "load everything else that you need in upper memory if you want to use AWEUTIL"?

Loading everything needed into upper memory is the actual recommendation. Depending of the SBK soundfont that I use it's around 578-590KB with AWEUTIL loaded.

Using a real mode UMB driver such as UMBPCI works if one doesn't want to use EMM.

Gmlb256 wrote on 2021-09-29, 19:32:

ericvids wrote on 2021-09-29, 19:31:

Is it possible to use AWEUTIL itself in upper memory using LOADHIGH? Or is the recommendation more like "load everything else that you need in upper memory if you want to use AWEUTIL"?

Loading everything needed into upper memory is the actual recommendation. Depending of the SBK soundfont that I use it's around 578-590KB with AWEUTIL loaded.

Using a real mode UMB driver such as UMBPCI works if one doesn't want to use EMM.

Ok, good to know.

I've updated the github with all .EXEs now. (It compiles slightly different because I now make an integrity check on the MID file size that I wasn't able to do in previous versions.)

first off thanks for your efforts, I've been playing around with it on my 386, but I too have been running into memory issues, Wolfenstein 3D itself seems to be running fine, but in Spear of Destiny the game crashes with "MM_GetPTr: Out of memory!" as soon as I try to increase the view size beyond a certain point, I can play with a smaller view without any crashes though, "noxms" in the command line makes no difference, loading the mouse driver high lets me increase the view size one more step so it's defenitly connected to how much memory is free.

EMM386 is loaded, and the vanilla SPEAR.EXE works fine.

imi wrote on 2021-09-30, 18:41:

first off thanks for your efforts, I've been playing around with it on my 386, but I too have been running into memory issues, Wolfenstein 3D itself seems to be running fine, but in Spear of Destiny the game crashes with "MM_GetPTr: Out of memory!" as soon as I try to increase the view size beyond a certain point, I can play with a smaller view without any crashes though, "noxms" in the command line makes no difference, loading the mouse driver high lets me increase the view size one more step so it's defenitly connected to how much memory is free.

EMM386 is loaded, and the vanilla SPEAR.EXE works fine.

Hi, I'm in the middle of making a big(ish) feature update, but for the meantime, can you test this build on your computer to see if the memory issues are gone? (I certainly reclaimed a tick of memory on the signon screen.)

Savegames from an older SPEARCM.EXE won't be compatible, the memory map has changed so much for this version.

I'm just learning now about a lot of things that other people already know in more than two decades of wolf3d hacking. 😀

this one seems to be working fine 😀 thank you

but like you said the savegames don't work, so I guess I gotta finish my playthrough on the smaller view ^^

imi wrote on 2021-10-01, 10:44:

this one seems to be working fine 😀 thank you

but like you said the savegames don't work, so I guess I gotta finish my playthrough on the smaller view ^^

Sorry about that, id's savegame code is very finicky that way. How much smaller is the view?

----

By the way, version 1.30 is up! -- with support for modern WASD controls!

You can also now press Tab to view your Kills, Secrets, and Treasures progress. (No, I did not make an automap. Yet. I know Super 3D Noah's Ark has one, but given the extra memory needed for MIDI playback and the Spear of Destiny memory issues, I don't know if it's possible. The goal is to make it work on PCs where vanilla Wolf3D/SoD already works, so I'm still iffy on this.)

Wolf3D purists who consider the above features as "cheating" need not complain -- separate .EXE files without these features are also available. 😀

ericvids wrote on 2021-10-01, 21:03:

No, I did not make an automap. Yet. I know Super 3D Noah's Ark has one, but given the extra memory needed for MIDI playback and the Spear of Destiny memory issues, I don't know if it's possible. The goal is to make it work on PCs where vanilla Wolf3D/SoD already works, so I'm still iffy on this.

Wolf3D v1.0 had a not useful overhead map view as a debug cheat code with Tab+O, you can see the remnant of this code in WL_DEBUG.C which is completely disabled with #if 0. This was a leftover from the alpha version which used Tile16 chunks.

Gmlb256 wrote on 2021-10-01, 21:23:

ericvids wrote on 2021-10-01, 21:03:

No, I did not make an automap. Yet. I know Super 3D Noah's Ark has one, but given the extra memory needed for MIDI playback and the Spear of Destiny memory issues, I don't know if it's possible. The goal is to make it work on PCs where vanilla Wolf3D/SoD already works, so I'm still iffy on this.

Wolf3D v1.0 had a not useful overhead map view as a debug cheat code with Tab+O, you can see the remnant of this code in WL_DEBUG.C which is completely disabled with #if 0. This was a leftover from the alpha version which used Tile16 chunks.

Thanks for the tip. I can also look at the reconstructed S3DNA source. But honestly I'm not sure if I should be doing it -- I want to give Wolf3D without automap a chance myself, because whenever I play it on ECWolf I just end up relying too much on the automap. Conversely, the Quake games don't have an automap, but I enjoyed finding secrets there. (Though the full 3-D may play a role there.)

For one, I want to see if the rest of Wolf3D is fun to memorize old-school (maybe not the dreaded E2L8 maze, but the rest of the game). I know I used to 100%-complete Episode 1 as a kid without ever checking a map. (And part of what made it not fun was not knowing how many secrets I've already gotten until I exit the level, hence the Tab feature.)

I'm also thinking that there are better things to implement than an automap: for example, auto-detecting secrets that are completely inaccessible and removing them from the secrets tally (i.e., the E2L8 maze bugs), and maybe a "hint" key to guide me to the nearest secret or something. But in any case, these things are already way beyond the original goal of this MPU-401 mod, and others have definitely done the features I mentioned (I am aware of at least one that's compatible with old systems, the "Better Wolf3D" mod, but I haven't tried it myself), so I don't know if I'll get around to it.

ericvids wrote on 2021-10-01, 21:03:

Sorry about that, id's savegame code is very finicky that way. How much smaller is the view? […]
Show full quote

imi wrote on 2021-10-01, 10:44:

this one seems to be working fine 😀 thank you

but like you said the savegames don't work, so I guess I gotta finish my playthrough on the smaller view ^^

Sorry about that, id's savegame code is very finicky that way. How much smaller is the view?

----

By the way, version 1.30 is up! -- with support for modern WASD controls!

You can also now press Tab to view your Kills, Secrets, and Treasures progress. (No, I did not make an automap. Yet. I know Super 3D Noah's Ark has one, but given the extra memory needed for MIDI playback and the Spear of Destiny memory issues, I don't know if it's possible. The goal is to make it work on PCs where vanilla Wolf3D/SoD already works, so I'm still iffy on this.)

Wolf3D purists who consider the above features as "cheating" need not complain -- separate .EXE files without these features are also available. 😀

it's not tooooo bad, I had to reduce the view size by 3 steps, still playable 😀
also thanks for providing seperate .exe files, I indeed like torturing my self with the janky old controls x3

ericvids wrote on 2021-10-01, 21:03:

By the way, version 1.30 is up! -- with support for modern WASD controls!

Legend!!

Um... *sheepishly*... I have to break savegame compatibility again, but hopefully this will be the last time. I figured out how to make the data segment identical between the MPU-only version and the modern-controls version. I also figured out how to make the dataseg identical between Apogee-version and Activision-version data files. So while my .EXEs will NOT be compatible with savegames from the original .EXEs, they WILL be compatible with each other for the same game, and HOPEFULLY all future versions of wolfdosmpu.

Also, I realized that the setting of my BC compiler was 80386, which might cause the executables to not work on 286 machines. (In fact, the Activision releases may not have worked on 286s either. I don't know, I never tried.) I flipped the switch to 80286 and hope that nothing breaks. The EXEs are slightly bigger, which makes me concerned, but I hope it's nothing.

Version 1.32 is now up. I labeled it an RC (Release Candidate) instead of a beta because I'm more or less happy with it at this point.

I'd like to ask for everyone's help to ensure that these EXEs work on the target hardware... better if you can test on a 386 or 286, even.

EDIT: I've confirmed that Activision-version .EXEs (and indeed previous versions of wolfdosmpu) do not work on 286 emulation mode in DOSBox-X. But version 1.32 does work. 😀