Akuma wrote on 2021-09-24, 15:22:

That's easier said then done. The symbols/overlay must be stripped first and the header must be patched to reflect the new size., then LZEXE is possible.

I think that he knows that the debug symbols must be stripped prior for release with Borland C++ and the new release of wolfdosmpu already has a smaller EXE size. I have also done this prior without any problems when I was tinkering with the Wolf3D source code years ago.

First things first: 'I think it does not really matter'
I enjoy the hell out of it 😀

But if you want to go down that road, you might as well do it right.
It is still 42k too big in its unpacked form and in order to mimic the original release well:

1. search for the 'COMPAQ' string
2. search for 0x5 0x00 0x00
3. everything after that can be stripped.
4. update the exe-header to reflect new size
5. then compress with LZEXE v1.00 and not the v0.91

1LzExe v1.00a [1991] SPEAR.EXE     112835   1993-01-05
2LzExe v0.91  [1990] SPEARCM.EXE   114812   2021-09-24

The new one will be even smaller ~500 bytes

1LzExe v1.00a [1991] SPEARCM.EXE   114268   2021-09-24 

EDIT:
Small note, my savegames dont work after updating (don't know if it was mentioned)

Akuma wrote on 2021-09-24, 18:13:

First things first: 'I think it does not really matter' I enjoy the hell out of it :) […]
Show full quote

First things first: 'I think it does not really matter'
I enjoy the hell out of it 😀

But if you want to go down that road, you might as well do it right.
It is still 42k too big in its unpacked form and in order to mimic the original release well:

1. search for the 'COMPAQ' string
2. search for 0x5 0x00 0x00
3. everything after that can be stripped.
4. update the exe-header to reflect new size
5. then compress with LZEXE v1.00 and not the v0.91

Copy code to clipboard

1LzExe v1.00a [1991] SPEAR.EXE     112835   1993-01-05
2LzExe v0.91  [1990] SPEARCM.EXE   114812   2021-09-24

The new one will be even smaller ~500 bytes

Copy code to clipboard

1LzExe v1.00a [1991] SPEARCM.EXE   114268   2021-09-24 

Clever EXE compression trick! 😀 To be fair I didn't mention the version of LZEXE.

Not even the recreated Wolf3D versions by NY00123 do this.

Akuma wrote on 2021-09-24, 15:22:

That's easier said then done. The symbols/overlay must be stripped first and the header must be patched to reflect the new size., then LZEXE is possible.

Hi, I have just read about this, actually. But since you mentioned "doing it right"... I actually ended up reading posts in another forum where they attempt to recreate the original .EXE files, and they discovered that there is an option within Borland C++ itself to remove the debugging information. This prevents the overlay from being generated at all. I'm gonna integrate it into my next release. (As for LZEXE version, I might stick to 0.91e as that's the official one that the author, Fabrice Bellard, offers on his website.)

But having said that, and with your mention that save games from previous versions do NOT work (*gasp*! I never actually saved any of my games, so I never knew...), I'm clearly descending into a giant rabbit hole, deciding what is necessary in this hack or not. When I decided on adding LZEXE, I assumed that the goal was simply practical -- to make it possible to transport the .EXE via floppies.

For now I'm tempted to just change the savegame filename to prevent people from reusing their saves across versions. I currently don't know what the consensus is about savegame binary formats -- should I even attempt being compatible with vanilla for this (I feel like I should), or do other mod authors generally ignore this? Also, it's pretty strange why it's not loading correctly -- apparently the variables I've introduced into the data segment to track MIDI state actually mess up the memory structure everywhere else (when they really shouldn't)... I have to look into this more.

EDIT: Actually it's something else... the mere fact that I output a version string on game exit SOMEHOW messes the data segment. I removed it and suddenly savegames from vanilla work fine. What the effin' heck.

ericvids wrote on 2021-09-24, 20:18:

For now I'm tempted to just change the savegame filename to prevent people from reusing their saves across versions. I currently don't know what the consensus is about savegame binary formats -- should I even attempt being compatible with vanilla for this (I feel like I should), or do other mod authors generally ignore this?

AFAIK most mod authors at the time when "modding" the DOS version were only messing with data structures such as gamestate that are located in WL_DEF.H, which affect compatibility with savegame files. In this case you should be compatible with vanilla.

Sticking with the original LZEXE after stripping the debug symbols with the compiler should be good enough, you don't need to use that trick Akuma showed.

Gmlb256 wrote on 2021-09-24, 19:55:

To be fair I didn't mention the version of LZEXE.

I know it was just an observation 😉

ericvids wrote on 2021-09-24, 20:18:

Hi, I have just read about this, actually. But since you mentioned "doing it right"... I actually ended up reading posts in another forum where they attempt to recreate the original .EXE files, and they discovered that there is an option within Borland C++ itself to remove the debugging information. This prevents the overlay from being generated at all. I'm gonna integrate it into my next release. (As for LZEXE version, I might stick to 0.91e as that's the official one that the author, Fabrice Bellard, offers on his website.)

Even better 😀
(however its not Fabrice Bellard that chose the compression tool version, it was John Carmack)

ericvids wrote on 2021-09-24, 20:18:

But having said that, and with your mention that save games from previous versions do NOT work (*gasp*! I never actually saved any of my games, so I never knew...)

Its in development, better to find these sooner than later 😁

I saw there's about a 2k difference, is it possible to put the code into a TSR? Then you would not have to touch the executables at all. Just run the 'mytsr.com' and then the main program wolf3d.exe or spear.exe.

Akuma wrote on 2021-09-25, 12:14:

I saw there's about a 2k difference, is it possible to put the code into a TSR? Then you would not have to touch the executables at all. Just run the 'mytsr.com' and then the main program wolf3d.exe or spear.exe.

Speaking of putting code into a TSR, there is one that implements hardware mixing for the GUS called UltraWolf but that doesn't work on version 1.4 and requires patching the original executables. 😁

The music are completely different and the digitized sound are played without proper resampling though.

Funny you guys should mention putting the code in a TSR -- I actually designed my protocol with that in mind, because I also wanted to support IMF games like Keen 4-6 and Duke Nukem II. 😀 I've been specifically placing my code where I can easily understand their disassembly, so I can replace key functions with interrupt calls to a TSR if need be. The only problem there is that I'm a beginner at this low-level stuff, and the more I thought of doing it like that, the more I just want to "get it over with", which is why I ended up with a plain source mod. 😀

Of course, I don't mind it if someone else takes the current work and writes a TSR version. I'm just happy enough to get it working as it is now.

Anyway, I just made a new release. I now completely reuse existing (unused) state variables, making the data segment as pristine as possible. As a result, the savegame issue is fixed for the Activision releases. (I hope -- I don't have a nice batch of savegames with possible special cases.)

I also made a very small change to the data sending poll routine -- maybe that will make the AWEUTIL TSR cooperate with it now?

ericvids wrote on 2021-09-25, 14:42:

Anyway, I just made a new release. I now completely reuse existing (unused) state variables, making the data segment as pristine as possible. As a result, the savegame issue is fixed for the Activision releases. (I hope -- I don't have a nice batch of savegames with possible special cases.)

I also made a very small change to the data sending poll routine -- maybe that will make the AWEUTIL TSR cooperate with it now?

Nice that you got the WOLF3DCM.EXE size very close to the original 1.4 Activision version, it's 99 bytes less. 😀

Sadly AWEUTIL TSR still freezes the computer just after pressing a key during the signon screen. 🙁

Gmlb256 wrote on 2021-09-25, 15:40:

Sadly AWEUTIL TSR still freezes the computer just after pressing a key during the signon screen. 🙁

I have an inkling to what the problem is, but can you try this one? (See attached file.) Also try running it using permutations of the NOXMS and NOEMS options. Let me know the exact behavior.

If nothing still works, I might have to chalk it up to AWEUTIL being, well, an awful util...

ericvids wrote on 2021-09-25, 21:57:

I have an inkling to what the problem is, but can you try this one? (See attached file.) Also try running it using permutations of the NOXMS and NOEMS options. Let me know the exact behavior.

If nothing still works, I might have to chalk it up to AWEUTIL being, well, an awful util...

Downloaded attached file, wolfdosmpu finally works with AWEUTIL (nice pun considering the compatibility btw). Thanks for the effort! 😀

However it ocasionally freezes the computer without using the parameters so far.

Gmlb256 wrote on 2021-09-25, 22:23:

However it ocasionally freezes the computer without using the parameters so far.

If it doesn't freeze at all when using a certain combination of those NOXXX parameters, I think I'll consider it "good enough" and just warn people with AWE cards to use those switches. Since you did mention that AWEUTIL dislikes programs that use protected mode and XMS, I don't think there's anything I can do to fix it, short of rewriting id's memory manager (which I heard Carmack wrote because he famously distrusted the compiler). 😜

It does have a bad implication on a possible TSR implementation -- that just might be an inferior solution after all, given that I have to wrangle with AWEUTIL for first dibs on NMIs. Just thinking about it gives me a headache.

ericvids wrote on 2021-09-25, 22:41:

Gmlb256 wrote on 2021-09-25, 22:23:

However it ocasionally freezes the computer without using the parameters so far.

If it doesn't freeze at all when using a certain combination of those NOXXX parameters, I think I'll consider it "good enough" and just warn people with AWE cards to use those switches. Since you did mention that AWEUTIL dislikes programs that use protected mode and XMS, I don't think there's anything I can do to fix it, short of rewriting id's memory manager (which I heard Carmack wrote because he famously distrusted the compiler). 😜

It does have a bad implication on a possible TSR implementation -- that just might be an inferior solution after all, given that I have to wrangle with AWEUTIL for first dibs on NMIs. Just thinking about it gives me a headache.

Did further testing and with NOXMS only (which I didn't test prior) and expanded memory manager loaded, and works much better.

Gmlb256 wrote on 2021-09-26, 03:02:

Did further testing and with NOXMS only (which I didn't test prior) and expanded memory manager loaded, and works much better.

Ah, is that regular EMM386 that comes with DOS? If so, good to know!

If no further issues arise in about a week, I'll remove the ominous BETA word from the release (not that it really makes any difference 😉 )

UPDATE: I found another issue -- the Read This feature on shareware/registered versions (and occasionally on the commercial version, after beating an episode) would always shrink the view window. Fixed in 1.22

Ok, I discovered a real problem with savegames, and I wonder if anyone here can help me with it:

I initially thought that the Spear of Destiny savegames are already compatible (the games were loading and saving fine), but upon further inspection, I noticed that all of the static objects (treasures, barrels, etc.) are all gone.

Frustrated, I finally tried to build a SPEAR.EXE from https://bitbucket.org/gamesrc-ver-recreation/ … f3d/src/master/, and found that one of my variables is getting placed in the wrong section of the data segment. In particular, the _ylookup table is being relocated in a different portion of the .EXE.

In the original:

1 3A81:99B5       _fontnumber
2 3A81:99B7       _fontcolor
3 3A81:99B8       _ylookup
4 3A81:9B48       _displayofs
5 3A81:9B4A       _ca_levelbit
6...
7 3A81:EC0D       _fastpalette
8 3A81:EC0F       _bordercolor
9 3A81:EC11       _linewidth
10 3A81:EC13       _pelpan
11 3A81:EC15       _bufferheight
12 3A81:EC17       _bufferwidth

In my .EXE:

1 3A77:99B5       _fontnumber
2 3A77:99B7       _fontcolor
3 3A77:99B8       _displayofs
4 3A77:99BA       _ca_levelbit
5...
6 3A77:EA7D       _fastpalette
7 3A77:EA7F       _bordercolor
8 3A77:EA81       _ylookup
9 3A77:EC11       _linewidth
10 3A77:EC13       _pelpan
11 3A77:EC15       _bufferheight
12 3A77:EC17       _bufferwidth

The only way I can ensure that savegames are compatible is if the data segment (3A81 in the original, 3A77 in mine) is generated exactly the same. The reason is that id's crappy algorithm actually saves raw pointers (e.g., for its object linked list) and expect them to just work.

I have no idea how to force a variable to be placed elsewhere in the memory map on Borland C++ 3.1. Is anyone familiar with this issue?

Given this problem, I think I'm going to have to give up on savegame compatibility if no one knows of a solution. 🙁

Some bad news, I'm no longer able to use the recent versions of wolfdosmpu. It freezes right after pressing any key on the signon screen when displaying "Working...".

Only version 1.00 and 1.10 currently works for me on DOS. 🙁

Gmlb256 wrote on 2021-09-28, 02:28:

Some bad news, I'm no longer able to use the recent versions of wolfdosmpu. It freezes right after pressing any key on the signon screen when displaying "Working...".

Only version 1.00 and 1.10 currently works for me on DOS. 🙁

Yikes, I may have had a regression at some point.

Can you try this executable? If you can check on both your real MPU401 and on AWEUTIL, that would be great...

ericvids wrote on 2021-09-28, 02:43:

Can you try this executable? If you can check on both your real MPU401 and on AWEUTIL, that would be great...

I tested the EXE that you posted, however it only works with the real MPU-401 after a long initialization and it isn't guaranteed.

Okay, that's really strange... how long is the delay? Also, what do you mean by it "isn't guaranteed", does the program just hang or does it continue without MIDI playing?

How about this one? Is the delay still there? And does it not work with AWEUTIL anymore with NOXMS? (I thought I fixed a bug...)

ericvids wrote on 2021-09-28, 03:13:

Okay, that's really strange... how long is the delay?

How about this one? Is the delay still there? And does it not work with AWEUTIL anymore with NOXMS? (I thought I fixed a bug...)

The delay is no longer there (it was about 2-4 seconds), but the initialization issue still persist and could not get AWEUTIL working with this anymore no matter which parameter. 🙁

Seems that it's related to the MPU-401 wait loops.