sub_9A6 does indeed seem to be a procedure to set an Adlib register. AL contains the register number, AH contains the value that the register needs to be set to. All those IN instructions are simply delays to wait for the OPL2 chip to be ready(delays, like those documented by the chip documentation). Almost no registers are destroyed, except AL becoming the value in AH.

sub_89E loads 0x12 adlib registers specified at location DS:171 and onwards (starting at Adlib register 0x80+[DS:171], 0x80+[DS:172] and onwards) with the value 0xFF.

loc_1EB seems to be the startup routine. It check AX for the 'drive letter status' (according to http://stackoverflow.com/questions/3715618/ho … ram-into-memory ), stores AX at word_2C6 when not zeroed, otherwise leaves it at value 0xA21F. Then it calls sub_1C8 to process the extension in the program's filename, loads 0x20 into Adlib register 0x01, clears register 0xBD(Drumkit information register) and loads register 0x08 with value 0x40. Then it saves CS on the stack to provide a 32-bit return pointer for RETF instructions and calls sub_211. Upon return it sets BX to 0xF00 and executes a far return(to the address at SS:SP, first IP, then CS that's pushed on the stack).

sub_1C8 seems to search the string pointer(char *var=&referencedstring-2; in C/C++) at [BP+(n<<1)+2] for a '.' character within the first 9 characters. n being increased after each loop. When it finds a '.' character, it replaces the following 3 bytes with the values in word_2C3 and byte_2C5, probably replacing the extensions of files found into the end-of-string(NULL byte) to terminate the string and filling the remaining two extension bytes with 1E, 0E(Record seperator, Shift out ASCII characters), then continuing the loop.
- So it's processing the referenced filenames pointed by a table at [BP+(n<<1)+2], terminating each string with a dot in it after the dot, adding a Record seperator and Shift out ASCII character(according to http://www.asciitable.com/ ) to each string with a '.' within the first 9 characters, probably for some list processing routine.
Edit: Looking at it again, it's probably processing the application filename from the MS-DOS PSP segment, replacing the second and third bytes into the Record seperator and Shift out ASCII character as a seperate string. The filename itself is reduced to become everything up to and including the dot of the filename.

sub_211 saves the flags on the stack(to be restored later), disables interrupts, and calls sub_81F, clears AX and clears byte_19A, returning to the called procedure while restoring the flags stored previously.

sub_81F saves DS on the stack for restoration before returning, points DS to CS(probably for the subroutine to use DX as a pointer into CS, when needed), loads CX to process 9 items in a loop. Then it calls sub_98B with DX being 8 to 0 in a loop. Finally it returns to the calling function, restoring DS in the process.

sub_98B loads AX with the value at memory location DS:[(2*DX)+0x15F], then copies it into CX. AL is loaded with (the low 8-bits of DX provided by the caller function)+0xA0 to provide a register index, then SI is set to a copy of the calculated register/value to set and sub_9A6 is called to set the adlib register. Upon return AX is restored to the value read at the start of the function, 0x10 is added to the register that was written and AH is set to the register set in the call to sub_9A6. Then it passes-though(executes) sub_9A6 on the resulting register/value pair, updating the same register with 0x10 added to it's original value, before letting sub_9A6 return to the caller of sub_98B.

Looking at the combined information of sub_81F and sub_98B, this might be processing all 9 adlib channels and producing a note OFF, then immediately a note-on on said channel, thus triggering a note, with information about it at the word variable located at DS:[(2*DX)+0x15F].

Edit: Whoops, bit 3 in register A0+ is part of the frequency LSB. So it's toggling the frequency between the selected frequency and 0x10 higher instead? Why would it do that? First load frequency LSB n, then n+16(wrapping around 256)?

Excellent work, superfury!

superfury wrote:

sub_89E loads 0x12 adlib registers specified at location DS:171 and onwards (starting at Adlib register 0x80+[DS:171], 0x80+[DS:172] and onwards) with the value 0xFF.

Yup, I've seen this before with MegaRace and Dune. If you also noticed during the initialization of Dune or MegaRace, all the instrument FNUMs are at 87. Before any notes are played, all the channels are grabbing the values where the driver stores the FNUM registers, and for what ever reason, the values are initially stored as 0x157 aka 343, which is "C" in HERAD FNUM Frequency Table. But since the FNUMs are split into two bytes on the register map, the driver interprets this value as Block number 1, FNUM 87, instead of FNUM 343.

superfury wrote:

loc_1EB seems to be the startup routine. It check AX for the 'drive letter status' (according to http://stackoverflow.com/questi […]
Show full quote

loc_1EB seems to be the startup routine. It check AX for the 'drive letter status' (according to http://stackoverflow.com/questions/3715618/ho … ram-into-memory ), stores AX at word_2C6 when not zeroed, otherwise leaves it at value 0xA21F. Then it calls sub_1C8 to process the extension in the program's filename, loads 0x20 into Adlib register 0x01, clears register 0xBD(Drumkit information register) and loads register 0x08 with value 0x40. Then it saves CS on the stack to provide a 32-bit return pointer for RETF instructions and calls sub_211. Upon return it sets BX to 0xF00 and executes a far return(to the address at SS:SP, first IP, then CS that's pushed on the stack).

sub_1C8 seems to search the string pointer(char *var=&referencedstring-2; in C/C++) at [BP+(n<<1)+2] for a '.' character within the first 9 characters. n being increased after each loop. When it finds a '.' character, it replaces the following 3 bytes with the values in word_2C3 and byte_2C5, probably replacing the extensions of files found into the end-of-string(NULL byte) to terminate the string and filling the remaining two extension bytes with 1E, 0E(Record seperator, Shift out ASCII characters), then continuing the loop.
- So it's processing the referenced filenames pointed by a table at [BP+(n<<1)+2], terminating each string with a dot in it after the dot, adding a Record seperator and Shift out ASCII character(according to http://www.asciitable.com/ ) to each string with a '.' within the first 9 characters, probably for some list processing routine.
Edit: Looking at it again, it's probably processing the application filename from the MS-DOS PSP segment, replacing the second and third bytes into the Record seperator and Shift out ASCII character as a seperate string. The filename itself is reduced to become everything up to and including the dot of the filename.

It looks like Dune's Gold driver does the same subroutines before initializing the Adlib Registers as well. Now, both MegaRace and Dune use compressed HSQ files for the music, but HSQ doesn't retain the file extension for the uncompressed music file, which is loaded into memory, so it makes sense that it's cutting off the file extension, since the uncompressed music files don't really have one.

superfury wrote:

sub_211 saves the flags on the stack(to be restored later), disables interrupts, and calls sub_81F, clears AX and clears byte_19 […]
Show full quote

sub_211 saves the flags on the stack(to be restored later), disables interrupts, and calls sub_81F, clears AX and clears byte_19A, returning to the called procedure while restoring the flags stored previously.

sub_81F saves DS on the stack for restoration before returning, points DS to CS(probably for the subroutine to use DX as a pointer into CS, when needed), loads CX to process 9 items in a loop. Then it calls sub_98B with DX being 8 to 0 in a loop. Finally it returns to the calling function, restoring DS in the process.

sub_98B loads AX with the value at memory location DS:[(2*DX)+0x15F], then copies it into CX. AL is loaded with (the low 8-bits of DX provided by the caller function)+0xA0 to provide a register index, then SI is set to a copy of the calculated register/value to set and sub_9A6 is called to set the adlib register. Upon return AX is restored to the value read at the start of the function, 0x10 is added to the register that was written and AH is set to the register set in the call to sub_9A6. Then it passes-though(executes) sub_9A6 on the resulting register/value pair, updating the same register with 0x10 added to it's original value, before letting sub_9A6 return to the caller of sub_98B.

Looking at the combined information of sub_81F and sub_98B, this might be processing all 9 adlib channels and producing a note OFF, then immediately a note-on on said channel, thus triggering a note, with information about it at the word variable located at DS:[(2*DX)+0x15F].

Edit: Whoops, bit 3 in register A0+ is part of the frequency LSB. So it's toggling the frequency between the selected frequency and 0x10 higher instead? Why would it do that? First load frequency LSB n, then n+16(wrapping around 256)?

Jepael shared with me some preliminary disassembly work on the Dune drivers and this subroutine also pops up as well. It looks like sub_211 is the Main Mute function, and that sub_81F is for silencing all the channels, and finally sub_98B mutes just the channel.

I know that MegaRace triggers NoteOff events by turning on the highest bit in the MIDI Pitch, but Dune just zeros out the pitch so I'm not entirely sure what that 0x10 is doing, since the subroutines are identical in both drivers.

EDIT: Maybe that 0x10 is used to write the KeyOff for the AdLib registers?

Looking at it again, it loads register A* and B* after all. It's adding 0x10 to the register number.

DX contains the channel to set, DS:[(2*DX)+0x15F] the value that's to be set to register A*. The byte after that address, DS:[(2*DX)+0x160] contains the value for register B*.

I'm slowly getting used to IDA and reverse-engineering the driver(s).

I did want to figure out a HERAD glitch that I've been noticing for a while now.

There's an interesting glitch that occurs in Skyholder (PAGA.HSQ). One of the instruments produces the wrong note an octave higher than it should when the song repeats. The first time the song plays, the instrument plays the notes normally, but when the song loops, the instrument plays the first note of the phrase an octave higher, even though it's reading the same MIDI data from the first time. This required further investigating...

Looking at my multitrack recordings of the song, I found the offending instrument on channel 2 and that the MIDI instrument being triggered is number 5. Here's the HERAD instrument data.

101 5B 00 02 06 06 03 01 04 01 0E 01 01 00 01 00 
203 03 0F 02 01 06 09 00 00 01 01 00 00 03 01 00 
302 00 F4 04 01 04 01 00

As you can see, there's an F4 for our transpose macro, so that means the instrument goes down an octave.

Let's look at the MIDI data:

100 90 2F 6F 00 C0 04 81 3F 80 2F 01 90 2F 77 81 
23F 80 2F 86 01 90 47 4B 00 C0 05 02 80 47 05 90 
347 47 03 80 47 02 90 40 56 0B 80 40

Immediately after looking at the MIDI data, I noticed something was off. Note On events were occuring BEFORE the instrument change. When this happens, HERAD plays the note, but has to load the macros from the previous instrument which are still in memory. Immediately after the Note On event, the correct macro values are loaded, but nothing happens until the next event, because the note has already been triggered.

By switching the MIDI events around from this:

101 90 47 4B 00 C0 05 02 80 47

To this:

101 C0 05 00 90 47 4B 02 80 47

This fixes the glitch by loading the instrument first before playing the note, therfore all the macros are locked in before any notes play.

Let's recap: having that program change after the Note On event is not the best way to change instruments. Even though it will very quickly load the instrument thanks to the zero delta-time, the Note On event has already been triggered so when the instrument changes, it already has the macros from the previous instrument. The macros parameters do update along with the instrument change, but in this setup, it is too late and the timers for the macros do nothing until the next Note On event.

I realized this occurs in the song a lot more than I thought, as a lot of the instrument changes in track 2 are after the Note On event. As a result, the first note of this instrument will always transposed incorrectly. Remember how I said the note plays correctly the first time the song plays? Well, the first note transposes correctly simply because the previous instrument was also transposed an octave lower, so the correct value just happened to be already loaded in memory.

Could it be they implemented the MIDI instrument differently, taking effect on running notes(as the adlib does, affecting running notes by modifying registers), instead of taking effect on all notes started(note on events) after it on the channel(as the MIDI specification does)? Also, MIDI starts out as instrument 0(piano) usually.

superfury wrote:

Could it be they implemented the MIDI instrument differently, taking effect on running notes(as the adlib does, affecting running notes by modifying registers), instead of taking effect on all notes started(note on events) after it on the channel(as the MIDI specification does)? Also, MIDI starts out as instrument 0(piano) usually.

I think you're right. I think the instrument change happens immediately. I'll make a quick test song file to confirm.

EDIT: You're absolutely right, superfury. The instrument changes running notes, immediately changing the registers, but any macros that modify registers don't work until the next Note On event.

I made a quick video to show how the glitch works. https://www.youtube.com/watch?v=qLqawHE9Cqg
I added a really exaggerated pitch slide macro to the second instrument but the Note On event has to trigger it even though it's loaded in memory.

I've attached some IDA *.idb files to help out with the disassembly. The first two files, DUNEADL.idb and DUNEAGD.idb were provided to me by Jepael. Hopefully, he won't mind me posting these files. The third file is MegaRace's driver DFADP.idb, with some disassembly work by me, namely just searching through the file and identifying identical subroutines from the other two files. If anyone can help cross-reference and make sure that these subroutines are correct and can help identify more, that would be greatly appreciated.

I have a list of all the known register blocks in the driver listed here, which will help out

http://www.vgmpf.com/Wiki/index.php/UserWiki: … P.HSQ.22_Driver

Qbix: Attachment removed as it got a broken file

Sorry for the delay, I've been super busy with real life. I'll reupload the IDB files soon.

As for HERAD Gold Driver reverse engineering, there's definitely something weird going on in terms of how the OPL channels work. Normally, for the regular OPL2 driver there's nine MIDI tracks and therefore, nine discrete channels of sound for the OPL2. Really straight forward fixed voice allocation. Dune's OPL3 driver seems to be doing some sort of dynamic voice allocation, as I just rendered some WAV multitracks from WORMSUIT.AGD and while there's 15 MIDI tracks, there's 16 OPL3 audio channels. After comparing the converted MIDI file to the multitrack recording, there's definitely something going on where the driver is sending different parts of a MIDI track to different channels. For example, MIDI track 8 is sending notes to OPL channel 2, then OPL channel 6, channel 8, then back to channel 2 before finishing up on channel 1.

I'm finally making some progress with disassembling the MegaRace driver! I figured out a lot of stuff and I'm currently documenting the hell out of it so we can figure out how it works.

Turns out one of the lookup tables - in this case, the one located at 0x71 (00 01 02 08 09 0A 10 11 12 03 04 05 0B 0C 0D 13 14 15) - is actually used to cycle through OPL registers.

Here's the subroutine where I figured this out, located at 0000:089E

1ClearSusRel     proc near           
2                 mov     si, 171h        ; places 171 in SI
3                 mov     cx, 12h         ; places 12 in CX
4                 mov     ah, 0FFh        ; places 0xFF at high byte in AX
5
6 loc_8A6:                               
7                 lodsb                   ; grabs the byte at 0x71, which is 00, and places it in the low byte of AX, which gives us FF00.  SI increments to 172.
8                 add     al, 80h         ; add 80, which gives us FF80.  The lookup table at 0x71 is for cycling through all the OPL registers and adding each byte, one at a time to 80, thus giving us OPL registers 80-95.
9                 call    ChipWrite       ; writes FF at OPL register 80
10                 loop    loc_8A6         ; do this 18 times, for OPL registers 80-95.
11                 retn
12 ClearSusRel     endp

This clears out the Sustain and Release registers with FFs, and it uses the table at 0x71 to cycle through each register, one byte at a time.

More interesting stuff is coming up, but it's going to take a while as there's still a lot more remaining that I need to run through DOSBox's debugger.

Hello SynaMax!

There are good news, I've added SOP format support into AdPlug and MIDIPLEX 😀 This format is also similar to MIDI (more to XMIDI, because of note durations) and it have multi-track structure, logically same as HERAD.

The format handler consists of two parts:
1. Event-to-OPL engine (Ad262Sop driver) by Park Jeenhong
2. Multi-track event replaying routine by me (the same is used in my MUS and MDI handlers)

The last is written from scratch by me, it's lightweight and easy to understand, and also it's pretty stable now.

I'm ready to start working on HERAD format handler for AdPlug, but there is one thing where I'm stuck...

SQX-1 decompression algorithm is still not researched. I've implemented an universal HSQ/SQX unpacker - tried to understand SQX-1 opcodes, but still no success. The decompressed data at first looks good, but only for few bytes.

Also I attached decompression utility and source code here, maybe someone could figure out the actual SQX-1 algorithm.

Glad to hear from you, binarymaster!

Since Dune's engine is very similar to KGB, I wonder if we can find the decompression algorithm for SQX in the same spot where it occurs in Dune for HSQ.

I'm currently still working on reverse engineering MegaRace's HERAD driver in DOSBox's Debugger.

Several new things that I discovered are:

1) When HERAD loads a song, it grabs the segment that the song is located, as well as the song's file size, but also two other dwords. These are the word located at 0x4000 in the song and a word located -8000 bytes above the driver. What the hell is going on here?! I haven't seen any other subroutines that mess around with this data but it just seems to store these words starting at 0x2BC. The song segment and song file size are normal, but grabbing random dwords at 4000 bytes into the song and -8000 bytes before the driver is loaded into memory doesn't make any sense at all. Perhaps it's a sanity check that ultimately went unused?

2) Definitely parts of Dune and KGB remain in MegaRace's driver, namely the flag responsible for changing songs, and I believe the function to fade out songs as well! The flag for changing songs is still implemented in the game, and if the game were to load the next song, it would work. However, for any song that doesn't use loop points, enabling this flag effectively removes the loop and the song stops playing when it reaches the end.

3) For whatever reason, the mystery 0x2 byte at offset 0x15 does quite a lot of stuff for the driver, namely to get the absolute addresses for the current MIDI positions, to load the offset for the song's speed and to help generate the dwords for some of the instrument macros. I don't know why the driver could just move 2 into a register instead of grabbing the byte from the driver, but I don't know if it's more efficient this way or not.

4) The mystery lookup table at 0x25 has sorta been solved! Each dword is the offset to a different subroutine based on the MIDI event. The subroutine at 0x3E5 calls one of the jumps, depending on what's going on in the song. For example, if a NoteOn occurs, the driver then points to 0x65D (or 0x55D if you're looking at the file) and then the executes those instructions.

5) I've run into a slight problem with my documentation and my addresses. When the driver is loaded in memory, it's actually loaded at 3B08:0100, instead of 3B08:0000, so as a result, all the offsets that are referenced in the driver that I have documented are off by 0x100, so I'll eventually need to change that, but as for right now, I'm sticking with the offsets based off of the DFADP.HSQ file.

We're slowly but surely getting there!

The table at 0x25 is now completely figured out. Each word is an offset to a subroutine in the driver that handles a particular MIDI Event:

06CF = NoteOff (80)
065D = NoteOn (90)
08E5 = Polyphonic Aftertouch (A0)
08E5 = Control Mode Change (B0)
05D2 = Program Change (C0)
072E = Aftertouch (D0)
07DD = Pitch Bend (E0)
06F5 = End of Track (FF)

The driver grabs the MIDI track positions starting at 0xB4 and uses those offsets to load the MIDI event bytes in the song. Once the driver has an event byte loaded in memory, it does an AND operation with 70, then shifts one bit to the right three times. This value, plus the offset of this table equals which subroutine to go to! So for example, a NoteOn would be 90. 90 ∧ 70 = 10; shift right by 1, three times = 2; 2 + table offset = NoteOn subroutine.

Note that Polyphonic Aftertouch and Control Mode Change share the same offset. That is because HERAD doesn't support those MIDI events so the sequencer uses this table to ignore the events and proceed with playing the event as a regular NoteOn.

I'm hoping to finish documenting the driver this weekend, but my schedule may change thanks to real life, etc.

I've written a small DOS program to load a driver and song to memory, and call the driver to play the song.

So far I just loaded Dune Adlib driver (DUNEADL.___) and ARRAKIS.SDB and it appears to work.
I think there are no regular Adlib songs so that's why ADL driver and SDB song.
Most likely it can be easily modified to load and use the other drivers too.
I even managed to patch the driver in memory so that the original subroutine to write chip registers is actually a callback to whatever I want it to be.
So this enables logging of OPL register writes during each driver call, so it enables exporting the music as RAW/DRO/IMF/VGM files (my personal order of preference).

I know Dune sets the timer interrupt rate to about 200 Hz, but calling the music driver at this rate sounds much too slow.
Do you know at what rate the driver should be called, and more importantly, how, like does the game call it several times in one timer interrupt and is the amount of calls per timer interrupt a constant or can it vary?

Jepael wrote:

I've written a small DOS program to load a driver and song to memory, and call the driver to play the song. […]
Show full quote

I've written a small DOS program to load a driver and song to memory, and call the driver to play the song.

So far I just loaded Dune Adlib driver (DUNEADL.___) and ARRAKIS.SDB and it appears to work.
I think there are no regular Adlib songs so that's why ADL driver and SDB song.
Most likely it can be easily modified to load and use the other drivers too.
I even managed to patch the driver in memory so that the original subroutine to write chip registers is actually a callback to whatever I want it to be.
So this enables logging of OPL register writes during each driver call, so it enables exporting the music as RAW/DRO/IMF/VGM files (my personal order of preference).

I know Dune sets the timer interrupt rate to about 200 Hz, but calling the music driver at this rate sounds much too slow.
Do you know at what rate the driver should be called, and more importantly, how, like does the game call it several times in one timer interrupt and is the amount of calls per timer interrupt a constant or can it vary?

That's awesome! I can't wait to try the program out! After doing some debugging, I couldn't get INT 08 (system timer to do anything) but I did get INT 16 (located at 01FA:C854) to consistently update the driver on every fifth time it was called, while the game was paused. I tried doing TIMERIRQ in DOSBox's debugger but nothing happened.

If it helps, if you set the song speed to 1, the approximate BPM of the song will be 500 bpm. I say approximate because after a few seconds it eventually starts to go out of sync. But, doing 500 / SongSpeed is a good way of getting a BPM that's pretty darn close to the actual song.

Since I'm still new to debugging, any advice about interrupts or timers would be greatly appreciated.

What kind of help you need with the system timer interrupt? In what programming language? Can I see what you have done so far? Do I ask too much?

Edit: As it turns out, I had a mistake - I was loading the MSB/LSB to timer the wrong way. So the correct rate is about 200 Hz (without going into details why it is not exactly 200Hz).

Glad to hear you were able to find out the issue, Jepael!

Here's my work in progress ASM file, documenting/commenting of the HERAD driver from MegaRace. It's still pretty rough and not all the subroutines have been documented yet. Also, since the driver in memory is offset by 100 bytes, all my comments will have the incorrect offset, so this will need to be fixed, as currently they reflect the offsets if we were looking at the decompressed file in a hex editor. Keeping that in mind, it's not too confusing to read.

Quick update:

After analyzing the MegaRace driver, I became suspicious if the driver actually supports Aftertouch MIDI events, which I know are used in both Dune and KGB but are non-existent in MegaRace. So to find out, I converted MORNING.SDB from Dune so that it'll play in MegaRace. Sure enough, Aftertouch events are ignored so the expressiveness in some of the instruments in that song are lost. The main instrument that uses it is that clarinet-like patch that gets louder when it sustains a note. The instrument will still play but it just won't get louder like it does in Dune. I've attached the song here in case you want to try it out in MegaRace.

To recap, the main differences between Dune/KGB's driver and MegaRace's driver is:

- NoteOffs are truncated (velocity byte is missing)
- Drum keymap support
- Transpose Macro functions differently depending on value
- Aftertouch is not supported
- Song Fade Out is not supported

If anyone is interested, here's a livestream I did of me reverse engineering MegaRace's HERAD driver. It's two hours long but I made some interesting discoveries during this session.

https://www.youtube.com/watch?v=__ttnM-XAXc

Finally, the universal HSQ / SQX decompressor is done!

Big thanks goes to -=CHE@TER=- who helped a lot with disassembling the original unpacker code from KGB game 😊
https://www.extractor.ru/ipb/index.php?showtopic=8767

I converted disassembly to Delphi code, and now it works excellent! Attached archive contains the source code and compiled Win32 binary executable.

@synamax nice stream!

YES! Thanks so much, binarymaster!! I'm so glad we finally have support for SQX! I already tried it out on some other KGB assets such as sprites and it works perfectly. 😀 Excellent work!

Things are a bit busy for me at the moment, but I'm hoping to continue working on documenting the MegaRace driver this weekend.