VOGONS

Common searches


Dune "HERAD" Ad Lib Music Hacking

Topic actions

First post, by synamax

User metadata
Rank Newbie
Rank
Newbie

Hey guys,

I'm currently working on reverse engineering the "HERAD" Ad Lib music format made by Cryo Interactive. This format was used in three DOS games: Dune, KGB, and MegaRace. Binarymaster at VGMFP's wiki started the initial reverse engineering work and I've taken up the bulk of the work for the past several months. We made some great strides and we now have the ability to add custom music to Dune and MegaRace. Ultimately, I would love to see AdPlug support the format and maybe even an instrument editor or music editor as I think the format is fairly flexible and easy to support.

For those who are interested in looking at how HERAD works, here's the wiki page where I've put all my documentation. http://www.vgmpf.com/Wiki/index.php?title=HERAD

Basically, HERAD's music notation is regular MIDI, however Pitch Bend events only use one byte (MSB only) and in MegaRace, the Note Off events have the velocity byte truncated. The instruments are formatted very similarly to Ad Lib's INS format, but there are exclusive macros that modify specific parameters of the instrument patch, such has modulator/carrier output levels, feedback scaling, automated pitch bends, etc. These macros are what give HERAD its expressive sound. MegaRace also introduces MIDI keymapping, which makes drum sequencing a lot easier.

I'm looking for additional help with reverse engineering the music. If anyone else that's well-versed in Ad Lib programming (especially Ad Lib Gold 1000), or if anyone is interested in looking at the disassembled music drivers, that would be greatly appreciated. The majority of the instrument data has been figured out but some of the Dune data that I'm running into doesn't make any sense, so any help would be appreciated.

Last but not least, here's an example of a custom music track in MegaRace. I wrote a short MIDI loop in FL Studio and hex edited a brand new HERAD music file together with that MIDI data. Then I compressed it in Cryo's HSQ format and injected the music data into the MEGARACE.DAT file and volia!

Reply 2 of 152, by synamax

User metadata
Rank Newbie
Rank
Newbie
Jepael wrote:

Hi, I believe I know enough about OPL chips and assembly to help, so do you have anything specific I should take a look at?

The last three bytes of every instrument are currently unknown. Dune uses these bytes but in MegaRace they're always the same for every instrument. Whenever I modify them, they seem to have no effect other than throwing in garbage data in the OPL register. I don't know if they have anything to do with Ad Lib Gold control settings or not.

I've attached the uncompressed Ad Lib, Ad Lib Gold, and Sound Blaster drivers for Dune and the Ad Lib and Sound Blaster drivers for MegaRace. Disassembling these files should hopefully give us a few answers.

Every instrument definition has a panning byte at 0x11 for the upper four bits at OPL register 0xC0. In MegaRace, all the instruments have this set to 3, which makes sense, as 3 is both speakers for OPL3. The Ad Lib Gold songs for Dune, seem to utilize this byte correctly as well, with 1 or 2 or 3 for either left speaker, right speaker or both speakers respectively. But the regular OPL2 songs for Dune have really weird values that don't make sense, like 2F or 73, so I'm wondering if they affect a different register altogether...or perhaps it's just garbage data.

Here's the links to the music from Dune and MegaRace:

Dune Music files
MegaRace Music files

Attachments

  • Filename
    drivers.7z
    File size
    6.25 KiB
    Downloads
    104 downloads
    File license
    Fair use/fair dealing exception

Reply 3 of 152, by Stretch

User metadata
Rank Member
Rank
Member

Can you convert the music into general midi?

PC1: Win98SE - Via Apollo Pro Mobo - Pentium II 233 - 256 MB - Voodoo 3 1000 - Acoustic Edge PSC706
PC2: Win98SE - ASRock 775i65G R3.0 - Celeron 2.2 GHz - 2048 MB - Geforce FX5700 - Audigy 2 ZS

Reply 4 of 152, by Jepael

User metadata
Rank Oldbie
Rank
Oldbie

The links to music contain .mid files.

MT-32 midi files could be closest to general midi, but the other files are just note data for device specific music drivers, so no other synth may play them correctly.

Reply 5 of 152, by synamax

User metadata
Rank Newbie
Rank
Newbie
Stretch wrote:

Can you convert the music into general midi?

Not general MIDI, but you can convert HERAD into MIDI, however you will lose the OPL instruments, so percussion instruments and sound effects are going to be played as regular melodic MIDI instruments.

Here's an example MIDI file that I made from converting the HERAD data in PAGA.SDB (Skyholder) to MIDI. This was all done manually, using a hex editor.

Reply 6 of 152, by synamax

User metadata
Rank Newbie
Rank
Newbie

Made some great progress today. I finally figured out two unknown values in the instrument chunk! The very last two bytes in each instrument definition seem to scale the output level for the modulator and carrier respectively whenever they're affected by aftertouch MIDI events. Aftertouch really isn't used a lot in HERAD so that's why I didn't make the connection until now.

I also figured out how one of the instrument macros works. One of the neat features of HERAD is that you can set up an automated pitch bend, which is very useful for drums and sound designing. One of the bytes (0x23) controls the range of the pitch bend. When it's set to 0, it just simply increments the FNUM up or down by 1. However, I didn't know what exactly it was doing when the byte was set to 1. It was definitely doing a coarser pitch bend but I didn't know exactly how it works...until now.

HERAD's MIDI Sequencer uses a table of frequency numbers (FNUMs) to assign each MIDI note in a octave to a corresponding frequency that the OPL2 can generate. For example: C = 343, C# = 348, D = 353, etc. Well, the coarse pitch bend macro works on the same principle, however it adds 4 extra FNUMs between each MIDI note, so instead of cycling through a 12-note scale, it goes through a 60-note scale with each note having it's own corresponding FNUM. I recorded all the FNUMs and put them in a handy table.

We're almost there in completely figuring out this format!

Reply 7 of 152, by synamax

User metadata
Rank Newbie
Rank
Newbie

Good news, bad news. The bad news is that I thought I figured out two unknown values last time I posted, but I later realized that I was incorrect and I only figured out one of them. As it turns out, the Aftertouch in HERAD affects the modulator output level only and does not affect carrier in any way. So that means we still have two unknown values remaining in the instrument chunk. I'm starting to believe more and more that these values are garbage or meta data that's ignored by the sequencer, but I'm still not entirely convinced by that.

The good news is that I did figure out a big difference between HERAD v1 (Dune/KGB) and HERAD v2 (MegaRace). In the instrument chunk, offset 0x22 controls the root note of an instrument, so for example, if you have a drum sound that you want to play at a particular pitch or you need your bass to be played an octave lower, you can set this byte to the pitch you want and the sequencer will play the instrument accordingly. Dune and KGB's MIDI sequencer will take any note that's thrown at it and change the pitch. If the note that needs to be changed is out of range on the MIDI keyboard, HERAD will just play note C1 instead. MegaRace's sequencer however introduces some very different functionality when it comes to handling the root note. From value ranges D1-FF to 01-30, the root note works exactly the same as Dune and KGB. However, from D0 to B9 and 31 to 90, the pitches cycle through again but this time, MIDI Note pitches are ignored, so any note that is triggered in the sequencer has it's pitch overridden by the root note control.

The value range 91-B8 produces the same functionality, except if a MIDI Control Change event occurs, no notes after it will be played until the song repeats. I don't see any MegaRace instruments that contain any values in this range so I think this behavior is a glitch and is not used.

I still need some help with disassembling the music driver. Please feel free if you want to contribute.

Reply 8 of 152, by synamax

User metadata
Rank Newbie
Rank
Newbie

Another unknown value for HERAD has been discovered! Remember that I stated in my previous post that Aftertouch only affects the modulator output level? That was incorrect and I was actually right the first time; the last byte does control the carrier output level via MIDI aftertouch events!

The reason why I didn't noticed this is because another offset was affecting the carrier output level. The 0x1F offset controls the output level via MIDI NoteOn velocity and if it is set to 0, no volume changes will occur, including aftertouch events! Once that byte was set to a valid value, I loaded my music test file into Dune's assets and sure enough, the aftertouch events affected the carrier output level!

This leaves just two remaining unknown offsets in HERAD's instrument chunk...the mysterious 0x25 offset and one of the values in the drum keymap.

Reply 9 of 152, by synamax

User metadata
Rank Newbie
Rank
Newbie

Believe it or not, I discovered that one of my previous instrument values that I had documented, had actually turned out to be another macro!

For the longest time, I thought offset 0x1B was a byte that was ignored, but it actually is used in Dune and KGB! It's scales the feedback based off of MIDI Aftertouch events. The reason why I didn't noticed this sooner is that I'm normally used to hacking with MegaRace, which doesn't use any MIDI aftertouch in any of the songs. On the other hand, Dune and KGB use aftertouch a fair amount, so once I started debugging Dune, I grabbed one of the instruments that I ripped from WORMSUIT.AGD. I noticed that the feedback was changing every time aftertouch events were playing, so sure enough, when I started modifying that byte at 0x1B, the causation was obvious.

On the other hand, the byte at 0x25 is still puzzling me. I even took the Ad Lib Gold version of WORMSUIT and deleted all the instrument definitions, with the exception of two instruments that happen to use that offset. Changing the value in either of the instruments did absolutely nothing. Nothing audibly noticeable happened nor any OPL register values changed. I'm really starting to think this is a meta value or something, but that being said, the fact that it's not used on a lot of instruments and that so far every byte before it has a purpose makes me question otherwise.

Reply 10 of 152, by DracoNihil

User metadata
Rank Oldbie
Rank
Oldbie

I wanted to ask, will this reverse engineering project lead to a possible loader for AdPlug? I'd like to beable to listen to the Dune files endlessly.

Steam Profile
YouTube Channel
Seal of Nehahra

Reply 11 of 152, by synamax

User metadata
Rank Newbie
Rank
Newbie
DracoNihil wrote:

I wanted to ask, will this reverse engineering project lead to a possible loader for AdPlug? I'd like to beable to listen to the Dune files endlessly.

That's what I'm hoping for! I don't know how to program so right now, we're relying on others to come up with code for a loader for AdPlug or even a DOS player much like RDOSPlay but with better playback and support for MegaRace. I would love to see an instrument editor and a program to convert MIDI to HERAD as well.

If anyone wants to help out with coding an AdPlug loader or standalone player, please feel free to chime in.

Reply 12 of 152, by Jepael

User metadata
Rank Oldbie
Rank
Oldbie
synamax wrote:
DracoNihil wrote:

I wanted to ask, will this reverse engineering project lead to a possible loader for AdPlug? I'd like to beable to listen to the Dune files endlessly.

That's what I'm hoping for! I don't know how to program so right now, we're relying on others to come up with code for a loader for AdPlug or even a DOS player much like RDOSPlay but with better playback and support for MegaRace. I would love to see an instrument editor and a program to convert MIDI to HERAD as well.

If anyone wants to help out with coding an AdPlug loader or standalone player, please feel free to chime in.

The last time I checked the MIDI playing code was already supporting three different MIDI based formats. Either it requires extending that or creating a new player code for just this format, because each format has their own "MIDI synth" code, meaning, how different MIDI events end up as OPL chip register writes.

Reply 13 of 152, by synamax

User metadata
Rank Newbie
Rank
Newbie
Jepael wrote:
synamax wrote:
DracoNihil wrote:

I wanted to ask, will this reverse engineering project lead to a possible loader for AdPlug? I'd like to beable to listen to the Dune files endlessly.

That's what I'm hoping for! I don't know how to program so right now, we're relying on others to come up with code for a loader for AdPlug or even a DOS player much like RDOSPlay but with better playback and support for MegaRace. I would love to see an instrument editor and a program to convert MIDI to HERAD as well.

If anyone wants to help out with coding an AdPlug loader or standalone player, please feel free to chime in.

The last time I checked the MIDI playing code was already supporting three different MIDI based formats. Either it requires extending that or creating a new player code for just this format, because each format has their own "MIDI synth" code, meaning, how different MIDI events end up as OPL chip register writes.

I think it requires writing new player code, since Pitch Bend events are three bytes long instead of MIDI's standard four and MegaRace's version of HERAD does this as well for Note Off, so we need to have a player that can support both versions.

Reply 14 of 152, by binarymaster

User metadata
Rank Newbie
Rank
Newbie

AdPlug's current MIDI replaying code is more GM oriented (e.g. multiple MIDI voices per channel vs. OPL voices), and it still doesn't support some events.

@synamax, you can create feature request issue for HERAD here:
https://github.com/adplug/adplug/issues

One interesting thing is the HERAD is already mentioned in TODO:
https://github.com/adplug/adplug/blob/14e57b8 … 4f251f7394/TODO

by Stas'M

Reply 15 of 152, by synamax

User metadata
Rank Newbie
Rank
Newbie

So, I posted the issue, but according to Malvineous there's no AdPlug developers, so someone else will have to code a patch.

binarymaster, I understand you're working on MIDIPLEX (which is an awesome program, I use it quite often). Do you think you'll be able to have support for both versions of HERAD implemented soon?

Reply 18 of 152, by binarymaster

User metadata
Rank Newbie
Rank
Newbie
synamax wrote:

Fantastic! I can't wait to try it out 😀

https://github.com/stascorp/MIDIPLEX/releases/tag/v0.4.2-pre

I've been made a pre-release build, you can try it out. Planning to add some more formats for v0.4.2.

by Stas'M

Reply 19 of 152, by synamax

User metadata
Rank Newbie
Rank
Newbie
binarymaster wrote:
synamax wrote:

Fantastic! I can't wait to try it out 😀

https://github.com/stascorp/MIDIPLEX/releases/tag/v0.4.2-pre

I've been made a pre-release build, you can try it out. Planning to add some more formats for v0.4.2.

Fantastic! I'm glad to see that the song speed is recognized correctly as well! MIDIPLEX works perfect as a HERAD to MIDI converter.

Would it be possible to implement the option of saving the music notation data in either HERAD version 1 or version 2? I also noticed that when converting a MIDI file to HERAD, the pitch bend events still have both the MSB and LSB bytes.

Excellent work so far, binarymaster!