Dune "HERAD" Ad Lib Music Hacking

Getting old DOS games working. (DOSBox topics belong in DOSBox areas below, not here).

Dune "HERAD" Ad Lib Music Hacking

Postby synamax » 2016-9-07 @ 05:41

Hey guys,

I'm currently working on reverse engineering the "HERAD" Ad Lib music format made by Cryo Interactive. This format was used in three DOS games: Dune, KGB, and MegaRace. Binarymaster at VGMFP's wiki started the initial reverse engineering work and I've taken up the bulk of the work for the past several months. We made some great strides and we now have the ability to add custom music to Dune and MegaRace. Ultimately, I would love to see AdPlug support the format and maybe even an instrument editor or music editor as I think the format is fairly flexible and easy to support.

For those who are interested in looking at how HERAD works, here's the wiki page where I've put all my documentation. http://www.vgmpf.com/Wiki/index.php?title=HERAD

Basically, HERAD's music notation is regular MIDI, however Pitch Bend events only use one byte (MSB only) and in MegaRace, the Note Off events have the velocity byte truncated. The instruments are formatted very similarly to Ad Lib's INS format, but there are exclusive macros that modify specific parameters of the instrument patch, such has modulator/carrier output levels, feedback scaling, automated pitch bends, etc. These macros are what give HERAD its expressive sound. MegaRace also introduces MIDI keymapping, which makes drum sequencing a lot easier.

I'm looking for additional help with reverse engineering the music. If anyone else that's well-versed in Ad Lib programming (especially Ad Lib Gold 1000), or if anyone is interested in looking at the disassembled music drivers, that would be greatly appreciated. The majority of the instrument data has been figured out but some of the Dune data that I'm running into doesn't make any sense, so any help would be appreciated.

Last but not least, here's an example of a custom music track in MegaRace. I wrote a short MIDI loop in FL Studio and hex edited a brand new HERAD music file together with that MIDI data. Then I compressed it in Cryo's HSQ format and injected the music data into the MEGARACE.DAT file and volia!
User avatar
synamax
Newbie
 
Posts: 69
Joined: 2016-9-07 @ 04:57

Re: Dune "HERAD" Ad Lib Music Hacking

Postby Jepael » 2016-9-07 @ 19:12

Hi, I believe I know enough about OPL chips and assembly to help, so do you have anything specific I should take a look at?
Jepael
Oldbie
 
Posts: 1182
Joined: 2005-6-15 @ 19:28
Location: Finland

Re: Dune "HERAD" Ad Lib Music Hacking

Postby synamax » 2016-9-08 @ 03:28

Jepael wrote:Hi, I believe I know enough about OPL chips and assembly to help, so do you have anything specific I should take a look at?


The last three bytes of every instrument are currently unknown. Dune uses these bytes but in MegaRace they're always the same for every instrument. Whenever I modify them, they seem to have no effect other than throwing in garbage data in the OPL register. I don't know if they have anything to do with Ad Lib Gold control settings or not.

I've attached the uncompressed Ad Lib, Ad Lib Gold, and Sound Blaster drivers for Dune and the Ad Lib and Sound Blaster drivers for MegaRace. Disassembling these files should hopefully give us a few answers.

Every instrument definition has a panning byte at 0x11 for the upper four bits at OPL register 0xC0. In MegaRace, all the instruments have this set to 3, which makes sense, as 3 is both speakers for OPL3. The Ad Lib Gold songs for Dune, seem to utilize this byte correctly as well, with 1 or 2 or 3 for either left speaker, right speaker or both speakers respectively. But the regular OPL2 songs for Dune have really weird values that don't make sense, like 2F or 73, so I'm wondering if they affect a different register altogether...or perhaps it's just garbage data.

Here's the links to the music from Dune and MegaRace:

Dune Music files
MegaRace Music files
Attachments
drivers.7z
(6.25 KiB) Downloaded 35 times
User avatar
synamax
Newbie
 
Posts: 69
Joined: 2016-9-07 @ 04:57

Re: Dune "HERAD" Ad Lib Music Hacking

Postby Stretch » 2016-9-09 @ 11:16

Can you convert the music into general midi?
Win98SE - Intel SE440BX-2 - Pentium II 233 Mhz - 256 MB System Ram - Voodoo 3 1000 AGP - Yamaha YMF-724 PCI - Soundblaster 32 ISA
Stretch
Member
 
Posts: 256
Joined: 2013-11-16 @ 00:57

Re: Dune "HERAD" Ad Lib Music Hacking

Postby Jepael » 2016-9-09 @ 16:29

The links to music contain .mid files.

MT-32 midi files could be closest to general midi, but the other files are just note data for device specific music drivers, so no other synth may play them correctly.
Jepael
Oldbie
 
Posts: 1182
Joined: 2005-6-15 @ 19:28
Location: Finland

Re: Dune "HERAD" Ad Lib Music Hacking

Postby synamax » 2016-9-10 @ 00:34

Stretch wrote:Can you convert the music into general midi?


Not general MIDI, but you can convert HERAD into MIDI, however you will lose the OPL instruments, so percussion instruments and sound effects are going to be played as regular melodic MIDI instruments.

Here's an example MIDI file that I made from converting the HERAD data in PAGA.SDB (Skyholder) to MIDI. This was all done manually, using a hex editor.
User avatar
synamax
Newbie
 
Posts: 69
Joined: 2016-9-07 @ 04:57

Re: Dune "HERAD" Ad Lib Music Hacking

Postby synamax » 2016-9-12 @ 02:34

Made some great progress today. I finally figured out two unknown values in the instrument chunk! The very last two bytes in each instrument definition seem to scale the output level for the modulator and carrier respectively whenever they're affected by aftertouch MIDI events. Aftertouch really isn't used a lot in HERAD so that's why I didn't make the connection until now.

I also figured out how one of the instrument macros works. One of the neat features of HERAD is that you can set up an automated pitch bend, which is very useful for drums and sound designing. One of the bytes (0x23) controls the range of the pitch bend. When it's set to 0, it just simply increments the FNUM up or down by 1. However, I didn't know what exactly it was doing when the byte was set to 1. It was definitely doing a coarser pitch bend but I didn't know exactly how it works...until now.

HERAD's MIDI Sequencer uses a table of frequency numbers (FNUMs) to assign each MIDI note in a octave to a corresponding frequency that the OPL2 can generate. For example: C = 343, C# = 348, D = 353, etc. Well, the coarse pitch bend macro works on the same principle, however it adds 4 extra FNUMs between each MIDI note, so instead of cycling through a 12-note scale, it goes through a 60-note scale with each note having it's own corresponding FNUM. I recorded all the FNUMs and put them in a handy table.

We're almost there in completely figuring out this format!
User avatar
synamax
Newbie
 
Posts: 69
Joined: 2016-9-07 @ 04:57

Re: Dune "HERAD" Ad Lib Music Hacking

Postby synamax » 2016-9-21 @ 04:04

Good news, bad news. The bad news is that I thought I figured out two unknown values last time I posted, but I later realized that I was incorrect and I only figured out one of them. As it turns out, the Aftertouch in HERAD affects the modulator output level only and does not affect carrier in any way. So that means we still have two unknown values remaining in the instrument chunk. I'm starting to believe more and more that these values are garbage or meta data that's ignored by the sequencer, but I'm still not entirely convinced by that.

The good news is that I did figure out a big difference between HERAD v1 (Dune/KGB) and HERAD v2 (MegaRace). In the instrument chunk, offset 0x22 controls the root note of an instrument, so for example, if you have a drum sound that you want to play at a particular pitch or you need your bass to be played an octave lower, you can set this byte to the pitch you want and the sequencer will play the instrument accordingly. Dune and KGB's MIDI sequencer will take any note that's thrown at it and change the pitch. If the note that needs to be changed is out of range on the MIDI keyboard, HERAD will just play note C1 instead. MegaRace's sequencer however introduces some very different functionality when it comes to handling the root note. From value ranges D1-FF to 01-30, the root note works exactly the same as Dune and KGB. However, from D0 to B9 and 31 to 90, the pitches cycle through again but this time, MIDI Note pitches are ignored, so any note that is triggered in the sequencer has it's pitch overridden by the root note control.

The value range 91-B8 produces the same functionality, except if a MIDI Control Change event occurs, no notes after it will be played until the song repeats. I don't see any MegaRace instruments that contain any values in this range so I think this behavior is a glitch and is not used.

I still need some help with disassembling the music driver. Please feel free if you want to contribute.
User avatar
synamax
Newbie
 
Posts: 69
Joined: 2016-9-07 @ 04:57

Re: Dune "HERAD" Ad Lib Music Hacking

Postby synamax » 2016-9-24 @ 20:07

Another unknown value for HERAD has been discovered! Remember that I stated in my previous post that Aftertouch only affects the modulator output level? That was incorrect and I was actually right the first time; the last byte does control the carrier output level via MIDI aftertouch events!

The reason why I didn't noticed this is because another offset was affecting the carrier output level. The 0x1F offset controls the output level via MIDI NoteOn velocity and if it is set to 0, no volume changes will occur, including aftertouch events! Once that byte was set to a valid value, I loaded my music test file into Dune's assets and sure enough, the aftertouch events affected the carrier output level!

This leaves just two remaining unknown offsets in HERAD's instrument chunk...the mysterious 0x25 offset and one of the values in the drum keymap.
User avatar
synamax
Newbie
 
Posts: 69
Joined: 2016-9-07 @ 04:57

Re: Dune "HERAD" Ad Lib Music Hacking

Postby synamax » 2016-10-01 @ 05:14

Believe it or not, I discovered that one of my previous instrument values that I had documented, had actually turned out to be another macro!

For the longest time, I thought offset 0x1B was a byte that was ignored, but it actually is used in Dune and KGB! It's scales the feedback based off of MIDI Aftertouch events. The reason why I didn't noticed this sooner is that I'm normally used to hacking with MegaRace, which doesn't use any MIDI aftertouch in any of the songs. On the other hand, Dune and KGB use aftertouch a fair amount, so once I started debugging Dune, I grabbed one of the instruments that I ripped from WORMSUIT.AGD. I noticed that the feedback was changing every time aftertouch events were playing, so sure enough, when I started modifying that byte at 0x1B, the causation was obvious.

On the other hand, the byte at 0x25 is still puzzling me. I even took the Ad Lib Gold version of WORMSUIT and deleted all the instrument definitions, with the exception of two instruments that happen to use that offset. Changing the value in either of the instruments did absolutely nothing. Nothing audibly noticeable happened nor any OPL register values changed. I'm really starting to think this is a meta value or something, but that being said, the fact that it's not used on a lot of instruments and that so far every byte before it has a purpose makes me question otherwise.
User avatar
synamax
Newbie
 
Posts: 69
Joined: 2016-9-07 @ 04:57

Re: Dune "HERAD" Ad Lib Music Hacking

Postby DracoNihil » 2016-10-04 @ 11:08

I wanted to ask, will this reverse engineering project lead to a possible loader for AdPlug? I'd like to beable to listen to the Dune files endlessly.
User avatar
DracoNihil
Oldbie
 
Posts: 1117
Joined: 2011-10-09 @ 14:26
Location: Independence, KS

Re: Dune "HERAD" Ad Lib Music Hacking

Postby synamax » 2016-10-04 @ 13:04

DracoNihil wrote:I wanted to ask, will this reverse engineering project lead to a possible loader for AdPlug? I'd like to beable to listen to the Dune files endlessly.


That's what I'm hoping for! I don't know how to program so right now, we're relying on others to come up with code for a loader for AdPlug or even a DOS player much like RDOSPlay but with better playback and support for MegaRace. I would love to see an instrument editor and a program to convert MIDI to HERAD as well.

If anyone wants to help out with coding an AdPlug loader or standalone player, please feel free to chime in.
User avatar
synamax
Newbie
 
Posts: 69
Joined: 2016-9-07 @ 04:57

Re: Dune "HERAD" Ad Lib Music Hacking

Postby Jepael » 2016-10-04 @ 18:25

synamax wrote:
DracoNihil wrote:I wanted to ask, will this reverse engineering project lead to a possible loader for AdPlug? I'd like to beable to listen to the Dune files endlessly.


That's what I'm hoping for! I don't know how to program so right now, we're relying on others to come up with code for a loader for AdPlug or even a DOS player much like RDOSPlay but with better playback and support for MegaRace. I would love to see an instrument editor and a program to convert MIDI to HERAD as well.

If anyone wants to help out with coding an AdPlug loader or standalone player, please feel free to chime in.


The last time I checked the MIDI playing code was already supporting three different MIDI based formats. Either it requires extending that or creating a new player code for just this format, because each format has their own "MIDI synth" code, meaning, how different MIDI events end up as OPL chip register writes.
Jepael
Oldbie
 
Posts: 1182
Joined: 2005-6-15 @ 19:28
Location: Finland

Re: Dune "HERAD" Ad Lib Music Hacking

Postby synamax » 2016-10-04 @ 20:22

Jepael wrote:
synamax wrote:
DracoNihil wrote:I wanted to ask, will this reverse engineering project lead to a possible loader for AdPlug? I'd like to beable to listen to the Dune files endlessly.


That's what I'm hoping for! I don't know how to program so right now, we're relying on others to come up with code for a loader for AdPlug or even a DOS player much like RDOSPlay but with better playback and support for MegaRace. I would love to see an instrument editor and a program to convert MIDI to HERAD as well.

If anyone wants to help out with coding an AdPlug loader or standalone player, please feel free to chime in.


The last time I checked the MIDI playing code was already supporting three different MIDI based formats. Either it requires extending that or creating a new player code for just this format, because each format has their own "MIDI synth" code, meaning, how different MIDI events end up as OPL chip register writes.


I think it requires writing new player code, since Pitch Bend events are three bytes long instead of MIDI's standard four and MegaRace's version of HERAD does this as well for Note Off, so we need to have a player that can support both versions.
User avatar
synamax
Newbie
 
Posts: 69
Joined: 2016-9-07 @ 04:57

Re: Dune "HERAD" Ad Lib Music Hacking

Postby binarymaster » 2016-10-07 @ 07:28

AdPlug's current MIDI replaying code is more GM oriented (e.g. multiple MIDI voices per channel vs. OPL voices), and it still doesn't support some events.

@synamax, you can create feature request issue for HERAD here:
https://github.com/adplug/adplug/issues

One interesting thing is the HERAD is already mentioned in TODO:
https://github.com/adplug/adplug/blob/1 ... f7394/TODO
by Stas'M
User avatar
binarymaster
Newbie
 
Posts: 54
Joined: 2010-3-07 @ 19:03

Re: Dune "HERAD" Ad Lib Music Hacking

Postby synamax » 2016-10-08 @ 00:38

So, I posted the issue, but according to Malvineous there's no AdPlug developers, so someone else will have to code a patch.

binarymaster, I understand you're working on MIDIPLEX (which is an awesome program, I use it quite often). Do you think you'll be able to have support for both versions of HERAD implemented soon?
User avatar
synamax
Newbie
 
Posts: 69
Joined: 2016-9-07 @ 04:57

Re: Dune "HERAD" Ad Lib Music Hacking

Postby binarymaster » 2016-10-09 @ 05:34

I'm already on the way to implement it.
by Stas'M
User avatar
binarymaster
Newbie
 
Posts: 54
Joined: 2010-3-07 @ 19:03

Re: Dune "HERAD" Ad Lib Music Hacking

Postby synamax » 2016-10-09 @ 13:20

Fantastic! I can't wait to try it out :)
User avatar
synamax
Newbie
 
Posts: 69
Joined: 2016-9-07 @ 04:57

Re: Dune "HERAD" Ad Lib Music Hacking

Postby binarymaster » 2016-10-09 @ 22:06

synamax wrote:Fantastic! I can't wait to try it out :)

https://github.com/stascorp/MIDIPLEX/re ... v0.4.2-pre

I've been made a pre-release build, you can try it out. Planning to add some more formats for v0.4.2.
by Stas'M
User avatar
binarymaster
Newbie
 
Posts: 54
Joined: 2010-3-07 @ 19:03

Re: Dune "HERAD" Ad Lib Music Hacking

Postby synamax » 2016-10-10 @ 04:21

binarymaster wrote:
synamax wrote:Fantastic! I can't wait to try it out :)

https://github.com/stascorp/MIDIPLEX/re ... v0.4.2-pre

I've been made a pre-release build, you can try it out. Planning to add some more formats for v0.4.2.


Fantastic! I'm glad to see that the song speed is recognized correctly as well! MIDIPLEX works perfect as a HERAD to MIDI converter.

Would it be possible to implement the option of saving the music notation data in either HERAD version 1 or version 2? I also noticed that when converting a MIDI file to HERAD, the pitch bend events still have both the MSB and LSB bytes.

Excellent work so far, binarymaster!
User avatar
synamax
Newbie
 
Posts: 69
Joined: 2016-9-07 @ 04:57

Next

Return to DOS

Who is online

Users browsing this forum: linkdex [Bot] and 1 guest