Thank you so much, Staticblast! I'll definitely give those settings a shot. Yeah, Cryo games act weird with fast CPUs, for example the speed of the opponent cars in MegaRace are dependent on the clock speed.
Awesome, thanks for the help on the elusive Surround chip, Jepael! To answer your question, 0xDE is only found in two of the songs, WORMSUIT.AGD and MORNING.AGD. The rest of the songs use the value 0xD6 instead which is still too large for the YM7128.
As for the routine for where it's read, I have no idea where to find that. I wish I knew how to read assembly so I can disassemble the driver files. The only thing I can make out is at the very end of the file where the driver calls the 388 port for the Adlib card.
OK the first byte is written to gold control register 8. Basically it selects muting, stereo modes (using TDA8425 audio processor) and how mixer (which in this context I presume TDA8425 output) channels are routed to left and right outputs.
I also confirmed the rest 31 bytes are just dumped to surround chip.
The difference is, 0xDE selects "spatial stereo" and 0xD6 selects "pseudo stereo".
OK the first byte is written to gold control register 8. Basically it selects muting, stereo modes (using TDA8425 audio processor) and how mixer (which in this context I presume TDA8425 output) channels are routed to left and right outputs.
I also confirmed the rest 31 bytes are just dumped to surround chip.
The difference is, 0xDE selects "spatial stereo" and 0xD6 selects "pseudo stereo".
This is fantastic!! The entire header for HERAD is now completely solved; thanks so much, Jepael!
Just for clarification, are the YM7128 register values that I posted earlier in the correct endian or do they have to be rearranged?
Ok, now we need to disassemble the drivers to reconstruct into a readable C code (or it could be easier to write from scratch?), and write new AdPlug replayer. 😀
Ok, now we need to disassemble the drivers to reconstruct into a readable C code (or it could be easier to write from scratch?), and write new AdPlug replayer. 😀
That's one option. I already have some kind of understanding of it, as I thought you people were already disassembling it, I just took a look at the reverb chip part.
Does it have to be readable? I once wrote a player for some game music, by doing the exactly same operation in C than assembly. OK, so things like "MOV AX,BX" would just be easy as "AX=BX;" with variables but the code flow will be a pain to get right, as on many occasions there are a mixture of calls, jumps etc with many points of entry. And jump tables based on music data.
Something I have thought is that why rewrite it at all? The binary exists and plays music files, and the music files exist. A lightweight x86 CPU emulator could load and execute the binary driver, and let the emulator catch OPL chip writes. I think some player already used this kind of approach, but it's more complex. Doesn't most SID players work like this as well, run a minimal memory image dump of the game playing music? It would be fine for a standalone player to include a PC emulator (CPU+necessary hardware like timers), but maybe not for AdPlug. OK, not so universal, and it would still need some stubs to actually call the driver. Downside is the complexity, as someone will want 386/pmode support at some point, so anyway a full blown core would be needed from some other emulator project. And distributing game files is illegal anyway, so one must legally own the game for having music files and player code, but distributing a .VGM file or somesuch to reproduce the music, I don't really know.
The binary exists and plays music files, and the music files exist. A lightweight x86 CPU emulator could load and execute the binary driver, and let the emulator catch OPL chip writes. I think some player already used this kind of approach, but it's more complex.
And distributing game files is illegal anyway, so one must legally own the game for having music files and player code, but distributing a .VGM file or somesuch to reproduce the music, I don't really know.
Something I have thought is that why rewrite it at all?
There are major reasons to rewrite it:
1. Ability to compile the playback code on different platforms and architectures (that's why I like AdPlug 🙄 )
2. Relative easy integration with other projects (emulators, games, players, etc.)
3. Better understanding of the format and the way of interpretation
4. Exclusive satisfaction writing your own new code that will replicate the functionality of the great player written many years ago by clever programmers (it's like cloning a mammoth - we have an ability and we have desire! 😁 ), plus an exciting excursion into AdLib chip programming
5. Sharing your code with the community of AdLib programmers (this direction is really being popular nowadays)
6. Help all those guys who searched around the Internet for the HERAD solution many times (me, SynaMax, and many others)
There is an inexpressible feeling of happiness when you're done something great like that. 😊 I believe SynaMax got the same feeling when he's done reversing the most part of the format, and also when released the entirely new music using it. I got that feeling recently when I almost completed an implementation of MUS / IMS / MDI replayer for AdPlug, and I'm enjoying music in these formats played back with my own custom build of AdPlug Winamp plugin right now. 🤣
Well... definitely an emotional buffer overflow in my post ^^)
For what it's worth, here is the source code to my old, never-released, work-in-progress, FM player for DOS, written for DJGPP (32 bit C for DOS). It does play Dune's OPL2 and OPL3 files, or as close as I could get them back then. I won't have the time to do much with it anymore, but maybe it can be useful for your efforts.
Last edited by NewRisingSun on 2017-02-11, 18:22. Edited 1 time in total.
The binary exists and plays music files, and the music files exist. A lightweight x86 CPU emulator could load and execute the binary driver, and let the emulator catch OPL chip writes. I think some player already used this kind of approach, but it's more complex.
To my knowledge, no one has really looked into the disassembly of the drivers from Dune or MegaRace. Looking at this particular program, from what I can tell, it looks like the guy just copied the uncompressed driver from the game and place it inside a simple program. I believe RDOSPlay is implemented pretty much the same way.
binarymaster wrote:There are major reasons to rewrite it:
1. Ability to compile the playback code on different platforms and architectures (that's […] Show full quote
Jepael wrote:
Something I have thought is that why rewrite it at all?
There are major reasons to rewrite it:
1. Ability to compile the playback code on different platforms and architectures (that's why I like AdPlug 🙄 )
2. Relative easy integration with other projects (emulators, games, players, etc.)
3. Better understanding of the format and the way of interpretation
4. Exclusive satisfaction writing your own new code that will replicate the functionality of the great player written many years ago by clever programmers (it's like cloning a mammoth - we have an ability and we have desire! 😁 ), plus an exciting excursion into AdLib chip programming
5. Sharing your code with the community of AdLib programmers (this direction is really being popular nowadays)
6. Help all those guys who searched around the Internet for the HERAD solution many times (me, SynaMax, and many others)
I also agree, I think a rewrite will be beneficial as we can implement other features that other players don't have, like a seek bar for playback, muting channels, instrument preview, etc. We understand most of the format and how it works, the only real questions that remain is how HERAD uses the Adlib timers, and how HERAD knows how many FNUMs to change when executing a MIDI pitchbend.
It also occured to me, another benefit to a rewrite is we can possibly introduce a new HERAD format. Basically attach a new header to the file so that detecting the different versions between HERAD 1 and 2 would be much easier and we can attach info like title and artist name.
binarymaster wrote:
There is an inexpressible feeling of happiness when you're done something great like that. 😊 I believe SynaMax got the same feeling when he's done reversing the most part of the format, and also when released the entirely new music using it.
You're absolutely right about that. Reverse engineering the format was a lot of hard work, but it was so exciting and rewarding to finally understand the format. Writing new music for the format was also a lot of fun, it's crazy to hear something I wrote and hear it play in MegaRace, 🤣.
NewRisingSun wrote:
For what it's worth, here is the source code to my old, never-released, work-in-progress, FM player for DOS, written for DJGPP (32 bit C for DOS). It does play Dune's OPL2 and OPL3 files, or as close as I could get them back then. I won't have the time to do much with it anymore, but maybe it can be useful for your efforts.
Thanks so much for sharing! I'm so glad to see the mystery byte 0x25 from the instrument chunk is listed as "dummy" in the code. I always wanted to have someone else make sure that value was unused.
To my knowledge, no one has really looked into the disassembly of the drivers from Dune or MegaRace. Looking at this particular program, from what I can tell, it looks like the guy just copied the uncompressed driver from the game and place it inside a simple program. I believe RDOSPlay is implemented pretty much the same way.
Yeah that would be the easiest way on a DOS machine to get the original files played back, or even stored as some other playable format (DRO/IMF/RAW or other register dump format). I even tried that with the SB driver, but writing the program with free tools like NASM wasn't exactly pleasant and calling the driver to load the music data hanged dosbox, so I never got around to get back to it. Oh well, calling driver card init succeeded though..
synamax wrote:
We understand most of the format and how it works, the only real questions that remain is how HERAD uses the Adlib timers, and how HERAD knows how many FNUMs to change when executing a MIDI pitchbend.
I don't think it uses Adlib timers at all. Currently I would say the game code is supposed to call the driver play routine, perhaps in a timer interrupt. But for the exact timing, that would require analysing the game binary instead.
synamax wrote:
It also occured to me, another benefit to a rewrite is we can possibly introduce a new HERAD format. Basically attach a new header to the file so that detecting the different versions between HERAD 1 and 2 would be much easier and we can attach info like title and artist name.
Can you briefly explain how to differentiate between those files if there is no knowledge about the version?
I like your idea, but I think people just want to play the original files from the game, instead of downloading new files with extra header/footer for indicating version and other metadata. Would a separate file with extension like .heradinfo suffice? Maybe it would make more sense if there was a program to generate songs in these formats. As there are only small amount of these files, maybe these can be identified by just file extension, file name (though the game has no names stored for the files so the names are arbitrary), or with a MD5 hash or something.
I think people just want to play the original files from the game, instead of downloading new files with extra header/footer for indicating version and other metadata. Would a separate file with extension like .heradinfo suffice?
Yep, that's right. Original files are compressed: Dune and MegaRace uses HSQ compression, KGB uses SQX compression (it is similar to HSQ, but have some differences). According to the header structures of compressed files it's also possible to detect actual format, so I think we don't need any special file extensions.
We understand most of the format and how it works, the only real questions that remain is how HERAD uses the Adlib timers, and how HERAD knows how many FNUMs to change when executing a MIDI pitchbend.
I don't think it uses Adlib timers at all. Currently I would say the game code is supposed to call the driver play routine, perhaps in a timer interrupt. But for the exact timing, that would require analysing the game binary instead.
Gotcha, that makes sense. A while back I made a test song that was simply 4 quarter notes per measure played at speed 0x1. Looking at the DRO file, I saw that each quarter note lasted for 120ms - this is how I figured out that the song speed value set to 1 is 500bpm (60,000 ms (1 minute) / Tempo (BPM) = Delay Time in ms for quarter-note beats, so 60,000ms / 500bpm = 120ms) After a while, the delays got more messy in the DRO file as the song progressed so after several seconds, two delays started to pop up, a 119ms delay and a 1ms delay following it. It does this for a while until goes back to 120ms delays...it often goes back and forth like this. That's why the tempo that I calculate (500bpm / HERAD song speed value = Tempo) doesn't exactly matches up to actual Adlib playback, thanks to the weird way how the sound card handles timers.
binarymaster wrote:It's quite simple, see this comment:
https://github.com/adplug/adplug/issues/39#is … mment-252757465
- the instruments structure […] Show full quote
Jepael wrote:
synamax wrote:
It also occured to me, another benefit to a rewrite is we can possibly introduce a new HERAD format. Basically attach a new header to the file so that detecting the different versions between HERAD 1 and 2 would be much easier and we can attach info like title and artist name.
Can you briefly explain how to differentiate between those files if there is no knowledge about the version?
Yes, binarymaster's program MIDIPLEX uses the instrument detection method for determining HERAD versions. The first byte in each instrument is the "mode" of the instrument, in Dune and KGB songs, it's 0x0, but in MegaRace it's either 0x1 or 0xFF.
That being said, I just recently discovered two Adlib Gold songs from Dune that break this rule. WORMSUIT.AGD and WORMINTR.AGD have some instruments that start off with a value of 0x4. I have no idea if this changes anything in the game yet, but I do know that WORMINTR doesn't load at all in MIDIPLEX because of this. Also, since Dune's regular OPL2 driver treats this first byte like a dummy value (same for MegaRace, as long as it's not 0xFF), I feel this isn't the best way to detect versions.
Ultimately, I think Jepael came up with a better way.
Jepael wrote:
I like your idea, but I think people just want to play the original files from the game, instead of downloading new files with extra header/footer for indicating version and other metadata. Would a separate file with extension like .heradinfo suffice? Maybe it would make more sense if there was a program to generate songs in these formats. As there are only small amount of these files, maybe these can be identified by just file extension, file name (though the game has no names stored for the files so the names are arbitrary), or with a MD5 hash or something.
I like that, I think new file extensions would work really nicely. Dune and KGB wouldn't have to change, but MegaRace would have to be something different...thank goodness there's only 6 songs. MegaRace's music files aren't as widely distributed as Dune and KGB's song files, so that shouldn't cause too much confusion.
Since new songs are now a reality (thanks to MIDIPLEX), perhaps all new songs from here on out should have new extensions, like *.HRD, or *.HR2. That way, when I'm looking at new song files, I can immediately tell which HERAD version that song is in. That should help make version detection easier too.
One last thing, I figured something out in the beginning section of Cryo's Adlib driver.
Gotcha, that makes sense. A while back I made a test song that was simply 4 quarter notes per measure played at speed 0x1. Looking at the DRO file, I saw that each quarter note lasted for 120ms - this is how I figured out that the song speed value set to 1 is 500bpm (60,000 ms (1 minute) / Tempo (BPM) = Delay Time in ms for quarter-note beats, so 60,000ms / 500bpm = 120ms) After a while, the delays got more messy in the DRO file as the song progressed so after several seconds, two delays started to pop up, a 119ms delay and a 1ms delay following it. It does this for a while until goes back to 120ms delays...it often goes back and forth like this. That's why the tempo that I calculate (500bpm / HERAD song speed value = Tempo) doesn't exactly matches up to actual Adlib playback, thanks to the weird way how the sound card handles timers.
Well, there's two things that explain that.
DRO time granularity is 1ms, so for example, if a piece of code writes a port once every 0.1 ms, it will group the writes into DRO file as ten writes, wait 1ms, ten writes, wait 1ms.
Another thing is that the game's timer may not be exactly a multiple of 1ms, and it takes a finite amount of time to write the data so sometimes it just happens that DRO 1ms timer ticks while sound driver writes a bunch of data to card, so some amount is written before and some amount after it. Think of it like DRO has exact 1ms timing, but game has maybe 119.99 ms timing, or something like that.
I figured out the game has some timer rate setting code to set it at about 200Hz, but that is just the default.
I hate to ask but can someone provide me a standard MIDI file of ARRAKIS from Dune? (The song that plays in your home palace) I'm making my own cover of the song and there's a part to it I don't understand and need to look at it visually.
“I am the dragon without a name…”
― Κυνικός Δράκων
I hate to ask but can someone provide me a standard MIDI file of ARRAKIS from Dune? (The song that plays in your home palace) I'm making my own cover of the song and there's a part to it I don't understand and need to look at it visually.
No problem at all. I'll upload it later this evening.
Here ya go! I attached the MIDI files from both the MT-32 version and the Adlib verison. The Adlib Gold version is exactly the same as the Adlib OPL2 (with the exception of Surround Module data, which is not needed). Thanks to binarymaster for ripping the MT-32 midi file!
Jepael wrote:Well, there's two things that explain that. […] Show full quote
synamax wrote:
Gotcha, that makes sense. A while back I made a test song that was simply 4 quarter notes per measure played at speed 0x1. Looking at the DRO file, I saw that each quarter note lasted for 120ms - this is how I figured out that the song speed value set to 1 is 500bpm (60,000 ms (1 minute) / Tempo (BPM) = Delay Time in ms for quarter-note beats, so 60,000ms / 500bpm = 120ms) After a while, the delays got more messy in the DRO file as the song progressed so after several seconds, two delays started to pop up, a 119ms delay and a 1ms delay following it. It does this for a while until goes back to 120ms delays...it often goes back and forth like this. That's why the tempo that I calculate (500bpm / HERAD song speed value = Tempo) doesn't exactly matches up to actual Adlib playback, thanks to the weird way how the sound card handles timers.
Well, there's two things that explain that.
DRO time granularity is 1ms, so for example, if a piece of code writes a port once every 0.1 ms, it will group the writes into DRO file as ten writes, wait 1ms, ten writes, wait 1ms.
Another thing is that the game's timer may not be exactly a multiple of 1ms, and it takes a finite amount of time to write the data so sometimes it just happens that DRO 1ms timer ticks while sound driver writes a bunch of data to card, so some amount is written before and some amount after it. Think of it like DRO has exact 1ms timing, but game has maybe 119.99 ms timing, or something like that.
I figured out the game has some timer rate setting code to set it at about 200Hz, but that is just the default.
Ahh, I forgot about the time granularity for DRO. I heard that's why some people like to stick with VGM as that format runs at 44100 samples per second.
04 = Decreases output level by every 1 note
03 = Decreases output level by every 2 notes
02 = Decreases output level by every 4 notes
01 = Decreases output level by every 8 notes
FF = Increases output level by every 8 notes
FE = Increases output level by every 4 notes
FD = Increases output level by every 2 notes
FC = Increases output level by every 1 note
Of course, 3F is the quietest value for the OPL2/3 and 00 is the loudest.
