The waveform roms are 8 bit. So even if SC55 uses 12 bit of sample, the additional 4 bits (to 12 bits) doesn't come from the same rom chip.
maybe, I am misunderstanding you, but one possible explanation for that is what I mentioned few times before, i.e. how 12-bit data are encoded in 8-bit inside Yamaha TG100 Waverom: two 12-bit samples are stored as 3 bytes (24 bits) and you need to unpack them to 4 bytes (two 16-bit samples) in order to play as PCM data:
until recently I was thinking the exact same encoding applies for VSC bank, but I was wrong - in any way, i know now the proper encoding for VSC and need to finally finish and release my tool for extracting playable and proper PCM data from decrypted VSC bank, as I promised about 1 month ago here:
Please, could you speed me up with your analysis with telling me the size of "TRAIN" tone from SC55 Waverom and give me the first 32 bytes (or so) of its encoded/encrypted PCM data bytes (reordered with proper address order). thanks!
BTW, just as a side note: the structure(s) with size of 0xd8 defines the tones - it's read in function GetToneNames() from VSC.sys. Also, there is function AssignVariation() that set the tone to GM2 or do GM to GS tone, etc.There is a lot that can be learned from looking at disassembled code of VSC in Windows what the synth engine is doing.
mattwwrote on 2020-11-25, 18:20:maybe, I am misunderstanding you, but one possible explanation for that is what I mentioned few times before, i.e. how 12-bit da […] Show full quote
The waveform roms are 8 bit. So even if SC55 uses 12 bit of sample, the additional 4 bits (to 12 bits) doesn't come from the same rom chip.
maybe, I am misunderstanding you, but one possible explanation for that is what I mentioned few times before, i.e. how 12-bit data are encoded in 8-bit inside Yamaha TG100 Waverom: two 12-bit samples are stored as 3 bytes (24 bits) and you need to unpack them to 4 bytes (two 16-bit samples) in order to play as PCM data:
until recently I was thinking the exact same encoding applies for VSC bank, but I was wrong - in any way, i know now the proper encoding for VSC and need to finally finish and release my tool for extracting playable and proper PCM data from decrypted VSC bank, as I promised about 1 month ago here:
Hmmm... now i am totaly confused. NewRisingSun wrote:
"The SC-55 wave ROM uses 8.25 bits per sample: 8 bits of mantissa for every sample, plus 4 bits of base 2 exponent (i.e. left-shift count) for every 16 samples. The encountered maximum shift count is ten, making the decompressed wave ROM samples 18 bits wide, even in the original SC-55. Mantissa and exponent are in the same ROM chip."
So i will try both, the link with 12To16 and NewRisingSun's explanation. I asume, that the MSB four bits of the "middle byte" belogs to first byte data, and the LSB 4 bits of the middle byte belogs to the 3-th byte.
I will try this on the adress-decrypted SC55 Waveform-Rom.
1. 19-bit address line scrambling
2. in each data byte the bits of its value are shuffled in order "1,2,7,3,5,0,4,6"
3. PCM data are encoded according to the algorithm in function decode_sample() here: https://git.redump.net/mame/tree/src/devices/ … d/rolandpcm.cpp
* in case of VSC we have:
1. and 2. are according to the algorithms in my code here: Re: About Roland Virtual Sound Canvas 3
3. as I mentioned in my first post to you, I haven't released that yet, but I know the correct PCM encoding now. So, all 3 points are known for VSC.
* in case of SC55 Waverom:
1. 20-bit address line scrambling, at least based on @NewRisingSun code
2. as far as I understand @NewRisingSun post - still unknown
3. without 2. nothing really can be said about 3.
so, I think you are misunderstanding point 2. is found for SC55/SC88 and you're trying to achieve point 3.
@mattw, thank you so much for these further li nks and informations. I 100 % agree to your statements. But I have build a rompler synth based on SC55 sf2. with an UC3A512 Microcontroller. And what can i say about it, is, that things are very time-critical, especialy in therms of number of voices. You do not have much time, to put up to 24 voices into the mix (plus lookUp normalization), fetching samples and in case of no root-key linear interpolation.
In case of SC55 the data-fetching and the mix is done by IC24, (custom PCM Chip) an ordinary FPGA. I cannot belive, that data is scrambled too 🙁 What can I imagine is - as you already meansured - that data into the FPGA is internal is shuffeld too. It will be hard to figure out, very hard.
yeah, but even weaker hardware like CM-32P that we know for sure how it works (MAME has full emulation of it) is fast enough to handle both address and data (de)scrambling. However, if @NewRisingSun is right, then data scrambling for SC55 is 20-bit compared to 19-bit for CM-32P, but that's very small difference in terms of performance, if there is really any.
that data into the FPGA is internal is shuffeld too. It will be hard to figure out, very hard.
actually, data value byte shuffling is easier than address line scrambling, because it's only 8 bits, i.e. 8 positions and then it's only 40320 possible permutations. You can brute-force that in seconds (maybe even milliseconds) if you know 3 (or more) consecutive bytes scrambled and their descrambled value. I did test that technique with CM-32P, but there it's easy, because unscrambled data contain the "CM32P" string.
I wanted to attempt such brute-force on SC55 and that's why I asked this:
but currently SC55Waverom data-address-line descrambled with @NewRisingSun code looks still too random at least to me - for example it starts with so many almost consecutive to each other 0x59 values - I just cannot imagine PCM data structured like that, even if we know how to descramble 0x59. Anyway, I will wait answer from @kitrinx and/or @NewRisingSun, because it's possible I misunderstood something from their analysis.
Last edited by mattw on 2020-11-25, 23:09. Edited 2 times in total.
you are absolutely right, yes it would be faculty of 8! but there are still 4 bits in addition to the 12 bits. What still is a "black hole" is the order of the 3 bytes that you menasured.
if we assume, that bytes are hard-wired scrambled, than the "middle byte" (2 nibbles) are also scrambled in the same bit-order. And all these facts do not make it easier 😀
mattwwrote on 2020-11-25, 22:38:yeah, but even weaker hardware like CM-32P that we know for sure how it works (MAME has full emulation of it) is fast enough to […] Show full quote
yeah, but even weaker hardware like CM-32P that we know for sure how it works (MAME has full emulation of it) is fast enough to handle both address and data (de)scrambling. However, if @NewRisingSun is right, then data scrambling for SC55 is 20-bit compared to 19-bit for CM-32P, but that's very small difference in terms of performance, if there is really any.
that data into the FPGA is internal is shuffeld too. It will be hard to figure out, very hard.
actually, data value byte shuffling is easier than address line scrambling, because it's only 8 bits, i.e. 8 positions and then it's only 40320 possible permutations. You can brute-force that in seconds (maybe even milliseconds) if you know 3 (or more) consecutive bytes scrambled and their descrambled value. I did test that technique with CM-32P, but there it's easy, because unscrambled data contain the "CM32P" string.
I wanted to attempt such brute-force on SC55 and that's why I asked this:
but currently SC55Waverom data-address-line descrambled with @NewRisingSun code looks still too random at least to me - for example it starts with so many almost consecutive to each other 0x59 values - I just cannot imagine PCM data structures like that, even if we know how to descramble 0x59. Anyway, I will wait answer from @kitrinx and/or @NewRisingSun, because it's possible I misunderstood something from their analysis.
i took also a look, and puted into audacity, and no, no recocgnizable sound, but that doesn't mean anything. It would be a big step, if adress-line descrambeling would be correct at least
i took also a look, and puted into audacity, and no, no recocgnizable sound, but that doesn't mean anything. It would be a big step, if adress-line descrambeling would be correct at least
figure it out, because I was not able to find the descrambling code in "cm-32_p__1.0.0.am27c512.7d.ic9" ROM. CM-32P uses Intel P8098, which essentially is Intel MCS-96 (80196) compatible Microcontroller. I looked at the assembler code, but was not able to find where the descrambling is done. It was much easier to me, doing the same for VSC. that's why it's really interesting to hear the whole story from "Valley Bell", but there is no email or anything for contact in the MAME code - I guess he doesn't want to be bothered exactly with such questions.
We have already finished decrypting and decompressing the SC-55 wave ROM data, and are now in the process of interpreting the instrument parameters from the control ROM and translating them into corresponding SoundFont generators. At least to the extent that this is possible -- the SoundFont specification lacks some of the capabilities that would be necessary for an accurate conversion. To wit: five instead of four envelope periods, looping modes other than forward, Attack ramping that is convex in amplitude rather than decibels, actually ignoring Note Off events for drum notes instead of clumsily trying to approximate that with Release period durations, and we have not even come to the LFO and filter parts.
This code snippet will decrypt and decompress the three MiB of wave ROM data from both the SC-55 and SC-55mkII.
1SC55Wave::SC55Wave (const uint8_t* encryptedWaveROMs) { 2 // Decrypt the encrypted wave ROMs by re-ordering both address and data bits. The first 32 bytes of each wave ROM are not encrypted. 3 // Expect the encrypted wave ROMs as an array of 3 MiB of data, each wave ROM's data directly following the previous wave ROM's. 4 std::vector<int8_t> decryptedWaveROMs(0x300000); 5 for (uint32_t encryptedAddress =0; encryptedAddress <0x300000; encryptedAddress++) { 6 if ((encryptedAddress &0xFFFFF) <0x20) 7 decryptedWaveROMs[encryptedAddress] =encryptedWaveROMs[encryptedAddress]; 8 else { 9 static const uint8_t dataOrder [ 8] ={ 2, 0, 4, 5, 7, 6, 3, 1}; 10 static const uint8_t addressOrder [22] ={ 2, 0, 3, 4, 1, 9, 13, 10, 18, 17, 6, 15, 11, 16, 8, 5, 12, 7, 14, 19, 20, 21 }; 11 12 uint8_t decryptedData =0; 13 uint32_t decryptedAddress =0; 14 for (uint8_t bit =0; bit < 8; bit++) decryptedData |=(encryptedWaveROMs[encryptedAddress] >> dataOrder[bit] &1) <<bit; 15 for (uint8_t bit =0; bit <22; bit++) decryptedAddress |=( encryptedAddress >>addressOrder[bit] &1) <<bit; 16 17 decryptedWaveROMs[decryptedAddress] =decryptedData; 18 } 19 } 20 21 // Decompress the decrypted wave ROMs to yield 18-bit sample data. 22 // Structure of each (decrypted) 1 MiB wave ROM: 23 // - Header from $000000-$0003FF; 24 // - Two packed 4-bit base-2 exponents (i.e. left-shift counts) per byte from $00400 to $7FFF; 25 // - 8-bit mantissa bytes from $08000 to $FFFFF. 26 // The packed exponent byte address for each mantissa byte is simply the mantissa byte's address right-shifted by five bits, so that one exponent byte applies to 32 mantissa bytes. 27 // Within those 32 mantissa bytes, the exponent byte's lower nibble applies to the first 16, the higher nibble to the second 16 mantissa bytes. 28 std::vector<int16_t> waveData; 29 for (uint32_t address =0; address <0x300000; address++) waveData.push_back(decryptedWaveROMs[address] <<(decryptedWaveROMs[address >>5 &0x007FFF | address &0x300000] >>(address &0x10? 4: 0) &0x0F) >>2); 30}
mattw wrote:
that's why it's really interesting to hear the whole story from "Valley Bell", but there is no email or anything for contact in the MAME code - I guess he doesn't want to be bothered exactly with such questions.
from where did you jump to adress $1DED0 ? From where did you know, that in your given example the pareameter adress of the second piano 1 sample is $1DED0?
Just by searching for the bytes from Cloudschatze's spreadsheet in the control ROM.
Amazing work! So, it's 22-bit address line scrambling, not like your initial/previous code that had only 20-bit address line scrambling. In any way big kudos and thanks for sharing how you analyzed that based on the logic analyzer logs.
What you think about we make SC55/SC55MKII to VSC sound-bank conversion? maybe @kitrinx will be interested to help me with that, because of his already comprehensive analyses of decrypted VSC bank:
actually, maybe SC55/MKII WaveROM to VSC soundbank is pointless, because VSC lacks resolution or if it doesn't at least currently we don't know how to enable it. however, Yamaha S-YXG50 supports both 16-bit resolution and GS. Fortunate, Yamaha S-YXG50 was fully hacked, but Unfortunately all the information is in Russian speaking forums (if you don't speak Russian, then it's very hard to understand with Google Translate) and it's not complete - I mean steps and details are missing, but as I mentioned even in my initial post here:
that is how they made even Yamaha TyRUS emulator based on patching Yamaha S-YXG50 and its soundbank. I am not sure how OK is to share any files regarding that here, but one key file to start with, is "yamaha_syxg50_tbl_exploded.7z" and how to enable 16-bit resolution support for Yamaha S-YXG50 was explained in another series of posts in Russian speaking forums - I spent hours with Google Translate, but I still didn't fully understand it. In any way making our own tools for Yamaha S-YXG50 is definitely of interest for the community.
nothing really new (at this point) in this post - i've just ported @NewRisingSun code to C and use the same style as the code for CM-32P and MT-32. So, just to save time to anyone else interested in that, the source code is attached.
[EDIT] on 2nd glance at my own port and because I forgot bits 20 and 21 of the address line scrambling in the macro definition - it's obvious they are not needed. So, in reality it's 20-bit scrambling. with that note made to prevent confusion, the corrected code attached as well - you can compare them to see what I mean - basically with and without bit 20 and 21 it's the same output. So, they are not needed really needed. both source code generate the same output and it's more syntax, not semantics correction.
Hi mattw, great work, thank you so much, but NewRisingSun is right when he pointed out, the work is far from done. Next steps would be to uncover SC55 Control-ROM wave-form coefficients, as well as parameters for the LFO (frequency, pitch) and others. Detuning, chorus and other params must yet found in Control ROM , to get an accurate sound of this old beast 😀
Last edited by Stiletto on 2020-11-27, 23:20. Edited 1 time in total.
i.e. tool that exacts each sound from decrypted VSC soundbank to playable WAV file and because I previously stated several times here wrongly the sounds are 12-bit encoded, just to tell the whole story behind it and how at the end I double confirmed the correct encoding with even "forensic" evidence.
So, in WAV specification "8-bit data is unsigned ", but in VSC soundbank data are stored as signed. That means making playable WAV file from decrypted VSC soundbank is as simple as convert "signed char" to "unsigned char" that is as simple in C as doing:
1data[i] ^ 0x80
what bothered me though initially that the result is not byte-by-byte the same as Microsoft Roland-licensed sounds in GM8.DLS (please, note that GM.DLS in WinXP is the 16-bit equivalent of the 8-bit GM8.DLS).
So, here my forensic investigation started - it turned out Microsoft (or Roland before gave the sounds to Microsoft) used "Cool Edit Pro 2.0" (or older version, 2.0 was the oldest version I found and used during my investigation) to convert Roland Sounds from signed to unsigned - that leaves unmistakable and unique signature, because "Cool Edit" internally is using 32-bit floating point processing, which leads to that signature rounding error. So, here is the definitive proof - take a sound from decrypted VSC soundbank - for example "TRAIN" ( 7 183 bytes in size), which I used during my investigation and then do:
1A) TRAIN sound from Decrypted VSC soundbank --> load the raw data from it in "Cool Edit Pro 2.0" --> safe to WAV 2B) GM8.DLS --> extract TRAIN sound, it's already in WAV format
Sound data as result of "A" and "B" are the same, byte-by-byte identical!! So, to convert sound from Decrypted VSC soundbank to WAV without any rounding error it's enough to do:
1data[i] ^ 0x80
I was very excited about the above finding, because forensically it gives the connection between the sounds in Microsoft GM8.DLS and those in VSC soundbank!
[EDIT] the above also allows to convert VSC soundbank to DLS bank, i.e. how to essentially make GM8.DLS without rounding error using decrypted VSC soundbank as data source. another tool, if someone is interested, can easily do using the information in this forum thread.
Guys, I've just run the port of @NewRisingSun code I made and posted few posts above on the SC88 Waveroms - it almost works! It seems data value bit scrambling is the same, but address line scrambling is a little different. So, SC55 descrambled Waverom includes date:
1990-09-12
and currently with SC88 Waverom and SC55 code I get:
1-913.1.903-
I still cannot figure out based on that how to fix address line descrambling for SC88, but definitely SC88 is very very close. I guess the year for SC88 is either 1991 or 1993.
[EDIT] attached that code for people who want to experiment and fix the address line descrambling for SC88
I still cannot figure out based on that how to fix address line descrambling for SC88, but definitely SC88 is very very close. I guess the year for SC88 is either 1991 or 1993.
