I'm trying to implement MIDI reverb and chorus effects into UniPCemu's MPU(Using the SoundFont 2.04 format). For some reason, when enabling the effect(by removing/commenting out the break statement in the loop calling MIDIDEVICE_getsample), it sounds very weird when the effects are used.
Btw, I've noticed that it uses one low pass filter state for all four reverb&chorus streams(4 reverb times 4 chorus is 16 streams in total). This might add some noise to the sound, but it shouldn't have such a strange fm-modulation-like effect?
I think I've managed to fix the chorus/reverb effects(might be very heavy on the host CPU though, leading to slow processing and audio dropouts due to that):
I've improved the effects a bit, while also adding a buffer (actually some kind of timeline log) containing the different reverb/chorus's pitch bended state of the main channel at the different points in time(with ~0.06 seconds of reverb leading at an extra buffer of 20MB in total memory required for all 24 voices together to be logged(around 700KB for each voice), with channels having a quality of maximum 50kHz samplerate).
The main loop is at rows 332(applyMIDIlowpassfilter) and 254("modulationratio = MIDIDEVICE_chorussinf(chorusreverbreversesamplerate,chorus,0); //Pitch bend default!"), according to the profiler. They take 10% and 14-15% of CPU time, respectively. Anyone knows if there's a way to speed that up further? It's already using lookup tables as much as I could think of (MIDIDEVICE_chorussinf function), precalculated in MIDIDEVICE_generateSinusTable(). The low pass filter is mainly heavy due to it's calculations:
Also, it still seems to have little problems with applying volume using a factor? Some stuff sounds too loud and other instruments sound too soft? It mainly runs using the dB2factor and factor2dB for it's volume calculations: https://bitbucket.org/superfury/unipcemu/src/ … und.h?at=master
For some reason some of the channels gets (+)inf inside it's saved modulation information(voice->modulationratiosamples[chorus] entries 0-1 and at some point 0-4)? Anyone can explain why this is happening?
Although it's running faster now, I notice that songs using reverb pull the CPU emulation down from 100% (at 4.77MHz accuracy) to 80-90% with reverberation used only (playing Jazz Jackrabbit MIDI tracks using the MIDI player). Playing songs that use a combination of reverb and chorus even pull it down much further: down to up to 18% speed(huge Soundfont rendering latency). Anyone can see why this is so heavy? As usual the profiler tells me that the heavy lines are mostly the MIDIDEVICE_chorusinf macro and low pass filter being applied(lines 245&240(8.1% at 245, with 1.9% being at line 240) and line 318 being 2.6%). Is there any way to speed this up further?
The current rendering algorithm (for every of the 24 MIDI channels):
1//How many steps to keep! 2#define SINUSTABLE_PERCISION 3600 3#define SINUSTABLE_PERCISION_FLT 3600.0f 4#define SINUSTABLE_PERCISION_REVERSE (1.0f/SINUSTABLE_PERCISION_FLT) 5 6float sinustable[SINUSTABLE_PERCISION*2]; //10x percision steps of sinus! 7sword chorussinustable[SINUSTABLE_PERCISION*2][4][2]; //10x percision steps of sinus! With 1.0 added always! 8float sinustable_percision_reverse = 0.5f; //Reverse lookup! 9 10void MIDIDEVICE_generateSinusTable() 11{ 12 word x; 13 byte choruschannel; 14 for (x=0;x<NUMITEMS(sinustable);++x) 15 { 16 sinustable[x] = sinf((float)((x/SINUSTABLE_PERCISION_FLT))*360.0f); //Generate sinus lookup table! 17 sinustable[x+SINUSTABLE_PERCISION] = sinf((float)((x/SINUSTABLE_PERCISION_FLT))*360.0f); //Generate sinus lookup table! 18 for (choruschannel=0;choruschannel<4;++choruschannel) //All channels! 19 { 20 chorussinustable[x][choruschannel][0] = (sword)((sinf((float)((x/SINUSTABLE_PERCISION_FLT))*360.0f)+1.0f)*choruscents[choruschannel]); //Generate sinus lookup table, negative! 21 chorussinustable[x][choruschannel][1] = (sword)(chorussinustable[x][choruschannel][0]+1200.0f); //Generate sinus lookup table, with cents base added, negative! 22 chorussinustable[x+SINUSTABLE_PERCISION][choruschannel][0] = chorussinustable[x][choruschannel][0]; //Generate sinus lookup table, positive! 23 chorussinustable[x+SINUSTABLE_PERCISION][choruschannel][1] = chorussinustable[x][choruschannel][1]; //Generate sinus lookup table, with cents base added, positive! 24 } 25 } 26 sinustable_percision_reverse = SINUSTABLE_PERCISION_REVERSE; //Our percise value, reverse lookup! 27} 28 29//Absolute to get the amount of degrees, converted to a -0.0 to 2.0 scale! 30#define MIDIDEVICE_sinf(value) sinustable[(uint_32)(((fmodf(value,360.0f)*sinustable_percision_reverse)+1.0f)*SINUSTABLE_PERCISION_FLT)]; 31 32//Absolute to get the amount of degrees, converted to a -1.0 to 1.0 scale! 33#define MIDIDEVICE_chorussinf(value, choruschannel, add1200centsbase) chorussinustable[(uint_32)(((fmodf(value,360.0f)*sinustable_percision_reverse)+1.0f)*SINUSTABLE_PERCISION_FLT)][choruschannel][add1200centsbase] 34 35OPTINLINE static void MIDIDEVICE_getsample(int_32 *leftsample, int_32 *rightsample, int_64 play_counter, uint_32 totaldelay, float samplerate, sword samplespeedup, MIDIDEVICE_VOICE *voice, float Volume, float Modulation, byte chorus, byte reverb, float chorusreverbvol, float chorusreverbreversesamplerate, byte filterindex) //Get a sample from an MIDI note! 36{ 37 //Our current rendering routine: 38 INLINEREGISTER uint_32 temp; 39 INLINEREGISTER int_64 samplepos; 40 float lchannel, rchannel; //Both channels to use! 41 byte loopflags; //Flags used during looping! 42 static sword readsample = 0; //The sample retrieved! 43 sword modulationratiocents; 44 word speedupbuffer; 45 46 if (chorus==0) //Main channel? Log the current sample speedup! 47 { 48 writefifobuffer16(voice->effect_backtrace_samplespeedup,signed2unsigned16(samplespeedup)); //Log a history of this! 49 } 50 51 if (play_counter>=0) //Are we a running channel? 52 { 53 if (readfifobuffer16_backtrace(voice->effect_backtrace_samplespeedup,&speedupbuffer,(uint_32)totaldelay,(filterindex==(CHORUSREVERBSIZE-1))) && chorus) //Try to read from history! Only apply the value when not the originating channel! 54 { 55 samplespeedup = unsigned2signed16(speedupbuffer); //Apply the sample speedup from that point in time! Not for the originating channel! 56 } 57 } 58 59 samplepos = play_counter; //Load the current play counter! 60 if (voice->modenv_pitchfactor && (chorus==0)) //Gotten a modulation envelope to process to the pitch?
…Show last 252 lines
61 { 62 modulationratiocents = voice->modenv_pitchfactor; //Apply the pitch bend to the sample to retrieve! 63 } 64 else if (voice->modenv_pitchfactor && chorus) //Both? 65 { 66 modulationratiocents = MIDIDEVICE_chorussinf(chorusreverbreversesamplerate,chorus,0); //Pitch bend default! 67 modulationratiocents += voice->modenv_pitchfactor; //Apply pitch bend as well! This also adds the base of 1200 cents required to work! 68 } 69 else if (chorus) //Chorus only has modulation of the pitch as well? 70 { 71 modulationratiocents = MIDIDEVICE_chorussinf(chorusreverbreversesamplerate,chorus,1); //Current modulation ratio! 72 } 73 else 74 { 75 modulationratiocents = 1200; //No modulation by default! 76 } 77 78 //Apply pitch bend to the current factor too! 79 modulationratiocents += samplespeedup; //Speedup according to pitch bend! 80 modulationratiocents -= 1200; //Make us correct! 81 82 //Apply the new modulation ratio, if needed! 83 if (modulationratiocents!=voice->modulationratiocents[chorus]) //Different ratio? 84 { 85 voice->modulationratiocents[chorus] = modulationratiocents; //Update the last ratio! 86 voice->modulationratiosamples[chorus] = cents2samplesfactorf((float)modulationratiocents); //Calculate the pitch bend and modulation ratio to apply! 87 } 88 89 samplepos = (int_64)(samplepos*voice->modulationratiosamples[chorus]); //Apply the pitch bend to the sample to retrieve! 90 91 //Now, calculate the start offset to start looping! 92 samplepos += voice->startaddressoffset; //The start of the sample! 93 94 //First: apply looping! 95 loopflags = voice->currentloopflags; 96 if (voice->has_finallooppos && (play_counter >= voice->finallooppos)) //Executing final loop? 97 { 98 samplepos -= voice->finallooppos; //Take the relative offset to the start of the final loop! 99 samplepos += voice->finallooppos_playcounter; //Add the relative offset to the start of our data of the final loop! 100 } 101 else if (loopflags & 1) //Currently looping and active? 102 { 103 if (samplepos >= voice->endloopaddressoffset) //Past/at the end of the loop! 104 { 105 if ((loopflags & 0xC2) == 0x82) //We're depressed, depress action is allowed (not holding) and looping until depressed? 106 { 107 if (!voice->has_finallooppos) //No final loop position set yet? 108 { 109 voice->currentloopflags &= ~0x80; //Clear depress bit! 110 //Loop for the last time! 111 voice->finallooppos = samplepos; //Our new position for our final execution till the end! 112 voice->has_finallooppos = 1; //We have a final loop position set! 113 loopflags |= 0x20; //We're to update our final loop start! 114 } 115 } 116 117 //Loop according to loop data! 118 temp = voice->startloopaddressoffset; //The actual start of the loop! 119 //Loop the data! 120 samplepos -= temp; //Take the ammount past the start of the loop! 121 samplepos %= voice->loopsize; //Loop past startloop by endloop! 122 samplepos += temp; //The destination position within the loop! 123 //Check for depress special actions! 124 if (loopflags&0x20) //Extra information needed for the final loop? 125 { 126 voice->finallooppos_playcounter = samplepos; //The start position within the loop to use at this point in time! 127 } 128 } 129 } 130 131 //Next, apply finish! 132 loopflags = (samplepos >= voice->endaddressoffset) || (play_counter<0); //Expired or not started yet? 133 if (loopflags) //Sound is finished? 134 { 135 //No sample! 136 return; //Done! 137 } 138 139 if (getSFSample16(soundfont, (uint_32)samplepos, &readsample)) //Sample found? 140 { 141 lchannel = (float)readsample; //Convert to floating point for our calculations! 142 143 //First, apply filters and current envelope! 144 applyMIDILowpassFilter(voice, &lchannel, Modulation, filterindex); //Low pass filter! 145 lchannel *= Volume; //Apply ADSR Volume envelope! 146 lchannel *= voice->initialAttenuation; //The volume of the samples! 147 lchannel *= chorusreverbvol; //Apply chorus&reverb volume for this stream! 148 lchannel *= VOLUME; //Apply general volume! 149 //Now the sample is ready for output into the actual final volume! 150 151 rchannel = lchannel; //Load into both channels! 152 //Now, apply panning! 153 lchannel *= voice->lvolume; //Apply left panning, also according to the CC! 154 rchannel *= voice->rvolume; //Apply right panning, also according to the CC! 155 156 //Clip the samples to prevent overflow! 157 if (lchannel>SHRT_MAX) lchannel = (float)SHRT_MAX; 158 else if (lchannel<SHRT_MIN) lchannel = (float)SHRT_MIN; 159 if (rchannel>SHRT_MAX) rchannel = (float)SHRT_MAX; 160 else if (rchannel<SHRT_MIN) rchannel = (float)SHRT_MIN; 161 162 //Give the result! 163 *leftsample += (int_32)lchannel; //LChannel! 164 *rightsample += (int_32)rchannel; //RChannel! 165 } 166} 167 168byte MIDIDEVICE_renderer(void* buf, uint_32 length, byte stereo, void *userdata) //Sound output renderer! 169{ 170 const float MIDI_CHORUS_SINUS_BASE = 2.0f*(float)PI*CHORUS_LFO_FREQUENCY; //MIDI Sinus Base for chorus effects! 171#ifdef __HW_DISABLED 172 return 0; //We're disabled! 173#endif 174 if (!stereo) return 0; //Can't handle non-stereo output! 175 //Initialisation info 176 float pitchcents, currentsamplespeedup, lvolume, rvolume, panningtemp; 177 INLINEREGISTER float VolumeEnvelope=0; //Current volume envelope data! 178 INLINEREGISTER float ModulationEnvelope=0; //Current modulation envelope data! 179 //Initialised values! 180 MIDIDEVICE_VOICE *voice = (MIDIDEVICE_VOICE *)userdata; 181 sample_stereo_t* ubuf = (sample_stereo_t *)buf; //Our sample buffer! 182 ADSR *VolumeADSR = &voice->VolumeEnvelope; //Our used volume envelope ADSR! 183 ADSR *ModulationADSR = &voice->ModulationEnvelope; //Our used modulation envelope ADSR! 184 MIDIDEVICE_CHANNEL *channel = voice->channel; //Get the channel to use! 185 uint_32 numsamples = length; //How many samples to buffer! 186 ++numsamples; //Take one sample more! 187 byte currentchorusreverb; //Current chorus and reverb levels we're processing! 188 int_64 chorusreverbsamplepos; 189 190 #ifdef MIDI_LOCKSTART 191 //lock(voice->locknumber); //Lock us! 192 #endif 193 194 if (voice->VolumeEnvelope.active==0) //Simple check! 195 { 196 #ifdef MIDI_LOCKSTART 197 //unlock(voice->locknumber); //Lock us! 198 #endif 199 return SOUNDHANDLER_RESULT_NOTFILLED; //Empty buffer: we're unused! 200 } 201 if (memprotect(soundfont,sizeof(*soundfont),"RIFF_FILE")!=soundfont) 202 { 203 #ifdef MIDI_LOCKSTART 204 //unlock(voice->locknumber); //Lock us! 205 #endif 206 return SOUNDHANDLER_RESULT_NOTFILLED; //Empty buffer: we're unable to render anything! 207 } 208 if (!soundfont) 209 { 210 #ifdef MIDI_LOCKSTART 211 //unlock(voice->locknumber); //Lock us! 212 #endif 213 return SOUNDHANDLER_RESULT_NOTFILLED; //The same! 214 } 215 if (!channel) //Unknown channel? 216 { 217 #ifdef MIDI_LOCKSTART 218 //unlock(voice->locknumber); //Lock us! 219 #endif 220 return SOUNDHANDLER_RESULT_NOTFILLED; //The same! 221 } 222 223 224 #ifdef MIDI_LOCKSTART 225 lock(voice->locknumber); //Actually check! 226 #endif 227 228 if (voice->VolumeEnvelope.active == 0) //Not active after all? 229 { 230 #ifdef MIDI_LOCKSTART 231 unlock(voice->locknumber); 232 #endif 233 return SOUNDHANDLER_RESULT_NOTFILLED; //Empty buffer: we're unused! 234 } 235 //Calculate the pitch bend speedup! 236 pitchcents = (float)(channel->pitch%0x1FFF); //Load active pitch bend (unsigned), Only low 14 bits are used! 237 pitchcents -= (float)0x2000; //Convert to a signed value! 238 pitchcents /= 128.0f; //Create a value between -1 and 1! 239 pitchcents *= cents2samplesfactorf(voice->pitchwheelmod*pitchcents); //Influence by pitch wheel! 240 241 //Now apply to the default speedup! 242 currentsamplespeedup = voice->initsamplespeedup; //Load the default sample speedup for our tone! 243 currentsamplespeedup *= cents2samplesfactorf(pitchcents); //Calculate the sample speedup!; //Apply pitch bend! 244 voice->effectivesamplespeedup = (sword)currentsamplespeedup; //Load the speedup of the samples we need! 245 246 //Determine panning! 247 lvolume = rvolume = 0.5f; //Default to 50% each (center)! 248 panningtemp = voice->initpanning; //Get the panning specified! 249 panningtemp += voice->panningmod*((float)(voice->channel->panposition-0x2000)/128); //Apply panning CC! 250 lvolume -= panningtemp; //Left percentage! 251 rvolume += panningtemp; //Right percentage! 252 voice->lvolume = lvolume; //Left panning! 253 voice->rvolume = rvolume; //Right panning! 254 255 if (voice->request_off) //Requested turn off? 256 { 257 voice->currentloopflags |= 0x80; //Request quit looping if needed: finish sound! 258 } //Requested off? 259 260 //Apply sustain 261 voice->currentloopflags &= ~0x40; //Sustain disabled by default! 262 voice->currentloopflags |= (channel->sustain << 6); //Sustaining? 263 264 VolumeEnvelope = voice->CurrentVolumeEnvelope; //Make sure we don't clear! 265 ModulationEnvelope = voice->CurrentModulationEnvelope; //Make sure we don't clear! 266 267 int_32 lchannel, rchannel; //Left&right samples, big enough for all chorus and reverb to be applied! 268 269 float samplerate = (float)LE32(voice->sample.dwSampleRate); //The samplerate we use! 270 271 float chorusreverbreversesamplerate, playcounterfltchorusreverbreversesamplerate; 272 chorusreverbreversesamplerate = MIDI_CHORUS_SINUS_BASE*(1.0f/samplerate); //To multiple to divide by the sample rate(conversion from sample to seconds) 273 274 byte chorus,reverb; 275 uint_32 totaldelay; 276 277 //Now produce the sound itself! 278 for (; --numsamples;) //Produce the samples! 279 { 280 lchannel = 0; //Reset left channel! 281 rchannel = 0; //Reset right channel! 282 playcounterfltchorusreverbreversesamplerate = (float)voice->play_counter*chorusreverbreversesamplerate; //Preconvert the play counter to floating point! 283 for (currentchorusreverb=0;currentchorusreverb<CHORUSREVERBSIZE;++currentchorusreverb) //Process all reverb&chorus used(4 chorus channels within 4 reverb channels)! 284 { 285 chorusreverbsamplepos = voice->play_counter; //Load the current play counter! 286 totaldelay = voice->totaldelay[currentchorusreverb]; //Load the total delay! 287 chorusreverbsamplepos -= (int_64)totaldelay; //Apply specified chorus&reverb delay! 288 VolumeEnvelope = ADSR_tick(VolumeADSR,chorusreverbsamplepos,((voice->currentloopflags & 0xC0) != 0x80),voice->note->noteon_velocity, voice->note->noteoff_velocity); //Apply Volume Envelope! 289 chorus = (currentchorusreverb&0x3); //Current chorus channel! 290 reverb = (currentchorusreverb>>2); //Current reverb channel! 291 ModulationEnvelope = ADSR_tick(ModulationADSR,chorusreverbsamplepos,((voice->currentloopflags & 0xC0) != 0x80),voice->note->noteon_velocity, voice->note->noteoff_velocity); //Apply Modulation Envelope! 292 MIDIDEVICE_getsample(&lchannel,&rchannel, chorusreverbsamplepos, totaldelay, samplerate, voice->effectivesamplespeedup, voice, VolumeEnvelope, ModulationEnvelope, chorus, reverb,voice->chorusreverbvol[currentchorusreverb],playcounterfltchorusreverbreversesamplerate, currentchorusreverb); //Get the sample from the MIDI device! 293 } 294 //Clip the samples to prevent overflow! 295 if (lchannel>SHRT_MAX) lchannel = SHRT_MAX; 296 if (lchannel<SHRT_MIN) lchannel = SHRT_MIN; 297 if (rchannel>SHRT_MAX) rchannel = SHRT_MAX; 298 if (rchannel<SHRT_MIN) rchannel = SHRT_MIN; 299 ubuf->l = lchannel; //Left sample! 300 ubuf->r = rchannel; //Right sample! 301 ++voice->play_counter; //Next sample! 302 ++ubuf; //Prepare for the next sample! 303 } 304 305 voice->CurrentVolumeEnvelope = VolumeEnvelope; //Current volume envelope updated! 306 voice->CurrentModulationEnvelope = ModulationEnvelope; //Current volume envelope updated! 307 308 #ifdef MIDI_LOCKSTART 309 unlock(voice->locknumber); //Lock us! 310 #endif 311 return SOUNDHANDLER_RESULT_FILLED; //We're filled! 312}
Used FIFO reading back pitch bend data(16-bit value):
1byte readfifobuffer16_backtrace(FIFOBUFFER *buffer, word *result, uint_32 backtrace, byte finalbacktrace) 2{ 3 uint_32 readposhistory; 4 if (__HW_DISABLED) return 0; //Abort! 5 if (buffer==0) return 0; //Error: invalid buffer! 6 if (buffer->buffer==0) return 0; //Error invalid: buffer! 7 if (allcleared) return 0; //Abort: invalid buffer! 8 9 fifobuffer_save(buffer); //Save the current status for restoring, if needed! 10 11 if (buffer->lock) 12 { 13 WaitSem(buffer->lock) 14 if (fifobuffer_INTERNAL_freesize(buffer)<((buffer->size-1)-(backtrace<<1))) //Filled enough? 15 { 16 readposhistory = (int_64)buffer->readpos; //Save the read position! 17 readposhistory -= (int_64)(backtrace<<1); //Trace this far back! 18 for (;readposhistory<0;) //Invalid? 19 { 20 readposhistory += buffer->size; //Convert into valid range! 21 } 22 readposhistory = SAFEMOD(readposhistory,buffer->size); //Make sure we don't get past the end of the buffer! 23 buffer->readpos = readposhistory; //Patch the read position to the required state! 24 readfifobuffer16unlocked(buffer,result); //Read the FIFO buffer without lock! 25 fifobuffer_restore(buffer); //Restore the saved state, we haven't changed yet! 26 if (finalbacktrace) //Finished? 27 { 28 readfifobuffer16unlocked(buffer,result); //Read the FIFO buffer without lock normally! 29 } 30 PostSem(buffer->lock) 31 return 1; //Read! 32 } 33 PostSem(buffer->lock) 34 } 35 else 36 { 37 if (fifobuffer_INTERNAL_freesize(buffer)<((buffer->size-1)-(backtrace<<1))) //Filled enough? 38 { 39 readposhistory = (int_64)buffer->readpos; //Save the read position! 40 readposhistory -= (int_64)(backtrace<<1); //Trace this far back! 41 for (;readposhistory<0;) //Invalid? 42 { 43 readposhistory += buffer->size; //Convert into valid range! 44 } 45 readposhistory = SAFEMOD(readposhistory,buffer->size); //Make sure we don't get past the end of the buffer! 46 buffer->readpos = readposhistory; //Patch the read position to the required state! 47 readfifobuffer16unlocked(buffer,result); //Read the FIFO buffer without lock! 48 fifobuffer_restore(buffer); //Restore the saved state, we haven't changed yet! 49 if (finalbacktrace) //Finished? 50 { 51 readfifobuffer16unlocked(buffer,result); //Read the FIFO buffer without lock normally! 52 } 53 return 1; //Read! 54 } 55 } 56 return 0; //Nothing to read! 57}
For some reason, enabling this rendering corrupts the CPU emulation in some way. Anyone can see what's going wrong here?
Edit: Disabling the readfifobuffer16_backtrace by making it return 0 always(adding a "return 0;" statement at it's start) makes Prince of Persia continue, but with a buggy screen instead of crashing at the point the flames are displayed.
Edit: Disabling the chorus/reverb effects and the readfifobuffer16_backtrace functions at the same time (making them always return 0) doesn't fix the CPU bugging out. So the problem shouldn't be at the renderer, but instead in something that has to do with MIDI(maybe the new voice allocation?)