FreddyV wrote on 2021-02-28, 12:53:Hi, […]
Show full quote
Hi,
Exactly, I am worried about the memory left for the samples and partition, even if Mod Master compress the partition, with 512Kb of RAM, Mod Master can'ty load all the samples if the file is 300Kb.
I started to write this code in 1993, then I did not write it with "Drivers" we can simply add or remove.
The Pascal part is doing the SoundCard init, and the assembly the sound output for example.
The Music replay play all the format with an internal file format. (Can't be splitted as well)
I have the project to do pascal library to use the Sierra SCI drivers, to do a Benchmark and test software.
I would like to work on game graphic code optimization, like having Sierra games or Prince of Persia faster....
The, if I have graphic code with the SCI video driver, why not doing a Graphical "Demo" of Mod Master.
I also had the idea to put the current Mod Master coide in the Mod Master 2.3 interface (Much more advanced, it is in VGA) but small motivation for it.
Mod MAster XT code changed a lot compared to Mod Master 2.3 (Mod Master XT is a modification of Mod Master 2.2)
Here's some ideas to tackle the memory / samples issue, I'm sure you have thought and maybe even tried to or implemented a few of these already, but I'll write down here what I can come up with:
* 1: You could add support for EMS (expanded memory)
The LIM 4.0 EMS specification supports up to 32MB of RAM, and is fully supported on 8086+ systems...
* 2: Resample each sample during loading to lower sample rate
During the loading of the MOD file and the samples therein,
you could downsample the samples to fit into the available memory.
Eg, if you have say 400Kb of memory available for samples, and you're loading a mod file with 650kb of samples,
for each sample, multiply the KHz of the original sample by the factor of 0.6153 (400 / 650) and resample the sample during loading, and you'll end up loading all the samples into memory with a total size of 399.945Kb...
It does reduce the quality of the samples, and you'll end up with samples of different KHz (dunno if your engine supports this), but, at least you'll be able to play the whole MOD file in slightly lower quality...
If your engine doesn't support having different KHz for individual samples, you could also resample them so that they are all the same KHz, but reduced so they all fit into memory...
You could vary the amount of resampling to a lower sample rate with some weights based on the sample in question:
Samples with low frequencies, like a drum kick or a low bass line, can be downsampled a little bit more than samples that are of medium or high frequencies...
You could create a non-linear scale like a logarithmic scale that is used during the analysis stage to determine the resampling rates for each sample prior to loading all samples, but after you've made an inventory off all the samples and their
sample rates and their mean fundamental frequency (eg low values are like a drum kick, a bass line etc... / medium for leads, pads, voices, etc... / high for cymbals, hi-hats etc...). You just calculate the "mean fundamental frequency" of the
sample in question, which is a very simple and fast algorithm. You don't need to calculate the mean fundamental frequency with all the sample-values of the sample in question, but just calculate it with getting a few thousand points spread
equally over the length of the sample, so it only takes a very short time to get a decent mean fundamental frequency, and you can also at the same time get the mean volume from this data for use as described below...
Finally, you could optimize this a bit more by adding some weights based on the number of times the sample is used in the song and the volume of the sample (eg if it's only used as a faint sound in the background of the sound-scape) with
samples that are used a lot and/or do not have low volumes getting a bit more KHz than the others, all via a logarithmic scale...
* 3: Make a dynamic sample loading/unloading engine
Before loading the samples, make an inventory of them, their sizes, KHz, and when they are used on the timeline of the song, and you analyze this information and you build an array of what samples are used where on the timeline of the song,
and you load/unload samples dynamically while the song plays, you could create a separate thread or add some code to your audio engine loop that does this while the song is playing.
It's a bit like Bethesda games, like Fallout 3/4 and The Elder Scrolls series, where the terrain and 3d models/textures are loaded dynamically based on your location on the world map and are loading in/out of memory as you move on the world
map.
This won't always work, but, if you can combine it with the point above (resampling the samples to lower rates during loading), you can end up with a hybrid system, where prior to loading the samples, you do your analysis, and you end up with 2 groups of samples:
* resident samples: These are used continuously throughout the song and should not be loaded/unloaded during playback. You load all these samples into memory prior to playback, but you use the "Resample each sample during loading to lower sample rate" technique to load them, with an intelligent algorithm using the non-linear scale and the mean fundamental frequencies of the samples.
* dynamic samples: These samples are the remaining samples, they are not used frequently, and, at all times during the song playback, there is enough time between playing the sample and playing the sample the next time, to unload and load it into memory between those 2 points. These samples are, of course, not loaded before starting playback, and are loaded/unloaded during playback in a reserved memory area with a size that has been calculated by the algorithm I just described, based on the total size of all the samples in the MOD, the amount of available memory for samples, the amount of memory used for the "resident samples" that are loaded using the sample-rate reduction algorithm, etc...
* 4: Recode your current Turbo Pascal User Interface code into C or C++
Not only is compiled, optimized C or C++ faster than Turbo Pascal, it also makes much smaller code,
as it doesn't add all the run-time pascal stuff into the executable, eg, it's linked to dynamic/shared libraries that have their own memory space,
so you end up with a tiny executable and a very small memory footprint for your UI code.
So the amount of memory used by the UI that is currently written in Turbo Pascal will be much smaller, giving you more memory for your audio-engine.
Another big difference is that C and C++ have VERY advanced and modern compilers (eg the latest versions of GCC and G++ or Clang/LLVM).
These compilers produce code that is tiny and VERY optimized with a near-infinite amount of optimizations that the Turbo Pascal compiler lacks...
A very experienced C/C++ developer who knowns his compiler very well and has a good understanding of exactly how his code gets translated into machine code,
can produce executables that are "nearly" as fast and small as those written in assembly language...
You could decide to open-source your code (If you want to, or just the User Interface part, not the audio engine) and put it up on GitHub,
so others can contribute to it, for starters, I myself am eager to contribute as you've read in my previous posts...
This way, you can focus on developing and optimizing the audio engine, and leave the UI stuff to others, so you don't have to waste your time on UI programming,
except for when it's necessary (eg the UI code for the playback part of the application)
I think a hybrid of point 2 and 3 would be a good start, with an analysis pass before loading the samples, that has an advanced algorithm for determining the memory layout for the song and all the data to be loaded into it, and adding support for EMS would be awesome too, for those who have EMS memory on their systems...
I don't know if Mod Master XT already supports EMS, haven't checked, so forgive my ignorance on the matter if this is the case! 🤣 😉
Cheers,
Terrence
PS: Have you a reply regarding my previous post about the GPL-licensed, C++ based UI using the engine as a library ???
Cheers,
Terrence
