VOGONS


A modern DOS Quake engine by Qbism

Topic actions

First post, by truth_deleted

User metadata

Qbism further developed and enhanced the software renderer of the Quake 1 (Q1) engine, the core of a game released by ID Software in 1996. This announcement is related to the DOS port developed by Qbism, particularly to Build 46 of his Super8 game engine (qs8 screenshots) and subsequent builds toward his Windows port. Of particular interest is the colored lighting model which was introduced in Build 53. However, qbism's SVN repository has incremental changes between these two builds, and changes after #53, which allowed the backporting of code to Build 46 which is known to build a DOS binary. After setting up a development environment to build qbism's engine, I made modifications to source code and makefile, including a few minor fixes.

In this DOS version, qbism's enhanced software renderer includes features beyond static colored lighting by use of lit formatted files, including a limited capability to load large custom maps, special effects such as translucent particles and transparent water, support for better skymaps, and enhancements for playback of music. I didn't test the music playback additions nor the networking capabilities, since this focus is on single player and the ability to play the original Q1 campaign or to load custom mods/maps.

Engoo is the other DOS based Q1 engine with an enhanced software renderer. This is leilei's project. These are the two current, major projects for development of the software renderer of Q1. There is collaboration among leilei and qbism, so these two projects share some features, in others they diverge. In particular, the DOS based version of engoo (v228) includes texture filtering and (dynamic) colored lighting. The dynamic colored lighting is not easily supported by the DOS memory model, so this is a great achievement. In both ports, qs8 and engoo, this feature has been a difficult task, a balance between game stability, performance, and realism of the lighting model. For use in dosbox or a low powered DOS system, then qbism's static lighting model should perform reasonably well. Qs8 employs a different lighting model than engoo, as discussed by leilei: "qbism uses [an] intensity/hue lookup, incidentally the resulting lightmap looks similar to PowerVR PCX2 running Quake2".

These two projects are important to developing the software renderer of a 3d game engine, especially since they run in dosbox, a dos emulator for games which runs on a large number of computing systems. It may be necessary to run dosbox with memory set to 63mb. In dosbox, qs8 will run best with the following command lines:
quake -mem 46 -heap 100000

To build the qs8 dos port, I first downloaded build 46 from qbism's SVN repository and the DJGPP development environment from rugxulo (DJGPP203.7z). Created a directory in Windows XP called c:\quake\ and moved the build 46 source code there and the development environment to a subdirectory, c:\quake\gcc\. Note that I patched build 46 at this step. Next, downloaded the bash shell, sed and file utilities, and moved them to c:\quake\gcc\bin\. Third, ran the batch file DJGPP.BAT, located in c:\quake\gcc\, while in the XP command shell and then ran bash.exe to obtain the capabilities of the bash shell language. Last, entered the directory with the qs8 source code and ran: make -f MAKEFILE. It will build a binary, also required is an installation of the original Quake 1 game and a set of lit formatted files so that qs8 (or engoo) will produce colored lighting. These .lit files are moved to the directory where the Q1 game is installed, in a subdirectory called \maps\ under \ID1\, such as c:\quake\ID1\maps\. Backup the original quake and cwsdpmi executables and replace with the attached binaries. Also, there are a few additional files with the binary package which are required and these are moved to their corresponding directory locations. In quake, set gamma to 0.7 at the console or use the brightness slider under the menu item - video options.

Edit: fixed errors specific to my source code modifications and updated the attached files (7/3/14).

Reply 1 of 54, by truth_deleted

User metadata

Attached patch and DOS quake binary which includes dynamic color lighting. Based on modified code from qbism's SVN repository (note that the dithering table was not compatible with DOS in its current form and was removed).

Edit: attached another quake binary which supports fog, but this is an experimental version and not for general use. It is untested because I do not have a playable map with fog.

Edit2: removed test binaries (b59+experimental fog and b59+dlights).

Last edited by truth_deleted on 2014-07-16, 22:39. Edited 1 time in total.

Reply 3 of 54, by truth_deleted

User metadata

It depends on the map size and which features are active. 😀

Reply 4 of 54, by truth_deleted

User metadata

Attached patch and quake binary which includes qbism's dynamic color light with dithering, brush model fix, and fog. The fog patch is untested. This version, qs8 DOS-5, also contains refinements to the color lighting and fog features, so this is the version to download for testing. If there are any software errors, then an older version may help isolate the cause.

Edit: updated the attachments (7/4/14) to include additional bits of qbism's SVN code (up to Release #89).

Edit 2: removed binary since there is a newer build below.

Last edited by truth_deleted on 2014-07-16, 22:40. Edited 1 time in total.

Reply 5 of 54, by truth_deleted

User metadata

Attached patch and quake binary which include improvements to the lighting and clipping code (qbism super8 SVN repository). The fog code is untested. This is version qs8 DOS-6 (quake will report as version DOS-5, but the modified date on this binary is 7/6/14). If there are any software errors, then an older version may help isolate the cause. Instructions are in the above posts.

I don't expect to add further features because of performance and stability in-game. However, this version requires testing for bugs on different maps. If any bugs exist, then please test with an older version to help isolate the cause. It would not be too difficult to revert back to older versions of source code.

This qs8 DOS-6 version also supports some of the very large custom maps, but others have long file names which are not supported in DOS. This is for single player only, although it may be possible to disable many of the features via console parameters so that multiplayer is working properly.

Edit 2: removed binary since there is a newer build below.

Last edited by truth_deleted on 2014-07-16, 22:40. Edited 1 time in total.

Reply 7 of 54, by leileilol

User metadata
Rank l33t++
Rank
l33t++

Not surprising really, Qbism targeted recent Core architecture by default. and never had an opportunity to do low-end testing. He couldn't see the drawback for sacking the assembly code

apsosig.png
long live PCem

Reply 8 of 54, by truth_deleted

User metadata

Attached an optimized build along with corresponding patch. Backup the original quake binary and cwsdpmi files, and then replace with those in the attached archive. Also, replace the super8.cfg with that in the archive. This version is for testing only and it has better performance.

Edit 2: removed binary since there is a newer build below.

Last edited by truth_deleted on 2014-07-16, 22:40. Edited 1 time in total.

Reply 9 of 54, by qbism

User metadata
Rank Newbie
Rank
Newbie

Hi, this is my first time here. I finally got a chance to try your build on dosbox. It's amazing to see colored light on the DOS platform. I was able to build this with djgpp within dosbox (by renaming vsnprintf.c to 'vsnprint.c' due to 8 char filename limit). BTW, could a current full source code be posted? I attempted to apply the various patches in order but had some rejections.

Certain chunks of current super8 source would be beneficial, version190 on svn. The newer lighting calculation looks much better, and the drawspan functions are a tad faster.

The asm D_Drawspans16 and some of the other span functions could be brought back. It would likely be faster in DOS depending on how DJGPP optimizes the hand-tuned C functions. It would be interesting to see a comparison.

Reply 10 of 54, by truth_deleted

User metadata

Sorry the patches didn't apply cleanly. Attached is an updated patch which applies against the source code in the original post and updates it to DOS-6o.

I tried the new lighting code but it generated a binary which reported a page fault error.

Reply 11 of 54, by truth_deleted

User metadata

Attached a 2nd optimized build (DOS-7o) along with the corresponding patch. Backup the original quake binary and cwsdpmi files, and then replace with those from the attached archive. Also, replace the super8.cfg with that in the archive. This version is for testing only and it has better performance.

Edit 2: removed binary since there is a newer build below.

Last edited by truth_deleted on 2014-07-16, 22:41. Edited 1 time in total.

Reply 12 of 54, by truth_deleted

User metadata

Attached another optimized build (DOS-7o-2) along with the corresponding patch. Updated runtime library from djgpp 2.03 to 2.04 and also included working music playback by CD audio (uses console commands not the menu items).

Backup the original quake binary and cwsdpmi files, and then replace with those from the attached archive. Also, replace the super8.cfg with that in the archive. This version is for testing only.

Edit: removed binary since there is a newer build below.

Last edited by truth_deleted on 2014-07-16, 22:42. Edited 1 time in total.

Reply 13 of 54, by qbism

User metadata
Rank Newbie
Rank
Newbie

In a quick test, this is running faster. CD music is working.

leileilol wrote:

Not surprising really, Qbism targeted recent Core architecture by default. and never had an opportunity to do low-end testing. He couldn't see the drawback for sacking the assembly code

True, although now I've got access to a P4 XP machine for testing. (realizing Pentium 4 isn't exactly super-retro...) Recently asm is added back as a compiler switch option. A bit off-topic but may be helpful here.

Reply 14 of 54, by truth_deleted

User metadata

Updated test build to DOS-8, including a built-in memory extender (wdosx) which replaces the external use of the cwsdpmi memory extender. Seems to increase in-game responsiveness. Also, adapted the music menu items for the updated cd audio function. Last, removed the unused fog code which should improve load times.

Refer to above posts on running the binary and other instructions.

Edit: removed the attached "qs8 DOS binary (build DOS-8 + no fog + wdosx memory extender)".

Last edited by truth_deleted on 2014-07-16, 22:37. Edited 1 time in total.

Reply 15 of 54, by leileilol

User metadata
Rank l33t++
Rank
l33t++

Depends on which Pentium 4. Willamette is 'super retro' because it's their hype backlash era when it didn't live up to Pentium III performance, except for optimized applications that did not exist at all at the time.

also truth you're calling your binaries "qbsim" 😀

apsosig.png
long live PCem

Reply 16 of 54, by truth_deleted

User metadata

The latest build is attached (qs8 DOS-9) along with an updated "super8.cfg" file. It is important to verify that the "fov" parameter is set to 90 and "snd_speed" set to 11025. These settings may be verified by viewing the super8.cfg file or the better method by viewing in-game (by typing the parameter name in the quake console). If you have a fast computer system, then the parameter "_snd_mixahead" may be changed to a lower value than 0.3, such as 0.15; it may produce a better sound. For typical systems, it is also suggested to set _vid_wait_override and vid_nopageflip to 0. The color lighting has several parameters, but two important ones are r_clcolorweight and r_clintensity; their values may be changed by 0.1 or 0.2 in either direction, but it is not recommended to change their values by a larger value, and verifying by restarting in an appropriate map with color lighting (such as e1m1).

I've tested other source code changes, such as reversion to assembly functions, but did not find a significant effect on performance. I now believe that the dynamic lighting and translucent water effects are the major bottlenecks in terms of performance and that they lessen any impact of some of the other assembly functions. The attached binary is fairly well optimized in this case and the impact of these added features is reasonable, perhaps the same impact of running vanilla quake without assembly optimization. It should be noted that these impacts vary within and between maps. These tests were done at 640x400 resolution, although it is possible to run at lower resolutions on slower systems.

Edit: attached patches (fpu precision and default particles); includes reversion to original particles, speed optimizations, and a correction to the ported color lighting code.

Reply 17 of 54, by meisterister

User metadata
Rank Newbie
Rank
Newbie

First of all, *bump*.

Second of all, I think that it would be an interesting project (that I would start if I had the time and/or skill) to try to write a quake engine specifically optimized for the 486. As I have heard, quake may well be the single most pentium-optimized game in existence. It would be cool to see what optimizations would be needed to make the game playable on lower end 486 hardware (meaning closer to 100 MHz than 133 MHz 😀 ).

Based on the Ultimate 486 Benchmark Comparison, I think that such an engine would be best suited by fixed point math, since the Pentium's ALU offered less improvement than its FPU.

Dual Katmai Pentium III (450 and 600MHz), 512ish MB RAM, 40 GB HDD, ATI Rage 128 | K6-2 400MHz / Pentium MMX 166, 80MB RAM, ~2GB Quantum Bigfoot, Awful integrated S3 graphics.

Reply 18 of 54, by leileilol

User metadata
Rank l33t++
Rank
l33t++
meisterister wrote:

Second of all, I think that it would be an interesting project (that I would start if I had the time and/or skill) to try to write a quake engine specifically optimized for the 486.

I'd really like to see this. Here's some nice inspiration to get you going with someone doing an excellent job at 'porting' Quake2 to a inferior-to-486 computer 😀

apsosig.png
long live PCem

Reply 19 of 54, by qbism

User metadata
Rank Newbie
Rank
Newbie

Restricting resolution to <64K pixels like 320x200 allows the speed gain of fixed-point spans.

/*==============================================
// Fixed-point D_DrawSpans
//PocketQuake- Dan East
//fixed-point conversion- Jacco Biker
//unrolled- mh, MK, qbism
//============================================*/


static int sdivzorig, sdivzstepv, sdivzstepu, sdivzstepu_fix;
static int tdivzorig, tdivzstepv, tdivzstepu, tdivzstepu_fix;
static int d_zistepu_fxp, d_zistepv_fxp, d_ziorigin_fxp;
static int zistepu_fix;

#define FIXPOINTDIV 4194304.0f //qbism- thx to Dan East, Jacco Biker

//524288.0f is 13.19 fixed point
// 2097152.0f is 11.21
//4194304.0f is 10.22 (this is what PocketQuake used)
//8388608.0f is 9.23

void UpdateFixedPointVars16( int all )
{
// JB: Store texture transformation matrix in fixed point vars
if (all)
{
sdivzorig = (int)(FIXPOINTDIV * d_sdivzorigin);
tdivzorig = (int)(FIXPOINTDIV * d_tdivzorigin);
sdivzstepv = (int)(FIXPOINTDIV * d_sdivzstepv);
tdivzstepv = (int)(FIXPOINTDIV * d_tdivzstepv);
sdivzstepu = (int)(FIXPOINTDIV * d_sdivzstepu);
sdivzstepu_fix = sdivzstepu*16;
tdivzstepu = (int)(FIXPOINTDIV * d_tdivzstepu);
tdivzstepu_fix = tdivzstepu*16;


}
d_ziorigin_fxp = (int)(FIXPOINTDIV * d_ziorigin);
d_zistepv_fxp = (int)(FIXPOINTDIV * d_zistepv );
d_zistepu_fxp = (int)(FIXPOINTDIV * d_zistepu );

zistepu_fix = d_zistepu_fxp * 16;
}

void D_DrawSpans16_FP (espan_t *pspan) //qbism from PocketQuake
{
int count, spancount, spancountminus1;
unsigned char *pbase, *pdest;
fixed16_t s, t;
int zi, sdivz, tdivz, sstep, tstep;
int snext, tnext;
pbase = (unsigned char *)cacheblock;
//Jacco Biker's fixed point conversion

// Recalc fixed point values
UpdateFixedPointVars16( 1 );
do
{
pdest = (unsigned char *)((byte *)d_viewbuffer + (screenwidth * pspan->v) + pspan->u);

// calculate the initial s/z, t/z, 1/z, s, and t and clamp
Show last 128 lines
		sdivz = sdivzorig + pspan->v * sdivzstepv + pspan->u * sdivzstepu;
tdivz = tdivzorig + pspan->v * tdivzstepv + pspan->u * tdivzstepu;
zi = d_ziorigin_fxp + pspan->v * d_zistepv_fxp + pspan->u * d_zistepu_fxp;
if (zi == 0) zi = 1;
s = (((sdivz << 8) / zi) << 8) + sadjust; // 5.27 / 13.19 = 24.8 >> 8 = 16.16
if (s > bbextents) s = bbextents; else if (s < 0) s = 0;
t = (((tdivz << 8) / zi) << 8) + tadjust;
if (t > bbextentt) t = bbextentt; else if (t < 0) t = 0;

//End Jacco Biker mod

// Manoel Kasimier - begin
count = pspan->count >> 4;
spancount = pspan->count % 16;
// Manoel Kasimier - end
//count = pspan->count;
// if (count >= 16)
// spancount = 16;
//else
//spancount = count;

while (count-- >0) // Manoel Kasimier
{
// calculate s/z, t/z, zi->fixed s and t at far end of span,
// calculate s and t steps across span by shifting
sdivz += sdivzstepu_fix;
tdivz += tdivzstepu_fix;
zi += zistepu_fix;
if (!zi) zi = 1;

snext = (((sdivz<<8)/zi)<<8)+sadjust;
if (snext > bbextents)
snext = bbextents;
else if (snext < 16)
snext = 16; // prevent round-off error on <0 steps from causing overstepping & running off the edge of the texture

tnext = (((tdivz<<8)/zi)<<8) + tadjust;
if (tnext > bbextentt)
tnext = bbextentt;
else if (tnext < 16)
tnext = 16; // guard against round-off error on <0 steps

sstep = (snext - s) >> 4;
tstep = (tnext - t) >> 4;

pdest += 16;
pdest[-16] = pbase[(s >> 16) + (t >> 16) * cachewidth]; s += sstep; t += tstep;
pdest[-15] = pbase[(s >> 16) + (t >> 16) * cachewidth]; s += sstep; t += tstep;
pdest[-14] = pbase[(s >> 16) + (t >> 16) * cachewidth]; s += sstep; t += tstep;
pdest[-13] = pbase[(s >> 16) + (t >> 16) * cachewidth]; s += sstep; t += tstep;
pdest[-12] = pbase[(s >> 16) + (t >> 16) * cachewidth]; s += sstep; t += tstep;
pdest[-11] = pbase[(s >> 16) + (t >> 16) * cachewidth]; s += sstep; t += tstep;
pdest[-10] = pbase[(s >> 16) + (t >> 16) * cachewidth]; s += sstep; t += tstep;
pdest[ -9] = pbase[(s >> 16) + (t >> 16) * cachewidth]; s += sstep; t += tstep;
pdest[ -8] = pbase[(s >> 16) + (t >> 16) * cachewidth]; s += sstep; t += tstep;
pdest[ -7] = pbase[(s >> 16) + (t >> 16) * cachewidth]; s += sstep; t += tstep;
pdest[ -6] = pbase[(s >> 16) + (t >> 16) * cachewidth]; s += sstep; t += tstep;
pdest[ -5] = pbase[(s >> 16) + (t >> 16) * cachewidth]; s += sstep; t += tstep;
pdest[ -4] = pbase[(s >> 16) + (t >> 16) * cachewidth]; s += sstep; t += tstep;
pdest[ -3] = pbase[(s >> 16) + (t >> 16) * cachewidth]; s += sstep; t += tstep;
pdest[ -2] = pbase[(s >> 16) + (t >> 16) * cachewidth]; s += sstep; t += tstep;
pdest[ -1] = pbase[(s >> 16) + (t >> 16) * cachewidth]; s += sstep; t += tstep;
// Manoel Kasimier - end

s = snext;
t = tnext;
// Manoel Kasimier - begin
}
if (spancount > 0)
{
// Manoel Kasimier - end
// calculate s/z, t/z, zi->fixed s and t at last pixel in span (so
// can't step off polygon), clamp, calculate s and t steps across
// span by division, biasing steps low so we don't run off the
// texture

spancountminus1 = spancount - 1;
sdivz += sdivzstepu * spancountminus1;
tdivz += tdivzstepu * spancountminus1;
zi += d_zistepu_fxp * spancountminus1;
//if (!zi) zi = 1;
//z = zi;//(float)0x10000 / zi; // prescale to 16.16 fixed-point
snext = (((sdivz<<8) / zi)<<8) + sadjust;
if (snext > bbextents)
snext = bbextents;
else if (snext < 16)
snext = 16; // prevent round-off error on <0 steps from causing overstepping & running off the edge of the texture

tnext = (((tdivz<<8) / zi)<<8) + tadjust;
if (tnext > bbextentt)
tnext = bbextentt;
else if (tnext < 16)
tnext = 16; // guard against round-off error on <0 steps

if (spancount > 1)
{
sstep = ((snext - s)) / ((spancount - 1));
tstep = ((tnext - t)) / ((spancount - 1));
}


pdest += spancount;
switch (spancount)
{
case 16: pdest[-16] = pbase[(s >> 16) + (t >> 16) * cachewidth]; s += sstep; t += tstep;
case 15: pdest[-15] = pbase[(s >> 16) + (t >> 16) * cachewidth]; s += sstep; t += tstep;
case 14: pdest[-14] = pbase[(s >> 16) + (t >> 16) * cachewidth]; s += sstep; t += tstep;
case 13: pdest[-13] = pbase[(s >> 16) + (t >> 16) * cachewidth]; s += sstep; t += tstep;
case 12: pdest[-12] = pbase[(s >> 16) + (t >> 16) * cachewidth]; s += sstep; t += tstep;
case 11: pdest[-11] = pbase[(s >> 16) + (t >> 16) * cachewidth]; s += sstep; t += tstep;
case 10: pdest[-10] = pbase[(s >> 16) + (t >> 16) * cachewidth]; s += sstep; t += tstep;
case 9: pdest[ -9] = pbase[(s >> 16) + (t >> 16) * cachewidth]; s += sstep; t += tstep;
case 8: pdest[ -8] = pbase[(s >> 16) + (t >> 16) * cachewidth]; s += sstep; t += tstep;
case 7: pdest[ -7] = pbase[(s >> 16) + (t >> 16) * cachewidth]; s += sstep; t += tstep;
case 6: pdest[ -6] = pbase[(s >> 16) + (t >> 16) * cachewidth]; s += sstep; t += tstep;
case 5: pdest[ -5] = pbase[(s >> 16) + (t >> 16) * cachewidth]; s += sstep; t += tstep;
case 4: pdest[ -4] = pbase[(s >> 16) + (t >> 16) * cachewidth]; s += sstep; t += tstep;
case 3: pdest[ -3] = pbase[(s >> 16) + (t >> 16) * cachewidth]; s += sstep; t += tstep;
case 2: pdest[ -2] = pbase[(s >> 16) + (t >> 16) * cachewidth]; s += sstep; t += tstep;
case 1: pdest[ -1] = pbase[(s >> 16) + (t >> 16) * cachewidth]; s += sstep; t += tstep;
break;
}

}
} while ((pspan = pspan->pnext) != NULL);
}