QDOS/Q2DOS/H2DOS with 3Dfx OpenGL on QEMU

[kjliew]

viewtopic.php?f=33&t=59935#p668661

As the extension to the above discussion, I simply tried recompiling my existing Glide3x guest wrapper with DJGPP GCC and fixed up all the plumbing for DJGPP DPMI implementation.

Update (2019/05/23): Fixed black screen issue when Q2DOS started directly into 3Dfx OpenGL.
Update (2019/05/25): QEMU 3Dfx main thread now includes the latest DJGPP DXE in the prebuilt binaries for guest wrappers.
viewtopic.php?f=24&t=60950#p680830

In theory, this should also work for other DJGPP DOS ports that use MesaFX/OpenGL for 3Dfx, such as QDOS/QWDOS/H2DOS.

With QEMU WHPX acceleration, the game was only a little faster than running in Win9x guest with QEMU TCG on AMD FX8300 CPU. The DJGPP DXE is a Glide3x driver, so at the moment, it will only work with Glide3x wrapper from host and for WIN64 this means only dgVoodoo2. Anyway, this is at least 2X more FPS than running on DOSBox SVN with voodoo chip emulation. Software rendering is very fast on QEMU WHPX acceleration, up to VESA 1024x768, with a decent desktop class CPU, but with different rendering quality.

Update: Attached Mesa-5.0.2 with Glide2x.dxe.

[kjliew]

Updated Glide3x DXE which fixed black screen issue when Q2DOS started directly into 3Dfx OpenGL.

I also tested that the Glide3x DXE worked for all 3 flavors of DOS OpenGL that come with Q2DOS, fxMesa 3.4.2/Mesa 6.0.2/Sage. Mesa 6.0.2 is the fastest and best quality on QEMU 3Dfx.

Update: Tested QDOS and H2DOS and the same driver works for all.

[kjliew]

As I get more into DJGPP DXE stuffs, I think I had come to know more about how things work. I built myself a DJGPP cross compilers based on GCC 7.4.0/binutils 2.30 so that I don't need to run the tools from VM. I can just build and test things out from my MSYS2/mingw-w64 on Windows 10 64-bit.

While DXE is the equivalent of DLL for DOS, it has one major downside which makes DXE not truly portable as of DLL on Windows. For DXE to be truly portable, DXE3GEN should not be used with -U to allow unresolved symbols. Unfortunately, for Glide3x and Mesa 3D DJGPP projects both were built with "DXE3GEN -U" to defer unresolved symbols until they were linked with the main program. That means the results GL.DXE and GLIDE3X.DXE are dependent on the exact location of the symbols from the main program, in such a way GAME.EXE <- GL.DXE <- GLIDE3X.DXE. In other words, you cannot take GL.DXE/GLIDE3X.DXE from H2DOS and use them for Q2DOS because the exact addresses for _symbols_abc_ may be different between GLH2DOS.EXE and Q2.EXE.

The Glide3x DXE guest wrapper for QEMU 3Dfx is truly portable because I don't use "DXE3GEN -U" to get it. Instead, I used a trick to resolve the missing symbols dynamically during runtime. The guest wrapper is simple enough that there are only 2 missing symbols that I need to resolve to replicate my own version of __djgpp_nearptr_enable(). If I had more free time, then I might look into Mesa 3D to see if it is also possible to make it truly portable, so that we can have one last stable version of GL.DXE that works with 3Dfx OpenGL for all Mesa DOSGL games.

[kjliew]

i recompiled Mesa-5.0.2 with Glide2x for those games to work on Linux with OpenGlide. Mesa-5.0.2 is the last stable version of Mesa that still support Glide2x. Those games may already have very good Linux native ports, but for those like the simplicity of DOS in QEMU VM, there you go. And, QEMU KVM would provide decent performance in Linux.

So far, I have tested with:
- QDOS 1.09
- Q2DOS 3.24
- GLH2DOS 1.59

The Mesa GL.DXE required linkage of unresolved symbols from the main EXE. Presumably, all those games were built similarly that the common symbols were exported through DXE_EXPORT_TABLE as I inspected the latest source code of QDOS. GLIDE2X.DXE remains fully portable, but it doesn't find any use outside of this specially built Mesa GL.DXE.

OpenGlide Glide2x implementation is not stellar, so there are minor issues with texture corruption and chromekey on the rendering. The texture corruption did not show up using 3Dfx MiniGL on Windows with GLQuake/Quake2, but using Mesa DLL on Windows would see the same issue. On the other hand, dgVoodoo2 Glide2x looked fine with Mesa.

[szo]

While DXE is the equivalent of DLL for DOS, it has one major downside
which makes DXE not truly portable as of DLL on Windows. For DXE to be
truly portable, DXE3GEN should not be used with -U to allow unresolved
symbols. Unfortunately, for Glide3x and Mesa 3D DJGPP projects both
were built with "DXE3GEN -U" to defer unresolved symbols until the
were linked with the main program.

One problem here is DJGPP's libc is not a dynamic library, and I'm sure
that there are others. stuff like memcpy, memset, etc are no problem,
but especially the stdio stuff would be problem, and the djgpp-specific
stuff would be problematic too. So we had to rely on the exe to export
the missing symbols. (Or, instead of manually dlopen()ing the library,
one can link with the dxe import library, which means any changes would
mean re-linking..)

If you post your solution to such problems, we might be able to use
them. (Also would like to see the source to your glide2x stuff.)

That means the results GL.DXE and GLIDE3X.DXE are dependent on the
exact location of the symbols from the main program, in such a way
GAME.EXE <- GL.DXE <- GLIDE3X.DXE. In other words, you cannot take
GL.DXE/GLIDE3X.DXE from H2DOS and use them for Q2DOS because the exact
addresses for _symbols_abc_ may be different between GLH2DOS.EXE and
Q2.EXE.

In fact we specifically keep the exported symbols list for the dxes
synchronized between uhexen2, qdos and q2dos, so you actually really
can exchange glide3x and gl dxes between those projects. (And yes, I
understand the basic concern that this requires a coordinated effort
between those projects.)

[kjliew]

One problem here is DJGPP's libc is not a dynamic library, and I'm sure
that there are others. stuff like memcpy, memset, etc are no problem,
but especially the stdio stuff would be problem, and the djgpp-specific
stuff would be problematic too.

Yes, this is also the problem for Watcom11 DOS OVL. For the Glide2x/Glide3x OVL/DXE guest wrappers, the solution is to avoid using any of such functions which require linking with the standard C libs and re-implement those functions locally.

If you post your solution to such problems, we might be able to use
them. (Also would like to see the source to your glide2x stuff.)

Sure, the package will definitely include Glide2x on next update. The existing package released 20190525 already has the Glide3x DXE source code and the clib.h already gives clues on the implementation. I had updated the code since then to add a few more symbols that resolved dynamically from main EXE, so that the code can call __djgpp_nearptr_enable() instead of writing my own version.

In fact we specifically keep the exported symbols list for the dxes
synchronized between uhexen2, qdos and q2dos, so you actually really
can exchange glide3x and gl dxes between those projects.

Yes, you are right. I did eventually come to realize that after inspecting QDOS source code. I am not interested in GLIDE3X.DXE because that requires real 3Dfx hardware. The Mesa/Sage3D GL.DXE is still portable among the 3 games.

[szo]

@kjliew:
OK, sorry, I hadn't noticed that the link was at viewtopic.php?f=24&t=60950#p680830
Will look at the sources in there.

The mesa gl.dxe you posted here seems to lack a few features such
as addition and exporting of DMesaGetProcAddress/fxMesaGetProcAddress
as well as certain bug fixes ,etc. In case you are interested, here
is the latest mesa (and glide) sources I use (last upd.: 2019-07-07):
http://uhexen2.sourceforge.net/devel/li ... _gl.tar.gz

And are the binaries built for glide2x (last upd.: 2019-07-07):
http://uhexen2.sourceforge.net/tmp/2x.zip
They also include the experimental glide2x dxe builds (for real hw,
to be tested yet.)