VOGONS


First post, by quadpixels

User metadata
Rank Newbie
Rank
Newbie

Hello VOGONS Forum,

I've hit myself against the wall trying to find the secrets in Gunmetal (1998, Mad Genius) for some time, but the game's secrets are still way too hard for me (it's said that games from that era are "punishingly hard" compared to today's games)
So, I decide to try it the hard way: treat the game run through DosBox as a white-box (meaning I can see the entire emulated physical memory and machine code), and hopefully with a bit of background knowledge on game programming, and enough time, I would be able to figure out part of how the game works.
This game has both Windows 95 and DOS versions, and I find working with the DOS version a bit easier than the Windows version.

So far, I have done the following things to my copy of DosBox:
- Imported the project into Eclipse for comfortable editing and navigation (in Ubuntu 18.04 + XFCE)
- Figured out a way to create an OpenGL window for showing information plumbed from the running DosBox (a memory visualizer for now)

With that, I had some visual impressions of the run-time memory layout, and some of the findings I have are as follows:
- Textures are all stored in memory (if you zoom in you can see some in-game textures from the memory visualization)
- A frame buffer is also in the memory; its address may change from run to run but it is pretty easy to spot

For next steps I plan to: 1) figure out what happens within a game frame; I imagine to see things like game logic (BSP tree traversal, game status update, etc), then rasterizing the entire frame into the frame buffer. The rasterization process may be used for a boundary of a logical frame. When this step is done, hopefully I can find some of the most executed code (or stack traces, just like the profilers used to generate FlameGraphs do) that are doing the interesting things for the game. 2) Another possibility is to figure out the game's resource and scene format, like what things are the scene that don't move and what could move, and how secret areas are defined and where they are (the original purpose of this effort.) There should be hints in the game's scene loading routines, or the subroutine that draws the minimap on screen. The memory contents are what connect 1) and 2).

This post is for note-keeping purposes for my poking around with DosBox (along with my fork on github with the changes)... I'm not sure how long this journey would take, but hopefully this post may be useful for you, (:

Attachments

Reply 1 of 16, by kolderman

User metadata
Rank l33t
Rank
l33t

It's not they were punishingly hard, just they were intentionally supposed to be a challenge, so you could reap satisfaction from beating them. You needed to develop strategy, tactics, skill and solve puzzles that actually worked your brain and physical reflexes.

Modern "games" are more like a kind of mental illness where you can (literally) buy quick dopamine fixes in the form of loot boxes and similar rewards, clothing and apparel for your avatar and stupid in game dancing, or simply paying to win.

Reply 2 of 16, by krcroft

User metadata
Rank Oldbie
Rank
Oldbie

Very interesting Quadpixels.

I can see lots of avenues where this can go as.. visualizing, querying, game-specific resource moding, locking or setting values (ie: cheat system), and so on!

Looking forward to following your progress; keep us posted!

Reply 3 of 16, by quadpixels

User metadata
Rank Newbie
Rank
Newbie
kolderman wrote on 2020-04-19, 11:44:

It's not they were punishingly hard, just they were intentionally supposed to be a challenge, so you could reap satisfaction from beating them. You needed to develop strategy, tactics, skill and solve puzzles that actually worked your brain and physical reflexes.

Modern "games" are more like a kind of mental illness where you can (literally) buy quick dopamine fixes in the form of loot boxes and similar rewards, clothing and apparel for your avatar and stupid in game dancing, or simply paying to win.

Hmm maybe I wasn't being accurate enough; I was referring to a walkthrough that remarks "some secrets are placed in obscure locations". The difficulty specifically for me was different, though: when I played this game many years ago I did not even know how to strafe; I was enlightened with the modern WASD + mouse-based control a few years later (I guess it was when Half Life 1 and Counter Strike came out; year 2001?), and when I play this game now I would remap the keys to this "modern" way.

A bit more update on Gunmetal: I visualized the frame buffer and I can actually see the game perform scanline fill of triangles during the middle of a frame... the game also seems to be doing sophisticated tricks like sorting and grouping lists of triangles for speedy rendering.
One most interesting thing I found is:
- The function at 0160:283c90 at run time seems to be the map drawing routine. In this function, instruction 0160:285060 ("mov [eax], dl") draws one pixel on the screen.
Still a long way to go before having something more interesting to show.

Reply 4 of 16, by quadpixels

User metadata
Rank Newbie
Rank
Newbie

Some very tiny update (mostly to indicate this thread is still alive):
According to the Cutter disassembler the function 0160:283c90 has a big loop that repeats [0x52149c] times.
The routine seems to be drawing one "section" of the map at a time and [0x52149c] seems to be the number of "sections" in a scene and if I understand correctly Level 1 has 170 "sections"; the hangar has 58 "sections".
I had to set the CPU type to Simple so that it executes only 1 instruction at a time and that I can always grab the memory values I need.
Will investigate the line-drawing code next.

Reply 5 of 16, by quadpixels

User metadata
Rank Newbie
Rank
Newbie

The minimap-drawing routine seems to have the following pattern (using Ghidra output):

function 0160:283c90() {
for (fstack164 = 0 to *0x490eac) { // Chunk of map?
for (istack220 = 0 to *pistack224) { // Object in a chunk?
for (istack208 = 0 to *(istack192 + 0x6a) { // Wireframe of an object?
for (istack172 = 0 to *(istack148 + 0x71) { // Draws a line?
}
}
}
}
}

There is a very interesting function that gets called frequently which appears to be doing coordinate transformation:

80: fcn.0027cc50 ();
0x0027cc50 fld dword [edx]
0x0027cc52 fmul qword [ebx + 0x18]
0x0027cc55 fadd qword [ebx]
0x0027cc57 fld dword [edx + 4]
0x0027cc5a fmul qword [ebx + 0x30]
0x0027cc5d faddp st(1)
0x0027cc5f fld dword [edx + 8]
0x0027cc62 fmul qword [ebx + 0x48]
0x0027cc65 faddp st(1)
0x0027cc67 fstp qword [eax]
0x0027cc69 fld dword [edx]
0x0027cc6b fmul qword [ebx + 0x20]
0x0027cc6e fadd qword [ebx + 8]
0x0027cc71 fld dword [edx + 4]
0x0027cc74 fmul qword [ebx + 0x38]
0x0027cc77 faddp st(1)
0x0027cc79 fld dword [edx + 8]
0x0027cc7c fmul qword [ebx + 0x50]
0x0027cc7f faddp st(1)
0x0027cc81 fstp qword [eax + 8]
0x0027cc84 fld dword [edx]
0x0027cc86 fmul qword [ebx + 0x28]
0x0027cc89 fadd qword [ebx + 0x10]
0x0027cc8c fld dword [edx + 4]
0x0027cc8f fmul qword [ebx + 0x40]
0x0027cc92 faddp st(1)
0x0027cc94 fld dword [edx + 8]
0x0027cc97 fmul qword [ebx + 0x58]
0x0027cc9a faddp st(1)
0x0027cc9c fstp qword [eax + 0x10]
0x0027cc9f ret

If the above code is expressed in matrix form:

[ eax        ]   [ ebx        ]   [ ebx + 0x18  ebx + 0x30  ebx + 0x48 ]   [ edx        ]
[ eax + 0x08 ] = [ ebx + 0x08 ] + [ ebx + 0x20 ebx + 0x38 ebx + 0x50 ] X [ edx + 0x04 ]
[ eax + 0x10 ] [ ebx + 0x10 ] [ ebx + 0x28 ebx + 0x40 ebx + 0x28 ] [ edx + 0x08 ]

It seems like something along the lines of "Vector3 world_position = position + orientation * local_position".
I wonder what shows up if I draw a point cloud with the output coordinates ..

Reply 6 of 16, by quadpixels

User metadata
Rank Newbie
Rank
Newbie

The output vertices of the aforementioned function (27cc50) appears to have a strong correlation with the minimap drawn on the game.
Will continue digging deeper.

Attachments

Reply 7 of 16, by MrFlibble

User metadata
Rank Oldbie
Rank
Oldbie

I fell in love with Gunmetal when I discovered the demos a couple years ago, nice to see someone doing some serious reverse-engineering analysis! It's a pity the game did not get the attention it deserved back when it came out. BTW, anyone ever tried to contact the original devs?

DOS Games Archive | Free open source games | RGB Classic Games

Reply 8 of 16, by quadpixels

User metadata
Rank Newbie
Rank
Newbie

Now showing the vertex data of the level extracted from the mini map in wireframe..
The findings since last time are: 1) the inner loop does draw one polygon (which should only either be a triangle or a quad) at a time, 2) by short-circuiting a variable during the loop it's possible to force the game to draw the entire mini map rather than just the parts the player has seen.

This wireframe format is still not efficient for locating the secrets though, guess I need to closely look at the data structure of the map 😁

Attachments

Last edited by quadpixels on 2021-04-28, 19:51. Edited 2 times in total.

Reply 9 of 16, by quadpixels

User metadata
Rank Newbie
Rank
Newbie
MrFlibble wrote on 2021-03-28, 15:11:

I fell in love with Gunmetal when I discovered the demos a couple years ago, nice to see someone doing some serious reverse-engineering analysis! It's a pity the game did not get the attention it deserved back when it came out. BTW, anyone ever tried to contact the original devs?

Ah thanks! I have not tried to contact the devs yet.

Reply 10 of 16, by quadpixels

User metadata
Rank Newbie
Rank
Newbie

Took a closer look at the minimap-drawing and found some data structures:

1) Information regarding a group of objects, 24 bytes size, starting address is *(ESI+152), count is stored in 0x490eac.

struct ObjectGroup {
int object_info_count; // offset 0
struct ObjectInfo** ptr; // offset 4
char unknown[16]; // offset 8
};

2) Information for one Object, 128 bytes size, starting address is pointed to by iStack192 (*(ESP+0x39c))

struct ObjectInfo {
glm::dvec3 position; // offset 0
glm::dmat3 orientation; // offset 24
int unknown; // offset 96
glm::vec3* vertex_buffer; // offset 100 (0x64)
short unknown1; // offset 104 (0x68)
short face_info_count; // offset 106 (0x6a)
FaceInfo* face_info; // offset 108 (0x6c)
char unknown[16]; // offset 112
};

3) Information for one face, 128 bytes size. A face can have 3 or 4 vertices (triangle or quad). Pointed to by iStack148 (*(esp+0x3c8))

struct FaceInfo {
char unknown[108];
IndexBuffer* index_buffer; // offset 108 (0x6c)
char unknown1; // offset 112 (0x70)
char vertex_count; // offset 113 (0x71)
char unknown2[15];
};

4) Index buffer. 16 bytes in size. To be used with #2. Pointed to by the pointer in #3.

struct IndexBuffer {
short vertex_indices[4];
char unknown[8];
};

With the aforementioned info one will be able to "reconstruct" the vertex information of the game scene. This includes game objects like the player's RPV, items that can be picked up. The procedure can be seen in more detail here.
I was happy to find out that the transformation part seems to work the way as I had imagined 😁 It's also a bit curious why the game uses a mix of double precision and single precision floating point numbers?

maybe_secret_1.png
Filename
maybe_secret_1.png
File size
33.38 KiB
Views
3147 views
File license
CC-BY-4.0

I tried to show the information related to an object and a face. As is shown above it's not very straightforward what the data mean. Perhaps some of the unknown fields are an index to a trigger that ends a level, opens a door, reveals a secret, etc. It's not very easy to manually inspect all those data. Would need to look into other functions to really know. Guess I will start by looking all references to 0x490eac.

Reply 11 of 16, by alberthamik

User metadata
Rank Newbie
Rank
Newbie
MrFlibble wrote on 2021-03-28, 15:11:

I fell in love with Gunmetal when I discovered the demos a couple years ago, nice to see someone doing some serious reverse-engineering analysis! It's a pity the game did not get the attention it deserved back when it came out. BTW, anyone ever tried to contact the original devs?

I have managed to get in contact with a couple of the developers myself. I have done a couple speedruns of the game and am aiming to go a bit further with the game. I'm interested in making it easier to play this game without all the issues like the CPU timing nonsense, which I've learned through the help of someone else is best handled with virtual machine CPU acceleration (KVM and whatever stuff for Windows) for some reason. I'm no programmer though, just looking to bundle together a foolproof package of stuff to load it up however best, which might be QEMU if the nightly had fixed that PS/2 input bug.

Reply 12 of 16, by DracoNihil

User metadata
Rank Oldbie
Rank
Oldbie

As someone who recently streamed this game to completion under Wine, I'd really look forward to getting this engine reverse engineered, as well as the original executable.

There are quite a lot of problems with Gunmetal execution-wise that make it a pain to run on anything other than a Pentium II class machine with a fast SVGA card.

It'd be even great if there was some way to edit the maps because if you pay attention to the log files that Gunmetal outputs, a lot of the levels, even after applying the last patch, have a bunch of problems being reported in them.

“I am the dragon without a name…”
― Κυνικός Δράκων

Reply 13 of 16, by alberthamik

User metadata
Rank Newbie
Rank
Newbie

I had actually already messaged quadpixels prior but to iterate here, I'm looking to also fix a couple bugs myself, like crashes that can occur from certain explosions sometimes, and maybe even fix up the infamous issues that plague the Stealth RPV armor. Real long shot on some things though, like maybe fixing a couple of the renderer issues for the glide renderer. But ultimately, I'd really love to find a way to fix the timing issues for the Windows version so we can stop having to mess with emulators for the MS-DOS version.

Reply 14 of 16, by MrFlibble

User metadata
Rank Oldbie
Rank
Oldbie
alberthamik wrote on 2021-06-08, 17:28:

I have managed to get in contact with a couple of the developers myself.

Cool! Are they in any position to release the game as freeware? Or maybe get it sold again, like via GOG.com?

DOS Games Archive | Free open source games | RGB Classic Games

Reply 15 of 16, by alberthamik

User metadata
Rank Newbie
Rank
Newbie

Sadly not, they lost the rights to the game pretty quickly to whoever as apart of the terms of their closure. I have no clue who holds the rights to Gunmetal now. It would be nice to see a rerelease of it someday but I'd prefer it to happen after somebody manages to fix the major bugs. And maybe some of the gameplay bugs like stealth armor. I hate retro game rereleases where they just bundled up the original game with DOSBox, especially given this game doesn't run great in DOSBox at all.

Reply 16 of 16, by slartibardfast0

User metadata
Rank Newbie
Rank
Newbie

hi @alberthamik sorry, i've still no access to the PM system so i couldn't reply to your message.

wrapping in more accurate sleep/time functions with a kernel32 wrapper might be possible
i'm fairly busy with work at the moment, but I promise i will take a look at possible opensource
kernel32 wrappers & gunmetal's exe in ghirdra on sunday afternoon!

do you have any idea what the ideal specs were originally? ideally we'd base our pinning on real data.
i've a p1 233 /w voodoo 1, and a _very throttleable_ P3 1.4ghz /w voodoo5/fx5800 on hand

many thanks,
-dc