Fixing games by unpacking executables: Wizardry VII demo

Having problems with a specific game or application? Post your problems here!

Fixing games by unpacking executables: Wizardry VII demo

Postby Akuma » 2019-7-24 @ 18:49

The Wizardry VII version I have kept rebooting when I tried to run it in DOSBOX. I initially blamed DOSBOX, but that was not the case. This method can be used on all software. Long story short, I fixed it. Here is how I did it.

Installing the game:

I installed the game from both floppies ending up with the game in C:\DSAVANT.

I tried running the game but it kept rebooting for some reason. So I thought maybe a bit fell over during the installation. After reinstalling it to another directory I did a sha256sum: all files were the same. And since I did not get an error during installation both times I concluded it was fine.

I viewed the 'wizardry.bat' batch file that essentially points to 'ds.exe', this seems to be the main executable that runs the game.


Examining the executable: 'ds.exe'

So I took a look with the debugger and the code did not make sense at all. Then opened it up with a hex-editor and could see clearly that the entropy of the exe-header was higher than normal. Thus the file might be packed with some kind of exe-compression utility.

This usually means a lot of anti-debugging tricks and other nasty things to lock or reboot the computer. >>Possible problem found<<.
So instead of doing this by hand, which I'm really terrible at and always takes up longer than I think it does, I took the easy route and used a generic exe unpacker.


Unpacking the executable in DOSBOX: ds.exe

I had a bunch lying around from way back.Tried all 20+ of them in DOSBOX, and got nothing but reboots and cpu-locks. So I decided to prep a real MS-DOS environment the one the executable would expect.

So I downloaded 'Rufus (rufus-1.1.7f.exe)' from https://rufus.ie . Then created a boot-able USB-drive with the MS-DOS option. Copied the game and my folder of generic unpackers on it, and rebooted into MS-DOS on an old laptop.

For fun I tried running it hoping for a different result, but no: *reboot*. Cycling through all unpackers, I finally lucked out with CyberWare Universal unPacker v3.4 by Alex A.P. (cup34.com). ( I needed a couple of tries though)

sha256sum: CUP34.COM
6ebba3c6ce961db17a356d8b78f5f9cc3f7852165daf66eb84592e1497599d66

Used flags:
/7 = step by step tracer using real mode 80386 CPU emulator
/s = use another method to determine executable image size

---MS-DOS---
Code: Select all
$ cup34.com /7 /s ds.exe ds_unpck.exe

CyberWare Universal unPacker v3.4 by Alex A.P (c) CyberWare, Inc.  (24-Apr-97)

Unpacking file 'DS.EXE' into the file 'DS_UNPCK.EXE'...

 Phase one:
   - Analyzing source file...
   - Searching for entry point...

 Phase two:
   - Analyzing the method of packing...
   - Unpacking program...

 Phase three:
   - Analyzing relocation structure...
   - Writing new executable file...

 Phase four:
   - ReloCS:ExeIP   = 0000:4B3A
   - ReloSS:ExeSP   = 0FF6:F198
   - Relocations    = 12
   - Image length   = 69152
   - New file size  = 69232

Thanx for using CyberWare products...



Finally unpacked it, this took forever with all those cpu-locks and reboots. Its just using the right tool for the job.
Then I decided to run the unpacked executable and got an error, but no reboot.

---MS-DOS---
Code: Select all
$ ds_unpck.exe

Your program caused a divide overflow error.
If the problem persists, contact your program vendor.



When I ran the executable in DOSBOX: the game started :-D

---
If you read this because you want to fix you game read no further. Download com/exe decompression utilities from the urls below and start unpacking.

Code: Select all
Tools that unpack most common packers are:

* UNP 4.12B Executable file restore utility, written by Ben Castricum,
* CyberWare Universal unPacker v3.4 by Alex A.P (c) CyberWare, Inc.  (24-Apr-97)
* IUP v0.6.7 Intelligent EXE/COM UnPacker by Frank Zago, copyleft 1996
* TEU V1.82e - The Executables' Unpacker - (C) 1996-98 by JVP

https://www.exetools.com/unpackers.htm
https://www.sac.sk/

---

Continuing the restoration:

Examining the unpacked executable 'DS_UNPCK.EXE':

So I took a look again with a hex editor, and the entropy of the exe-header was normal, but still somewhat compact. I gave it a spin in DOSBOX and with great joy I saw the game loading. I closed DOSBOX again and looked at the executable. Examining the exe-header (first 28-bytes) the original file size should be at offset 03,04,05. http://www.delorie.com/djgpp/doc/exe

offset | 00 01 02 03 04 05 06 07 08 09 0C 0B 0C 0D 0E 0F
---------------------------------------------------------
0000000 | 4d 5a 70 00 88 00 0c 00 05 00 2e 0e ff ff f6 0f
0000020 | 98 f1 00 00 3a 4b 00 00 1c 00 00 00 3b 01 a0 04

The 'DS_UNPCK.EXE' header shows: 70 00 88

executable pagesize=512 bytes

offset 03: bytes in the last page = 70h = 112 in decimals.
offset 04: either 0 (+0) or 1 (+256)
offset 05: number of pages = 88 = 136 including the last page.

The real size is:
offset 03: 70h = 112 * 1 = 112
offset 04: 00h = 0 * 256 = 0
offset 05: 88 = 135 * 512 = 69120 (without last page 136-1=135)
----------------------------------
69232

Exactly the size it is ... hmm, the unpacker might have corrected it by rebuilding the executable or it was never there to begin with and the unpacker calculated it. It could also have some extra data at the end, leftovers from the unpacking routine or from dumping memory. So the file size could be off by a bit too. Well thats a dead end, back to the entropy then.


Examining the EXE-header:

https://en.wikipedia.org/wiki/DOS_MZ_executable

The environment of an EXE program run by DOS is found in its Program Segment Prefix. EXE files normally have separate segments for the code, data, and stack. Program execution begins at address 0 of the code segment, and the stack pointer register is set to whatever value is contained in the header information (thus if the header specifies a 512 byte stack, the stack pointer is set to 200h).

The 200h is a common value, and usually filled with relocation items or empty space. I don't know if its a default but I'm missing some empty space indicating that the EXE-header might be optimized.

So looking at our header at 08-09

offset | 00 01 02 03 04 05 06 07 08 09 0C 0B 0C 0D 0E 0F
---------------------------------------------------------
0000000 | 4d 5a 70 00 88 00 0c 00 05 00 2e 0e ff ff f6 0f
0000020 | 98 f1 00 00 3a 4b 00 00 1c 00 00 00 3b 01 a0 04

It shows 05 00 for the number of paragraphs in the header.

Number of paragraphs in the header. The program's data begins just after the header, and this field can be used to calculate the appropriate file offset. The header includes the relocation entries. Note that some OSs and/or programs may fail if the header is not a multiple of 512 bytes.

Allright, it does not specify that it uses a multiplier of 16 because the file seems to start at 80h. (05x16=80)=50h.

So if the stack pointer is set to 200h for compatibility reasons. That would mean that the original value could well be set at 200h. Something to keep in mind.


Page Aligning the EXE-header

Examining the facts I came to the following conclusion:

1. The exe-file could be larger than its original, leftover slack caused
by unpacking method or memory dump.
2. The exe-header could be optimized by unpacker, original could be
at 200h for compatibility reasons.

The latter could be easily solved by UNP Executable file restore utility

UNP 4.12B Executable file restore utility, written by Ben Castricum,
usage: UNP command [options] [[d:][\path]Infile] [[d:][\path]Outfile]

commands : l = load and save
options : -p+ = align header data on a page

---DOSBOX---
Code: Select all
$ unp412b.exe l -p+ ds_unpck.exe ds_page.exe

UNP 4.12B Executable file restore utility, written by Ben Castricum, 08/27/95

processing file : DS_UNPCK.EXE
DOS file size   : 69232
file-structure  : executable (EXE)
EXE part sizes  : header 80 bytes, image 69152 bytes, overlay 0 bytes
new file size   : 69664
writing to file : DS_PAGE.EXE


Lets take a look at our files:

filename | size | description
--------------------------------------------
DS.EXE | 75360b | original
DS_UNPCK.EXE | 69232b | unpacked
DS_PAGE.EXE | 69664b | page aligned


Determining original EOF and stripping trailing garbage:

At this point I needed some reference material so I started hunting for patches on the internet and found an official one, just not from an official site (did they ever have one ?) https://www.tk421.net/wizardry/files.shtml

Hey look, there is a 'pentium reboot' patch 'cds_pent.zip' and a home made 'divide overflow patch'. I downloaded both of them. Lets begin with the 'pentium reboot' patch

---DS.TXT---
Code: Select all
This is the Wizardry 7 - Crusaders of the Dark Savant Pentium reboot patch.
   Install Crusaders, copy file DS.EXE into the Dark Savant
   directory and overwrite the already existing DS.EXE file.
 
    .... Sir-Tech, September, 1994

The date on the file however states dec 6 1994, which I found to be weird. So I tried a couple of other sites just to make sure. But everyone of them states dec 6 1994.

I opened 'PATCH\DS.EXE' file with a hexeditor. The header was aligned at 200h and 1 relocation item. 'DS_PAGE.EXE' was at 050h and had 12 relocation items.) Then I looked at the end of the file:' STACK OVERFLOW, INCREASE STACK SIZE' and then ends on '0a 0d'.

Looked at 'DS_PAGE.EXE' (the page aligned/unpacked executable) and did a search on the 'overflow' string then looked for the 0a 0d. Noted the offset at 10e56h (69206 decimals and 206246 in octals).

Well well, it looks like the file ends on 69206 which is the exact size of the 'PATCH\DS.EXE' file. Lets fix the size in the-header. I copied DS_PAGE.EXE to DS_SIZE.EXE and opened DS_SIZE.EXE in the hexeditor to edit the header:

Calculating the real size 69206:

#Pages = 69206/512 rounded up = 136d = 88h
#Remainder = ((69206/512)-135) * 512 = 86d = 56h

offset 03 must be set to 56
offset 04 must be set to 00 (remainder < 256)
offset 05 must be set to 88

I saved the DS_SIZE.EXE and exited the hexeditor. To get our file to 69206 we need to strip the overlay using UNP:

UNP 4.12B Executable file restore utility, written by Ben Castricum,
usage: UNP command [options] [[d:][\path]Infile] [[d:][\path]Outfile]

commands : l = load and save
options : -r+ = remove overlay data

---DOSBOX---
Code: Select all
$ unp l -r+ ds_size.exe ds_strip.exe

UNP 4.12B Executable file restore utility, written by Ben Castricum, 08/27/95

processing file : DS_SIZE.EXE
DOS file size   : 69664
file-structure  : executable (EXE)
EXE part sizes  : header 512 bytes, image 68694 bytes, overlay 458 bytes
new file size   : 69206
writing to file : DS_STRIP.EXE


Then tested the executable and it started the game, nice :)

Since we have two files with the same size I wanted to know how these files differ from each other. If they altered the source and recompiled it, the exe would most likely have a different size. Comparing them would result in a text flood of differences. If they patched the binary the size would be the same. Comparing them would show minor differences.

So I diffed them hoping to find a couple of NOP's or some other minor alteration indicating the 'pentium reboot' patch.

PATCH\DS.EXE
< 0000000 4d 5a 56 00 88 00[01]00 20 00[4b]0e ff ff f6 0f
< 0000020 98 f1 00 00 3a 4b 00 00 20 00 00 00 00 00 00 00
< 0000040[3b 4b]00 00 00 00 00 00 00 00 00 00 00 00 00 00
< 0000060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
< 0000100 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

DS_STRIP.EXE
> 0000000 4d 5a 56 00 88 00[0c]00 20 00[2e]0e ff ff f6 0f
> 0000020 98 f1 00 00 3a 4b 00 00[1c]00 00 00[3b 4b]00 00
> 0000040[66 0c]00[10 4c 0d]00[10 52 0d]00[10 58 0d 00 10]
> 0000060[5e 0d]00[10 78 0d]00[10 7a 0d]00[10 7c 0d 00 10]
> 0000100[7e 0d]00[10 90 0d]00[10 94 0d]00[10 ff ff ff ff]

Wow, the files are exactly the same except for some relocation items. Lets check it by running PATCH\DS.EXE in MS-DOS to make sure.

---MS-DOS---
Code: Select all
$ patch\ds.exe

Your program caused a divide overflow error.
If the problem persists, contact your program vendor.


It seems the 'pentium reboot' patch wasn't a patch at all, the reboot was a side effect of EXE-compression they used to protect it. They just distributed the 'original uncompressed exe-file' . Well Scooby another mystery solved.


Restoring the EXE-header:

I copied 'DS_STRIP.EXE' to 'DS_FIXED.EXE' and start marking the differences:

PATCH\DS.EXE
< 0000000 4d 5a 56 00 88 00[01]00 20 00[4b]0e ff ff f6 0f
< 0000020 98 f1 00 00 3a 4b 00 00 20 00 00 00 00 00 00 00
< 0000040[3b 4b]00 00 00 00 00 00 00 00 00 00 00 00 00 00
< 0000060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
< 0000100 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

DS_FIXED.EXE
> 0000000 4d 5a 56 00 88 00[0c]00 20 00[2e]0e ff ff f6 0f
> 0000020 98 f1 00 00 3a 4b 00 00[1c]00 00 00[3b 4b]00 00 1
> 0000040[66 0c]00[10 4c 0d]00[10 52 0d]00[10 58 0d]00[10 2-5
> 0000060 5e 0d]00[10 78 0d]00[10 7a 0d]00[10 7c 0d]00[10 6-9
> 0000100 7e 0d]00[10 90 0d]00[10 94 0d]00[10 ff ff ff ff] 10-13

The sixth byte indicates the number of relocation items.As you can see from the 'PATCH\DS.EXE', there is only 01 at 20h the [3b 4b] which is also in our 'DS_FIXED.EXE' at pos 1Bh. The rest of the relocation items are not needed to run it. They could be removed by the software guys at Sir-Tech. Seeing they went through all the trouble protecting this file. If someone can elaborate on this I am all ears.

Now I count 13 relocation items, but clearly the last one is bogus seeing all those ff's.

Finishing up:
1. Lines 40,60,100 can be cleared with zeroes.
2. The relocation counter on offset 06h can be set to 01.
3. The only relocation item can be moved from 1Bh to 20h.
4. The relocation offset on 18h can be changed from 1c to 20.

DS_FIXED.EXE
> 0000000 4d 5a 56 00 88 00 01 00 20 00[2e]0e ff ff f6 0f
> 0000020 98 f1 00 00 3a 4b 00 00 20 00 00 00 00 00 00 00
< 0000040 3b 4b 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Which leaves us with one byte at 0Ah, reading from the EXE format:

0Ah-0Bh
Number of paragraphs of additional memory that the program will need. This is the equivalent of the BSS size in a Unix program. The program can't be loaded if there isn't at least this much memory available to it.

Ours is smaller but it was set by the unpacker when it rebuild the EXE-header.
Changing this last byte from the patch file results in identical files, but we knew that already ;)

sha256sums: PATCH\DS.EXE 3d945f1132d2eb4df6f0dbf9da8ccaa3de03046584c423d11c83c0d0b48e05d3
sha256sums: DS_FIXED.EXE 3d945f1132d2eb4df6f0dbf9da8ccaa3de03046584c423d11c83c0d0b48e05d3

Lets take a look at our files:

No.| filename | size | description
-----------------------------------------------
| PATCH\DS.EXE | 69208b | original (6 dec 1994)
0. | DS.EXE | 75360b | original (14 sept 1992)
1. | DS_UNPCK.EXE | 69232b | unpacked
2. | DS_PAGE.EXE | 69664b | page aligned
3. | DS_SIZE.EXE | 69664b | filesize altered in header
4. | DS_STRIP.EXE | 69208b | garbage stripped from exe
5. | DS_FIXED.EXE | 69208b | EXE-header fixed

That's it folks, I've proven that these files are the same. The file runs fine in DOSBOX after unpacking, but knowing that's it 100% original again I feel even better :)


Fixing the divide overflow error:

Lets examine the unofficial 'divide overflow' for differences:

DS.EXE
< 0033220 01 46 3b 17 74 f6 8b d6 8b c1 b9 37 00 [f7 f1] 83

DS_DOFIX.EXE
> 0033220 01 46 3b 17 74 f6 8b d6 8b c1 b9 37 00 [90 90] 83

The game runs fine in MS-DOS, but running it in DOSBOX is a lot slower. I need more cycles to mitigate this and that's not what I want. There might be other side effects, but I have not encountered them. So I decided to experiment, divide overflow means that the resulting value is too big to fit. All digits are filled and we start over either negative or back at zero. How to fix this?

1. The type must be changed to make it fit (but I lack the knowledge)
2. I could increase the divider causing the result to fit the type.

After some quick experimenting:

DS.EXE
< 0033220 01 46 3b 17 74 f6 8b d6 8b c1 b9 37[00]f7 f1 83

DS_MYFIX.EXE
> 0033220 01 46 3b 17 74 f6 8b d6 8b c1 b9 37[04]f7 f1 83

This worked for my Intel(R) Core(TM)2 Duo but yours could be higher or lower. The best part, I do not experience a slowdown in dosbox, it runs fine with 3000 cycles.


Conclusion:

EXE-packers/protections cause problems, they might run fine on originally targeted hardware. But when this changes and they become faulty, they cause cpu-locks and unwanted reboots. The anti-debugging features and ASM-shenanigans were designed as part of copy protection to prevent debugging and the alteration of files. So if your game does not run. Unpack it! and try again.

Thanks for reading and leave any comments if you'd like. Especially when you are a veteran in RE/ASM please enlighten me on all major and minor problems you find :)


Akuma

Edited: fixed some typo's and inconsistencies
Akuma
Newbie
 
Posts: 31
Joined: 2019-7-24 @ 14:47

Re: Fixing games by unpacking executables: Wizardry VII demo

Postby Beegle » 2019-7-24 @ 20:50

Very thorough assessment, and a fun read too!
Thank you for doing this. Although I don't play Wizardry myself, I'm sure this will be useful to someone down the line.
User avatar
Beegle
Member
 
Posts: 457
Joined: 2013-12-01 @ 02:15

Re: Fixing games by unpacking executables: Wizardry VII demo

Postby ripsaw8080 » 2019-7-24 @ 20:57

Did you try the cputype=386_prefetch setting? It can handle the anti-debugger prefetch tricks. The prefetching core was recently improved in SVN, so be sure to try it with a current source build of DOSBox as well. That one setting change could have saved you a lot of work, although it seems you enjoyed the work enough to write at length about it. ;)
User avatar
ripsaw8080
DOSBox Author
 
Posts: 4389
Joined: 2006-4-25 @ 23:24

Re: Fixing games by unpacking executables: Wizardry VII demo

Postby Akuma » 2019-7-25 @ 09:49

@Beegle:
Thank you for reading, I thought I wouldnt care about the feedback, but I do ;)

@ripsaw8080:
Ohh you are so right sir, running with the cputype=386_prefetch makes the game load without problems.
If I had known this sooner that would have saved me some trouble, but as you stated I like a good challenge :D
Although I hoped I could unpack it in DOSBOX now, that still breaks unfortunately.
Akuma
Newbie
 
Posts: 31
Joined: 2019-7-24 @ 14:47


Return to DOSBox Games/Apps

Who is online

Users browsing this forum: No registered users and 2 guests