Cli2Nop...Hrmmm

Getting old DOS games working on modern hardware. (DOSBox topics belong in DOSBox areas below, not here).

Cli2Nop...Hrmmm

Postby Nicht Sehr Gut » 2002-7-02 @ 19:20

Cli2Nop is something of a mystery to me. Actually, it’s more like a container of extremely potent chemicals that could be quite handy and yet, quite hazardous to your health. The container is poorly labeled and rumored to be leaking.

For NT/2000 owners however, this is their only option (without stepping over to another OS) for some titles. I seem to remember that Cli2Nop had different “levels” of patching (or hacking).

The less “aggressive” patching will be more stable, but perform poorly. Very aggressive patching will be quite smooth, but will crash very quickly.

From what I understand it “replaces interrupt disable instructions in an executable file image with no-ops”, but the problem arises from the way Cli2Nop presumptuously replaces the code (that may not be worded quite right, but hopefully you understand what I’m saying).

It seems to me that someone should be able to patch the executable “by hand”. IOW, use Cli2Nop at a minimal level to allow the program to run, and then manually patch the remainder of the program over an extended period of time to increase performance. This would be repeated until they had the best possible patch for that particular OS. They would then release a script (or some equivalent) to allow others to create an identical executable without the manual patching,

It still wouldn’t replace a full-blown “Basic Pentium PC” emulator (which is what some of these really need), it would at least provide some relief for those who are otherwise “plain stuck”.
User avatar
Nicht Sehr Gut
l33t
 
Posts: 3630
Joined: 2002-6-30 @ 17:32

Postby Snover » 2002-7-02 @ 21:45

That would be excellent, but CLI2NOP's source is closed. However, I imagine it would be rather straightforward to rewrite it -- all you need to do is to embed a disassembler and thread the code through it. I imagine it could be done on the fly, though I don't know how it would interface within the game.

Still, isn't there an even better solution to, for example, take the CLI instruction and turn them into something..else. I mean, I don't know that much about coding, but I have seen assembly (ugh) and it seems to me that it's quite straightforward to find the cli commands and manipulate them. I guess it all depends on the frequency. I'm not that well versed on the topic, though, so don't take my words too seriously. (Plus I'm only paying half-attention -- damn that Judge Judy! ;)).
Snover
l33t++
 
Posts: 5213
Joined: 2002-6-30 @ 04:47

Postby Harekiet » 2002-7-03 @ 13:08

Well there's a big problem with that since CLI is a single byte instruction 0xFA. You can't just go search for that byte and replace it with another instruction like NOP or something. Big chance you'd seriously screw up a lot of other things :)

Better release some patched executables for the most troubled executables so they'll run. Wonder if it would also be possible to someway get the better winXP virtual interrupt flag could be implemented in win2k.
User avatar
Harekiet
DOSBox Author
 
Posts: 1050
Joined: 2002-7-01 @ 07:14
Location: Fryslan

Postby Nicht Sehr Gut » 2002-7-03 @ 15:27

Harekiet wrote:Well there's a big problem with that since CLI is a single byte instruction 0xFA. You can't just go search for that byte and replace it with another instruction like NOP or something. Big chance you'd seriously screw up a lot of other things


Well that apparently is the problem. I was suggesting a minimal level of patching with Cli2Nop, then a "handmade" patch to cover the remaining items.

[B]Better release some patched executables for the most troubled executables so they'll run.


That was my idea. Only problems are 1) it would be quite tedious to do (whomever was going to try would need a _lot_ of patience) and 2) the process would have to be repeated for every executable (while patching Duke Nukem 3D might help to patch Blood, they're probably not "interchangeable").

[B]Wonder if it would also be possible to someway get the better winXP virtual interrupt flag could be implemented in win2k.

That's probably an even better idea, if it's possible. One shot deal and it would cover multiple titles.
User avatar
Nicht Sehr Gut
l33t
 
Posts: 3630
Joined: 2002-6-30 @ 17:32

Postby Stiletto » 2002-7-03 @ 20:46

To repost this again (for those who haven't seen it):

Frequently Asked Questions about CLI2NOP
- Created by "Stiletto" -
----------------------------------------

Q. What, exactly, is the "CLI2NOP problem" - a.k.a. the VDM interrupt problem?

A. To quote the README file of SoundFX 2000 (It's called README for a reason):
"Under Windows NT the virtual DOS machine (VDM) maintains a virtual interrupt enable state for each DOS box. Execution of a CLI or STI instruction causes a trap to the VDM, which then updates the virtual interrupt state. Many programs save the interrupt state before disabling them by executing a pushf instruction. Unfortunately, due to a limitation in the Intel architecture when running a process in protected mode at a privilege level above zero, the popf instruction that would normally restore the interrupt state does not cause a trap and hence the virtual interrupt state in the VDM gets out of step with what the program expects and further virtual interrupts such as timer, mouse or sound card are not delivered to the program."

Q. What does CLI2NOP do?

A. To paraphrase SoundFX 2000's manual again:
"SoundFX 2000's solution to this problem is to fix the application by patching the application using CLI2NOP. CLI2NOP is a batch file script and utility that searches for and replaces interrupt disable instructions in an executable file image with no-ops. It does this to avoid the freezes that can occur with some applications as a result of the VDM interrupt problem. This program attempts to work around this problem for several common games."

In summary - it "no-op's" the CLI commands present in the binary code of the executable. This works around the sticking points but changes the code, usually leaving the application with a tendency to freeze, and freeze frequently. So the question is, will you trade freezing at one point with freezing at another?

Q. When do I need to run CLI2NOP?

A. When most compatibility lists of software, such as at http://www.ntcompatible.com, suggest that you do so. Generally speaking, it is a problem that largely affects BUILD engine based games (Duke Nukem 3D, Blood 3D) and other popular games which use the same engine as DOOM, like DOOM, DOOM2, Hexen, and other games. That is, any game which is a protected-mode application and requires DPMI emulation to be enabled when you attempt to run the game, say, enabled with a utility such as VDMSound Launchpad. So the two factors to pay attention to are: 1.) if it requires DPMI and 2.) it freezes / locks up sometimes at "start-up." ALSO NOTE: the CLI2NOP application is only required under Windows NT 4.0 and Windows 2000. Windows XP includes a "feature" similar to CLI2NOP that is built in to their "virtual DOS machine" (VDM) emulation, which is called on the fly, rather than to be executed preemptively like CLI2NOP. VDMSound now includes the ability to call this feature of Windows XP from within VDMSound. Make sure you get the latest update. If you do not have Windows XP, you must (for now) use CLI2NOP.

Q. Where can I get the program CLI2NOP?

A. By using a search engine such as Google (http://www.google.com), you can find locations on the Internet that carry CLI2NOP in and of itself. This, however, is strictly not permitted under copyright law without the full (trial or otherwise) SoundFX package. Software Systems has a - understandably - dim view of spreading CLI2NOP without the full package. Thus, the only place I can suggest is at the creator's, Software Systems, website: http://www.softsystem.co.uk

SoundFX 2000, itself, is not strictly required in order for CLI2NOP to work. But you must download it in order to use CLI2NOP. That said, VDMSound can be used in place of SoundFX 2000 in order to provide the sound card emulation necessary to run the program that you are trying to run. You can find VDMSound at its website: http://ntvdm.cjb.net

Q. How do I install CLI2NOP?

A. CLI2NOP is not normally installed by the SoundFX 2000 setup program so you should use your favorite Zip file decompression utility (such as WinZip) to extract CLI2NOP.BAT and PATCH.EXE from the "Patch" folder of the SoundFX 2000 distribution archive. You can create a folder that includes PATCH.EXE and CLI2NOP.BAT and list this folder in your path settings. Alternately, you can put CLI2NOP.BAT and PATCH.EXE in the folder that includes the executable for the application that you want to patch. Both files should be placed in the same directory.

Q. How do I use CLI2NOP?

A. CLI2NOP should be run from a Command Prompt window in the directory of the file that is to be patched. At the command prompt enter:

cli2nop [options] file-to-patch [patched file]

The only option is "-2" which enables additional processing using a more aggressive search/replace. (In particular, it replaces "pushfd, cli" combinations with "pushfd, nop", instead of replacing "pushfd, pop eax, cli" combinations with "pushfd, pop eax, nop".) For example, SoundFX 2000 suggests that this setting be used with DOOM and DOOM2, but suggests that it not be used with Duke Nukem 3D.

If no output file is specified then the input file will be modified.

An example of this would be using "cli2nop -2 doom.exe doomnt.exe" for DOOM.

Once you have VDMSound installed, right click on your new executable and choose "Run With VDMS". The game will start, loading VDMS to emulate the sound.

You can also of course use SoundFX 2000 to emulate the sound, since at this time you have it as well on your computer - consult their instructions to install SoundFX 2000.

If you need help setting up the game with its setup program, you're on your own!

Q. Are there any problems in using CLI2NOP to work around the VDM interrupt problem?

A. To quote the SoundFX 2000 manual again: "Removing the interrupt disable sequences can render the application liable to race conditions and contentions that the original programmer had designed to avoid. Consequently the application may be unstable and hence crash more after being patched. Further the patch algorithm is very simple and can possibly replace instructions or data that are essential to normal operation."

When creating PIF settings for applications patched with CLI2NOP you should usually enable the "Compatible timer hardware operation" option after patching. This reduces the frequency of crashes caused by timer interrupt race conditions. It does tend to slow the app slightly so experimentation is worthwhile. A final reminder: if no output file is specified when using CLI2NOP, then the input file (the original game executable) will be modified. You have been warned.

Q. Are there any advanced features you can tell me about?

A. No. Seriously, the definitions of the switches used in executing PATCH.EXE are available by typing "patch.exe -h" while at the Command Prompt, should you wish to use PATCH.EXE for your own purposes.

Thanks To:
Software Systems, creators of SoundFX 2000 (http://www.softsystem.co.uk) - your README file, Help file, and CLI2NOP batch file were invaluable in the creation of this FAQ. Thanks also to Lawrence Rust of Software Systems for his quick feedback on the initial version of this FAQ.

Vlad Romascanu, creator of VDMSound (http://ntvdm.cjb.net)

If you must, contact Vlad or I through the VDMSound User's Forum on Bravenet or the messageboards on SourceForge, linked to from the VDMSound website, if you have any further questions regarding CLI2NOP. You can also contact Software Systems at their contact email address: soundfx@softsystem.co.uk. However, this should answer all your questions.

Stiletto - 04/26/02
"I see a little silhouette-o of a man, Scaramouche, Scaramouche, will you
do the Fandango!" - Queen

Stiletto
User avatar
Stiletto
l33t
 
Posts: 4361
Joined: 2002-7-01 @ 21:57

Postby Snover » 2002-7-04 @ 06:10

Thanks for the FAQ, Stiletto. I'm gonna mark this thread "Useful" (thereby forcing it to stick at the top of the forum) and, eventually, add the thing as a separate thing in a file library.
Snover
l33t++
 
Posts: 5213
Joined: 2002-6-30 @ 04:47

Virtual Interrupt Flag

Postby Snover » 2002-7-05 @ 00:15

Is the Windows XP Virtual Interrupt Flag implemented in Windows 2000 SP3?
Snover
l33t++
 
Posts: 5213
Joined: 2002-6-30 @ 04:47

Postby Stiletto » 2002-7-10 @ 16:57

That calls to mind this thread:
http://groups.google.com/groups?threadm=5.0.2.1.2.20020608134608.009e4050%40pop.gmx.net

I'll take that as a "no." Unless Vlad's doing his own implementation.

Stiletto
"I see a little silhouette-o of a man, Scaramouche, Scaramouche, will you
do the Fandango!" - Queen

Stiletto
User avatar
Stiletto
l33t
 
Posts: 4361
Joined: 2002-7-01 @ 21:57

Postby vladr » 2002-7-11 @ 03:16

Harekiet wrote:Wonder if it would also be possible to someway get the better winXP virtual interrupt flag could be implemented in win2k.


POPF is really not trapped in p-mode, which makes me believe that the XP "workaround" is merely a periodic check of whether interrupts have been disabled for more than x milliseconds, at which point XP would force a STI-equivalent. This woudl also explain the performance hit taken by games that work under XP (with performance issues) and freeze under NT/2k (the performance hit goes away under XP if CLI2NOP is used). Not only that, but I see no other way. A long time ago I suggested on the DosEmu list (also archived on groups.google.com) that a "timeout method" (liek the one that also appears to be used by XP) be combined with CLI2NOP-like functionality (i.e. detect, using the timeout approach, shich was the last CLI to execute and lock the machine, then replace that particular CLI with NOP in memory -- so you'd get one initial, unnoticeable performance hit for the first interrupt, but the issue would not arise again during the game's activity).

Trouble is, all this is implemented in the NT kernel (to save on the user-mode switches I guess -- priviledged instruction faults seem to be handled exclusively in k-mode, and in order for VDMS to implement the workaround (1) these faults would somehow have to find their way into user-mode in NTVDM.EXE's address-space, and (2) I should be able to inject code to perform the fake STI, yet another near-impossible thing to do).

So, one thumb up for Microsoft (did I say thumb? sorry, I meant finger. oh, never mind).

V.
User avatar
vladr
Oldbie
 
Posts: 894
Joined: 2002-6-30 @ 20:09
Location: Montréal, QC, CAN

Postby Nicht Sehr Gut » 2002-7-11 @ 03:29

Vlad. What's the viability of my theory? Is it possible to Cli2Nop a program at a minimal level, and then perform the same function "by hand" (or by Hex editor or whatever...), to get a reasonable compromise for certain titles that are a royal pain in the derriere.
User avatar
Nicht Sehr Gut
l33t
 
Posts: 3630
Joined: 2002-6-30 @ 17:32

Postby vladr » 2002-7-11 @ 03:39

Nicht Sehr Gut wrote:Vlad. What's the viability of my theory? Is it possible to Cli2Nop a program at a minimal level, and then perform the same function "by hand" (or by Hex editor or whatever...), to get a reasonable compromise for certain titles that are a royal pain in the derriere.


Yes, CLI2NOP can be used to identify "candidates", then hand-crafted patches can be applied (provided someone manages to fit an extra STI somehwere in there). Big pain if you ask me, though (especially since some games come compressed, i.e. pklite'd/lzexe'd/etc.)

V.
User avatar
vladr
Oldbie
 
Posts: 894
Joined: 2002-6-30 @ 20:09
Location: Montréal, QC, CAN

Postby Nicht Sehr Gut » 2002-7-11 @ 03:43

That's kind of what I thought. So the next time someone complains loudly about how they can't live without their Duke3D and they have some capacity to program, they will be, in fact, _volunteering_ to fix it.

Remember that everybody.
User avatar
Nicht Sehr Gut
l33t
 
Posts: 3630
Joined: 2002-6-30 @ 17:32

Programming C

Postby Snover » 2002-7-11 @ 04:18

I've finally decided to start learning C. (I need to know a program language, heh...) So, we'll see how well and how quickly I learn. (I tried learning it when I was, like, 9 or 10, which just didn't work -- I was too young. Hopefully I'm not now. :)) It should be interesting, in the least.
Snover
l33t++
 
Posts: 5213
Joined: 2002-6-30 @ 04:47


Return to DOS

Who is online

Users browsing this forum: No registered users and 2 guests