leileilol wrote:

kjliew wrote:

DOSBox is another good and near perfect Windows 98 emulator from my point of view. Its Voodoo chip emulation, though not quite perfect, provides decent acceleration for legacy Direct3D games that fall under the misery of Direct3D APIs immaturity during DX3 to DX5.

no

Why not? Some games work. You just may have to save often.

@leileilol
You have your right of choice and so do I. 😀 And, I did not mention Glide pass-through at all for DOSBox Direct3D acceleration if you are not aware that DOSBox has Voodoo chip emulation that run with real 3Dfx Win9x driver from Diamond Monster3D and that means acceleration for Direct3D from DX3 to DX5.

kjliew actually even tomb raider 5 works being a dx7 game

Current state of PCem lends itself well to running Windows 98 with things like emulated Voodoo hardware. It would do the job for me if I were into emulation..

PCem doesn't have virtualized voodoo like dosbox so the speed is incomparable between both.

robertmo wrote:

PCem doesn't have virtualized voodoo like dosbox so the speed is incomparable between both.

Does it even matter?

(OP asked why Windows is so difficult to emulate, not for the optimal solution running legacy Windows 9x software on new or non-x86 hardware)

This.

What do you mean by 'emulate'? If not on modern hardware, which hardware are you thinking of?

The title 'emulator' is generally given to programs which 'emulate' instructions of a different hardware. WINE has been mentined which is (I agree) a compatiblity layer, although ironically uses (or at least used to use) dosbox (which is an emulator!) for MS-DOS compatibility. I wouldn't call WINE an emulator tbh, it's as much an 'emulator' as a 'wrapper' is imo... i.e a glide wrapper 'emulates' the glide API, but not glide hardware... ergo it's called a 'wrapper' and not an 'emulator'.

It's not the Windows API (for Win9x) which is the problem here (SoftWindows95 did it back in mid-90's, on non x86 hardware 😀, Win10 does it today), it's all the hardware that might be used, and drivers that would be installed in Win9x to provide a truly versatile experience. Unless that hardware is physically present, and the running compatiblity layer has access to it (and knows how to work with it, i.e host kernel allows it), it needs to be emulated to fake Win9x and it's drivers to thinking it is present and behave as Win9x and its drivers expect it to behave. This requires aditional work outside of just having a Win9x compatibilty layer, and falls more into the realm of hardware emulation. As been mentioned, you need something to emulate the hardware, and the Windows compatiblity layer would sit on top of the emulated hardware....

To install Win9x, it would be expecting a certain hardware environment. To run a Win9x program (without installing Win9x), as long as that program isn't using some bespoke exotic hardware/API's and just relies on the standard Win9x API, this can esily be done even in Win10 (ymmv, but even today I use a program written for Win98 in Windows10, uses standard OpenGL 1.1, so I get hardware accelerated OpenGL in Win10 😀). Window theme is still Aero for some reason though 😕.

spiroyster wrote:

What do you mean by 'emulate'? If not on modern hardware, which hardware are you thinking of? […]
Show full quote

What do you mean by 'emulate'? If not on modern hardware, which hardware are you thinking of?

The title 'emulator' is generally given to programs which 'emulate' instructions of a different hardware. WINE has been mentined which is (I agree) a compatiblity layer, although ironically uses (or at least used to use) dosbox (which is an emulator!) for MS-DOS compatibility. I wouldn't call WINE an emulator tbh, it's as much an 'emulator' as a 'wrapper' is imo... i.e a glide wrapper 'emulates' the glide API, but not glide hardware... ergo it's called a 'wrapper' and not an 'emulator'.

It's not the Windows API (for Win9x) which is the problem here (SoftWindows95 did it back in mid-90's, on non x86 hardware 😀, Win10 does it today), it's all the hardware that might be used, and drivers that would be installed in Win9x to provide a truly versatile experience. Unless that hardware is physically present, and the running compatiblity layer has access to it (and knows how to work with it, i.e host kernel allows it), it needs to be emulated to fake Win9x and it's drivers to thinking it is present and behave as Win9x and its drivers expect it to behave. This requires aditional work outside of just having a Win9x compatibilty layer, and falls more into the realm of hardware emulation. As been mentioned, you need something to emulate the hardware, and the Windows compatiblity layer would sit on top of the emulated hardware....

To install Win9x, it would be expecting a certain hardware environment. To run a Win9x program (without installing Win9x), as long as that program isn't using some bespoke exotic hardware/API's and just relies on the standard Win9x API, this can esily be done even in Win10 (ymmv, but even today I use a program written for Win98 in Windows10, uses standard OpenGL 1.1, so I get hardware accelerated OpenGL in Win10 😀). Window theme is still Aero for some reason though 😕.

Thanks! I think I get this. But in principle all of this should be true for DOS/XP programs as well, shouldn't it? So perhaps to rephrase or recast the question a bit: what is it about the hardware and drivers often used by Win 9x programs that makes compatibility issues seemingly so much more prevalent than for earlier or later eras?

Many win95 programs use apis which aren't supported on NT. For one example, MapSL and MapLS which convert selector:offset addresses to linear and vice versa. NT only supports selector:offset addresses in ntvdm which is not compatible with win32 programs. The other major issue is win32 programs which load win16 dlls. This was common in the early win95 era but NT only supports the reverse (ie win16 programs that load win32 dlls).

WINE has been mentined which is (I agree) a compatiblity layer

To be a bit pedantic, wine is an implementation of win32. Microsoft themselves made 3, NT first, Win32s second and Win95 third. There are also 3 implementations of Win16, Win1/2/3/95, ntvdm and wine.

appiah4 wrote:

robertmo wrote:

PCem doesn't have virtualized voodoo like dosbox so the speed is incomparable between both.

Does it even matter?

Speed matters

captain_koloth wrote:

Thanks! I think I get this. But in principle all of this should be true for DOS/XP programs as well, shouldn't it? So perhaps to rephrase or recast the question a bit: what is it about the hardware and drivers often used by Win 9x programs that makes compatibility issues seemingly so much more prevalent than for earlier or later eras?

I'm outside of my comfort zone talking about this, but I'll tell you what I think I know...

It all boils down to device drivers. Windows has a much more complex interaction with many more device drivers than DOS had. Win9x uses VxD mainly (and also WDM in some cases for Win98 and WinME) and so for a decent emulation, either the Win9x device drivers need to be rewritten to act as wrappers/compatibility layers for the host they are running on (this is essentially what VBoxGuestAdditions are in virtual box), or the devices themselves need to be emulated for use of the original Win9x device drivers operating within the Win9x environment (and the emulation needs to be very good, otherwise BSOD's likely). This device emulation would have to be built into the VM/emulator... a lot of work given the myriad of possible devices one may want with Win9x.

DOS device drivers had much more free reign over the hardware and OS (much more low level). So much so that many DOS drivers talked directly to the hardware without the DOS kernel having any clue about what was happening as BIOS etc was directly exposed at the hardware level (usually implemented as TSR's to work with the DOS kernel, but depended on what the device was, and how the driver worked). In order for a device driver to function, it needs to be able to talk to two things, the hardware or device it was supplying functionality for (via interrupts or other hardware channels), and the OS kernel (for application memory allocation etc).

However, DOSbox is an emulator which provides A/V for various (popular at the time) hardware and a limited DOS environment (enough to play games), it is not a fully blown DOS environment (lacking in things such as it's commad interpreter command.com etc) which is why it is not recommended for anything other than games (and certainly not DOS device drivers, although depending on what the device is, i.e not hardware, you may get away with it). Games pretty much only require A/V and rarely (apart from some installers perhaps) a DOS environment in this manner. basically DOSbox can get away with it as it doesn't have to deal with device drivers (it provides the device support through it's support of various hardware written in code).

Windows was much more sophisticated in this regard and provided a much more complex kernel architecture, so drivers then had to start talking more with the kernel as they were not allowed as much free reign as they had in DOS days (although they were still allowed quite alot compared to todays more heavily guarded kernels). Early windows provided an interface (VxD) specifically for this. Win98 introduced WDM, as well as supporting VxD, however this WDM for the Win9x kernel was designed very differently to that of its 'enterprise' counterpart, NT.

NT became the de facto kernel for all subsequent MS OS's XP onwards (known as the NT family), and so even Win10 today shares much more heritage with XP than it does with Win9x. Win10 does however still use something called the WDM (introduced in Win98), but naming aside, the actual interface is probably very different (idk, never written a device driver before), but at the same time would probably share much more familiarity with XP/Win2000 WDM than Win98 WDM. This is why XP (and even Win2000) are much easier to provide VM support for (both use NT kernel's, so the architecture is similar... details will vary though). There is a much higher chance of an XP 'WDM' driver working in Win10 than a Win98/ME 'WDM' driver working... although I wouldn't put money on either tbh.

The whole WDM architecture changed quite drastically when Vista came out again iirc, which is why some Vista/7/8 drivers work fine in later OS's (less likely Vista ones in Win10, more likely Win7/8 in Win10), and likewise many Win2000 drivers worked in XP and sometimes Vista (however little nuances can mean they don't work which is why it is always best to use drivers for the OS they are designed to work with, Even if nothing really changed in the drivers code). I have yet to come across a Win9x WDM driver that works in XP (although some may exist, idk).

TL:DR

DOS is very different to Win9x in terms of device drivers (less complex, easier to emulate), Win9x is much more complex using VxD and WDM (ahrder to emulate), but this WDM is very different to XP's WDM (different to emulate than Win9x WDM). XP is part of the NT family so shares a lot more heritage with Win10 (NT kernel) than Win9x, which is why there aren't as many problems with XP programs working in Win10 (that and many of the runtimes are similar, albeit different versions, although even older versions of certain runtimes (.Net, msvcrt) form XP era are more likely to run on Win10 without help)... certainly not like there would be with Win9x programs.

Win9x is just that different to others, yet at the same time quite complex (when running as an entire environment) and harder to sandbox (what a compatibility layer would do). This doesn't mean applications written for Win9x can't run fine using various software layers though... just that programs that require higher level device functionality (graphics/sound etc) are harder to get working, and more importantly a Win9x environment running in it's entirety requires much more sophisticated emulation than todays NT family, with substantially different device driver interfaces (VxD, WDM) than XP (WDM) onwards.

Spiroyster did a really good job of this, but I'm going to throw in my explanation as part addendum, part re-stating, such that hopefully any fuzzy bits become more clear. Here goes...

The Windows API has been a lot more stable since Win 2000, and especially XP. When Microsoft brought the consumer and workstation mainline OSes together for good in XP, that meant the entire Windows userbase was pretty much all running on the same NT-based API. So, apps written for 2K/XP and up all "just work" most of the time.

There are exceptions... and that comes down to what dependencies that application has. Does it need particular hardware -- and thus particular drivers? Is there a post-XP driver, or will the XP driver itself work on modern Windows? What about runtime libraries? The C or .NET runtime, Java VM, DIrect X library, etc., must exist and run on the modern Windows environment. If it does, no problem -- the app should just work.

I.e., for XP-era and up, there's a good chance that you don't need any emulation at all. Perhaps at best you need to enable "Compatibility Mode." I confess I don't actually know what this does under the hood -- I've always assumed it's partly lying to the application about what version of Windows this is (like SETVER from back in DOS) and customizing some common APIs to act more like older versions where they may have changed subtly over time. This may or may not be an accurate representation of what's really going on.

So that's "newer than 95" covered. What about older?

Well, as said before, DOS was trying to be the glue that bound applications to lots of different hardware -- and it did this by ensuring the least common denominator of hardware. You get BIOS calls, system routines, and a little housekeeping for free. If that's enough to do what you want to do, then all you really need to do is proxy x86 instructions (handling differences between real and protected mode as necessary), trap those syscalls and emulate them with your own routines, and you're good to go.

As mentioned above, this isn't enough for games, which need to believe there's an AdLib OPL2 chip, or Sound Blaster DSP, MPU-401 interface, VGA card, etc. In this case, DOSBox has to emulate that hardware and translate hardware-level I/O calls meant for that hardware to abstracted bitmap and PCM stream handles on the host OS. Not trivial, but still relatively easy since it's a small list of simple and well-supported hardware.

Win 9x is different. If you take a simple business application, it might run on modern Windows because the subset of API it uses still exists. But, if it starts pulling in 16-bit code (which modern Windows has dropped support for), or needs Direct X version or other runtime libraries that refuse to run on modern Windows, then it can't just run anymore, and you need to look at emulation.

Emulation is a broad term, though. There's API emulation, which just takes function calls from the app, massages them to be compatible with modern equivalents, then passes them on to be handled by the host OS. Then there's instruction proxying and trapping -- which will selectively pass on some instructions to be executed directly (increment, subtract, copy to/from memory, etc.) and consume others (like direct hardware I/O calls, etc.). The latter requires emulating of hardware -- creating virtual registers and I/O ports and IRQs, and acting like that hardware interface to the application.

For Win9x applications to be happy, you either have to emulate the entire API as it existed in Win9x -- which is not something anyone has done, AFAIK. Maybe WINE does this? I'm not sure -- I would've figured it's NT-based, but I don't really know. Someone mentioned SoftWindows -- maybe that did it, but I suspect it's not an option for many people today anyway.

... or you have to emulate the hardware environment of an x86 machine circa late 90s, early 2000s. That is a lot more complex than the minimal hardware you need for DOS support. There's a LOT more variety than just "SB16 and VGA BIOS". And old habits died hard, so there's a lot of software (and of course drivers!) that still wanted to reach past the Windows abstraction to touch the hardware directly.

Really the only way to effectively emulate Win95, for example, is to emulate the entire computer. Certain products do this -- or did. VirtualBox, PCEm, and VMware for example. But the commercial offerings are targeted towards business users that have long since moved on from 9x. So, for example, while you still pick Windows 95 from the list of target OSes in VMware, there's no driver support for it, and it's really low on the list of things VMware cares about so it might or might not work. (I hit some stumbling blocks along the way and gave up for a better alternative.)

Better to not bother with being a translation layer and emulate the raw metal entirely. But, emulating even a Pentium, at the hardware level, is a fairly arduous task still today. It may be a while before this is really feasible with the kind of performance you would get on actual hardware. Not to mention the herculean task of emulating such a variety of hardware devices accurately. With the problems hardware emulation has WRT parallelism, the ideal solution -- for now, at least -- is still FPGA-based. There are projects out there recreating bits and pieces of PCs, and entire older non-PC machines, so this could be reality. It would even be possible for it to be modular, such that the busses actually exist as they did in real life -- allowing you to use real or virtualized ISA or PCI cards for 100% authenticity and a path to authentic recreations over time.

That's my hope. I really want an FPGA 486 and FPGA Pentium MMX. Then REAL DOS / Win3.1 and Win95 emulation will be possible.

Jo22 wrote:

Well, "compatibility layer" is probably the most precise term (+1 to you). 😀

Wikipedia wrote:

Wine (recursive backronym for Wine Is Not an Emulator) is a free and open-source compatibility layer that aims to allow computer programs (application software and computer games) developed for Microsoft Windows to run on Unix-like operating systems. Wine also provides a software library, known as Winelib, against which developers can compile Windows applications to help port them to Unix-like systems.

I guess I made an assumption 😐
And yes, I believe "emulate" does mean to "mimic", so you are correct on that.

kjliew wrote:

@leileilol
You have your right of choice and so do I. 😀 And, I did not mention Glide pass-through at all for DOSBox Direct3D acceleration if you are not aware that DOSBox has Voodoo chip emulation that run with real 3Dfx Win9x driver from Diamond Monster3D and that means acceleration for Direct3D from DX3 to DX5.

You mean from that patch? The old MAME rasterizer isn't something that could be associated with 'acceleration', and that only covers V1.

robertmo wrote:

PCem doesn't have virtualized voodoo like dosbox so the speed is incomparable between both.

No. Glide passthrough != 'virtualized'. It's also not a reasonably accurate way to do things and you still miss out on a lot of games.

You're both missing the point of the thread with your dosbox misleading entirely. By that point if you're going through the huge trouble preparing a Win9x install in a buggy unintended unsupported process for a non-Win9x-running HLE DOS (not Win9x) emulator that does not support Win9x, you're better off rom hunting to use something better suited for it, such as PCem.

This thread is about the accessibility for running that Win9x gaming gap with less of this "perfect" duct-taped-emulator/VM fuss. Wine-based solutions (WineVDM, Boxedwine, Wine under Linux etc) and library replacements/tweaks (dgVoodoo2, DXWnd etc) are the only realistic options I can see for the ease.

leileilol wrote:

You mean from that patch? The old MAME rasterizer isn't something that could be associated with 'acceleration', and that only covers V1.

I think you didn't get your facts right. While it is true that kekko's Voodoo chip emulation is based on MAME rasterizer, the implementation also includes an OpenGL code path that provides some acceleration. I won't say it is perfect, but for games that work (for eg. Tomb Raider II Win9x-DirectX5) it is perfectly playable and much better than software renderer. This means acceleration. PCem Voodoo emulation is all done in software, its playability is no match for DOSBox even if it is a more accurate emulation and supports for Voodoo2/SLI.

Yeah, that only covers V1, so what? What's the point of Voodoo2/SLI emulation in software if anything that plays is a slide-show? Or wish that we could live long enough to witness 10GHz break-through in CPU and we can finally enjoy the games in the glory of perfect Voodoo2/SLI emulation all in software at 60FPS. 😀

10GHz would only be twice faster, so 1 fps would get incredible speed boost to 2 fps 😉

SirNickity, Spiroyster:

THANK YOU! These answers are superlative! Answers exactly the original question I was asking.

One additional question:

For Win9x applications to be happy, you either have to emulate the entire API as it existed in Win9x -- which is not something anyone has done, AFAIK. Maybe WINE does this? I'm not sure -- I would've figured it's NT-based, but I don't really know. Someone mentioned SoftWindows -- maybe that did it, but I suspect it's not an option for many people today anyway.

I think I probably followed about 70% of your respective explanations but I'm not quite sure I understand this part of it. Can you elaborate on this? How exactly does it differ from emulating the hardware and running Win 9x on top of that emulation?

Emulating the hardware is much lower level. You're taking code at the CPU op-code level and making decisions about whether the host CPU can process that instruction directly (there's no harm in letting the host CPU add two numbers together), or if that instruction needs to be redirected to a proxy routine -- which would be the case if the emulated code is trying to... say... write to an I/O port for a piece of hardware that may not exist at all and so has to be emulated, or would interfere with the state of the hardware being used by the host OS.

Emulating an API is higher level. As a basic example, let's say the virtualized application is trying to call an API routine named "GetWindowsVersion". Well, for one, that API call might not exist anymore -- having been named to "GetWindowsNTVersion" for example. (Please bear in mind these are made up names, BTW, just for illustrative purposes.) So, the API compatibility layer will map those original calls to either internal routines (returning "Windows 95 build xxxx.xxx" for example), or map them to modern API calls with perhaps a little fudging of the parameters along the way.

API emulation essentially lets you run non-native applications directly on the native OS, by "pulling a fast one" during certain function calls that wouldn't ordinarily work if the app tried to run locally without any help. It can get as extreme as in the case of WINE, where you're providing an entire Windows API on a non-Windows OS. This usually requires that the instructions themselves are directly executable on the underlying hardware -- there's no x86-to-ARM translation or anything like that. It's just a switcheroo of API functions to alternative ones, which in the end all boil down to patching the location of calls to the correct (emulated API) code segments in memory so the desired outcome is reached.

Hardware emulation can allow any code to run on any platform. Run code designed for a Pentium MMX on a Raspberry Pi, or less drastically, allow a generic x86-32 CPU to execute x86-32 code, but non-exclusively -- i.e., let two or more OSes run on the same hardware at the same time by emulating the "bare metal" that all but one of those OSes sees.

SirNickity wrote:

Emulating an API is higher level. As a basic example, let's say the virtualized application is trying to call an API routine n […]
Show full quote

Emulating an API is higher level. As a basic example, let's say the virtualized application is trying to call an API routine
named "GetWindowsVersion". Well, for one, that API call might not exist anymore -- having been named to "GetWindowsNTVersion" for
example. (Please bear in mind these are made up names, BTW, just for illustrative purposes.) So, the API compatibility layer will map
those original calls to either internal routines (returning "Windows 95 build xxxx.xxx" for example), or map them to modern API calls with
perhaps a little fudging of the parameters along the way..

Well explained! 😀 That reminds me of the issues of old Java versions (v6 ?) which retrieved the Windows text string instead of a version number.
https://news.ycombinator.com/item?id=8397664

Another really useful API that nolonger seems to be used is "PlatformID", which could have had avoided such nonsense issues beforehand.
"PlatformID" tells if the OS in question is MacOSX, Unix, Win32NT, Win32S, WinCE, XBox or Win32Windows (=Win32c).
https://docs.microsoft.com/en-us/dotnet/api/s … etframework-4.8