Basically, It still has several X11 build-related dependecies, but no third-party window manager is required anymore to run the apps under mac os.

The Wiki is devoid of any reference on the matter, but you should check the dll/winemac.drv folder inside wine sources. it's the default video driver for the macport builds.

I've gotten through the first 'header info' pass of NE loading and am tackling the relocation data. Other than RELOC, I think I have all the tables translated, with probably some bugs in the resource table. I know a lot about a given Win31 executable now, though I still don't have the functions imported list - apparently that's embedded in the reloc data. I do have the "modules imported" list; that was the easy part. Many of my sample Win31 exes are simple and straightforward, but I have one that just doesn't seem to closely follow the specification, so probably there are some flavors to learn long term.

When I'm done with the segment reloc tables, I'm hoping that will give me enough information to pretend to load and lay out the exe into memory and show off the problem of calling into system dlls; and once the problem is clear, hopefully solutions will be as well.

Onward!

Reloc tables and fake segment layout came out OK, but I ran into unexpected trouble trying to take "import User func #12" or "import Kernal func #2b" and find the export in USER.exe, etc. User doesn't actually export much, at least according to the spec I'm working from. Maybe it's "special" being an internal windows module, or maybe I'm just not understanding/missing something in the spec I'm working off of. I think I'll take one more look when I have free time this week, then it's time to dig into the Wine code and try to find the NE loader and User/Kernal/GDI function support.

If you have any hints that can help me out, they'd be most welcome!

Just so you know, you are not posting this with no interest from others. I have not posted much because I don't really have much to contribute at this point. I understand that it is far too early to say where this may go, if at all, but if I had someway to do modern installers for the Win only 16-bit games without legal/IP issues I would be very happy.

Thanks, collector! That dream and curiosity are the two things driving me.

I took a look at the Wine 16bit loader, and it looks the same as the spec I'm working from, so no big surprises or Eureka moments. That just means more work to be done, but I've started tackling the Wine learning curve. I'm also thinking I should take the time to figure out how to translate the resources in the file, I think that's the last block I'm not loading. There are a section of string that look like procedure names in USER.exe, but they appear to be a part of resources, not exports. Since the Wine loader didn't turn up anything, I think I should verify that assumption.

So, more obvious things to do before worrying about being stuck! 😀

Found the "small bug" that was preventing loading the exported names tables. Small issues of BYTE vs. WORD in the name tables... Was helped along by the book "Windows Internals", 1993, Matt Pietrek, who kept talking about looking in USER.exe at the exported names, so I knew it was supposed to work. Some binary investigation against the spec and it became obvious. Cheers to Matt!

Now I have the exports and can start working on mapping names to desired functions for later emulation. Sadly I didn't get the resources working yet; they'll have to wait for future time allocation. On top of that, I've just barely started digging into the Wine 16bit code, and the "How to build Dosbox" wiki, so making progress on all fronts.

And I now am able to resolve the relocs across dlls to turn import ordinals into exported function names! For the simple Win 3x apps I should now be able to look at memory layout and stubs to bridge these reloc calls to emulation.

1Imports: MODULE FUNC
2...
3KERNEL  LSTRCPY              KERNEL  OUTPUTDEBUGSTRING     USER   REGISTERCLASS     GDI    SETBKCOLOR
4KERNEL  MAKEPROCINSTANCE     GDI     PATBLT                USER   RELEASEDC         USER   SETFOCUS
5USER    MESSAGEBOX           USER    POSTQUITMESSAGE       GDI    SELECTOBJECT      GDI    SETTEXTCOLOR
6USER    OFFSETRECT           USER    PTINRECT              USER   SENDMESSAGE       USER   SETTIMER
7...

And bugs galore, I'm sure; when I stray outside of my simple samples, I get file load crashes, but those are probably just more off-by-a-byte or similar and corner cases in my table loading. Get a simple few working, then worry about the complex, that's my motto. 😀

Thank you for the work on wine in dosbox! A wine developer tried something very similar to your project here, but there is no available code, just a post somewhere on their mailing list. It may be worthwhile to email him once you have a demo running.

Also, danoon has been working on porting wine to SDL, presumably using part of the dosbox emulation (and a mini-linux kernel) as a go-between. The problem is wine is dependent on the X11 library and their code is not easily ported, at least this is my understanding and it is consistent with the lack of ports apart from OSX. I believe OSX wine is dependent on XQuartz which makes the port much simpler. These methods to mimic the X11 library in Windows (or other x86 system) seems reasonable as an approach, given the difficulty of replacing the X11 dependent wine code with another windowing library. The approach by hxdos extender has been successful at loading some win32 apps, but the number of functions required is too many for a small project; here is a blurb from the developer's page: "DPMILD32 will provide for the translation of the module names, so if a PE executable has references to KERNEL32.DLL, DPMILD32 will try to resolve these imports by loading DKRNL32.DLL, for example." Since wine has done the work already, it doesn't seem a simple path to implement the number of remaining functions, and instead take your approach.

Great work so far (from what you tell us anyways!). I expect this will result in not only a patch for DOSBox to add experimental Win31 support through Wine but also a patch against Wine shared with the Wine development team to improve its Windows 3.1x binary handling. Great stuff, I'll keep watching 😀

Thanks for the community support! I'm approaching the "barely some features working alpha version" of the research app, so I'll find a way to share that here soon. "Save early, save often" applies to banking code with the community, too, eh? 😀 And while I've dreamed of the DOSBox patch, it had never occurred to me it would be useful to the Wine community, too. You think big, my friend!

Offhand, does anyone know the details of how dosbox transfers from code being emulated to the emulated functions - the back channel? Or where in the dosbox source to go looking for it? From years ago I have a memory it was a specific port/port block rather than an interrupt or other mechanism. And any opinions whether I should use the same ports and just add new messages to the back channel, or use different ("the next block" of) ports for "windows 3.x emulation support"? I'll look into it, but if anyone can save me some time, much appreciated. And I'd love to connect with people who've tried similar projects, the more the merrier. 😀

Dosbox uses an invalid opcode that rather than faulting transfers control to a callback. You can find it on 1102 of prefix_none.h in the normal core. The callback is a 16-bit word provided by CALLBACK_Allocate. Each callback is an entire class so using 1 per DLL export is probably out. Maybe you could use a new invalid opcode with a 32-bit callback type with the callback number for the DLL and the ordinal number.

Fantastic, thanks for the details. I would have wasted lots of time trying to track down ports. 😜
That solution sounds interesting, I'll keep it in mind as I take a look.

Quadko wrote:

Found the "small bug" that was preventing loading the exported names tables. Small issues of BYTE vs. WORD in the name tables... Was helped along by the book "Windows Internals", 1993, Matt Pietrek, who kept talking about looking in USER.exe at the exported names, so I knew it was supposed to work. Some binary investigation against the spec and it became obvious. Cheers to Matt!.

Please report it back to the devs. The only good bug is a dead bug 😉

Thanks for the encouragement; sadly I am the dev in question and it was in my little research app. After losing a few days of distraction to it, I can say I was intimately aware of the bug. But I can also now happily say that particular bug is squashed. 😀

No code progress, just some planning and research. I'm going to try to polish off the basic resource loading in this final sprint; checklist looks like 1) Resources, 2) Refactor an item that's bugging me, 3) Layout and display graphically in some fashion the relocs including emulator stub, and then package and release the code. It's a VS2012 C# .NET 4 project, but it's pretty simple and I assume other C# & Mono compilers will have little or no problems with it. I'd be happy to hear real details whatever they are - hey, I'm happy if anyone actually takes a look at the app or code under any conditions. 😁

Now I'm just debating license for a silly tool like this - just PD it, CC, or something heavier? 😜 I doubt it matters other than to keep everyone provably legit if they ever want to use code from it, but I'm open to opinions. Of course everything that changed in a dosbox patch/private build would stay under the dosbox license, and I'm assuming I can mix Wine code in without license conflict, but I'm totally ignorant on the realities of the conflict between FOSS licenses other than to observe the occasional train wreck.

After that, I'll start to tackle some of the other DosBox and Wine related setup questions and try to get in place to start actual hacking around and building. Meanwhile I'm planning and designing pieces in my head.

One question for right now: Does anyone know a FOSS font I can use as the basic 'windows 3.1' replacement font under the dosbox license? I can research (or draw, they are raster, not vector IIRC) something to start, but if someone has some knowledge that'd help. I definitely need at least one to start, but I assume I'll need at least 3 to be serious: serif, sans serif, and symbol. Maybe more as I complete more research, I've totally forgotten what the win31 font landscape looked like.

The other question I'm working on but haven't researched yet is how to hook into DosBox's back channel communication - the invalid opcode mentioned above. Thanks again for the pointers, crazyc, I've found and started where you say. I can see using a single or small range of 16 bit words; I can see hooking a port to stuff data into before triggering the callback opcode; and worst case I assume I can push something on the stack, but that alteration of current emu state makes me uncomfortable. Of course I assume I can hook ports and access and alter stack pointers and memory locations, otherwise I'm in big trouble and have totally wrong designs "dancing like sugar plums in my head". 😀 But that's what emulators do, I think. If I do use a port to pass data, that'll make "Windows support" a hardware attachment as far as dosbox is concerned... a mental picture that amuses me for some reason. I envision it as an oversized thumb-drive sized dongle somehow hanging off the back of a PC XT 5160 with a microsoft & windows 3 era logo on it.

Does anyone have any helpful pointers for me to claim a port or port range in DOSBox, like maybe a "PORT_Allocate"? 😀

At the moment I think the reloc data will be pointed at a stub (one stub per windows module + function), and each stub will have the basic psudocode (assuming port method for params):

OUT ModuleID WORD - less than 65k modules allowed, seems reasonable
OUT FunctionID WORD - less than 65k funcs per module allowed, seems reasonable
CALLBACK OPCODE (FE 07)
RET(F)

That's in the ballpark of 10-16 bytes per stub, which should work fine. My simple Win31 program examples have 50 - 100 WinAPI calls, so 1K for stubs seems very reasonable and room to grow.

The Emulation side will know the module and function ordinal number and name for all (supported) windows api calls, and will use that to distinguish stubs from actual client dlls that have to be loaded. Wine hopefully comes in with functions to execute or at least provide a starting point to execute the windows funcs in the emulator; everything that doesn't have to be invented will help.

There will be a loader .com dos app that starts the load and calls into the emu code, something like "lwin3 APP_NAME", the new win emulator code will load the win3 app into memory, setup the stubs, load any client dll hierarchy, and init itself. If it doesn't run out of dos memory in dosbox between loading and runtime memory alloc, I think I can get away with just doing that.

There will be a config section in the dosbox.conf file (or a separate config file?) to setup any windows environment stuff - does anyone have advice or opinion on whether that belongs in the standard conf file or it's own file? I envision several sections, the emulator setup section like 'win3xemu', and [windows] and [system] sections to stand in for the windows ini files as necessary.

If everything works perfectly, I can imagine having similar emulation for this Win3.x 16 bit, Win3x w/ Win32s, Win9x, and WinXP someday - big dreams, but mostly want to avoid naming conflicts now through good planning. Suggestions welcome.

The initial version will stay in real mode, as I think it doesn't matter for a single app. If I have to run in 16 bit protected mode, I'll cross that bridge later. It's probably no problem, I just don't have any experience with protected mode. Opinions welcome there, too. Win3.0 ran in real mode, and I think apps didn't care, especially the games we'd be targeting. Not like it's Excel or Lotus123!

That's the plan after this research app is completed, and I'm trying to fly from one project stage to the other on the wings of enthusiasm to see how far I can get. Any glaring oversights anyone sees?

Got the base stuff going to load resources, menu resource loading is working, and in the middle of working on icons. Strings should be easy, dialog boxes moderate, and bitmaps & cursors just "more formats supported" after icons. Looking forward to a 3 day weekend and hopefully some nice time to make some progress. And painting the bathroom - I need an "I'd rather be coding" t-shirt, I think.

Here are some screenshots of the App. Everything but the Segment layout (lower right) is real data.

Screens are "Tables and Summary for Win3.1 NE files", "Binary", "Resources", "Exports", "Imports", and finally the "Segment Layout" showing 640k: Dosbox's portion in brown, Code segments in green, data segments in blue, and memory ranges with reloc calls to Windows in red. Unused memory available for the heap and program data in the remaining purple-ish color.

Segment layout underlying data is currently fake (an exe with 1 code & 1 data segment linked to 1 dll with 1 code and 1 data segment as well) to get the drawing code working; next I figure out how to lay out the real segments and put it in the structures I'm using for drawing and fine tune the display for real data. Plenty of ui & data bugs, unfinished data translation like resource types, and unhandled exceptions blocking loading legitimate sample exe's, but I've got some working samples (like Golf.exe card game pictured) that allow forward progress. Full size image

Got the real layout function working, and most of my sample games (including reversi, tetris, and minesweeper) nicely lay out into the first row of the Segment Layout - great samples, plenty of room to play with. And then there is sol.exe, an evil program - or maybe I have a bug. 😜 That 177k program file claims to take up far more than 640k in memory. Heh.

Am looking at the port option; after perusing the bochs port list I didn't see any huge unused areas, so I think trying for something like 0xFEDB (Freekin' Extension to Dos Box?) or some other random unmentioned port probably will work. I like this method, if all works out, because it won't require many entries in the backchannel opcode.

The test app is definitely coming to an end, bugfixes and more resource type translations are about the only further uses it will have, I think. I'll do some of that, but it's time I get a working dosbox compile setup going; I've got the wiki instructions I'm following. If all goes well I'll post app source code tomorrow.

Well, look at that - the whole DB range is free: use port DB00 or DBDB for DosBox? 😁 It'd be nice to get a port under 0xFF so as not to have to use DX during the out call: OUT imm8 AX vs. OUT DX AX; there are a few low ports not listed in that port list, but it's a crowded area. Heh, DB isn't listed, but is squeezed in the DMA port area, so long term maybe a bad idea. For hacking around is probably amusing, though.

Here's my current multi-byte port assembly to handle callback for each emulated windows reloc call. If we can use a low port, we can lose all dx references & the push/pop pair.

1PUSH AX
2PUSH DX
3MOV DX DBDBh
4MOV AX 1h ; *Parameter 1: DLL id, like '1 = kernal'
5OUT DX AX
6MOV AX 20h ; *Parameter 2: Func id, like '20h = exit program'
7OUT DX AX
8POP DX
9POP AX
10OPCODE FE 07 param#; Call into dosbox with proper syntax for my win emulation callback
11RET(F) ; appropriate ret to game - all relocs require a far call, IIRC, so retf should work

Quick counting puts this at about 20 bytes per stub, and about 100 stubs in my sample programs; without DX is just two less, I think, at 18 - wait, no, we gain a byte each on the port out imm8's so it is a wash size-wise, just affects the stack less. It might work as an optimization to jump near to a common FE07 dosbox op, but that doesn't save much. A real cheater way would be to remember the code's CS:IP position for FE07 when it is set it up by the emulated loader, and "know" behind the scenes with a table lookup which call the current CS:IP is supposed to be accessing. That sounds more fragile and less flexible than passing params, but probably less work than hooking ports, and I'm not seeing terrible consequences here at first. I'll have to think about that. That would reduce the stub code to just FE07 (param16) & RET(F), more like 5 bytes and no affecting the stack. I'm working on building the stub bytes currently, so am thinking through this.

edit: Of course, if I'm affecting the stack and changing registers anyway, I could push ax & bx and pass data through those, letting the emulator pop them back and avoiding the local pop/restore code. But maybe at that point I look at the earlier suggestion of a different invalid op with 4 bytes of immediate parameter data? The "win3 emulation call" with 7 bytes per stub and set the highest bit to switch between command and execute emulated function mode? Hm, more to consider. Any opinions?

I wouldn't use ports, I'm not sure how dosbox deals with IOPL but there could be issues if it's changed. Why don't you just place the ordinal number after the callback and just advance ip 2 bytes before returning? You could then do it in 6 bytes.

That's slick, and hadn't occurred to me at all. I thought I'd have to modify instruction set decoding to get more data, but of course not, why would I? I control CS:IP. 😀 Thank you!