VOGONS


MSYS2/mingw-w64 for Win64 DOSBox

Topic actions

First post, by kjliew

User metadata
Rank Oldbie
Rank
Oldbie

There was an old thread and Windows 64-bit DOSBox build seems to be in disaster.
64bit Vanilla Dosbox?

After years long of using MSYS/MinGW from MinGW.org (WinXP-x86->Win7-x86->Win10-x64), I decided to look for more modern GNU toolchain on Windows, especially ones that support native WIN64 builds. MSYS2/mingw-w64 seems to be quite decent complete with tons of latest prebuilt packages and PACMAN package management system ported from ArchLinux. Everything seems to be so good....., until let's build DOSBox SVN. 🙁

WIN64 DOSBox SVN built by mingw-w64-x86_64 gcc-5.3.0 immediately segmentation faulted on dynamic core (dynarec flavor). Normal core runs fine, but too slow for modern DOS4GW games or running Win98.
WIN32 DOSBox SVN built by mingw-w64-i686 gcc-5.3.0 segmentation faulted on Tomb Raider 1 cinematic intro (running with dynamic-x86). Normal core again, works fine.
Both ELF32 and ELF64 built from Ubuntu (gcc-4.8.4 amd64)) and Debian (gcc-4.7.2 i686) works without any issues, similar to the one built by legacy MSYS/MinGW gcc-4.9.3.

Since I just started with MSYS2/mingw-w64, I need to check if I can go back to older gcc for more precise comparison.

Anyone has any luck with Visual Studio for 64-bit WIN64 build?

Reply 1 of 37, by Dominus

User metadata
Rank DOSBox Moderator
Rank
DOSBox Moderator

make sure you compile SVN of Dosbox

Windows 3.1x guide for DOSBox
60 seconds guide to DOSBox
DOSBox SVN snapshot for macOS (10.4-11.x ppc/intel 32/64bit) notarized for gatekeeper

Reply 2 of 37, by NY00123

User metadata
Rank Member
Rank
Member

I don't think DOSBox is currently ready for being built as a WIN64 exe, in contrary to the case on Linux.

There's one difference between Windows and Linux when it comes to 64-bit C and C++ apps: sizeof(long) is 4 for WIN64 apps, but 8 in the Linux case. There aren't such differences when to comes to 32-bit apps.

I also won't be surprised if there are more, unrelated, reasons for problems.

Reply 4 of 37, by Dominus

User metadata
Rank DOSBox Moderator
Rank
DOSBox Moderator

I have no idea what you mean...

SVN is a file versioning system which Dosbox uses for storing their source code. So I meant that you should try the latest source code of Dosbox and not the last (years old) version (0.74).

Windows 3.1x guide for DOSBox
60 seconds guide to DOSBox
DOSBox SVN snapshot for macOS (10.4-11.x ppc/intel 32/64bit) notarized for gatekeeper

Reply 5 of 37, by kjliew

User metadata
Rank Oldbie
Rank
Oldbie
Dominus wrote:

make sure you compile SVN of Dosbox

You bet. I am always on latest SVN.

NY00123 wrote:

There's one difference between Windows and Linux when it comes to 64-bit C and C++ apps: sizeof(long) is 4 for WIN64 apps, but 8 in the Linux case. There aren't such differences when to comes to 32-bit apps.

I am well aware of such 64-bit data model difference between WIN64 and ELF64. I thought the configure scripts should have taken care of such differences. The generated config.h would provide such #define and the rest of codes would always honor that.

Anyway, I got to spend some time getting a better gdb running on MSYS2/mingw-w64 to debug this. Will report back when I have time for this.

Just curious, even Visual Studio has no luck of getting WIN64 build? 😕

Reply 6 of 37, by gulikoza

User metadata
Rank Oldbie
Rank
Oldbie

dynamic-x86 is significantly faster than dynrec...as 32-bit apps will be supported in Windows for the foreseeable future there's really little reason to use it aside from knowing it's 64-bit 😀 It was never really tested in win64 and while there is some code to support it, the calling conventions are different so it's quite possible something is messed up.

http://www.si-gamer.net/gulikoza

Reply 8 of 37, by Dominus

User metadata
Rank DOSBox Moderator
Rank
DOSBox Moderator

Azarien: again, SVN is way to version files. Dosbox and many other projects use it when I write "use the SVN version of Dosbox, I mean that you should use the latest source of Dosbox. Nowhere SDL was mentioned (Btw SDL does not use SVN to version its source code, it uses Mercurial).

Windows 3.1x guide for DOSBox
60 seconds guide to DOSBox
DOSBox SVN snapshot for macOS (10.4-11.x ppc/intel 32/64bit) notarized for gatekeeper

Reply 10 of 37, by kjliew

User metadata
Rank Oldbie
Rank
Oldbie
gulikoza wrote:

dynamic-x86 is significantly faster than dynrec...

Well, yeah I know that, but still considerable faster than normal core. As CPU performance continues to improve over time, there will be a time that ones no longer care about the performance difference between dynamic-x86 and dynarec core. I am more towards the convergence of build toolchain to rid myself of having to maintain 2 separate version of runtime libraries on Windows. If MSYS2/mingw-w64 had been more consistent as the legacy MSYS/MinGW from MinGW.org, then I would not hesitate to move my future Windows development completely to WIN64.

At least, I have such guarantee on Linux. And who knows, if the *new* Microsoft would further accelerate the obsolescence of legacy compatibility. It has already happened for WoW32 and NTVDM on 64-bit Windows.

Azarien wrote:

yeah, I know what SVN is. But I was thinking of SDL, and this is my question: do I need a customized version of SDL for DOSBox?

It's off topic, but I can answer that. No, you don't.

Reply 11 of 37, by Azarien

User metadata
Rank Oldbie
Rank
Oldbie

If MSYS2/mingw-w64 had been more consistent as the legacy MSYS/MinGW from MinGW.org, then I would not hesitate to move my future Windows development completely to WIN64.

32-bit Windows version of DOSBox is more important because of compatibility. Yes, DOSBox is intended to be run on new systems, but it should remain retro-friendly and continue to work on older systems.

64-bit Windows version gives you nothing if you think of it.

Reply 12 of 37, by kjliew

User metadata
Rank Oldbie
Rank
Oldbie

Reviving this thread. It's 2018 and MSYS2/mingw-w64-x86_64 has matured over time. It has great package ecosystem and the latest GNU toolchain. WIN32 build (MSYS2/mingw-w64-i686) is now perfect, and all CPU cores - normal, dynarec, dynamic_x86 - work now.

WIN64 build (MSYS2/mingw-w64-x86_64) remains broken. CPU core normal works, dynarec crashed upon invocation, and this is with the latest official SVN version. We now have renewed focus on 64-bit build from MacOS users. Hopefully, things would get improved on the Windows side, too. We need the DOSBox developers to re-look into DOSBox CPU core dynarec implementation for different 64-bit flavors, LLP64 (WIndows) vs LP64 (Linux).

Reply 13 of 37, by kjliew

User metadata
Rank Oldbie
Rank
Oldbie

I think I found the problem. Unfortunately, I don't know how to fix it yet.

It most likely because of the differences in C ABI between MS WIN64 and System V AMD64 on the function calling convention.

MS WIN64 ABI
https://docs.microsoft.com/en-us/cpp/build/ov … ing-conventions
System V ABI
https://wiki.osdev.org/System_V_ABI

The x86_64 dynarec CPU core is currently coded for System V ABI, which works for major UNIX operating systems such as Linux, BSD & MacOS.

From src/cpu/core_dynarec/risc_x64.h

// register used for address calculations, if the ABI does not
// state that this register is preserved across function calls
// then define DRC_PROTECT_ADDR_REG above
#define FC_ADDR HOST_EBX

// register that holds the first parameter
#define FC_OP1 HOST_EDI

// register that holds the second parameter
#define FC_OP2 HOST_ESI

// special register that holds the third parameter for _R3 calls (byte accessible)
#define FC_OP3 HOST_EAX

// register that holds byte-accessible temporary values
#define FC_TMP_BA1 HOST_ECX

// register that holds byte-accessible temporary values
#define FC_TMP_BA2 HOST_EDX


// temporary register for LEA
#define TEMP_REG_DRC HOST_ESI

The registers that hold 1st & 2nd argument values are different. MS WIN64 uses RCX and RDX, if more are required, perhaps this will also affect FC_TMP_BA1&2, R8 and R9 may be required and the decoding of R8-R15 requires special byte prefixes. Hopefully, DOSBox devs can start looking into this and fix this for WIN64. MSYS2/mingw-w64-x86_64 is now very stable and well supported WIN32/WIN64 development environment.

Reply 14 of 37, by gulikoza

User metadata
Rank Oldbie
Rank
Oldbie

This is interesting, I do remember some ABI differences were coded in (check the define for _MSC_VER in gen_load_param_imm).
But yeah, most likely you would need to change the defines for FC_OP1 and FC_OP2. R8 and R9 are already handled in gen_load_param_* functions, you just need to add define for mingw-w64 compiler.

http://www.si-gamer.net/gulikoza

Reply 15 of 37, by kjliew

User metadata
Rank Oldbie
Rank
Oldbie
gulikoza wrote:

This is interesting, I do remember some ABI differences were coded in (check the define for _MSC_VER in gen_load_param_imm).
But yeah, most likely you would need to change the defines for FC_OP1 and FC_OP2. R8 and R9 are already handled in gen_load_param_* functions, you just need to add define for mingw-w64 compiler.

Yes, I already changed FC_OP1 and FC_OP2 to HOST_ECX and HOST_EDX. It solved the problem of instant segmentation fault upon switching to dynarec core, and from GDB I could confirm that it now handed off to "BlockReturn ret=core_dynrec.runcode(block->cache.start)" properly. However, the core just seemed to be hung. DOSBox did not crash, it just hung and could still accept keys, for eg. ctrl-F10 and ctrl-F9.

There must be some other dynarec cache block translation that depends on more that 2 input arguments, or the rest of the ABI definition such as register for LEA, 3rd parameter. or FC_TMP_BA1 and FC_TMP_BA2.

@Gulikoza, thanks for your attention to this old thread. I think it is time to get WIN64 DOSBOX working as well. Slow it maybe, it is however future-proof thanks to native 64-bit binary. Are you or @QBix the original author of dynarec core? I believe it would be fairly easy for those who understand dynarec core well to fix the ABI differences.

Reply 16 of 37, by gulikoza

User metadata
Rank Oldbie
Rank
Oldbie

Did you change the define in gen_load_param_* functions for mingw?

wd wrote the dynrec core, but I did the debugging as @wd did not have the environment at the time. So I could probably debug this, but I'm afraid I don't have anything of mingw-w64 set up at the moment...

edit: dynrec core is quite slower than dynamic. Since I don't see MS cutting support for w32 anytime soon, what advantages would w64 build provide?

http://www.si-gamer.net/gulikoza

Reply 17 of 37, by DosFreak

User metadata
Rank l33t++
Rank
l33t++

Check my Google Drive:
Post 670081

I've uploaded fully setup environments for mingw and mingw-w64 for compiling DOSBox.

How To Ask Questions The Smart Way
Make your games work offline

Reply 18 of 37, by kjliew

User metadata
Rank Oldbie
Rank
Oldbie
gulikoza wrote:

Did you change the define in gen_load_param_* functions for mingw?

Tried that. DOSBox still hung.

gulikoza wrote:

edit: dynrec core is quite slower than dynamic. Since I don't see MS cutting support for w32 anytime soon, what advantages would w64 build provide?

Honestly for now, none 😀 other than not having to maintain separate build env and libs. It's meant for the future, when CPU becomes really, really fast and everything should be native 64-bit. As for myself, I would like to be able to do direct comparison between WIN64 and Linux/x64 build.

MSYS2/mingw-w64-x86_64
https://www.msys2.org

This is really stable and very easy to setup. Much better than prior MinGW.org, though a little slow in user experience, CPU & IOs today are fast enough to ignore this. It has pacman packaging system and well supported. Lastest GCC, Clang, cmake, python2/3 and MS SDK headers etc. It is almost like a standard Linux build setup in Windows.

Reply 19 of 37, by kjliew

User metadata
Rank Oldbie
Rank
Oldbie

I have made some progress in debugging this and was able to start WIN64 DOSBox with dynarec core into the DOSBox's DOS prompt. However, launching a DOS program still ended up in segmentation fault.
Summarizing the issues so far:
1. In gen_load_param_reg, the encoding of "mov R8, reg&7" and "mov R8, reg&7" are incorrect. From GDB disassembly, they are actually "mov reg, R8" and mov reg, R9".
2. For WIN64 ABI, space is allocated on the call stack as a shadow store for callees to save those registers. There are cases that gen_call_function_setup/raw were thrashing the return address in stack, and this was a major issue for segmentation fault. One of the instruction is the "INT #imme8".

With those issues fixed, WIN64 DOSBox would boot up into the prompt on dynrec core. I believe there are more cases of stack thrashing for decoding other instructions. It is very tedious to track each of them down. Hopefully, this would give some insight to the DOSBox devs who are familiar with dynrec core to deal with issue #2 above once and for all.

Correction:
Actually, not segmentation fault, it was a hung. That's what make it tough to debug. Otherwise, segmentation fault would have landed into GDB and stack trace could be helpful sometimes....