VOGONS


OS X Core Audio Compile Problem

Topic actions

Reply 21 of 66, by krcroft

User metadata
Rank Oldbie
Rank
Oldbie
hail-to-the-ryzen wrote:

One problem of supporting several compilers is the vast amounts of regression testing. Every version is different.

Back in the early 2000s, I was writing production c++ from high-level pseduo-code and existing source from academic researchers (Some was fortran written in the 70s; other code was fresh Matlab and IDL). We had dual-CPU DEC Alpha kilowatt+ monsters running Linux and gcc, million-dollar octo-CPU Sparc NUMA beasts running Solaris with Sun's Forte compiler suite, and various i686 Intel and (soon) x86_64 SMP Opterons running RedHat with gcc and pgi (the Portland Group's optimizing compiler).

Every architecture and compiler permutation helped shake out more bugs and corner-cases in the code, as well as finding common-ground regarding c++ template STL syntax (as most compilers were still catching up then). It was a guantlet, and although it was a lot of work like you mentioned, our code was far better in the end for it.

The next thing we need is some form of blackbox testing: feed DOSBox a config and a DOS program that generates some capturable result (calculation or text output, screenshot, audio stream, etc), which the test harness compares against a know good result. This will allow automated and mass-testing across these permultations of compiler(s) and host(s). It also alllows capturing metrics about those tests (runtime, memory usage, latency, etc) - so additional regressions can be caught 'for free' in addition to catching functional regressions.

Test cases can start small and exercise the most critical functionality first, followed by more coverage as time and effort permits.

Last edited by krcroft on 2019-10-27, 17:06. Edited 2 times in total.

Reply 22 of 66, by jmarsh

User metadata
Rank Oldbie
Rank
Oldbie
krcroft wrote:

I wasn't aware DOSBox's core developers exclusively support Clang-based builds on OS X (happy if you can point me to that).

Similarly, I wasn't aware the core developers have declared gcc as /not supported/ on OS X.

Clang is not the only compiler that Apple has ever shipped. They used gcc long before then, and those builds worked perfectly fine with their headers. The fact that you're trying to use an incompatible build of gcc is not a DOSBox related issue, it's an Apple vs. Brew issue - suggest you go take it up with them.

Reply 23 of 66, by Dominus

User metadata
Rank DOSBox Moderator
Rank
DOSBox Moderator

It IS kind of redundant to test all these gcc versions on OS X, IMO. Since you need to install Apple's clang to get those gcc versions to install to begin with.
And yes, unsupported means "not supported by Apple" not "not supported by DOSBox".
It is nice what you are doing here but IMO you are going way over the top in this.

Windows 3.1x guide for DOSBox
60 seconds guide to DOSBox
DOSBox SVN snapshot for macOS (10.4-11.x ppc/intel 32/64bit) notarized for gatekeeper

Reply 24 of 66, by krcroft

User metadata
Rank Oldbie
Rank
Oldbie

Google's first hit for 'apple, how to install gcc', is to an Apple-hosted help forum where the number one response says 'brew install gcc' and then calling those binaries from /usr/local/bin.

https://discussions.apple.com/thread/8336714

When I use the term 'gcc', I'm talking about the as-released and built version from https://gcc.gnu.org/, which is what you get from 'brew install gcc'. I also consider manually compiling the gcc sources (from the same link) as having equal provenance, even if it needs to be built with clang.

If you're saying that Apple's modified version counts as 'gcc', then I would argue it's not. It's more like 'AppleGcc'. Yes, installing xcode does let you type 'gcc' on the command line (second link below), but it's just clang in disguise:

$ gcc --version
Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/c++/4.2.1
Apple LLVM version 10.0.0 (clang-1000.10.44.4)
Target: x86_64-apple-darwin18.2.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools

https://github.community/t5/Project-Developme … -mac/td-p/13382

I don't see a reference to the DOSBox's core developers stating that GNU gcc is explicitly not supported on OS X, and that only Apple's modified gcc (or clang-in-disguise) is supported. Do you have that reference or not?

I agree with taking up the issue with Apple: I'm happy to file a bug against their non-standards header files.

Reply 25 of 66, by Dominus

User metadata
Rank DOSBox Moderator
Rank
DOSBox Moderator

You only need to install proper gcc for the few cases that clang is not up to the task (like fortran and everything depending on it AFAIK).
And still you need to install Apple's clang then install homebrew and only then "brew install xyz" will do anything.
(Btw. for a more robust and proper installer MacPorts is better, as it tries to keep self contained and doesn't mess things up by going into /usr/local

Windows 3.1x guide for DOSBox
60 seconds guide to DOSBox
DOSBox SVN snapshot for macOS (10.4-11.x ppc/intel 32/64bit) notarized for gatekeeper

Reply 26 of 66, by Dominus

User metadata
Rank DOSBox Moderator
Rank
DOSBox Moderator

I don't see a reference to the DOSBox's core developers stating that GNU gcc is explicitly not supported on OS X, and that only Apple's modified gcc (or clang-in-disguise) is supported. Do you have that reference or not?

It could be interpreted that jmarsh meant this but he didn't and I suspect you know this.

Windows 3.1x guide for DOSBox
60 seconds guide to DOSBox
DOSBox SVN snapshot for macOS (10.4-11.x ppc/intel 32/64bit) notarized for gatekeeper

Reply 27 of 66, by krcroft

User metadata
Rank Oldbie
Rank
Oldbie
Dominus wrote:

It is nice what you are doing here but IMO you are going way over the top in this.

I honestly wasn't aware that building DOSBox with 'brew install gcc' would garner so much contention (at least from non-core-devs.. I don't see Qbix taking issue), and that doing so would be considered an over-the-top move.

Seriously, I don't get it. This is a bog-standard gcc... and it builds DOSBox just fine when I use Apple's own published method for detecting their AppleBlocks extension.

I'm not trying to use a compiler from some long lost platform or that I uncovered during an expedition to the c++ tomb's of egypt 🤣

Last edited by krcroft on 2019-10-27, 13:54. Edited 1 time in total.

Reply 28 of 66, by Dominus

User metadata
Rank DOSBox Moderator
Rank
DOSBox Moderator

For me, the whole testing thing is over the top. And you are not a core dev, so let us other non core devs chime in with our opinions. Overall a lot of hostility IMO which needs to be dialed down

Windows 3.1x guide for DOSBox
60 seconds guide to DOSBox
DOSBox SVN snapshot for macOS (10.4-11.x ppc/intel 32/64bit) notarized for gatekeeper

Reply 29 of 66, by jmarsh

User metadata
Rank Oldbie
Rank
Oldbie
krcroft wrote:

This is a bog-standard gcc... and it builds DOSBox just fine when I use Apple's own published method for dectecting their AppleBlocks extension.

It's not fine though, it's crippled - missing a standard feature that a macOS build is expected to have.

Have you checked if Apple's gcc (not clang) also supports the detection method you have added, or have you unintentionally disabled coremidi for OSX PPC builds?

Reply 30 of 66, by krcroft

User metadata
Rank Oldbie
Rank
Oldbie
Dominus wrote:

I don't see a reference to the DOSBox's core developers stating that GNU gcc is explicitly not supported on OS X, and that only Apple's modified gcc (or clang-in-disguise) is supported. Do you have that reference or not?

It could be interpreted that jmarsh meant this but he didn't and I suspect you know this.

I honestly thought that jmarsh was refering to the DOSBox team not supporting GNU gcc on OS X. That's why I've asked twice for that refrence.

GPL-licensed software by default has no support terms unless augmented with some form of SLA (like RedHat, SuSE and others offer paying customers), so I wasn't under the impression that Apple would ever support GNU gcc. (I get that Apple probably does officially support their ClangGCC-hybrid though.)

DOSBox and GCC are both GNU-licensed software and I expect no support from either. Interest and discussion? Sure!

And hey, if QBix says that I shouldn't use GNU gcc to build DOSBox, then that's a valuable data point as well. I was under the impression that this was the scenario to which jmarsh was referring when mentioning 'unspported configuration'.

Last edited by krcroft on 2019-10-27, 13:51. Edited 3 times in total.

Reply 31 of 66, by krcroft

User metadata
Rank Oldbie
Rank
Oldbie
jmarsh wrote:

Have you checked if Apple's gcc (not clang) also supports the detection method you have added, or have you unintentionally disabled coremidi for OSX PPC builds?

Good question and I appreciate the technical focus.
I'm not sure, however Apple's modified gcc needs to support the Blocks extension otherwise #include'ing Apple's coreMIDI header would similarly choke on the broken prototype syntax.

The preprocessor code snippet published by Apple, which I'm using verbatim, includes backward compatibility for pre 3.x compilers as well.

I can put together a stand alone code snippet to validate Apple's preprocessor statements that someone with PPC OSX can also try. Any proposed approach should have no change to Apple's modified gcc and clang builds targeting OS X.

edit:
I found a circa-2007 discussion mentioning Apple's customized gcc 4.2, and how brew can also install it via their package called 'apple-gcc42', so I will try adding a build for that in the CI workflow. If there's a more standard or current way to use the Apple-modified-gcc then I am happy to test that too.

Reply 32 of 66, by krcroft

User metadata
Rank Oldbie
Rank
Oldbie
Dominus wrote:

for a more robust and proper installer MacPorts is better, as it tries to keep self contained and doesn't mess things up by going into /usr/local

Thanks. I'm assuming this will install the Apple-modified gcc? (a valuable test scenario that jmarsh suggested) Are there some non-interactive command line steps you can suggest to install it? Do you know the path to the binaries, and their names (gcc vs gcc-4 gcc-42, ...)? This will be a huge help in adding this as a CI workflow.

The Mac workflow will ideally contain just one representative build per compiler family, so clang 10.x, GNU gcc 9.x, and, with yours and jmarsh's help, Apple-modified-Gcc 4.x.

Last edited by krcroft on 2019-10-27, 16:27. Edited 1 time in total.

Reply 33 of 66, by jmarsh

User metadata
Rank Oldbie
Rank
Oldbie
krcroft wrote:

I'm not sure, however Apple's modified gcc needs to support the Blocks extension otherwise #include'ing Apple's coreMIDI header would similarly choke on the broken prototype syntax.

The versions of xcode that bundle gcc don't bundle system headers that use blocks, obviously.

The preprocessor code snippet published by Apple, which I'm using verbatim, includes backward compatibility for pre 3.x compilers as well.

Sure, it's compatible - meaning it will compile. But it may not work as desired because it can exclude coremidi support when it is available.

test_blocks.c:

#include <CoreMIDI/MIDIServices.h>

#ifndef __has_feature
#define __has_feature(x) 0
#endif
#ifndef __has_extension
#define __has_extension __has_feature
#endif

#if __has_extension(blocks)

int main(int argc,char** argv) {
return 0;
}
#else
#error Compiler doesn't support blocks extension
#endif
mac-mini:~ $ gcc --version
i686-apple-darwin9-gcc-4.0.1 (GCC) 4.0.1 (Apple Inc. build 5493)
Copyright (C) 2005 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

mac-mini:~ $ gcc test_blocks.c
test_blocks.c:16:2: error: #error Compiler doesn't support blocks extension

In my opinion the proper "fix" for this is writing an autoconf test based on a .c file that #includes <CoreMIDI/MIDIServices.h> and if it fails to compile, exclude the coremidi stuff.

Reply 34 of 66, by krcroft

User metadata
Rank Oldbie
Rank
Oldbie
jmarsh wrote:

In my opinion the proper "fix" for this is writing an autoconf test based on a .c file that #includes <CoreMIDI/MIDIServices.h> and if it fails to compile, exclude the coremidi stuff.

Very nice; and thank you.

I prefer your solution because it directly tests the active compiler's ability to include and parse whatever Core MIDI header is on offer, without the risk of side-effects resulting from using a secondary or implied test of functionality. This covers the scenario you describe where non-Blocks coreMIDI headers are available and could infact be handled by a non-Blocks compiler regardless of the packager (brew, xcode, or MacPorts).

Reply 35 of 66, by Dominus

User metadata
Rank DOSBox Moderator
Rank
DOSBox Moderator

I was not near my computer since this discussion started and I'm just returning home. But I'll take a look at the collection of OS X SDKs I have to pinpoint when the blocks were added.
Because the SDKs might be another thing to check in configure.

Windows 3.1x guide for DOSBox
60 seconds guide to DOSBox
DOSBox SVN snapshot for macOS (10.4-11.x ppc/intel 32/64bit) notarized for gatekeeper

Reply 37 of 66, by krcroft

User metadata
Rank Oldbie
Rank
Oldbie

jmarsh,

I tested your suggestion (diff):
https://github.com/dreamer/dosbox-staging/com … a64b6a9bddb4620

And it's working well for clang:

checking if compiler supports -mno-ms-bitfields... yes
checking if compiler supports CoreMIDI headers... yes
checking for ALSA CFLAGS...
checking for ALSA LDFLAGS... -lasound -lm -ldl -lpthread

And brew-installed gcc:

checking if compiler supports -mno-ms-bitfields... yes
checking if compiler supports CoreMIDI headers... no
checking for ALSA CFLAGS...
checking for ALSA LDFLAGS... -lasound -lm -ldl -lpthread

Full output here:
https://github.com/dreamer/dosbox-staging/runs/276722463

(Haven't added MacPorts or xcode's gcc to the CI workflow yet; will tinker with that today - any tips on installing and using those from the command line appreciated)

Reply 39 of 66, by Dominus

User metadata
Rank DOSBox Moderator
Rank
DOSBox Moderator

ok, the MIDINotifyBlock was introduced with the OS X SDK 10.11. All SDKs before that don't have that (at least not in MIDIServices.h). Curiosly that's the SDK I'm using to build my 64bit snapshot of DOSBox (but with a deployment target of 10.7).

As for installing macports, see the travis install script for Exult https://github.com/exult/exult/blob/master/.t … /install-osx.sh that gives you the pointers

Notes: select the proper distfile for the OS X you are using, everything in # Install actual dependencies is what Exult needs, not necessarily you 😀
To install something use "sudo port -q install foobar" (-q for quiet)
For apple's gcc you'd use apple-gcc40 or apple-gcc42. For the others it's gcc5, gcc6,.., gcc9. Then you use "sudo port select --set gcc gcc9" to set the gcc you want. You might want to use some testing with "port select --list gcc" to see how that works and looks. "select" is also used for other things you can choose between different versions (clang, llvm, pygments, python, pyhton2, python3 and probably more).

Windows 3.1x guide for DOSBox
60 seconds guide to DOSBox
DOSBox SVN snapshot for macOS (10.4-11.x ppc/intel 32/64bit) notarized for gatekeeper