dosbox-staging (DOSBox Git repo)

Developer's Forum, for discussion of bugs, code, and other developmental aspects of DOSBox.

dosbox-staging (DOSBox Git repo)

Postby dreamer_ » 2019-10-02 @ 12:35

Hello everyone :) I am happy to announce repository dosbox-staging.

The purpose is not to create "yet another DOSBox fork", but to make it easier to develop and submit patches upstream. For this reason, master branch is kept 99% in sync with SVN (with new SVN commits regularly merged-in), but includes some changes to make it easier to use Git. We have:

  • svn/* branches - representing full development history 100% in sync with SVN upstream (to make the future synchronization easy), e.g. svn/trunk
  • svn:ignore props imported as .gitignore files (so your working tree won't be polluted with files generated by automake)
  • SVN tag paths imported as real Git tags (so it's cleaner than using git-svn; you can see tags directly in your git log, as is normal for Git repositories)
  • some high-priority patches imported as git commits, rebased on top of proper commits in svn/* branch history. For example, see ff5732 on vogons/openglide-r4258 branch. This way you can easily cherry-pick commits you're interested in instead of hunting down patches and applying them manually.
  • Automatic CI builds (via GitHub Workflows) for Linux (Ubuntu 16.04, 18.04), macOS (10.14) and Windows (Server 2019), using GCC (3 versions), Clang (2 versions) and MSVC (1 version ATM). We also have scripts, that cause CI to fail if your branch introduces new warnings to selected builds (so you can find build issues early).
  • Static code analysis (Clang) included in CI (ATM reports can be downloaded via build artefacts, they are not published automatically yet).
  • Visual Studio solution files compatible with Visual Studio 2019 and libraries installed via vcpkg (see INSTALL file for up-to-date instructions)

It's still early and there are plenty of areas to improve - but it's really useful already :)

If we're missing some feature you need: tell us or create a GitHub issue.
If you want to improve something in the CI area: send PR for review.
If you want some specific patch imported: tell us.
If you develop/maintain a DOSBox feature and would like push access to the repo: let's talk.
If you are involved with the development of DOSBox upstream and want to cooperate: let's talk (I am especially interested in establishing contributing guidelines for DOSBox upstream, codifying coding convention, etc).

I will be monitoring this thread, but you can also find us on #dosbox-staging channel in Luxtorpeda Discord Server.
Code: Select all
| ← Ceci n'est pas une pipe
User avatar
dreamer_
Newbie
 
Posts: 43
Joined: 2019-5-17 @ 20:19

Re: dosbox-staging (DOSBox Git repo)

Postby Dominus » 2019-10-02 @ 15:02

hmm, seems more like you want to move things away from the DOSBox devs as fast as possible...
User avatar
Dominus
DOSBox Moderator
 
Posts: 8006
Joined: 2002-10-03 @ 09:54
Location: Ludwigsburg

Re: dosbox-staging (DOSBox Git repo)

Postby dreamer_ » 2019-10-02 @ 17:14

If I wanted that, I would create full-blown hard fork and focus on my own thing, instead of maintaining code parity with SVN and submitting patches upstream. I find using SVN really unproductive and time-consuming, so wanted to make the development process as easy and streamlined as possible. Using GitHub makes it easy to review code while using Git makes it very easy to merge/rebase branches, cherry-pick commits, split or squash them, create patch series and maintain CI environment.
Code: Select all
| ← Ceci n'est pas une pipe
User avatar
dreamer_
Newbie
 
Posts: 43
Joined: 2019-5-17 @ 20:19

Re: dosbox-staging (DOSBox Git repo)

Postby gdjacobs » 2019-10-03 @ 02:27

You're committing to periodic rebasing against DOSBox SVN. I guess I don't see where you're saving yourself any pain.

Have you talked with the DOSBox devs to see what they use in SVN that they can't get in Git? If you can address their concerns and present clear benefits to using Git, you'll make it easier and more likely for them to transition their tool chain in the future.
User avatar
gdjacobs
l33t++
 
Posts: 6698
Joined: 2015-11-03 @ 05:51
Location: The Great White North

Re: dosbox-staging (DOSBox Git repo)

Postby Ringding » 2019-10-03 @ 07:10

I guess this is one of the reasons for doing this exercise.
Ringding
Newbie
 
Posts: 34
Joined: 2016-1-05 @ 21:02

Re: dosbox-staging (DOSBox Git repo)

Postby Dominus » 2019-10-03 @ 07:25

Ringding wrote:I guess this is one of the reasons for doing this exercise.

Well, this is not really helping when you don't talk with the devs but just do it and then offer a chance to talk about it. It may even just drive the devs to point at this and say "why do you want US to switch to git, just use this!"
And this might lead to fragmentation...
User avatar
Dominus
DOSBox Moderator
 
Posts: 8006
Joined: 2002-10-03 @ 09:54
Location: Ludwigsburg

Re: dosbox-staging (DOSBox Git repo)

Postby Ringding » 2019-10-03 @ 07:41

Probably. But I seem to remember a similar discussion a few months ago where the outcome was exactly this – demonstrate the value and go from there. I think this is what is being done here.
Ringding
Newbie
 
Posts: 34
Joined: 2016-1-05 @ 21:02

Re: dosbox-staging (DOSBox Git repo)

Postby Dominus » 2019-10-03 @ 07:44

Nah, the outcome was: talk to the devs and see how to make them see the light.
User avatar
Dominus
DOSBox Moderator
 
Posts: 8006
Joined: 2002-10-03 @ 09:54
Location: Ludwigsburg

Re: dosbox-staging (DOSBox Git repo)

Postby dreamer_ » 2019-10-03 @ 08:01

gdjacobs wrote:You're committing to periodic rebasing against DOSBox SVN. I guess I don't see where you're saving yourself any pain.

For my own code, that is not merge'able upstream yet: yes. But it would need to happen anyway (except it's easier to do in Git than by manually re-applying patches). And it gives me an incentive for sending patches back upstream.

For keeping master largely in sync with the state of trunk: actually no :). I am committing to keeping most of my code changes **away** from master branch and regularly merge-in imports from svn/trunk branch (this way Git's DAG represents what actually happens with the code). You can see clearly what I mean by looking at recent history of master branch:

Image

Creating merge commits (Named "Merge branch svn/trunk" in above screenshot) is painless - I have it largely automated (not 100% automated though, I review the results before pushing them to svn/trunk branch). You can review my import script yourself: link (if you'll find something wrong with it, don't hesitate to create an issue or send PR). I also have improved version of this script in development on my local branch, will send it for review for master when ready (yes, commits not originating from SVN are being reviewed before reaching master - krcroft is kind enough to help with that, you can see example here: #7).

gdjacobs wrote:Have you talked with the DOSBox devs to see what they use in SVN that they can't get in Git? If you can address their concerns and present clear benefits to using Git, you'll make it easier and more likely for them to transition their tool chain in the future.

Yes, briefly, some time ago when I first arrived on the forum. If I understand correctly they are not interested in moving to Git at all and that's the end of story. And I understand - if some random dude from the internet approached me and offered to change workflow of my project to tool Blargh, I would reject it as well. This repo is my way of showing, that transition to Git is beneficial to the project (and I can infer from various comments throughout this forum - desired by the community at large).

Dominus wrote:And this might lead to fragmentation...

I'm afraid it's too late for that :( Subversion usage already resulted in fragmentation. All active DOSBox "hard-forks" I found moved to Git. The only active "soft-fork" (DOSBox-ECE) is maintaining patches (in other words: doing git-rebase, but manually - without automation and benefit of automatic 3-way merge conflict resolution offered by Git).
Code: Select all
| ← Ceci n'est pas une pipe
User avatar
dreamer_
Newbie
 
Posts: 43
Joined: 2019-5-17 @ 20:19

Re: dosbox-staging (DOSBox Git repo)

Postby krcroft » 2019-10-04 @ 05:23

I see the dosbox-staging repository as preventing fragmentation by acting as a rigorous proving ground in which new code can be honed and improved - until its sufficiently polished to consider by the core developers.

git, GitHub, and now the CI workflow, which dreamer continues to expand upon, have given me a massive advantage that I didn't have before; one that would have been insurmountable for me to maintain as an individual.

Now, every change I commit automatically gets built on a pile of VMs running in Microsoft's server farm.. OS X with two releases of Clang, the last three releases of Ubuntu (19.04, 17.04, and 16.04), and now two versions of visual studio on Windows. The instrumented binary is fed through clang's analyzer which has caught memory leaks, dead code, and logical problems -- all of which turned out to be correct and I've since addressed them. It's even found issues with the single-header codecs I'm using, so those authors have also been busy making fixes too, and passing on their thanks!

And yet, somehow, I get all of that for free... around the clock... without having a fat power bill or rack of PCs grinding away in my house. My company only just recently achieved something similar in-house, at great cost.

As for my own code, dreamer has provided very helpful feedback in how it can be adjusted specifically for improving its odds of acceptance upstream. More specifically, the new Windows builds and package manager will let me externalize the audio patch's Opus dependence, reducing its footprint (atleast until I can replace the Opus libraries with the single-header Opus decoder, which dr_ is working on!).

This has been a game-changer for me and is a very positive addition to the DOSBox community.
User avatar
krcroft
Member
 
Posts: 404
Joined: 2017-4-29 @ 15:07
Location: Ogden's Retreat

Re: dosbox-staging (DOSBox Git repo)

Postby ripa » 2019-10-05 @ 15:27

Thank you dreamer_! Cloning the upstream repo from SVN is a pain in the ass (takes over an hour using git-svn), and maintaining patches with plain SVN sucks.
ripa
Oldbie
 
Posts: 558
Joined: 2005-4-18 @ 00:53
Location: Finland

Re: dosbox-staging (DOSBox Git repo)

Postby digger » 2019-10-05 @ 16:51

Dominus wrote:Nah, the outcome was: talk to the devs and see how to make them see the light.


Okay, I'm going to write the following rant with the highest of respect and appreciation for the upstream DOSBox developers. Please keep that in mind.

Convincing a developer that Git is so much better than SVN is like trying to explain to someone who's never eaten [INSERT_FAVORITE_FOOD_HERE] before how delicious it is. They need to experience it for themselves. If only they would have a more open mind and just give it a try already.

The mere fact that there are many more forks and patches of DOSBox floating out there than anyone can count right now, that alone would make the case for a more advanced and branche/review/merge-friendly source control system such as Git. Switching to Git isn't just something that would benefit the DOSBox core developers (it would!) It would also make life so much easier for everyone else who would like to contribute.

Okay, let me just bite my lip and flat out ask the DOSBox developers: what concrete concerns are currently keeping you from considering a migration of the DOSBox code base to Git? We'd definitely like to make you "see the light", but this does require an open mind from you as well.

If we could convince you that the performing of reviews of contributions as pull requests in Git, and the ease of merging them on approval, would drastically reduce the effort and time that you currently need to spend on evaluating and merging the vast backlog of proposed fixes, new features and improvements... Would that be enough for you to give it a shot? It's not like Git lacks any features that only SVN provides, right? It seems that resistance to change, having to (briefly) step out of your comfort zones, is currently the only barrier to adopting Git. Or am I missing something?

Some of all those patches floating around are highly requested popular features and fixes that have been proven to work and have been gathering dust, pending acceptance for years. Having a centralized "staging" project where they all could be integrated and thoroughly tested together seems like a very good idea to me. And as a nice bonus, it also serves like a good example to the still sceptical DOSBox developers of how well Git would suit their project.

If you're still not convinced of Git being a good fit for DOSBox, then at least give this dosbox-staging initiative a chance, I say!
User avatar
digger
Member
 
Posts: 226
Joined: 2010-2-12 @ 18:15
Location: Amsterdam, the Netherlands

Re: dosbox-staging (DOSBox Git repo)

Postby DosFreak » 2019-10-05 @ 17:14

Threads with agendas are likely to be closed and I'd rather not see this thread closed.

If you want to ask the devs something then as Dominus said PM or email them. Don't bother with the forum because you're just going around and around in circles. Talk to the devs and stop bitching about it.

As for patches, just because there are patches out there for DOSBox doesn't mean they should be in official DOSBox and usually the loudest complainers are those who shouldn't be listened to. I mostly see complaints come up for patches for 9x guest support or patches for less than a dozen games and the patch is only compatible with Windows or for the boatload of Daum patches but the patches must be included in DosBox now or the world will end and DOSBox is a POS and old because it doesn't support them but an unofficial ver of DOSBox has the patch so DOSBox should too! Why oh why does DOSBox not include the patch? Oh woe is me!. Keep it sane people.

So:
Post this list of "Highly Requested Popular Features" with the link to the patches
Post the # of games affected
Post how much the patches affects the codebase and is likely to affect guest and host compatibility
Verify license status
Verify OS compatibility Windows 95+ or XP+, OSX, Linux
Verify compiler compatibility with the patch for OS support so that the patch works for Windows 95+ or at least XP+, OSX and Linux
Verify SDL1.2 / SDL2 compatibility or if SDL has to be modified or recompiled
Who is going to maintain the patch in DOSBox, is it worth including in DOSBox?
Are the patches compliant with C++03?
etc
User avatar
DosFreak
l33t++
 
Posts: 10499
Joined: 2002-6-30 @ 16:35
Location: Your Head

Re: dosbox-staging (DOSBox Git repo)

Postby Qbix » 2019-10-05 @ 19:55

Can you change that dosbox-staging to not include our email addresses in the main tree and on each commit ? We have our authors file structured the way it is intentionally.
(with not writing down the email every where on purpose, I get enough spam already. Thanks)

For me the svn commit numbers are important. Can they be included in the commit title and maybe remove the somewhat unneeded mention of it being imported from
svn.net ?

turn:
Code: Select all
2019-09-02 16:50 Peter Veenstra        │ o Fix bug 512, reported by philipp. (checking wrong variable to see if malloc was a success)

into:
Code: Select all
2019-09-02 16:50 Peter Veenstra        │ o [4254] Fix bug 512, reported by philipp. (checking wrong variable to see if malloc was a success)

And how to do it by all commits by default when done by the core devs ? (is that even possible?)

There are several reasons why I currently don't see the need to switch to git. But the lack of an easy to reference (ever increasing) number is one from a practical point of view, as a lot of notes and personal workflow is based around it.

I use git for some other projects, so I am somewhat familiar with it. Unlike some posters in this thread, I don't consider it the best for everything.
Water flows down the stream
How to ask questions the smart way!
User avatar
Qbix
DOSBox Author
 
Posts: 10947
Joined: 2002-11-27 @ 14:50
Location: Fryslan

Re: dosbox-staging (DOSBox Git repo)

Postby krcroft » 2019-10-05 @ 20:39

Qbix wrote:the lack of an easy to reference (ever increasing) number ....

A push hook can apply a incrementing numbered tag to every commit to master, 4268 for example. Those tags can be programmatically referenced in future operations, so they carry the same functional use as SVN's commit numbers.

Multiple investigations can be publically spun off on parallel branches before merging back to master (when the tag increment occurs), so would not cause divergent tag numbering, despite on-going work in the same repository.

Git's DAG gives it mastery over branching and merging, essentially eliminating the burden for collaborators to keep their branches in sync with master.

DosFreak - I've always found your build guides very useful. Check out - https://github.com/dreamer/dosbox-stagi ... /build.yml
This would be perfect place for your mingw and other builds that aren't covered yet, giving DOSBox even more build coverage.
GitHub launches the builds in parallel across VMs resulting in (https://github.com/dreamer/dosbox-stagi ... =252618651)
User avatar
krcroft
Member
 
Posts: 404
Joined: 2017-4-29 @ 15:07
Location: Ogden's Retreat

Re: dosbox-staging (DOSBox Git repo)

Postby DosFreak » 2019-10-06 @ 11:56

Thanks.

Any time I write a guide or automate I'm always torn between making it simpler or making it more complex.

I'd like to keep the guide simple for the common person but for automation I may have to spin off a separate guide. Something to think about.

Recently got my test desktop operational and am currently revamping the storage on my server (added 2x more drives so 100TB usable now) and am starting back up on updating the guides for Windows and Linux for mingw, mingw-w64 and VS. Then I need to work on OSX. After that then I'll look more into the scripting and automation.
User avatar
DosFreak
l33t++
 
Posts: 10499
Joined: 2002-6-30 @ 16:35
Location: Your Head

Re: dosbox-staging (DOSBox Git repo)

Postby dreamer_ » 2019-10-08 @ 13:01

digger wrote:Some of all those patches floating around are highly requested popular features and fixes that have been proven to work and have been gathering dust, pending acceptance for years. Having a centralized "staging" project where they all could be integrated and thoroughly tested together seems like a very good idea to me.

That's my impression as well. I imported few of those patches as git commits for easy cherry-picking, but all patches from Vogons threads I tested so far need some work and polish (usually due to lack of code review - patches work only on Linux or require additional changes in buildsystem or documentation).

I just turned-on wiki on GitHub project, so we could collaborate on a list of high-priority patches/features and start planning on how to get them upstreamed: dosbox-staging/wiki/DOSBox-patches. Please confirm, that wiki edits are open to everyone.

DosFreak wrote:Post this list of "Highly Requested Popular Features" with the link to the patches

Well, patches maintained by DOSBox-ECE are good candidates here. Let's start listing what prevents each patch from being upstreamed on wiki. There are others as well. Personally, I am not going to post any patches on Vogons forum, because I find it unsuitable for proper code review - I will commit my changes to Git repo and submit them to patches queue in SourceForge project, as long as it'll be reasonable.

DosFreak wrote:Verify OS compatibility Windows 95+ or XP+, OSX, Linux

Windows 95 or XP? Why would anyone do that in 2019 with DOSBox 0.75.x? And I would add OS/2 to short list - you can't expect devs to test their implementation on these OSes and, at the same time, have buildystem incompatible with up-to date Visual Studio on Windows 10 or require non-upstreamed patches for SDL1.2 to build in MinGW.

Qbix wrote:Can you change that dosbox-staging to not include our email addresses in the main tree …

Done.

Qbix wrote:… and on each commit? …

GitHub does not expose committer nor author emails via web interface, spammers can't get your mails by scrapping GitHub site.

I tested sending email to user.sourceforge.net before using these mails and… it is disabled by default in SourceForge nowadays (I haven't realized that SF actually forwards those mails for some users) - so if you're spammed by mails forwarded via SF I suggest changing user settings - there's an option to make mails generated by SF (e.g. by people posting issues or patches for the project) be delivered.

Otherwise, downloading Git repo and scrapping mails from content is not harder than:
Code: Select all
svn log --limit=10 https://svn.code.sf.net/p/dosbox/code-0/dosbox | grep "^r" | cut -d "|" -f2 | xargs printf "%s@users.sourceforge.net\n" | sort -u

… so, I am not going to rewrite repo history and break interaction for users, who already cloned the repo.

If you would like to create fresh history for official DOSBox Git repository, then it's as easy as replacing domain in this script.

Qbix wrote:For me the svn commit numbers are important. Can they be included in the commit title and maybe remove the somewhat unneeded mention of it being imported from svn.net?


Before I'll explain why SVN revision numbers should be avoided: I don't know what program generates such unhelpful log and I hope it shows actual Git commit hashes somewhere else in UI, otherwise this log is pretty much useless and I would recommend using better software. If you need GUI to visualise DAG, then gitk is pretty good (despite ugly GUI), works on any OS, is very fast and is usually distributed with the official Git client by default (some distros have it in a separate package).

Git uses hashes to properly illustrate non-linear development process of software development - it's impossible to reliably assign total order in decentralized environment, and even in centralized environment it is misleading and results in user errors.

Mention of being imported from SVN is important (at least in transitional phase when SVN is still being used) - it utilizes Git feature called message trailers (more info in man page: git-interpret-trailers(1)).

This way you can identify a commit in log e.g. this way:

Code: Select all
$ git log --oneline --grep="@4254"
25e01bd0 Fix bug 512, reported by philipp. (checking wrong variable to see if malloc was a success)


But also use parse it for with purpose of automation. For example, GitHub utilizes it to automatically close issues when commit with trailer "Fixes: #<issue-number>" is pushed to master.

To make it easier for your workflow, I prepared a script that uses extensive git-log(1) options to format the log: git-rlog.sh. Example output:
Code: Select all
$ git rlog
a888d740 2019-10-08 02:06 Patryk Obara         [] Replace an authorsfile with a script                             
5275860e 2019-10-08 00:40 Patryk Obara         [] Add alternative for git-log for SVN users                       
f7a5080c 2019-10-04 00:04 Patryk Obara         [] Update allowed warnings limit                                   
a323e173 2019-10-03 23:44 Patryk Obara         [] Merge branch 'svn/trunk'                                         
60204619 2019-10-03 20:03 Peter Veenstra       [4267] - Fix url to forum. - Fix Bit8u instead of char weirdness for i..
7f979234 2019-10-03 16:35 ripsaw8080           [4266] Bit 2 of video status register always set. Satisfies a strange ..


Of course, refs can be specified according to gitrevisions(7), so operators like ".." and "..." work as expected, e.g.
Code: Select all
$ git rlog origin/svn/trunk..origin/svn/0_74_3
will show all the commits delivered to svn/0_74_3 branch since it branched-off the svn/trunk. Although it's a bit easier to use normal git-log, due to bash autocompletion (branch names, tag names, etc can be all auto-completed).

Also, please note that in Git, there's a convention for formatting commit messages (here's great article describing it in detail: https://chris.beams.io/posts/git-commit/) and editors (e.g. vim) help users with upholding this convention by automatic colouring and wrapping of commit messages.

And one more thing: SVN does not support distinction between author and committer (rendering SVN annotate almost useless), but Git does; git-rlog.sh displays committer and commit date to be in line with SVN; Git prefers to show author by default (which makes more sense in most contexts), but author and committer can be displayed e.g. with "git log --full".

Why SVN revisions should be avoided

SVN revisions seem like easy and nice feature, but they are in fact version control equivalent of "false friends" - they are misleading, and - combined with SVN "feature" called "Mixed Revision Working Copies" (which basically means: each directory in your SVN working copy is checked out in its own revision, so "svn update" performed in a subdirectory might leave you with a code state not intended by anyone and seemingly random breakage). Experienced SVN users usually trained themselves to avoid behaviour resulting in breakage, but new users experience it often.

Contrary to the popular practice amongst SVN users, repository revision numbers do NOT identify commits. Single revision change might span several SVN "branches" or SVN "tags" (for users, who don't know: SVN does not have branches or tags as features - only "copies" - trunk/branches/tags directories are only a popular usage convention). To identify an actual code change, a tuple <SVN path, SVN revision> is needed. Often path can be inferred from the context of discussion, but simply stating revision number *will* mislead someone into applying patch to the wrong path (and, btw, SVN lacks a feature for automatic application of patches - git has set of git-format-patch, git-am and git-apply commands to deal with it easily with added bonus of preserving author information and original commit message).

Another area where SVN revisions are misleading is "tag" handling. Users might see a tag being created with revision N, so what do they do? They might try to checkout their existing working copy to revision N, which might result in completely unrelated code. To correctly checkout tagged code in SVN, user needs to do this dance:

Code: Select all
$ svn log -v --stop-on-copy ^/dosbox/tags/RELEASE_0_74_2

# now, manually translate information:  "(from /dosbox/branches/0_74_2:4155)"
# into "^/dosbox/branches/0_74_2@4155",
# assuming the tagging was done correctly by original SVN committer

$ svn checkout ^/dosbox/branches/0_74_2@4155


(You might say: "just checkout tag path directly", but that risks you committing a change to a tag, which is a no-no).

In context of DOSBox this is not a bigger problem only because the rate of new patches arriving in svn/trunk is so slow and there are only few direct contributors.
Code: Select all
| ← Ceci n'est pas une pipe
User avatar
dreamer_
Newbie
 
Posts: 43
Joined: 2019-5-17 @ 20:19

Re: dosbox-staging (DOSBox Git repo)

Postby Kisai » 2019-10-09 @ 05:40

dreamer_ wrote:
digger wrote:Some of all those patches floating around are highly requested popular features and fixes that have been proven to work and have been gathering dust, pending acceptance for years. Having a centralized "staging" project where they all could be integrated and thoroughly tested together seems like a very good idea to me.

That's my impression as well. I imported few of those patches as git commits for easy cherry-picking, but all patches from Vogons threads I tested so far need some work and polish (usually due to lack of code review - patches work only on Linux or require additional changes in buildsystem or documentation).

I just turned-on wiki on GitHub project, so we could collaborate on a list of high-priority patches/features and start planning on how to get them upstreamed: dosbox-staging/wiki/DOSBox-patches. Please confirm, that wiki edits are open to everyone.


The problem with a lot of the patches posted on vogons, is that they only work against specific revisions of SVN, and only against vanilla SVN unless otherwise stated. Like when I "rebase" the patches I have on my hard drive, I often end up using the git tool anyway to apply the patches since the svn tool will just refuse outright to patch without a working svn pull. Anyway, after I apply the patches, I then do the reverse, and make one patch against the svn vanilla so that hopefully the next time it applies cleanly. This is why I haven't posted some of the WIP stuff to vogons, because prototype phases just change the respective dosbox.cpp or hardware.cpp files, and refined versions put a #define C_FEATURE #endif around an include so that less editing has to be done to the SVN and the patch tools can easier find where to patch in a feature.

dreamer_ wrote:
DosFreak wrote:Post this list of "Highly Requested Popular Features" with the link to the patches

Well, patches maintained by DOSBox-ECE are good candidates here. Let's start listing what prevents each patch from being upstreamed on wiki. There are others as well. Personally, I am not going to post any patches on Vogons forum, because I find it unsuitable for proper code review - I will commit my changes to Git repo and submit them to patches queue in SourceForge project, as long as it'll be reasonable.



The four main patches I always apply, in order:
SDL2 (and cd-rom compatibility patch)
NukedOPL
MT32
FluidSynth

Obstacles:

The FluidSynth patch will work on the 1.x version of Fluidsynth only, and specifically only the version with glib reverted to the pre-glib version if you want it to compile on MSVC (glib has fragile portability issues.) I did backport pieces of 1.1.11 and 2.0's drivers to the 1.1.6-noglib that I use when I build the fluidsynth patch. Since the fluidsynth developer seems insistent on depreciating and refactoring some of 1.1.x's functionality in 2.0. If FluidSynth is enabled, it should be shipped with a legally redistributable GM/GS SF2 font that can be used when it's turned on. There's no difference between SDL1 and SDL2 with the Fluidsynth patches, though some patches don't have the dosbox audio callback, and some don't have the external drivers. So obviously if you have a Fluidsynth version with the SDL2 driver integrated you can't compile it against SDL1.

For MT-32 is the lack of a "rom-less" version that can be permanently integrated. So on one hand MT-32 support is pretty much required to correctly emulate the intended audio of about 100 dos-era games, but because of the dubious copyright over the MT32 roms with Roland, there might never be a point where Roland blesses distribution of the MT32/CM32L ROMS with stock dosbox. I'm think people are smart enough to recognize this, but leaving it as a compiler directive may side-step it, and even having the functionality enabled behind a "experimental" switch in the configuration/command line may be the other solution. There's only one line in the MT-32 patch that changes between SDL1 and SDL2, and that's the line around the thread invocation.

There shouldn't be any obstacle for SDL2 other than the glide, or reelmagic patches needing different resolutions. The code may compile, but because different platforms handle video and audio differently, there's no guarantee that any specific patch produces audio/video with the same latency/quality.

dreamer_ wrote:
DosFreak wrote:Verify OS compatibility Windows 95+ or XP+, OSX, Linux

Windows 95 or XP? Why would anyone do that in 2019 with DOSBox 0.75.x? And I would add OS/2 to short list - you can't expect devs to test their implementation on these OSes and, at the same time, have buildystem incompatible with up-to date Visual Studio on Windows 10 or require non-upstreamed patches for SDL1.2 to build in MinGW.



If an OS is no longer under development I'd probably recommend not trying to support it unless someone is willing to actually support it themselves. Win95 and OS/2 are so old that the number of people who actually do this probably can't run a recent version of DOSBOX in the first place. Microsoft is officially no longer supporting Win7 this year, so people still using Win7, use it at their own risk. Binaries produced with MSVC 14.2 (2019) only produce binaries that work on Win10 (32-bit x86 , x86-64, ARM32, ARM64) unless the Windows SDK 140 (2015) is installed that enables XP binaries. MSVC can also use clang but I've not tried to build DOSBOX with it.
Kisai
Member
 
Posts: 134
Joined: 2010-5-05 @ 08:04

Re: dosbox-staging (DOSBox Git repo)

Postby krcroft » 2019-10-09 @ 08:08

Calling out for Windows build help!

I'm using a GitHub workflow to build DOSBox under MSYS2, MinGW 64 and 32bit, and I cannot solve two link errors, which appear to be looking for midi and serial port symbols.
I've tried linking with ld vs g++, static vs dynamic, explicitly including various windows libraries (gdi, ole, w2), explicitly adding gcc libraries (-static-libgcc -static-libstdc++), and combinations of these groups - but to no avail.

g++ as the linker appears to the doing the right thing by added these itself: -L/mingw64/lib -lmingw32 -lSDLmain -lSDL -mwindows -lpng -lz -lSDL_net -lopengl32; however midi and serial functions aren't being picked up under the -mwindows set of libraries.

You'll also notice the missing symbols start with __imp, where as some people online mention the actual symbols in the libraries do not start with __imp, so they flipped some declare spec (which I haven't figured out).

Hoping someone has already solved such issues or might have some tips on what to try next. Here's the final link line, and the full log is attached:

Code: Select all
2019-10-09T07:49:47.2647365Z g++  -fstack-protector -Wall -O2 -pipe -mno-ms-bitfields   -fstack-protector -Wall -O2 -pipe -o dosbox.exe dosbox.o  cpu/libcpu.a debug/libdebug.a dos/libdos.a fpu/libfpu.a  hardware/libhardware.a gui/libgui.a ints/libints.a misc/libmisc.a shell/libshell.a hardware/mame/libmame.a hardware/serialport/libserial.a libs/gui_tk/libgui_tk.a libs/decoders/libdecoders.a libs/decoders/internal/lib/libopusfile.a libs/decoders/internal/lib/libspeexdsp.a libs/decoders/internal/lib/libopus.a libs/decoders/internal/lib/libogg.a -L/mingw64/lib -lmingw32 -lSDLmain -lSDL -mwindows -lpng -lz -lSDL_net -lopengl32

2019-10-09T07:49:47.2648169Z C:/tools/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: gui/libgui.a(midi.o):midi.cpp:(.text$_ZN17MidiHandler_win329PlaySysexEPhy[_ZN17MidiHandler_win329PlaySysexEPhy]+0x63): undefined reference to `__imp_midiOutPrepareHeader'
2019-10-09T07:49:47.2648412Z C:/tools/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: gui/libgui.a(midi.o):midi.cpp:(.text$_ZN17MidiHandler_win329PlaySysexEPhy[_ZN17MidiHandler_win329PlaySysexEPhy]+0x94): undefined reference to `__imp_midiOutLongMsg'
2019-10-09T07:49:47.2648648Z C:/tools/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: gui/libgui.a(midi.o):midi.cpp:(.text$_ZN17MidiHandler_win324OpenEPKc[_ZN17MidiHandler_win324OpenEPKc]+0xa2): undefined reference to `__imp_midiOutOpen'
2019-10-09T07:49:47.2649130Z C:/tools/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: gui/libgui.a(midi.o):midi.cpp:(.text$_ZN17MidiHandler_win324OpenEPKc[_ZN17MidiHandler_win324OpenEPKc]+0x2d5): undefined reference to `__imp_midiOutGetNumDevs'
2019-10-09T07:49:47.2649368Z C:/tools/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: gui/libgui.a(midi.o):midi.cpp:(.text$_ZN17MidiHandler_win324OpenEPKc[_ZN17MidiHandler_win324OpenEPKc]+0x329): undefined reference to `__imp_midiOutGetDevCapsA'
2019-10-09T07:49:47.2649615Z C:/tools/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: gui/libgui.a(midi.o):midi.cpp:(.text$_ZN17MidiHandler_win324OpenEPKc[_ZN17MidiHandler_win324OpenEPKc]+0x35d): undefined reference to `__imp_midiOutOpen'
2019-10-09T07:49:47.2649848Z C:/tools/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: gui/libgui.a(midi.o):midi.cpp:(.text$_ZN17MidiHandler_win324OpenEPKc[_ZN17MidiHandler_win324OpenEPKc]+0x4b5): undefined reference to `__imp_midiOutGetDevCapsA'

2019-10-09T07:49:47.2650065Z C:/tools/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: hardware/serialport/libserial.a(misc_util.o):misc_util.cpp:(.text+0xcd): undefined reference to `__imp_getpeername'
2019-10-09T07:49:47.2650464Z C:/tools/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: hardware/serialport/libserial.a(misc_util.o):misc_util.cpp:(.text+0x126): undefined reference to `__imp_getsockname'


Here's the current workflow YAML that's driving this on GitHub.
- https://github.com/dreamer/dosbox-stagi ... /msys2.yml

And the graveyard of prior failed attempts and their logs:
- https://github.com/dreamer/dosbox-staging/actions
Attachments
msys2_mingw.txt
(415.78 KiB) Downloaded 24 times
User avatar
krcroft
Member
 
Posts: 404
Joined: 2017-4-29 @ 15:07
Location: Ogden's Retreat

Re: dosbox-staging (DOSBox Git repo)

Postby jmarsh » 2019-10-09 @ 08:24

Normally you'd get those functions from winmm.lib and ws2_32.lib, but I can't vouch for mingw having the same libs as VS. They're not linked by default (like kernel32.lib and friends) so not surprising they're not included by using -mwindows.
jmarsh
Member
 
Posts: 317
Joined: 2014-1-04 @ 09:17

Next

Return to DOSBox Development

Who is online

Users browsing this forum: Bing [Bot] and 3 guests