VOGONS


First post, by ssokolow

User metadata
Rank Member
Rank
Member

One of the hobby projects I've started to poke at again (mind you, very slowly) is an effort to build an installer builder for DOS. (InnoSetup, NSIS, WiX, and MS Setup 2.0 serve everything from Windows 3.1 onward, but I couldn't find anything for DOS that was both legally free and as polished as the professional installers of the period.)

Up until now, I've mostly been focusing on infrastructural bits where I don't need feedback. (writing the most compact alternative to graph.h that I can, finding some public domain unzipping code I can adapt to function in real-mode, surveying the sizes of period installers, etc.)

Now, I'm getting to the point where I need to start designing bits that would actually be experienced by other developers using the code to build installers, so I thought it'd be a good idea to start getting other people's input.

Since I'm taking a bit of a rose-coloured glasses approach to retro-nostalgia (trying to paper over things in the vein of "Really? Windows 9x didn't have Ctrl+A bound to "Select All"?), my goal is to have an installer that, like InnoSetup 1.2.x for Windows 3.1, is a single-file distributable rather than a Zip or self-extractor containing loose installer files.

Here's my idea so far:

  • Use Open Watcom C/C++ and a real-mode target with my own hand-tuned library of functions (many wrapping inline assembly) as a compromise between ease of development and size. (My research shows period installer sizes cluster around 15K or 50K and using DPMI or graph.h starts me off at a ~35K Hello World while I can get a very fancy-looking Hello World as a test for a set of INT 10h API wrappers in 1.6K with real-mode Open Watcom and it makes it trivial to support both cross-compilation and building on genuine hardware... 996 bytes of which is the Open Watcom C runtime at the smallest size I could accomplish.)
  • To make installers easy to distribute yet easy to pick apart, design the runtime as a Zip SFX stub that pulls human-readable control scripting from a scripts folder inside the Zip file.
  • To keep parser size down, have a compiler which converts InnoSetup-style declarative project definitions into an imperative scripting language designed to be easy to parse with minimal code.
  • To allow reusing syntax highlighting definitions and to share the tokenizer between the argument parser and the scripting interpreter, make the internal control scripting DOS batch syntax with a different set of internal commands.
  • To save on code footprint, using trees of folders inside the Zip file if a hierarchical key-value store is needed.
  • To allow debugging and unit testing, implement a command-line flag which writes status messages to a specified file, with COM1 being the default since DOSBox can map that to a telnet server for the integration test harness running on the host OS to connect to.
  • For nostalgia and retro-utility, design it to be able to replicate the look and feel of iconic installers of the period by providing drawing primitives based on INT 10h functions and bundling theme definitions for the compiler which are just libraries of script functions which satisfy a certain API. (I've already taken reference screenshots for the vast majority of my installers.)
  • Since the tokenizer will be doing half the work anyway in a language with no nested block syntax, I figured I might as well write a pretty-printing reformatter for the control scripting that could be hooked into something like ALE for Vim for format-on-save. (In which case, maybe I'll also add a switch to use it for reformatting .BAT files.)
  • More modern standards for code quality through a judicious use of ifdefs to hide things like near and far from splint and cccc. (Already set up and part of the build process when the Watcom WMAKE Makefile is run on Linux.)
  • Complete and polished Doxygen documentation (I take pride in how I document release-ready projects and, for something like the control scripting for the project, writing the documentation beforehand as a spec helps me to think through the implications of the design.)
  • Some kind of unattented install mode... probably by designing the API such that every user prompt is bound to a named variable and having a command-line flag which pre-fills the variables from a file and skips prompting the user. (I'll need to think about ways to make this compatible with name reuse in the context of looping.)

I've also been thinking that it'd make a nice way to quickly and easily build pretty DOS menus if I provide support for having a menu selection result in "write selected command to a batch file and then exit" so the conventional memory-freeing trick for menus can apply.

So, the kinds of questions I'm asking you guys are:

  • What kinds of things would you want to build with something like this?
  • What do you think of the "Control scripting that looks like a cleaned up DOS batch file" idea?
  • GOTO in DOS batch is nostalgic, but has the potential for ugly code and bugs akin to forgetting a break in a C switch. What's your opinion on supporting and promoting CALL :label as the recommended way, and having a label implicitly end the previous label's block? (ie. ":label" would have an implicit "exit /b" before it when CALLed and the reformatter would treat ":label" as akin to "def label() :" in Python... whether it'd indent the block body would be something I'd have to experiment with.)
  • Which do you find more unpleasant? Syntax highlighting where the list of valid keywords is significantly divergent from the language or a language that looks familiar but has the keywords behave differently? (eg. possibly having ERASE affect the screen rather than being a synonym for DEL or having VOL prompt to swap disks.)
  • How important is multi-volume (ie. floppy disk set) support to you?
  • There are a lot of three-letter extensions that FILExt has no entries for either beginning or ending with IH which could be used for things like ".tih = Theme for ..." and ".cih = Control Script for ...". What do you think of a name like "Installer4H"? (With a FAQ explaining that it's short for something like "Installer for Hobbyists" because, with Intel phasing out support for non-UEFI boot, hobbyists are the only people who care about writing new things for DOS anymore.)
  • ...other questions that will no-doubt reoccur to me when I'm not struggling with being over-tired.

Internet Archive: My Uploads
My Blog: Retrocomputing Resources
My Rose-Coloured-Glasses Builds

I also try to announce retro-relevant stuff on on Mastodon.

Reply 1 of 12, by BloodyCactus

User metadata
Rank Oldbie
Rank
Oldbie

i have a neat dos installer that has some of those requirements

http://mega-tokyo.com/si103.zip

its an installer rather than setup, as doing tricks like modifying config.sys/autoexec.bat can get tricky when it comes to multimenu stuff... and this was originally written in 97...

--/\-[ Stu : Bloody Cactus :: [ https://bloodycactus.com :: http://kråketær.com ]-/\--

Reply 2 of 12, by ssokolow

User metadata
Rank Member
Rank
Member
BloodyCactus wrote on 2020-08-29, 20:31:

i have a neat dos installer that has some of those requirements

http://mega-tokyo.com/si103.zip

its an installer rather than setup, as doing tricks like modifying config.sys/autoexec.bat can get tricky when it comes to multimenu stuff... and this was originally written in 97...

UI-wise, it's definitely better than the two frontends I found in old freeware/shareware archive CDs for putting a TUI in front of a self-extracting archive.

I'm not a fan of it apparently using a custom packfile format, but I'm still willing to mention it as a "best known option so far" effort in my Free Installer Creators for All Eras blog post... except for one small problem: I can't find explicit license information in what that file installs to. The two SFX TUIs I found explicitly call themselves freeware.

(Same problem as with the code from this page on making tinier Open Watcom binaries. Offered up by people who don't understand that, in the eyes of the law, no stated license = All Rights Reserved... which is a shame. I've already started to use bit of C and inline assembly to implement things like my own graph.h and my own strlen to shrink the binary size, so I don't anticipate problems implementing my own I/O, but I'm nowhere near skilled enough to create my own heap allocator and C runtime bootstrap.)

Given that your name is BloodyCactus and it says it's by Bloody Cactus Development Labs, I'll assume it's your work. Ideally, there are three things I'd like to see for inclusion on my list:

  1. A page I can permalink people to for an introduction to what it is and a "latest version" download.
  2. A license that allows redistribution of si103.zip and its successors so that people who link to you can also locally host a mirror in case you get hit by a bus. (Same principle as is recommended for academic paper authors who cite things. Keep backups of what you reference because dead links are a plague.)
  3. A license that explicitly allows installers built with it to be redistributed for any purpose including commercial.

(Currently, I have the un-published progress I've made on my installer under the Zlib license since it doesn't require people to bloat out their potentially floppy-bound binaries with license text but does require that any altered versions of the source be clearly marked as such.)

Internet Archive: My Uploads
My Blog: Retrocomputing Resources
My Rose-Coloured-Glasses Builds

I also try to announce retro-relevant stuff on on Mastodon.

Reply 3 of 12, by BloodyCactus

User metadata
Rank Oldbie
Rank
Oldbie

well yeah i wrote it in 97, its just a utility, freeware. nobody cared about that stuff back then. i dont care about distribution, like the masses of anything that got uploaded to bbs's back in the day.

also, i think your going down the wrong road trying to rewrite everything to your own functions. when the installer or script compiler runs its the only thing running, you dont need to optimise 10 bytes off strlen, you have all of base ram to yourself. I dont understand what you think the gains are for the massive amount of work and testing required to write your own heap allocator and rewriting libc functions.

--/\-[ Stu : Bloody Cactus :: [ https://bloodycactus.com :: http://kråketær.com ]-/\--

Reply 4 of 12, by ssokolow

User metadata
Rank Member
Rank
Member
BloodyCactus wrote on 2020-08-30, 15:02:

well yeah i wrote it in 97, its just a utility, freeware. nobody cared about that stuff back then. i dont care about distribution, like the masses of anything that got uploaded to bbs's back in the day.

Would you mind either saying that in a permalinkable download page I can link to from my blog post or building a new install bundle containing a license file to that effect?

BloodyCactus wrote on 2020-08-30, 15:02:

also, i think your going down the wrong road trying to rewrite everything to your own functions. when the installer or script compiler runs its the only thing running, you dont need to optimise 10 bytes off strlen, you have all of base ram to yourself. I dont understand what you think the gains are for the massive amount of work and testing required to write your own heap allocator and rewriting libc functions.

It's not about memory consumption but on-disk footprint. I want to cram as much functionality as possible into 15KiB or, failing that, overshoot it by as little as possible. (My survey of the sizes of real-world period installers including their control scripting shows a bimodal distribution with peaks at 15KiB and 50KiB and a minimum of 10.9KiB for one version of the Lemmings installer.)

As for rewriting libc functions, writing my own wrappers for INT 10h APIs let me do all the graphics I needed, plus a test screen for all of them, in 1.6KiB, while just a plain "Hello World" positioned at a non-default position costs over 35KiB with graph.h from Open Watcom's libc.

Likewise, rewriting a strlen suitable to the project is trivial. Just a while loop that tests whether the character is null and increments a counter. (I already use counted strings of the same design that INT 10h uses anyway, so the purpose of strlen is mainly for the function that makes a string slice (to borrow Rust terminology) from a C string. Counted strings are also useful for reducing the amount of copying needed to do parsing, since you don't need to inject a null to take a smaller piece of a larger string.)

The main reason I'd like to get rid of Open Watcom's C runtime bootstrap is that it's 996 bytes for a void main(void) {}, consisting of more than half the size of my graphics API+test binary. (I've implemented my own argc/argv handling because I wanted to share the code and Open Watcom doesn't provide a way to reuse the tokenizer they provide to build a parser for a script interpreter.)

Internet Archive: My Uploads
My Blog: Retrocomputing Resources
My Rose-Coloured-Glasses Builds

I also try to announce retro-relevant stuff on on Mastodon.

Reply 5 of 12, by BloodyCactus

User metadata
Rank Oldbie
Rank
Oldbie
ssokolow wrote on 2020-08-30, 16:37:

Would you mind either saying that in a permalinkable download page I can link to from my blog post or building a new install bundle containing a license file to that effect?

i put in a 'its freeware' in the install.txt and reuploaded it. i dont know how long that link will last, i have not bothered to archive all my stuff on the net.

ssokolow wrote on 2020-08-30, 16:37:

It's not about memory consumption but on-disk footprint. I want to cram as much functionality as possible into 15KiB

15kb.. thats odd because its not an even cluster size so your basically 'wasting' space on a hd which is going to be a multiple of 2. Only FAT12 will have a 1 or 2 sectors per cluster size to evenly fit 15kb. 15kb.. is great if your targeting physical media which 99% of the time floppy disks are fat12.. anything else will not be fat12. I think since your writing this today and not 30 years ago, you should probably round up to 16kb as that fits neatly into fat16/fat32 clustering without wasting space.

--/\-[ Stu : Bloody Cactus :: [ https://bloodycactus.com :: http://kråketær.com ]-/\--

Reply 6 of 12, by ssokolow

User metadata
Rank Member
Rank
Member
BloodyCactus wrote on 2020-08-30, 19:02:

i put in a 'its freeware' in the install.txt and reuploaded it. i dont know how long that link will last, i have not bothered to archive all my stuff on the net.

Thanks. I'll mirror a copy on my blog when I add it to the list.

BloodyCactus wrote on 2020-08-30, 19:02:

15kb.. thats odd because its not an even cluster size so your basically 'wasting' space on a hd which is going to be a multiple of 2. Only FAT12 will have a 1 or 2 sectors per cluster size to evenly fit 15kb. 15kb.. is great if your targeting physical media which 99% of the time floppy disks are fat12.. anything else will not be fat12. I think since your writing this today and not 30 years ago, you should probably round up to 16kb as that fits neatly into fat16/fat32 clustering without wasting space.

It's intended to work as a Zip SFX stub, so the stuff to unpack will be concatenated onto the end of it. (I say "intended" because I have plans for an unpacked fallback mode for rapid iteration during development.)

Having a self-contained single-file distributable without making an external unzipper mandatory, like the 16-bit versions of InnoSetup, was one of the design goals from the very beginning. Otherwise, I could just use V8Power to hack together something using batch files like FreeDOS does.

(That aside, I do also want it to be suitable for anyone who wants to write New Old Stock floppies and sell physical copies of their modern retro games. That's why I'm trying to keep the size down.)

Also, it's just a target, not a hard limit. If I can make it work in under 10.9KiB and beat the smallest self-contained installer I surveyed, that'd be even better. If I have to put it in the 15-50KiB range to accomplish the feature goals, so be it but I'll try to avoid it.

Internet Archive: My Uploads
My Blog: Retrocomputing Resources
My Rose-Coloured-Glasses Builds

I also try to announce retro-relevant stuff on on Mastodon.

Reply 7 of 12, by BloodyCactus

User metadata
Rank Oldbie
Rank
Oldbie
ssokolow wrote on 2020-08-30, 19:17:

(That aside, I do also want it to be suitable for anyone who wants to write New Old Stock floppies and sell physical copies of their modern retro games. That's why I'm trying to keep the size down.)

makes sense. I have an option in mine to reserve Xkb on disk one, if you choose to use disk sizes to save space for things like a readme.txt, the install app itself etc, and this reserved space doesn't bleed into subsequent images.

i think most people just created an installer of max size or had apps that fit under 1.44mb. i did test multiple 360 + 720 spanning images but that was a long time ago.

--/\-[ Stu : Bloody Cactus :: [ https://bloodycactus.com :: http://kråketær.com ]-/\--

Reply 8 of 12, by ssokolow

User metadata
Rank Member
Rank
Member
BloodyCactus wrote on 2020-08-30, 19:55:

makes sense. I have an option in mine to reserve Xkb on disk one, if you choose to use disk sizes to save space for things like a readme.txt, the install app itself etc, and this reserved space doesn't bleed into subsequent images.

*nod* Not a high priority yet, given that splitting across multiple floppies is more a v2.0 thing given the hassle involved in splitting Zip archives, but definitely something I was aware of from Microsoft Setup Toolkit from the Windows 3.1 SDK. (MS Setup 2.0 is, thankfully, freeware.)

Internet Archive: My Uploads
My Blog: Retrocomputing Resources
My Rose-Coloured-Glasses Builds

I also try to announce retro-relevant stuff on on Mastodon.

Reply 9 of 12, by BloodyCactus

User metadata
Rank Oldbie
Rank
Oldbie

hey I have my upload a permanent url

ftp://bloodycactus.com/pub/si103.zip

--/\-[ Stu : Bloody Cactus :: [ https://bloodycactus.com :: http://kråketær.com ]-/\--

Reply 10 of 12, by ssokolow

User metadata
Rank Member
Rank
Member
BloodyCactus wrote on 2020-08-31, 11:20:

hey I have my upload a permanent url

ftp://bloodycactus.com/pub/si103.zip

Thanks. 😀

(It's not ideal when browsers are phasing out built-in FTP support, but a local mirror will solve that.)

If I don't get side-tracked, I'll add it to the list later today.

Internet Archive: My Uploads
My Blog: Retrocomputing Resources
My Rose-Coloured-Glasses Builds

I also try to announce retro-relevant stuff on on Mastodon.

Reply 11 of 12, by BloodyCactus

User metadata
Rank Oldbie
Rank
Oldbie

a web page should be doable.

https://bloodycactus.com/files.html

oh man I feel like I took over your thread with my own stuff and I did not intend for it to be so 🙁

--/\-[ Stu : Bloody Cactus :: [ https://bloodycactus.com :: http://kråketær.com ]-/\--

Reply 12 of 12, by ssokolow

User metadata
Rank Member
Rank
Member
BloodyCactus wrote on 2020-08-31, 14:15:

a web page should be doable.

https://bloodycactus.com/files.html

Thanks. Barring any tweaks you propose, this is what the title line for the entry will look like:

Installer v1.03 by Bloody Cactus Development Labs (HTTPS, FTP, Local Mirror)

EDIT: Done.

BloodyCactus wrote on 2020-08-31, 14:15:

oh man I feel like I took over your thread with my own stuff and I did not intend for it to be so 🙁

No biggie. In the end, the primary goal of my project is to ensure that there are free options to meet all needs, so this is topical in the "provide the best possible 'in the mean time' option" sense. God only knows how long it'll take me to find time to finish mine.

Internet Archive: My Uploads
My Blog: Retrocomputing Resources
My Rose-Coloured-Glasses Builds

I also try to announce retro-relevant stuff on on Mastodon.