VOGONS

Common searches


First post, by Guest

User metadata

Hi,

For a long while now I've been been a fan of so-called abandonware and emulation and noticed that almost all other platforms have a method for auditing and organising roms, rom sets, or disk images (For example, Mame has ClrMAMEPro, TOSEC takes care of disk and tape based media generally and GOODRoms / RomCenter take care of binary cartridge dumps for systems like the Super Nintendo).

What I propose initially is a discussion about how we could consider achieving the same thing for DOS games, in conjunction with DOSBox. A good starting example as a point of reference would be the MAME frontend, MAME32 for Windows.

Questions must be raised though on the format in which games are stored - and whether Warez rips are valid for inclusion (should copy protection mechanisms prohibit the playing of a game).

Some initial thoughts:

1) Very old games which booted directly from floppy disk could simply be stored as a disk image.

2) Games to be stored as single compressed files and decompressed when a game is launched (with a cache for larger games).

3) Savegames would be derived from diff'ing the game folder on DOSBox exit.

4) Icons, artwork, history and screenshots would be nice. Another thing would be a configuration profile system (as in dosbox speed settings done intelligently and preset autoexec / config files).

5) CD Images as .ISO files. Perhaps investigate compression of ISO data.

6) Dealing with split sets - as in games which have had patches released or differing versions (international / cracked) - perhaps think about doing a similar thing to MAME.

OK - These are just a few thoughts, and if there is interest around, I'll set up a wiki. Unfortunately, I'm only a VB coder, and although I'd be proficient enough to write a front end with most of the features supported, I'd probably be shunned by the developer elite! 😉

So lets get discussing. It'd be great if we could help preserve history. It'd be a mammoth task, but if you look at sites like pouet.net - you'll find that huge databases of information regarding old school computer stuff is achievable with a bit of applied skill and structure.

rua (posting anonymously until my account is activated)

Reply 1 of 9, by HunterZ

User metadata
Rank l33t++
Rank
l33t++

My two cents:

It's a good idea, but due to the nature of PC games it would be extremely complex to come up with a good way to differentiate a "good rip/copy/whatever" of a PC game from a bad one. Some considerations that come to mind are:

  • PC games can have hundreds of files each, whereas even arcade ROMs are usually comprised of less than a dozen small files
  • Old games are often distributed in pre-installed form, which is equally valid for playing to an image of the original CD(s) or floppy disk(s)
  • Not all files (e.g. saved game, configuration data and readme files - or even executables in the case of games that can be run with third-party engines such as ScummVM) may be necessary for a game to run
  • Archive and image files come in many formats, each with many sub-options (e.g. zip/arj/rar, iso/bin-cue/nrg)
  • Achieving wide-spread support for any kind of standardized format for packaging and distributing old PC games will probably be an extremely difficult task

I also don't particularly like the idea of always running games from archives that are extracted and re-created automatically - for big games it would be slow and take up extra disk space. I see you addressed this concern with a caching idea, but I'm still skeptical 😀

The idea of cataloging old games somehow so that they can be recognized by auditing programs is a really good one though. Maybe something simple like building a database of game executable hashes would be a good start.

For me, some interesting potential fruits of such a project that come to mind are:

  • Could create frontends that can automatically generate game launch menus
  • Could create custom DOSBox configuration files on the fly from said frontends with special options that are necessary for some games (e.g. memory and sound options)
  • Could create auditing utilities that are able to report game version information and make the user aware of newer or alternate versions (e.g. via patches)
  • Could make a utility to link to a database such as the DOSBox compatability database or VOGONS threads to make user aware of issues with running old games in DOS, Windows, DOSBox, VDMSound, etc.

You probably already mentioned some of those points - I didn't read them all before posting.

I should also mention (in case anyone is interested) that all serious auditing utilities that I've used work by generating SHA-1 or CRC32 hashes (digital signatures/fingerprints generated by running an algorithm - usually mathmatical - on the data in a file) of the files being audited and then comparing them against a database of known hashes. It's much more reliable than other methods, such as recording filenames by themselves, because it actually looks at enough of the actual data in a file to differentiate it from others in the database.

Reply 2 of 9, by Snover

User metadata
Rank l33t++
Rank
l33t++

Re. #5: ISO (level 1) images can be compressed quite well by stripping out the error correction data prior to using a standard deflater (Z, ZIP, GZIP, RAR, BZIP2, LZMA, whatever). Removing the ECC data itself saves space, and also since the ECC is generally harder to compress than regular data I have had in my trials up to 30% improvement in compression storage (again, for level 1 -- level 2 (CDA, XA) stores no error correction data). There is a program I use called ECM (Error Correction Modeler) which removes and then reconstitutes the error correction data. It's pretty slick.

Yes, it’s my fault.

Reply 3 of 9, by hydr0x

User metadata
Rank Newbie
Rank
Newbie

honestly, i can't see this happen for various reasons, actually i'm part of the NSRT (SNES ROM auditing tool) team so i'm not a total newb 😉

keep in mind that i'm thinking of all games say up to 2000, if the program would only do disk-era games some problems wouldn't exist

- maximum file size, biggest thing auditing tools handle right now are Neo Geo and some MAME roms, and those are max. 500MB or something like that. if you're going to audit a game like Floyd you'll have 4 650MB files for just one game 🙁 any auditing tool operations on such a file would be really slow, like zipping, crc-checking and all that stuff

- number of games in db, current rom databases have a max of something like 10k-20k entries, if you think about pc, with up to a dozen versions of one game in just one country + dozens of country variations (just 3-10 for console roms) you're looking at a 500k-1mio enty db, and more important, your tool would have to probably rename 1 Million (!!) games in a batch 🙄

- what would you want to include? commercial releases? cd-rom games? dvd-games? freeware? shareware? add-ons? what about java games? flash games? what about downloadable contents (either official or inofficial) like scenarios and all that stuff, what to do with patches? that's a LOT of decisions (and thus a lot of people who are not 100% happy with the product) and also a lot of additional data to the db

so, all in all, i just think it's TOO BIG

but what about this idea instead:

Do mainly the same thing, but remove all file handling, auditing and so on functions... you might wonder what's left then, huh 😅

well, what's left?

an ULTIMATE PC GAME DATABASE

add all the stuff you considered for a frontend, add reviews and all the stuff you'd expect from such a db and you'd have a 100% legal excellent tool for retrogamers

if you code that thing to allow plug-ins someone could program a "start game" plug-in that opens DosBox with a specified or default conf file. then someone could collect conf files and pack them together and host them, and voila, you'd be already quite near to what you wanted, just without the auditing and automatic renaming stuff which is imho not really necessary and useful for pc games...

btw, about your .iso and floppy image idea, this would be a nice idea, but to do that you'd need an Emulator (DosBox?) which could handle those files directly, just like Atari/Amiga/C64 emulators. that would be really hard to though as far as i can tell

and a final thing to consider, do you really want the abandonware/oldgames community to become a ROM collectors scene like SNES and the others?

Reply 4 of 9, by mirekluza

User metadata
Rank DOSBox Moderator
Rank
DOSBox Moderator

Well, I cannot imagine this idea working ... It is a kind of universal-solution-for-everything idea. A lot of work not woth the benefits it would bring (supposing it would get to somehow usable state - which is not certain).
But do not let my opinion stop you, if you think it can lead somewhere, just start on it... But from my point of view there are better things to do,

The game database mentioned above would be usefull and feasible.

Mirek

Reply 5 of 9, by oneirotekt

User metadata
Rank Member
Rank
Member

Storing DOS games as single files has been discussed before, but I think it's problematic in many ways. Console and arcade ROMs are designed as black boxes to the user anyway, so it makes sense to just bundle up all the data into one read-only file. With a PC game, you've got a bunch of files sitting in the directory, and normal use might call for the user to look at, execute or mess with more than one of those. If Warlords 2 were stored in a single archive, how would I launch the Setup utility versus the Scenario Editor versus the main game? Also, most DOS games came on floppies or CD-ROM but needed to be installed to a hard drive. To say nothing of the problems of updating files within the archive when they change. And as for using preset settings stored in the ROM, what if people want to use Tandy sound instead of Adlib, etc etc?

I just don't think all of the neat tricks console emulator authors have devised apply to PC games, and the costs seem to outweigh the benefits. Square peg, round hole, apples-to-toaster-ovens.

Reply 7 of 9, by hydr0x

User metadata
Rank Newbie
Rank
Newbie

nah MobyGames is nice to use once in a while, but overall it's far from perfect , has some flaws and is missing a lot of stuff

additionally to that, it's online, and i was thinking about something you can use offline (maybe distribute it on dvd)

edit:

oh yeah, moby is also missing a lot of games (just saw 3D World Tennis is missing for example)

Reply 8 of 9, by priestlyboy

User metadata
Rank Oldbie
Rank
Oldbie

It would be really really hard at least for the present age to do anything about having a real collection of games or to be able to "detect" if you're using a good version of a game.

Floppy versions ran off of the HD *generally* exclusively. There are some floppy games that were boot-from-floppy only. (some protection scheme i think)

Some games installed/ran off of the CD but some CDs like the Sierra CD games could be copied over and modify the RESOURCE.CFG to make it run exclusively on the HD.

Not to mention ... CD ROMs are quite HUGE and generally DO NOT compress well. 600 mbs -> ~300 mbs.
It wouldn't be easy and there are billions of games and I found out recently that King's Quest 7 had at least 3 or 4 different versions! They were like 1.0, 1.4, 1.51, and 2.00b. Each CD had a Windows Version but the last one 2.00b had finally included a DOS-based version. 🤣. craziness.

I'd LOVE to be able to do something like that. I do recommend using D-Fend 2 for DOSBox Frontending but it would be "difficult" at the present time to have configs that work on everybody's system. I mean the list just goes on because MS-DOS was so dynamic and many people perfer different things. Tandy vs VGA and GUS vs SB. Not to mention current comupter architecture, type, CPU, etc the settings you use for speed may not work on another persons system. Graphics card and drivers may give different results. Sigh, 🤣.

But you could make an offline list of games, stats, details and probably put together something basic but I dunno how it would work. The only thing I've ever seen that could do that "successfully" but the emulator wasn't soo great was UberNES where people put a lot of time and effort to start up a Database with text about the games and scans of the cartridges.

I suppose looking at www.the-underdogs.org and www.mobygames.com and several other game communities you could cobble something together.

/Ieremiou

Btw: Mabus made this a long long time ago but I never got into using it.
http://members.home.nl/mabus/GameDB.htm
It's a Game Database Manager he had created but then lost interest in maintaining the idea. I think he lost the source to it too but I think it's the kind of Idea you're thinking of.

Ieremiou
----------
Helping Debug DOSBox.

Reply 9 of 9, by Snover

User metadata
Rank l33t++
Rank
l33t++

The best thing about the architecture of PCs is that it is so modular that you can do virtually anything with it.
The worst thing about the architecture of PCs is that it is so modular that you can do virtually anything with it.

Yes, it’s my fault.