PhysFS patch

[boyofdestiny]

Okay, I think I see now. So it can handle reading from the archives, and maintain something separate for the writes, keeping the original data untouched...

Similar to using unionfs + squashfs

http://www.linux-mag.com/cache/7409/1.html

I think something like this could be made transparent and multi-platform, using fuse

I think the benefit of it being separate from dosbox, is that you should get better compression efficiency with one large compressed file, as opposed to individual game directories and it should work with any software...

[ADDiCT]

I've been trying to get Terminator Future Shock/Skynet to work with the latest Gulikoza build and PhysFS. No chance. The problem is that even though the ZIP files are mounted (i was using one for hd data and one for cd data) and the game can be started, it will error out eventually because of one basic problem.

The PhysFS implementation somehow has problems with subdirectories in a ZIP file. With a DIR command in a subdir, the effects are quite strange. Sometimes i see the the contents of the parent directory, sometimes i see a random list of files with a length of 0 bytes. What's _really_ strange is that i can start executables even when i don't see them in a DIR listing. All this stuff is no big deal when the games "behaves", but for the two Terminator games it makes running them nearly impossible, as it seems they are looking for their own files in a similar way the DIR command does.

The behaviour i was describing looks like a bug to me. Gulikoza, is there a chance of you looking into this problem?

[gulikoza]

I can take a look.

[danielc]

So... is this a dead patch? Would be great if we could have compressed CD images. Something like a custom cuesheet would be a great implementation, as already mentioned using ZIP is better than NT-filesystem compression.

I.e.:

FILE "TRACK_01.ZIP" BINARY
TRACK 01 MODE1/2048
INDEX 01 00:00:00
FILE "TRACK_02.OGG" MP3
TRACK 02 AUDIO
INDEX 01 00:00:00

etc...

[fil]

yes, it would be good if we could mount a zip or 7z instead of the iso in a cue file, as iso is still big, not compressed at all

to get round it at the moment i have to mount twice, i mount a 7z or zip as the C drive, then i create an empty iso to go with the cue and ogg files on D, that is the only way of doing it at the moment, but would be a lot better and easier if the cue could contain a 7z or zip and mount that instead of an iso

hope someone could add this feature

[Serious Callers Only]

So is this dead?
It's a pity this was never included into upstream dosbox because imo this shouldn't in anyway be left to frontends, basically for consistency, availability and 'non-tie in'.

And ofc if it should be left to frontends it means you're delegating the basic functionality of mounting a filesystem away from dosbox.conf files.

The advantages are obvious: better compression, less fragmentation, making sure a pristine version of the game remains read only.

It's been over 10 years since this was posted for the first time and ofc no frontend implemented it and no frontend supports zip files in a sane way (without unzipping and rezipping). Well at least i think so, not quite sure what random frontend X only existing in Solaris does or not.

I'm sympathetic to the dosbox authors idea that this feature could be done at the filesystem level and then delegated to dosbox, but such a effort will be most likely platform dependent and impossible for the average user to configure or know about. While a cmd like imgmount would be there in the help listing the first time they started dosbox.

[rcblanke]

... It's been over 10 years since this was posted for the first time and ofc no frontend implemented it ...

That is incorrect, DBGL has been supporting PhysFS for quite some time now. That is, if you're using a DOSBox version with the PhysFS patch.

[Dominus]

Yeah, there seems to be a massive misconception of frontends at play...

[Serious Callers Only]

... It's been over 10 years since this was posted for the first time and ofc no frontend implemented it ...

That is incorrect, DBGL has been supporting PhysFS for quite some time now. That is, if you're using a DOSBox version with the PhysFS patch.

Oh? The not upstreamed patch that i can't simply apt-get install dosbox for (or for that matter apt-get install DBGL)? Sorry but that just underlines the point that this feature not being upstreamed just fragments capabilities of frontends. If it had been, oh, 5 years ago, all frontends on all platforms would have been supporting it since 4 years ago.
You aren't even supporting it yourself with dbgl mounting the filesystems itself, like wd wanted but instead depending on a very old patch for a inbuilt version of dosbox that you (or someone else, just more one point of failure when people lose interest) probably had to adjust multiple times over the years!

I'm not saying you don't have a linux port mind you. I'm saying people don't know about it, if they know about it, it's a near certainty that the dosbox fork you're depending on will stop updating in the future, and trying to put a 'full capacity' dbgl in the standard (not personal) repositories of any half decent distro would get vetoed because it's using 1 or more inbuilt patch over the thing its frontending. If you just remove the forked dosbox, people will just not know it's possible or be very annoyed that they have to download another thing manually from some random site in the internet and extract/install it in the right place.

It's going to be funny when MAME/MESS starts emulating a complete DOS machine and people start using it over dosbox because it's more out of the box friendly.

All that bitching said, i think my path is clear. I just have to try to maintain another dead patch i don't understand in my forked dosbox. Yay.

[Serious Callers Only]

Amusingly, i can't even download the updated patch from this thread (although i can post on it!) because i 'don't have have the required permissions to view the file'.

I don't know why i bother, but let me suggest github for merge requests again (about 6 or 7 years later than the last time), it's a much much better workflow than this diff patching file exchange forum even if used like a glorified svn.

[Dominus]

Why do you bother at all? The patch is dead, not one of the devs seems to have shown any interest in this. And it's a "might be nice to have but surely not critical" patch. If you don't understand it and you consider it dead yourself, just don't make your fork more complicated by maintining it. If you don't understand it, breakage will occur at some point and you are likely to miss it and then you will need to go regression hunting...
A lot of bitching over this dead patch...

[Serious Callers Only]

Well i bother because this is very useful for games. A way to make read only mediums work to store AND lower disk space and fragmentation AND play games? Yes please. I'm just emphatically suggesting this should be upstream for obvious reasons. Currently already things like SDL cdrom support are being retired. It wouldn't surprise me to see dvds being gone from all mobiles and even desktops in the near future (hell, they're already gone from NUCs and tablets). Whatever lowers disc usage is a good thing now since if you actually want to use the stuff, you should be digitalizing everything you own. Not to mention discs don't last that long anyway.

I was thinking of lifting patches from Daum now, but i'm leery. I only really want this and voodoo emulation, maybe the gus panning fix too (and why isn't that merged?). And it looks like Daum doesn't give patch files anyway (maybe wisely since they'd break all the time if tried to use with upstream).

Speaking of voodoo emulation, i think i remember a dosbox dev (wd?) saying that he wanted to create their own upstream software voodoo emulation, that seems to have never happened.

[Dominus]

Disk space and fragmentation is really a non issue these days... The read only medium part, yes, that could come in handy in theory, but mostly, IMO it's only semi useful, in practice. Again because mostly these days it's not that important anymore.
Adding an extra layer of possible problems is just not worth the small gain. But it's your time, spend it any way you like it...

[Serious Callers Only]

fragmentation is really a non issue these days...

Oh, that is going to be a problem again. The new large flash drives (128GB-1TB) work like log filesystems, their firmware perform something akin to 'defragmentation' on low write times when they're near full - and performance drops like a rock until done ofc - and they prefer to have special log oriented filesystems for maximum lifetime and performance (so you can forget formating as Ext4/NTFS if you know what's good for you).

And i happen to disagree that read only storage for games is not useful especially if you want to play them, it's the only way to make sure that you have a sandboxed pristine copy of the of game after install and something like this patch is the only way to still be able to use it without a workaround circus (unzip manually, rezip saves/forget about saves etc, or copy elsewhere mark everything read/write if you're offended by zip for some reason)

This would also be useful for something like the internet archive, which has a DCMA exception (thus the recent news about being able to play MAME roms on it). I doubt they will ever bother with dos unless they can separate game write calls elsewhere without copying the whole thing, ie: exactly what this patch does.

[Dominus]

That "circus" you mention might be a hassle to you but is just a snap of a finger for others.

As for the internet archive bothering about dos games... Where have you been the last six months? http://www.gamespot.com/articles/play-o ... 0-6424578/

[Serious Callers Only]

Hey, no skin off my back if you want to do useless work (if you try to have readonly archives) and maybe lose all progress saves and configurations and have to reinstall and redo it again 2 years later, whatever you want. I just want the latest diff patch visible for me now.

And you know what? If the internet archive did that to 400 dos games i bet they implemented something like this - it wouldn't do to have hundreds of users write saves and config files over the same directory. Or maybe they use client side saving of written files, either way, they need a fuse filesystem thing - or something clumsier like nuking the saves/settings each time on the port. Good for them i guess, but it's just one more sign of NIH from the amazingly fragmented dosbox patches ecosystem, that software developers are reimplementing a useful feature multiple times.

And it isn't even the first time iirc, i think there were multiple voodoo emulation attempts, high level and low level - with none getting into dosbox either. Now look, MAME is emulating the voodoo and the current dosbox patch is practically a copy paste of it. I guess there is nothing wrong with that, but even now, with experienced emulation coders doing the implementation work and it being actively used and developed by a popular project, it's not in dosbox upstream - and everyone gave up on it being there.

[DosFreak]

Don't make assumptions on the status of DOSBox development that you have no insight into.

Have fun using Mame\Mess as a replacement for DOSBox.

Thread closed.

[DosFreak]

Reopening but this is a patch thread not a bitchfest, if it continues then it'll be locked again.

[Serious Callers Only]

For whats it worth, here is a updated patch that compiles on current svn. As i've been looking at it i've also come to recognize that the code is too bitrotten and buggy to be the best approach and from experimentation i've realized zip/7z support is the wrong approach. I'm now investigating a bsd/macos/linux only solution using FUSE+squashfs+union fs. It would work just like this, but the files squashfs handles are a magnitude faster to read and consequently they can handle much larger files and dirs - it's used on live cds to compress the whole cd.

I may do a (much simpler, delegating all of the real work to external dependencies) patch to use in my ppa which i will share if it ever materializes. The only-in-my-head plan for this is to have mount of a .sqshfs file execute a parallel linux mount of a unionfs(writedir+squashfs) through fuse - so it doesn't need superuser. I may copy the ":" idea to reference paths inside the zip, it would be critical to not have to cd into things all the time.
On shutdown of dosbox that would be unmounted.

For frontend support what i think would be best would be for the frontend to search for 'dosbox.conf' files in the .sqshfs file 'added' to it, and taking the name of the directory that contains them or the name of the .sqshfs file itself if it's top level for a config that just mounts the directory to run with that conf file (and by default, merging the main .dosbox confile too). That would be a p. nice way to 'automount' a game filesystem (it would work with normal ones too, idk if DBGL already has something like this maybe).

Now you may notice that all of the functions of such a patch can be implemented in the frontend itself. Indeed, the search patch for a frontend will probably have to mount the thing itself, so it all least needs to mount a squashfs. I'm ambivalent about everything being delegated to the frontend or just partially. Maybe it will be easier in the frontend since at least partial support is needed for usability. And in a frontend we wouldn't need to umount and remount everytime we quit a game and started a new one on the same filesystem. So yes, probably best there.

In the future, you could maybe even configure stuff to update the .sqshfs with a different FUSE combination.

[jarreboum]

I'm just a user of this patch, via daum, and I hope the idea isn't abandoned.

It's very useful to have in a multi user environment, without having to worry of one user corrupting or deleting the saves and configurations from another. I simply keep the games read-only in /usr/local/share/games/dosbox and the modifications in ~/.dosbox/saves, mount them together and everybody is happy.