PhysFS patch

Here you can discuss the development of patches.

Re: PhysFS patch

Postby ADDiCT » 2009-6-22 @ 08:44

From a "normal user" point of view (=mine (; ): i always wanted to try this functionality, but the problem is that it seems to be kind of a "one way" solution. I think official DOSBox doesn't support the patch, so i'm "tied" to a special build. If official DOSBox (or a clean CVS) has a fix for a specific game, i can't play the zipped game with it, so i have to jump though hoops in order to play it (compile from source, keep compressed/uncompressed versions of the game, etc.). Hmmm... I wonder if it would be possible to automatically create binaries based on the latest official CVS, with specific patches applied. That would solve the issue i've described.
User avatar
ADDiCT
Oldbie
 
Posts: 1170
Joined: 2007-2-02 @ 15:07
Location: Retroland

Re: PhysFS patch

Postby gulikoza » 2009-6-22 @ 08:58

Yes, the easiest way is to include the patches in the cvs :wink:
Other than that there is no easy way for automatic patching...the patches break very easily, even changed whitespace can cause rejections. Also, the code that is not even patched can be changed and indirectly cause the patch to stop working correctly. Any additional functionality is better added with some form of plugins rather than automatic source patching and compiling. But that requires a working plugin system...again a lot of work.
User avatar
gulikoza
Oldbie
 
Posts: 1693
Joined: 2004-6-25 @ 14:53

Re: PhysFS patch

Postby wd » 2009-6-22 @ 09:26

Imo this functionality belongs to frontends, if at all.
wd
DOSBox Author
 
Posts: 10818
Joined: 2003-12-03 @ 21:23

Re: PhysFS patch

Postby ADDiCT » 2009-6-22 @ 10:33

Gulizoka: thanks for the infos. I think the problems you're describing are part of the reason why this stuff isn't included in the official DOSBox build.

wd, i think i can kinda understand your point of view. But i don't agree (; . What use would it be if the frontend supported zipped files? It'd have to unzip the complete set to a temporary location, find out which files have been changed, and then rezip everything (or at least the changed files). This'd be awkward for large games. If i correctly understand the way PhysFS works, it allows for "on the fly"-reading (transparent) of archived files, and will write any new or changed files to a special directory, which could then be backed up easily. I'm not an expert, but i guess that kind of "transparent" archive handling can't be implemented by a frontend.

I'm just thinking loud here. From my point of view, a DOSBox build based on the latest CVS with added patches (PhysFS, GLIDE, Direct3D, Built-in menu system, etc.) would be a dream come true.
User avatar
ADDiCT
Oldbie
 
Posts: 1170
Joined: 2007-2-02 @ 15:07
Location: Retroland

Re: PhysFS patch

Postby DosFreak » 2009-6-22 @ 10:36

Why is the "latest CVS" necessary? The latest official version of DOSBox with the patches (which gulikoza provides) should be enough.

For the very few cases where the "latest CVS" is required then the user can simply select that version in their frontend of choice or simple execute the executable if they don't use a frontend.
Game Acronym List
DosBox CVS Builds
DosBox Feature Request Thread
DosBox FAQ
PC Game Compatibility List
"People like coldplay and voted for the nazis, you can't trust people."
User avatar
DosFreak
l33t++
 
Posts: 9179
Joined: 2002-6-30 @ 16:35
Location: Your Head

Re: PhysFS patch

Postby wd » 2009-6-22 @ 10:41

but i guess that kind of "transparent" archive handling can't be implemented by a frontend.

Don't see why not.
wd
DOSBox Author
 
Posts: 10818
Joined: 2003-12-03 @ 21:23

Re: PhysFS patch

Postby h-a-l-9000 » 2009-6-22 @ 10:43

> Why is the "latest CVS" necessary?

Using an 'old' DOSBox version, even if it's a stable, somehow gives you a bad feeling ;)
1+1=10
h-a-l-9000
DOSBox Author
 
Posts: 4485
Joined: 2005-2-23 @ 00:14

Re: PhysFS patch

Postby ADDiCT » 2009-6-22 @ 12:34

wd wrote:
but i guess that kind of "transparent" archive handling can't be implemented by a frontend.

Don't see why not.
Well because the frontend can't get any control over how DOSBox reads files, can it? I mean, DOSBox expects files in a "normal" file system. So unless the frontend is capable of "mounting" zip files (by itself or via the host OS) it won't be able to provide files to DOSBox without first uncompressing the whole archive, right?

Anyway, as i've said, i'm just thinking aloud. The current situation kinda proves the point i was trying to make, though: right now, there is no way of having 0.73 with these "nice" patches (maybe the sources could be patched and compiled, but it doesn't look like this is something a non-developer is able to do easily). I know Gulizoka is working on it, and i'm very glad and thankful he does so. Still, the situation is like i've described.
User avatar
ADDiCT
Oldbie
 
Posts: 1170
Joined: 2007-2-02 @ 15:07
Location: Retroland

Re: PhysFS patch

Postby MiniMax » 2009-6-22 @ 17:25

I don't see what all the fuzz is about. Why is this a desired feature? For saving space?

On Windows, wouldn't you gain the same space saving by using a compressed file folder as the virtual C-drive?

On Linux/MacOSX, I assume you either have a similar compressed folder/filesystem in the kernel, or maybe a ZipFS in user-space?
DOSBox 0.73 ReadMe | DOSBox Wiki | DOSBox 60 seconds guide | How to ask questions
_________________
Lenovo M58p | Core 2 Quad Q8400 @ 2.66 GHz | Radeon R7 240 | LG HL-DT-ST DVDRAM GH40N | Win 10 Pro
User avatar
MiniMax
Moderator
 
Posts: 5128
Joined: 2004-1-18 @ 04:15
Location: Stockholm, Sweden, Europe, Earth Interests: Old games & young women

Re: PhysFS patch

Postby gulikoza » 2009-6-27 @ 21:33

I'd have to agree with MiniMax to some degree...I don't have my games zipped...

But still, I had to add a few features to make it more user friendly :happyhappy:. Here's the new version:
  • again updated to latest cvs (flag handling&stuff), including adding the new flag handling to physfsdrive
  • obey securemode when mounting from command line
  • free physfs memory on unmount
  • make mount syntax (hopfully :happy:) more user friendly:
    * mount no longer fails when trailing : is missing
    * use capture directory when writedir is missing (seems safer then dosbox directory)
    * automatically use physfs when mounting files (mount no longer fails with %s is not a directory, but will try to use physfs instead)
    * all this makes physfs mounting as simple as: 'mount c: filename.zip'
  • support drag&drop :happyhappy:. This uses mostly existing code, so far only supports *.zip & *.7z (7z are not working in my current test build, will try to update to physfs 2.0), more extensions can be added if so desired...The write dir is set to zip directory, this could possibly be improved to check if dest. is writable or use the capture dir instead (if dragging from a cd-rom for example)
You do not have the required permissions to view the files attached to this post.
User avatar
gulikoza
Oldbie
 
Posts: 1693
Joined: 2004-6-25 @ 14:53

Re: PhysFS patch

Postby rcblanke » 2009-6-27 @ 22:08

Very nice, greatly appreciated Guli!
gulikoza wrote:support drag&drop :happyhappy:
:sly: :happyhappy:
User avatar
rcblanke
Oldbie
 
Posts: 1282
Joined: 2005-4-01 @ 09:44
Location: Round 42

Re: PhysFS patch

Postby ADDiCT » 2009-7-12 @ 16:48

OK, i've played around with what PhysFS stuff for the first time. Most things are working great, but there are a few strange effects.

First, i'd like to answer MiniMax's question about "what the fuzz is about". ZIP compression is much more efficient than NT's built-in transparent compression. Example: X-Wing CD. The files on the CD use 68,2 MB (71.549.748 Bytes). With NT's compression, this decreases to 57,6 MB (60.491.440 Bytes). The same files zipped use up 38,5 MB (39.419.137 Bytes). I'm pretty sure this could be reduced further by using 7-zip compression. You can easily see the difference. And this doesn't even take defragmentation or "partially used disk blocks" into consideration. One large file is much easier to handle than many small ones, this is true for all FS's.

Another advantage of PhysFS is the fact that you can mount archives from read-only media. This makes it possible to store games on DVD or CD, and mount the ZIP files from there. New or changed files will be redirected to a local directory. It's also easy to maintain a "pristine" version of an installed game. You just install the game and zip it up without configuring it. After that, you configure the game, embed DOS32/A into the executables, etc. - the archive will be untouched, all configs or changed files will be written to the local hd. If you want to go back to the original version, just delete or move the local files.

If PhysFS would be added to official DOSBox, another great advantage would be multi-platform, of course (; .

One thing that's a bit strange is the way PhysFS handles archives with more than one directory in them. For example, i like to put my games in a "c:\games\<name>" directory. An archive containing two directories won't display files correctly on a "dir" command (after the first one or two dir levels), even though the files seem to be there (i can execute them etc.). This looks like a small bug.

I'll do some more testing, especially with demanding and large games. If PhysFS works as good as it is now, i'm gonna switch my DOS games collection to using PhysFS. There's no need to waste resources on uncompressed files when there's a good alternative IMO.

It'd be nice if the DOSBox devs would consider including PhysFS in the official DOSBox build. Gulikoza should be quite familiar with the code, so it should be possible to include it in a "clean" way.
User avatar
ADDiCT
Oldbie
 
Posts: 1170
Joined: 2007-2-02 @ 15:07
Location: Retroland

PhysFS patch/latest Gulikoza build: how to set write directory?

Postby ADDiCT » 2009-7-14 @ 09:41

I know this patch and the builds that are using it are not supported, but maybe someone can answer my question anyway. More info about the patch here.

So far, everything is working great when using PhysFS. One thing i can't figure out however is how to set the "writing dir". By default, Gulikoza's build writes to the "capture"-directory, and i can change that location via doxbox.conf. This is my current workaround.

From what i've read about PhysFS, it should be possible to configure the write directory by using a mount syntax like "mount c c:\write:c:\mygame.zip". This doesn't seem to work for me - when using that syntax, i get an empty c:-drive (which is interesting, because something is obviously mounted, otherwise i'd be getting a message from DOSBox). I'll have to consider a few more things. For once, i'm having long paths/spaces in paths, so i have to enclose path names in "" characters. I'd like to use relative paths, too, which works perfectly well for other DOSBox features.

Can anyone give me feedback about the correct mount syntax, and/or whether setting the write dir in the mount command with Gulikoza's latest build is working or not?

EDIT: think i've got it. The problem occurs when paths are enclosed in double quotes. As soon as these are involved, the mount command will only "see" the first directory (the one before the ":"). And i've found a workaround, too, though i wonder why it's working (; . The solution is to enclose everything after "mount c" in double quotes, like this: mount c "c:\write:c:\mygame.zip".

EDIT2: ahhh, no luck. My workaround only works if the zip file path doesn't contain spaces. If it does, DOSBox crashes (!). This is the first time i've had DOSBox crash on me. Looks like a bug in the handling of the "mount" command parameters. Allright, back to changing the "captures" option. Not a clean solution, but it works.

EDIT: some typos
Last edited by ADDiCT on 2009-7-31 @ 07:58, edited 1 time in total.
User avatar
ADDiCT
Oldbie
 
Posts: 1170
Joined: 2007-2-02 @ 15:07
Location: Retroland

Re: PhysFS patch/latest Gulikoza build: how to set write directory?

Postby gulikoza » 2009-7-26 @ 15:45

It's a problem with how the mount command is parsed. It won't work if " are used... :happyhappy:

edit: perhaps it would be best to have all patches related problems/feedback in that one thread?
User avatar
gulikoza
Oldbie
 
Posts: 1693
Joined: 2004-6-25 @ 14:53

Re: PhysFS patch/latest Gulikoza build: how to set write directory?

Postby ADDiCT » 2009-7-26 @ 18:39

Yeah, sorry for opening a new thread... Can this be moved by a mod or should i c/p in the patches thread?
User avatar
ADDiCT
Oldbie
 
Posts: 1170
Joined: 2007-2-02 @ 15:07
Location: Retroland

Re: PhysFS patch

Postby MiniMax » 2009-7-26 @ 19:42

Merged
DOSBox 0.73 ReadMe | DOSBox Wiki | DOSBox 60 seconds guide | How to ask questions
_________________
Lenovo M58p | Core 2 Quad Q8400 @ 2.66 GHz | Radeon R7 240 | LG HL-DT-ST DVDRAM GH40N | Win 10 Pro
User avatar
MiniMax
Moderator
 
Posts: 5128
Joined: 2004-1-18 @ 04:15
Location: Stockholm, Sweden, Europe, Earth Interests: Old games & young women

Re: PhysFS patch

Postby boyofdestiny » 2009-7-29 @ 19:24

I agree with wd. This should/could be a feature in a front-end. Under *nix, since running things with dosbox are automounted, it should be trivial even with something like a bash script.

Take the parameter (the zip file), extract it to $temp (use mktemp or whatever, hard code a temporary folder)
Then either take the program name as a parameter or just mount that as C:, let user run whatever he or she wants.

After dosbox has exited, merely rezip the contents, and overwrite the original zip in the process.

Under Windows, why not just modify this .bat posted here?
viewtopic.php?t=22383

Frankly, the win version should just have the above integrated. It would solve so many issues for newbies, as one could just right click an .exe, .com, .bat and have it automount and launch in dosbox.
User avatar
boyofdestiny
Newbie
 
Posts: 55
Joined: 2005-9-06 @ 11:04

Re: PhysFS patch

Postby gulikoza » 2009-7-30 @ 06:58

Reziping is not such a good idea. Something can go wrong and corrupt entire zip file. Also, you no longer have the original (unmodified) game data. This might work with some archiver which is able to produce incremental archives, so that the original archive stays unchanged (and may be located on a read-only medium for example) and the differences are later archived someplace else...
User avatar
gulikoza
Oldbie
 
Posts: 1693
Joined: 2004-6-25 @ 14:53

Re: PhysFS patch

Postby rcblanke » 2009-7-30 @ 08:54

..and lets not forget the time required to extract (and rezip) the archive, which can potentially take quite some time for larger games. Using physfs, mounting (7-)zips is similar to using imgmount for iso's, and runtime extraction of the data is very fast.
User avatar
rcblanke
Oldbie
 
Posts: 1282
Joined: 2005-4-01 @ 09:44
Location: Round 42

Re: PhysFS patch

Postby ADDiCT » 2009-7-30 @ 10:48

Indeed. Unzipping/Rezipping isn't similar to PhysFS functionality in any way (other than both handling archives). I agree that the plain Unzipping/Rezipping functionality should be part of a frontend of sorts, but the functionality PhysFS provides can't be implemented via a frontend (unless the frontend or host OS is able to transparently mount archives).
User avatar
ADDiCT
Oldbie
 
Posts: 1170
Joined: 2007-2-02 @ 15:07
Location: Retroland

PreviousNext

Return to DOSBox Patches

Who is online

Users browsing this forum: No registered users and 2 guests