VOGONS


First post, by wunderwaffe

User metadata
Rank Newbie
Rank
Newbie

Hi,

i'm trying to run an old DOS insurance-app under dosbox. The application is storing letters send to customers in the past in one directory. Currently there are around 13000 files in this directory.

DOSBOX runs fine with the application but as soon as I enter the 13k-dir it crashes - looks like an infinite loop or something - the process doesnt't die, it's just eating up processor-time and - though rather slowly - memory.
The same happens whenever I enter this directory at DOS-prompt and invoking "dir".

Any ideas?

Thx Ralf

PS: Dosbox version: 0.63

Reply 1 of 13, by eL_PuSHeR

User metadata
Rank l33t++
Rank
l33t++

13.000 files in a directory. I don't know if FAT entries supported a number that high (it seems an insane value to me). Why not using one PHYSICAL file to store them (like .dbf)?.

Intel i7 5960X
Gigabye GA-X99-Gaming 5
8 GB DDR4 (2100)
8 GB GeForce GTX 1070 G1 Gaming (Gigabyte)

Reply 2 of 13, by HunterZ

User metadata
Rank l33t++
Rank
l33t++

I really don't think you could put more than 1024 or so files in a directory in DOS... Hopefully Qbix or someone will comment.

Reply 3 of 13, by Kippesoep

User metadata
Rank Oldbie
Rank
Oldbie

There is no set limit to the size of a FAT directory. There is a limit only to the amount of files that may be stored in the root of the drive.

My site: Ramblings on mostly tech stuff.

Reply 4 of 13, by Qbix

User metadata
Rank DOSBox Author
Rank
DOSBox Author

as far as I know dosbox doesn't really have a limit. but it stores all files in the current directory
in a STL list I think. and then starts sorting all the entries.
Both will eat memory I think.

There is a limit on the amount of directories though.
But the structure wasn't designed with 13K directories in mind so it's quite possible that something breaks at those sizes.

Water flows down the stream
How to ask questions the smart way!

Reply 5 of 13, by wunderwaffe

User metadata
Rank Newbie
Rank
Newbie

Hi,

I tried the same on a modern machine (the other was a 500MHz 128MB computer). End of story is: The dir-command just goes on for ages, but at the end finishes allright.
So next question: I guess the low performance is due to the fact, that the cache is filled at the same time. Is there a way to disable the cache - at least for certain directories?

Thank you very much for your replies!

Bye Ralf

Reply 6 of 13, by Qbix

User metadata
Rank DOSBox Author
Rank
DOSBox Author

uhm yes that is possible, but only by modifying the source of dosbox.

I don't assume that is an option for you.

Water flows down the stream
How to ask questions the smart way!

Reply 7 of 13, by `Moe`

User metadata
Rank Oldbie
Rank
Oldbie

You are free to find a programmer who makes these modifications for you, that's the joy of open source. It's perfectly legal to do so. DOSBox is not hard to modify - any C++ programmer should be able to work on it.

Reply 8 of 13, by HunterZ

User metadata
Rank l33t++
Rank
l33t++

I took a look at the DOSBox source and ran away screaming. It's easy to compile but hard to understand (unless someone has added a lot of comments recently...).

Reply 9 of 13, by kekko

User metadata
Rank Oldbie
Rank
Oldbie

Actually the slowdown verifies even on mount command.

drive_cache.cpp, line 699, move the sort out of the loop (this fixes DIR command)

	for (Bitu i=0; i<dirSearch[dirID]->fileList.size(); i++) {
CreateEntry(dirFindFirst[dirFindFirstID],dirSearch[dirID]->fileList[i]->orgname);
// Sort Lists - filelist has to be alphabetically sorted, even in between (for finding double file names)
std::sort(dirFindFirst[dirFindFirstID]->fileList.begin(), dirFindFirst[dirFindFirstID]->fileList.end(), SortByName);
};

drive_cache.cpp, line 636, move the sort out of the loop (this fixes MOUNT command)

		while ((tmpres = readdir(dirp))!=NULL) {			
CreateEntry(dirSearch[id],tmpres->d_name);
// Sort Lists - filelist has to be alphabetically sorted, even in between (for finding double file names)
// hmpf.. bit slow probably...
std::sort(dirSearch[id]->fileList.begin(), dirSearch[id]->fileList.end(), SortByName);
}

This fixes both issues and doesn't seem to break anything.
I couldn't understand why the filelist should be sorted in between. Could anyone please explain?

Reply 10 of 13, by Qbix

User metadata
Rank DOSBox Author
Rank
DOSBox Author

from the comments it would appear for double filenames
so
file and File (on case sensitive filesystems.), but I'm not certain. Finsterr wrote this part and he is inactive for quite a while.
Although the whole cache will be removed oneday.

To remove the cache temporary.
look at drives.h
there is a drive_cache added to each drive.
you can replace it with a no-drivecache. This is used to check if bugs happened due to drive_cache.
And some lowmemory ports (pocketpc) use it to save some memory.

Water flows down the stream
How to ask questions the smart way!

Reply 11 of 13, by `Moe`

User metadata
Rank Oldbie
Rank
Oldbie
HunterZ wrote:

I took a look at the DOSBox source and ran away screaming. It's easy to compile but hard to understand (unless someone has added a lot of comments recently...).

😀

But honestly, DOSBox source is good in most parts. Comments are _not_ a good measurement for code quality, as they tend to be the first things that acquire bugs. The general structure is logical, hackiness is bearable, and there's some OO put to good use. It's not an example of perfect application design, but it IS good to understand (unless you are talking about the CPU, which is a science by itself).

Reply 12 of 13, by kekko

User metadata
Rank Oldbie
Rank
Oldbie

There's really no way to fix this before next release?

Reply 13 of 13, by Qbix

User metadata
Rank DOSBox Author
Rank
DOSBox Author

no.
Oneday I'll remove the drive_cache and add my new drive/dos system.

The sorts you mention can be removed probably for the case insensitive filesystem like windows.
But not in general.

Water flows down the stream
How to ask questions the smart way!