VOGONS


Application eating >500MB of ram

Topic actions

First post, by huglester

User metadata
Rank Newbie
Rank
Newbie

Hello everyone,

I have used dosbox for so many times, but firstly I met the problem 🙁

I have an old program, called SIMS II, it's a tool for security, it launches fine and works fine under DOSBOX, but I have problem with RAM.

When I start DOSBOX, it eats ~45MB of ram. When I star a program SIMS, it jumps to 53mb of RAM, and later it goes up, ~every 10 seconds it jumps 1-2MB up. After running for a while, it makes dosbox use ~800mb of ram, and up.... until windows xp kills it 🙁

For example as from starting to write this message, memory consumption went up from 45MB to 100MB of ram.. and going up all the way

I tried using loadfix -64 sims.exe
I also searched for memory limits in dosbox.conf - but I could only find low memory limit, I think they won't help me..

Maybe you guys have some suggestions on this problem?

Dosbox version: 0.74
Motherboard Intel (but also tried on asrock with same problem)
RAM - DDR2, 1GB
Windows XP SP3, with latest patches
output=surface
model=svga_s3

well the DOSBOX config is the default one...

Thank you very much!
Jaroslav

Reply 2 of 27, by huglester

User metadata
Rank Newbie
Rank
Newbie
h-a-l-9000 wrote:

A shot in the dark:
Tried core=normal?
Does the program switch the screen resolution frequently?

Hello
just tried your suggestion, but this won't help.. RAM usage goes from ~45 to 100mb in like 5 minutes... (I have also tried your's, MegaBuild v.5 - but same results) I have also tried all the types of "core=" - no luck with all of them 🙁

Well, I must say the program does not change resolution at all... if I understood you right?

The program starts, we see messages, some messages could be 'blinking' but there can be a situation that no changes are made on the monitor for 20minutes or more... but even at this situations - RAM usage goes up...

Thank you for your time and 'shots' 😀

Cheers, Jaroslav

Reply 4 of 27, by huglester

User metadata
Rank Newbie
Rank
Newbie
wd wrote:

May be the directory cache going crazy due to many open searches,
or some playing with vga registers that change resolution but are not visible.

Hello again 😀

'directory cache' - I've searched a google about it, is it the:
CTRL-F4 'Swap mounted disk-image (Only used with imgmount). Update directory cache for all drives!'? or I am not understanding something?

and playing with VGA registers, you mean play with the 'machine=' value?

Sorry, but I am new to these things

Thanks for your time and help!

Reply 6 of 27, by huglester

User metadata
Rank Newbie
Rank
Newbie
wd wrote:

Neither is user-configurable.

so it's something to do with recompilation etc...? maybe you have some examples or something like that? I'm ready to play with it.. since I need this software to run badly 🙁

thank you for any ideas!

cheers, Jaroslav

Reply 7 of 27, by wd

User metadata
Rank DOSBox Author
Rank
DOSBox Author

so it's something to do with recompilation etc...?

Recompiling without changing something is pretty useless.
But yeah i can make an extensive list of things that you *may*
fix in order to get rid of some possible memory leaks.

Reply 8 of 27, by huglester

User metadata
Rank Newbie
Rank
Newbie
wd wrote:
Recompiling without changing something is pretty useless. But yeah i can make an extensive list of things that you *may* fix in […]
Show full quote

so it's something to do with recompilation etc...?

Recompiling without changing something is pretty useless.
But yeah i can make an extensive list of things that you *may*
fix in order to get rid of some possible memory leaks.

Hello wd,
I would be very thankful if you could make that list, when you have some of free time for me. I'd try your ideas...
In a meantime I am still playing, by changing different config settings, but still no luck...

Thank you,
Jaroslav

Reply 10 of 27, by huglester

User metadata
Rank Newbie
Rank
Newbie
wd wrote:

- reimplement drive access without a drive cache
- add lazy-style resolution switching
- check regular runs with standard heap tracking tools (valgrind, purify etc.)

thanks for suggestions, but first two suggestions sound to difficult for me...
I will continue trying different dosbox configs etc etc, maybe I'll try to play with versions too.

I'll get back here soon. but one more question, are there any other options in config which *could* cause this problem?

Thank you very much

Reply 14 of 27, by ripsaw8080

User metadata
Rank DOSBox Author
Rank
DOSBox Author

Based on the demo version of SIMS II, the high memory usage is caused by the app searching for several files with FindFirst (INT 21/4E), which it does every few seconds. DOSBox's memory usage climbs steadily until it runs out of search slots and resets them, at which point the memory is released. With 1GB of physical RAM in WinXP, the only way I can see running out of memory before the reset is if there is little or no virtual memory configured in the OS.

AFAICT the app uses the file searches only to determine if a file exists based on error or success of the function call, but DOSBox doesn't have a way to know that and allocates resources for a search as usual. One option is to reduce the MAX_OPENDIRS constant in dos_system.h to lower the highest amount of memory used by open searches, with all caveats that entails.

Reply 15 of 27, by huglester

User metadata
Rank Newbie
Rank
Newbie
Jorpho wrote:

If it's some kind of drive cache problem, would running the program from a hard drive image instead of a mounted directory be a solution?

wd wrote:

Yes, the directory caching is not in effect for imgmounted drives.

Jorpho and wd,
thanks for your suggestions!

maybe I could then somehow make an .iso from my little program, but then I would need 'write' access to mounted iso.. But... I think there is no such possibility?

ripsaw8080 wrote:

Based on the demo version of SIMS II, the high memory usage is caused by the app searching for several files with FindFirst (INT 21/4E), which it does every few seconds. DOSBox's memory usage climbs steadily until it runs out of search slots and resets them, at which point the memory is released. With 1GB of physical RAM in WinXP, the only way I can see running out of memory before the reset is if there is little or no virtual memory configured in the OS.

AFAICT the app uses the file searches only to determine if a file exists based on error or success of the function call, but DOSBox doesn't have a way to know that and allocates resources for a search as usual. One option is to reduce the MAX_OPENDIRS constant in dos_system.h to lower the highest amount of memory used by open searches, with all caveats that entails.

ripsaw8080, thanks for your time for looking into the application!
Maybe there is a way to somehow reset 'memory opendir slots'?

Thanks for sharing the sollution about MAX_OPENDIRS, I will go and find howto, about compiling DosBOX and play with the option you said. I get back with the results 😀

Hopefully in 2-4 hours.. I'll start playing with low values like
#define MAX_OPENDIRS 128 and up (instead the original one: #define MAX_OPENDIRS 2048)

Have a nice day, evening everyone of you
and thank you VERY MUCH, you guys are saving 1 human live:))

Jaroslav

Reply 16 of 27, by ripsaw8080

User metadata
Rank DOSBox Author
Rank
DOSBox Author

In order to stop (not just reduce) the memory consumption, I made a small TSR program that works like this: if the filespec being sought contains any wildcards, the search is passed on to DOSBox; but if it's an individual filename the TSR gets its attributes and uses success or failure to set the search function result accordingly. Reading attributes does not use memory like searches do. It appears to work for the app in question, but the assumption that searching for an individual file is only used as a simple existence test could be wrong in other cases. Source code is included in the attached archive.

Edit: removed flawed workaround program.

Last edited by ripsaw8080 on 2010-10-15, 02:40. Edited 1 time in total.

Reply 17 of 27, by huglester

User metadata
Rank Newbie
Rank
Newbie
ripsaw8080 wrote:

In order to stop (not just reduce) the memory consumption, I made a small TSR program that works like this: if the filespec being sought contains any wildcards, the search is passed on to DOSBox; but if it's an individual filename the TSR gets its attributes and uses success or failure to set the search function result accordingly. Reading attributes does not use memory like searches do. It appears to work for the app in question, but the assumption that searching for an individual file is only used as a simple existence test could be wrong in other cases. Source code is included in the attached archive.

Hello ripsaw8080 😀
In am now trying to recompile dosbox with your suggestions, as for now I'm stuck at mintgw and other stuff before compiling... getting some erorrs about autoconf/m4, but still trying

As I understood, your fix should be launched before the application?
if I do so, when I start the application, I do not see messages, It's only application with the menu. and all the list with messages is empty...

Maybe there is something I can do to reproduce the problem?

Thank you once again ripsaw8080!

Jaroslav

Reply 18 of 27, by ripsaw8080

User metadata
Rank DOSBox Author
Rank
DOSBox Author

Sorry, didn't notice the list problem. I guess the app needs the DTA filled even for individual file searches.

I had an idea for a safer approach that calls FindNext after FindFirst for individual files (not wildcard searches). DOSBox closes the search slot and releases the memory when a search completes, and one call to FindNext will complete the search for a single file. I made another TSR to implement the idea, and it appears to work OK. Memory use still goes up steadily, but at a much slower rate; and the search slots are not eventually reset because they are being freed up. In my tests the increase in memory tops out after less than 2MB, but this is with a program I made that rapidly performs FindFirst calls. I didn't wait around to see if it eventually tops out for SIMS II; it would take awhile to be sure because the increases are around 4-8Kb every few seconds.

Attachments

  • Filename
    FFCLOSE.ZIP
    File size
    985 Bytes
    Downloads
    320 downloads
    File license
    Fair use/fair dealing exception

Reply 19 of 27, by huglester

User metadata
Rank Newbie
Rank
Newbie
ripsaw8080 wrote:

Sorry, didn't notice the list problem. I guess the app needs the DTA filled even for individual file searches.

I had an idea for a safer approach that calls FindNext after FindFirst for individual files (not wildcard searches). DOSBox closes the search slot and releases the memory when a search completes, and one call to FindNext will complete the search for a single file. I made another TSR to implement the idea, and it appears to work OK. Memory use still goes up steadily, but at a much slower rate; and the search slots are not eventually reset because they are being freed up. In my tests the increase in memory tops out after less than 2MB, but this is with a program I made that rapidly performs FindFirst calls. I didn't wait around to see if it eventually tops out for SIMS II; it would take awhile to be sure because the increases are around 4-8Kb every few seconds.

Hello ripsaw8080, it's me again 😀

Yesterday, before going to sleep, I have managed to compile dosbox with your suggestions, and it worked! I have watched how the memory grows up, and when it is ~110MB it drops to ~50, and it was continuing that way! I left computer on for about 4 hours, and went to bed, after I woke up, it was at ~80mb, and when it grew up to ~110 - it also drop down to ~50! Very nice 😀 (changed MAX_OPENDIRS to 128)

Now the comments about your last attached files... - it also WORKS! After I woke up, I downloaded your attached archive and started testing...
I'll paste some results, so It would be more clear, how it works:

So I have launched SIMS II using your application at: 9:39 - 47.568 MB (memory consumption at the very start) 
The graph below shows the memory consumption in 'Task Manager' for 1 minute

from 9:55 till 9:56
47.824 MB
47.964 MB
47.832 MB
47.964 MB
47.836 MB
47.972 MB
47.844 MB
47.980 MB

from 10:06 till 10:07
48.012 MB
48.144 MB
48.016 MB
48.152 MB
48.020 MB
48.156 MB
48.024 MB
48.160 MB

from 10:23 till 10:24
48.300 MB
48.432 MB
48.304 MB
48.440 MB
48.308 MB
48.444 MB
48.312 MB
48.448 MB
48.316 MB

from 10.53 till 10:54
(here we will see that the memory stopped growing)
48.684
48.552
48.684
48.552
48.684
48.552
48.684
48.552
48.684

from 11.54 till 11:55 (after an hour - same memory consumption)
48.684
48.552
48.684
48.552
.........
11:55

So I hope it won't start growing anymore!:) Today, later in the evening, I will get to the place, where I could test this program live! because at home I can't fully test it, since few devices should be connected to COM ports...

But as for now, I'm really very happy with the results! I think I'll be ready with the results in around 8-16 hours...

Thank you ripsaw8080 for your time and patience and efforts!

Jaroslav