DosBox: Error "not enough memory for internal tables" and how to fight it?   

Having problems with a specific game or application? Post your problems here!

DosBox: Error "not enough memory for internal tables" and how to fight it?   

Postby womak » 2013-5-26 @ 11:55

 
I Use DosBox 0.74 to run the old DOS programm, written in Fortran. Nor any graphics, sound, joysticks, no, only mathematical calculations. I need to take 1500 source files and get 1500 results files.
   In the first version I called DosBox from the my program each time where the file was ready for processing. It takes a lot of time.
   Then I decided to optimize the process: calculated 1500 source files created in parallel BAT-file in which prescribed that the DOS programm start up sequence and each time took the following file for processing. And in this BAT-file launched through DosBox. So, BAT-file runs to 109 the file, and then DosBox crashes with "not enough memory for internal tables". 
   What's the problem? 
    Section [dos] in the configuration file:  
xms = true  
ems = true  
umb = true  
keyboardlayout = none
User avatar
womak
Newbie
 
Posts: 9
Joined: 2013-5-26 @ 11:34
Location: Russia

Re: DosBox: Error "not enough memory for internal tables" and how to fight it?   

Postby Qbix » 2013-5-26 @ 14:59

Are you calling the bat files recursively ?
Water flows down the stream
How to ask questions the smart way!
User avatar
Qbix
DOSBox Author
 
Posts: 10827
Joined: 2002-11-27 @ 14:50
Location: Fryslan

Re: DosBox: Error "not enough memory for internal tables" and how to fight it?   

Postby womak » 2013-5-27 @ 01:11

No, I'm calling a BAT-file once and it is registered by three commands for the each of 1500 files:
Code: Select all
CHDIR c:\ETERNA30
echo off
copy c:\inp\14_1.inp c:\ETERNA30\ETERNA.INP
eterna3.exe > c:\1.txt
copy c:\ETERNA30\eterna.prn c:\dat\14_1.dat

copy c:\inp\14_2.inp c:\ETERNA30\ETERNA.INP
eterna3.exe > c:\1.txt
copy c:\ETERNA30\eterna.prn c:\dat\14_2.dat

copy c:\inp\14_3.inp c:\ETERNA30\ETERNA.INP
eterna3.exe > c:\1.txt
copy c:\ETERNA30\eterna.prn c:\dat\14_3.dat

copy c:\inp\14_4.inp c:\ETERNA30\ETERNA.INP
eterna3.exe > c:\1.txt
copy c:\ETERNA30\eterna.prn c:\dat\14_4.dat
Attachments
start.bat____.txt
(155.88 KiB) Downloaded 49 times
User avatar
womak
Newbie
 
Posts: 9
Joined: 2013-5-26 @ 11:34
Location: Russia

Re: DosBox: Error "not enough memory for internal tables" and how to fight it?   

Postby ripsaw8080 » 2013-5-27 @ 02:32

If applications set their file handle count to something greater than 20 then DOSBox uses some of its table memory to hold the handle table. Table memory is never released, and if an application is run repeatedly in the same session of DOSBox then the table memory can be depleted at a rate that depends on the number of file handles requested.

DOSBox could probably be improved so the table memory is released when an application terminates, but it could be complicated to implement given how the table memory is currently allocated. In any case, an improvement like that would rarely, if ever, benefit DOS games, which are DOSBox's target.

A workaround is to limit applications to 20 file handles or less, if possible.
User avatar
ripsaw8080
DOSBox Author
 
Posts: 4368
Joined: 2006-4-25 @ 23:24

Re: DosBox: Error "not enough memory for internal tables" and how to fight it?   

Postby womak » 2013-5-27 @ 03:03

I get it. I run the program 109 times in one session DosBox and Table memory is filled .
Hopefully in the next version of the program will be able to release Table memory.
In the case of "Virtual PC" such schema is working.
User avatar
womak
Newbie
 
Posts: 9
Joined: 2013-5-26 @ 11:34
Location: Russia

Re: DosBox: Error "not enough memory for internal tables" and how to fight it?   

Postby womak » 2019-1-15 @ 01:31

Dear ripsaw8080, developer!
Is there a solution to my problem? I'm talking about running lot of files in one DOS session?
Maybe there was an opportunity to clean the descriptions of files in the allocation table?
Really looking forward to your reply!
Thank!
User avatar
womak
Newbie
 
Posts: 9
Joined: 2013-5-26 @ 11:34
Location: Russia

Re: DosBox: Error "not enough memory for internal tables" and how to fight it?   

Postby ripsaw8080 » 2019-1-15 @ 02:23

There have been no changes in DOSBox SVN regarding how table memory is used for expanded file handle tables, so your process will still have the same problem. DOSBox is intended for games; however, even for an application, yours seems unusual.

It is important to remember that if your program does not resize its handle table to more than 20 handles then no table memory will be consumed and thus it won't eventually run out within the session. Perhaps your program only needs a few files open at a time, but the runtime code (Fortran in this case) is requesting more than 20 file handles because it wasn't told otherwise, and maybe you can limit handles to 20 with a compiler switch or so.
User avatar
ripsaw8080
DOSBox Author
 
Posts: 4368
Joined: 2006-4-25 @ 23:24

Re: DosBox: Error "not enough memory for internal tables" and how to fight it?   

Postby llm » 2019-1-16 @ 06:12

I Use DosBox 0.74 to run the old DOS programm, written in Fortran. Nor any graphics, sound, joysticks, no, only mathematical calculations. I need to take 1500 source files and get 1500 results files.
In the first version I called DosBox from the my program each time where the file was ready for processing. It takes a lot of time.


i don't think DOSBox will change its behavior, but maybe we could provide you with a better solution, if you give us more information

Questions/Ideas:
1. is your host system linux(Distro, Version?), Windows (Version), 32 or 64bit?

2. do you got the Fortran-Program source code?
ever tried to compile your programs to native Windows/Linux executables with current Fortran compiler (and get blasting fast calculation times due to todays optimization)
for example gfortran(gcc-frontend), ,g95, PGI, Intel Fortran (Linux is free) or others free compilers see: https://www.fortran.com/compilers.html
(if we know your os we could tell you the correct compiler, compilation-instructions,...)
2.1. what Fortran: 66, 77, 95, ...?
2.2. what compiler was(is) used to compiling the source (very legacy?)
2.3. is the code using many external libraries or just simple single file sources?
2.4. could you upload the source code of at least one of your programs

3. under Windows/Linux VMWare-Player (Free) + FreeDOS would be the "fastest" solution (VirtualPC is very old)

4. try using "MS-DOS Player" for Win32-x64 (QEMU based)
https://virtuallyfun.com/wordpress/2011 ... -player-2/ (Short info)
http://takeda-toshiya.my.coocan.jp/msdos/index.html (Homepage)

5. are these links in any form related? found them using eterna + fortran in google
http://www.eas.slu.edu/GGP/ETERNA34/MANUAL/ETERNA33.HTM
ETERNA - Earth tide analysis package for the analysis of hourly Earth tide observations for gravity and tilt. ETERNA can be used for adjusting tidal and meteorological parameters from Earth tide observations, the computation of Fourier spectra, and the visualization of observations, residuals and spectra. by Hans-Georg Wenzel

The ETERNA package has benn given a new and better structure, and several programs have been renamed (e.g. the former program ETERNA has been renamed to ANALYZE, the former program ETGTAB has been renamed to PREDICT, the former program PRETERNA has been splitted into programs DETIDE, DESPIKE and DECIMATE). The program LOAD89 (provided by O. Francis) has been included, which enables the computation of ocean loading effects from different ocean tide models (Schwiderski 1980, CSR3.20, FES952).

https://github.com/hydrogeoscience/pygtide -> pyhton port of ETERNA PREDICT using f2py converter
https://de.wikipedia.org/wiki/Hans-Georg_Wenzel
http://igets.u-strasbg.fr/soft_and_tool.php ETERNA 3.40 download, also an successor of ETERNA (latest release 2019) available

i can compile ETERNA 3.40/predict.f without any problems with latest(trunk aka developer) x86-64 gfortran on godbolt.org
predict.f (former etgtab.exe): see: https://godbolt.org/z/p8q0Be
analyze.f (former eterna.exe): see: https://godbolt.org/z/Mkdiuv
using "-std=legacy -ffixed-form -O2" as command line parameters

sincerely awaiting 5 answers/statements :)
llm
Newbie
 
Posts: 42
Joined: 2009-1-18 @ 16:57

Re: DosBox: Error "not enough memory for internal tables" and how to fight it?   

Postby womak » 2019-1-17 @ 07:29

Dear LIM
I sincerely thank you for your interest in my problem!
1. I am running under Windows XP 32 bits, but I am going to switch to Widnovs 10 64 bits.
2. I do not have the source code for this program. I do not program, except for BASIC))). I am a simple user.
3. I tried to work with OracleVM, but I have not been able to configure the system yet. I have not tried working with other virtual systems, but I will try to do it.
4. "MS-DOS Player", I just found out about it, thanks.
5. You are right, this is the same program. I will be very grateful if you can compile it to work in a modern operating system.
User avatar
womak
Newbie
 
Posts: 9
Joined: 2013-5-26 @ 11:34
Location: Russia

Re: DosBox: Error "not enough memory for internal tables" and how to fight it?   

Postby llm » 2019-1-17 @ 08:22

1 and 3: you should use vmware player - its the best and free for personal use, skip virtualbox(oracle) virtualpc, qemu, bochs, ... not worth the try for your case

2. Source is part of the release, can you upload all the files,samples somewhere? Zip file on http://www.tinyupload.com for example

5. Can you try the 3.40 version (still dos i think) from this page if its work for your case? Its more uptodate and easier to compile
It seems that the file path are hardcoded, but maybe it works, analyze.exe is the new name of the eterna.exe
Or is there a reason for not using the latest version?
llm
Newbie
 
Posts: 42
Joined: 2009-1-18 @ 16:57

Re: DosBox: Error "not enough memory for internal tables" and how to fight it?   

Postby womak » 2019-1-17 @ 22:47

1. Yes, I can use the latest version of the program. I did not use it earlier because the program had a failure, but I could not understand the reason for this failure. Whether this is an improperly prepared data file, or an incompatibility of software, the absence of certain libraries or incorrectly specified paths.

2. I did not understand the question a bit. The source of eterna3.4 is located at the link you provided. The source of eterna3 (this is the previous version with which I work) is located by link http://s000.tinyupload.com/index.php?file_id=09798393688480095800. The eterna.inp file is the raw data, the eterna.prn is the result. Sorry for my English.

5. I will try the latest version of the program, I was going to do it for a long time.
User avatar
womak
Newbie
 
Posts: 9
Joined: 2013-5-26 @ 11:34
Location: Russia

Re: DosBox: Error "not enough memory for internal tables" and how to fight it?   

Postby llm » 2019-1-18 @ 06:04

1. Yes, I can use the latest version of the program. I did not use it earlier because the program had a failure, but I could not understand the reason for this failure. Whether this is an improperly prepared data file, or an incompatibility of software, the absence of certain libraries or incorrectly specified paths.


maybe we or some fortran pro can work this out :)

2. I did not understand the question a bit. The source of eterna3.4 is located at the link you provided. The source of eterna3 (this is the previous version with which I work)


maybe you skipped 3.40 because of calculation problems, or you were not aware of the version at all

eterna.for (the source file from your ETERNA30.rar) compiles out of the box without errors/warnings
see: https://godbolt.org/z/NJocA3

install the windows gfortran compiler (takes 5min)
1. go to http://mingw-w64.org/doku.php/download/mingw-builds (click on "Installation: Sourceforge")
2. install the downloaded mingw-w64-install.exe
3. run "C:\Program Files (x86)\mingw-w64\i686-8.1.0-posix-dwarf-rt_v6-rev0\mingw-w64.bat" to open a ming-w64 shell (cmd with some environment settings)
the "i686-8.1.0-posix-dwarf-rt_v6-rev0" path could be different on your installation
4. run "gfortran --version" - should say something with version 8.1 blabla
click on image so see sample from my desktop: https://imgur.com/a/3upIomi

i can compile eterna 3.40 analyze.f with that and get a working exe that fails to run because of wrong file-pathes - can be easily fixed in source
eterna.for (from your ETERNA30.rar) needs some fixes/aka configuration
its only the last function "SUBROUTINE GEOEXT(IUN6,DEXTIM)" starting in line 4601 - (the MSFOR lines are for the old Microsoft Fortran compiler, LAHEY for some MS-DOS compiler and UNIX is what you need - i think) - but definitily doable

in short: its doable to have a 32/64bit version of your programm :)

!!!PLEASE BACKUP EVERYTHING BEFORE CHANGING!!! or ask for further support :)

-it could be that your exact ETERNA3.EXE is not the compilation result of the given eterna.for (source) - pathes could be different, some constants, ...? i cannot check that -> but i think comparing the result of the old+new version with the same input will show
-it could be that the math precision etc. differs between the old compiler used to build your ETERNA3.EXE compared to the current compiler - calculation results can be different
llm
Newbie
 
Posts: 42
Joined: 2009-1-18 @ 16:57

Re: DosBox: Error "not enough memory for internal tables" and how to fight it?   

Postby llm » 2019-1-18 @ 15:34

got your 32bit windows build ready

-i configured the source of eterna.for properbly
-removed binarydirt from the end of ETERNA.DAT (part of the uploaded files)

http://s000.tinyupload.com/?file_id=041 ... 2306925217

ETCPOT.DAT <- fixed version
ETERNA.inp <- sample input from you
eterna3_32.exe <- the Mingw-w64 gfortran 8.1 static build (-O2 optimization level, march=native)
run.bat <- run and see - see comments

Copy the files to a diffent folder, no other files needed to to run the 32bit version of eterna

produces the very same result (except timeing and the cache-file (that is compiler related)) as DOS ETERNA3.EXE - but under windows AND in 0 seconds, but i would compare the result of a 1500 files run with 16bit ETERNA3 with the result of the same input using eterna3_32 - to be safe - or upload more samples

Are you using other executeables from the eterna3 package?

you need to compile it yourself if the exe don't work, but its super simple

ETERNA 3.40 should also work after some path fixes :)

And please tell me more about your background: are you using eterna for private fun?, what about the 5years!! gap asking for help in this forum?
llm
Newbie
 
Posts: 42
Joined: 2009-1-18 @ 16:57

Re: DosBox: Error "not enough memory for internal tables" and how to fight it?   

Postby womak » 2019-1-21 @ 06:23

Dear Lim!
I am very grateful for your help!
maybe skipped 3.40 because of the calculation problems

Version 3.4 is significantly enhanced than version 3.0. Version 3.4 adds many new features. I knew that such a version existed, but until a certain point it was not laid out in free access. I found this by searching on ftp servers, but I was not sure that it was a working version.
Later IGETS laid out the full distribution of this program, but I was already engaged in other tasks.
in short: version 32 / 64bit version of your programm

!!! PLEASE BACKUP EVERYTHING BEFORE CHANGING !!! or ask for further support


Thanks, Lim! Most likely I will choose the second)))

produces the very same result (that is compiler related)) as DOS ETERNA3.EXE — ETERNA3 with eterna3_32 - it can be safe - or upload more samples

I uploaded another file, it is very "long"

Are you using the eterna3 package?

I do not work with other programs from this package.

ETERNA 3.40 should also work after some path fixes

Thank you very much!

And what about the 5years !! gap asking for help in this forum?


Eterna is a classic for calculating the parameters of the Earth tides. I use this tool sometimes for my research. After I realized that DosBox would not work, I switched to Virtual PC and counted time for about a week. Then there were other tasks. Recently I decided to find out if the DOSBox has changed. I asked this question on the forum. Now I have a sample of a very large number of files from seven years of observations. I will try to count them in a program that You have optimized. Thank you so much again.
User avatar
womak
Newbie
 
Posts: 9
Joined: 2013-5-26 @ 11:34
Location: Russia

Re: DosBox: Error "not enough memory for internal tables" and how to fight it?   

Postby llm » 2019-1-21 @ 07:54

hope the exe is working for you

I uploaded another file, it is very "long"


-44sec in dosbox/eterna3, 0sec using native eterna3_32 - do you have even larger samples? maybe the timing stuff isn't working properbly
-very same result (except the bechmarking) as dos eterna3.exe - use text/binary diff program on all output files of both executables

small difference:

if ETCPOT.DAT is missing: (just a test i made)

dos eterna3.exe outputs
Data transfer beyond end of file, FILE=ETCPOT.DAT, UNIT=4 RECORD=1, Position=0
in etpota
Called by eterna

from F77L3.EER file
if F77L3.EER is also missing eterna3.exe outputs
Cannot Open Error file "F77L3.EER", error # 94


according to the internet:
F77L3.EER is only used by "Lahey Fortran" compiler based dos executables.
If an error occurs, this file is used to print error messages


eterna3_32.exe (based on the gfortran compiler) does not use the F77L3.ERR file and does not give any feedback
but maybe the newer file operations are better and do not try to read a non openable, missing file and just ends

i would say test the 32bit version of eterna3 a while and lets see how eterna 3.40 perform later (weekend i think)
but in the end you should use the uptodate (last release from 6.1.2019) eterna package from http://ggp.bkg.bund.de/eterna/ i think (i don't know how compatible this all this is)
llm
Newbie
 
Posts: 42
Joined: 2009-1-18 @ 16:57

Re: DosBox: Error "not enough memory for internal tables" and how to fight it?   

Postby womak » 2019-1-23 @ 03:38

Dear Llm!
I actively test eterna32. I have already processed more than 11,000 files (a large recording length has been obtained in five years). Analysis of the results of calculations performed later.
Thanks for your help!
User avatar
womak
Newbie
 
Posts: 9
Joined: 2013-5-26 @ 11:34
Location: Russia

Re: DosBox: Error "not enough memory for internal tables" and how to fight it?   

Postby llm » 2019-1-23 @ 06:23

How much faster is the new exe in you 11k file case, fast enough or still very long to calculate?

With some minor tweaks it could be possible to calculate with several running eterna3_32 in parallel
or maybe 3.40 or the newest release are better optimized
llm
Newbie
 
Posts: 42
Joined: 2009-1-18 @ 16:57

Re: DosBox: Error "not enough memory for internal tables" and how to fight it?   

Postby Stiletto » 2019-1-23 @ 22:42

llm wrote:4. try using "MS-DOS Player" for Win32-x64 (QEMU based)
https://virtuallyfun.com/wordpress/2011 ... -player-2/ (Short info)
http://takeda-toshiya.my.coocan.jp/msdos/index.html (Homepage)


It's actually MAME-based.
"I see a little silhouette-o of a man, Scaramouche, Scaramouche, will you
do the Fandango!" - Queen

Stiletto
User avatar
Stiletto
l33t
 
Posts: 4271
Joined: 2002-7-01 @ 21:57

Re: DosBox: Error "not enough memory for internal tables" and how to fight it?   

Postby womak » 2019-2-10 @ 09:25

llm,
At the moment I have completed the calculations in this program. I processed more than 45,000 files of different lengths, from 14 to 360 days. The total calculation time was 14 days !!! If I worked in a virtual machine, then maybe, would consider a whole month. The program worked without fail. Now I am analyzing the result.
I want to thank you once again for your help.
Thank.
User avatar
womak
Newbie
 
Posts: 9
Joined: 2013-5-26 @ 11:34
Location: Russia


Return to DOSBox Games/Apps

Who is online

Users browsing this forum: No registered users and 0 guests