Chaosbreaker wrote on 2020-09-05, 23:54:

Hey everyone, […]
Show full quote

Hey everyone,

I've been lurking around on this site since forever, because more than once I've been able to find
a solution or workaround for a retrogaming problem I have been facing every now and then.

Decided to create an account now, because I'm having a problem that I just can't wrap my head around.

Disclaimer: During the golden age of DOS gaming, I was about 5 or 6, and although I've tinkered quite
a bit with systems back in the day, I never actually *really* knew what I was doing.

Long story short; I recently bought a brand new, sealed, never used Gateway Solo PIII laptop that I'm
*really* happy with. Replaced the harddrive with something a little heftier than standard, and crammed
it full with dos and early windows games.

As was to be expected, some games reintroduced me to the challenges that oldschool gaming posed
every now and again, with things like memory management and the such. Kinda 'relearned' how
everything worked, so had little problems.

But then came TES: Arena. It requires EMS to function, and also needs a whopping 603K of conventional
memory to not-crash. I've been able to accomplish one, and the other, but never at the same time.

Whenever I set config.sys to have DEVICE=C:\DOS\EMM386.EXE NOEMS, I have around 617K of
conventional memory, which is more than enough for Arena to run, but Arena needs EMS.
Loading all the programs that I've set up to load into the upper memory in config.sys and autoexec.bat
behave like they should, though.

HOWEVER

I enable EMS by changing config.sys to DEVICE=C:\DOS\EMM386.EXE RAM and my mem /c |more
output shows that absolutely *NOTHING* get's loaded into the upper memory, while all pointers in config.sys
and autoexec.bat are in fact set to load into the upper memory. This leaves me with only around 550K of
conventional memory, and is sure to crash the game whenever things get a bit 'busy' (like opening a map with
a lot of notes).

I have tried to use/install third party memory managers (QEMM, 386MAX, JEMM386 and JEMMEX), but I am
afraid they are (in one way or another) incompatible with the relatively modern hardware I'm using (ESS1980
soundcard, S3 Savage videocard, 192MB Ram). JEMM386 and JEMMEX seem to be violently incompatible with
Arena and just spew tons of text strings before brutally halting in an OVERFLOW. Shame, because especially
JEMMEX seemed to leave generous amounts of conventional memory intact. Having tried all these things,
it seems that 'good old' emm386 is my only viable option left.

Additional information: I have been playing around with the order of lines in config.sys and autoexec.bat, and
I have also been able to use CTMOUSE as a lightweight mouse driver, VIDE-CDD as a lightweight CD-driver and
the such, to minimize overhead and experiment with the results.

Besides of getting Arena work, I *really* want to know if the output I'm getting using mem /C |MORE
when EMM386 is set to RAM is normal. Is it normal that everything that I try to load into the upper memory
works fine when EMM386 is set to NOEMS, but whenever I set EMM386 to RAM, nothing seems to be loaded
into the upper memory anymore? Am I misunderstanding how this is supposed to work to begin with?

Additional info:

Memory information output in dos with EMM386 set to NOEMS (EMS off)
Modules using memory below 1 MB:

Name Total Conventional Upper Memory
MSDOS 16221 (16k) 16221 (16k) 0 (0k)
HIMEM 1120 (1k) 1120 (1k) 0 (0K)
EMM386 3120 (3k) 3120 (3k) 0 (0k)
COMMAND 2928 (3k) 2928 (3k) 0 (0k)
VIDE-CDD 20528 (20k) 0 (0k) 20528 (20k)
DISPLAY 8336 (8k) 0 (0k) 8336 (8k)
SETVER 448 (0k) 0 (0k) 448 (0k)
CTMOUSE 3104 (3k) 0 (0k) 3104 (3k)
MSCDEX 25888 (25k) 0 (0k) 25888 (25k)
Free 631808 (617k) 629808 (615k) 2000 (2k)

Memory information output in dos with EMM386 set to RAM (EMS on)
Modules using memory below 1 MB:

Name Total Conventional Upper Memory
MSDOS 16269 (16k) 16269 (16k) 0 (0k)
HIMEM 1120 (1k) 1120 (1k) 0 (0K)
EMM386 8368 (8k) 8368 (8k) 0 (0k)
COMMAND 2928 (3k) 2928 (3k) 0 (0k)
VIDE-CDD 20496 (20k) 20496 (20k) 0 (0k)
DISPLAY 8304 (8k) 8304 (8k) 0 (0k)
SETVER 416 (0k) 416 (0k) 0 (0k)
CTMOUSE 3104 (3k) 3104 (3k) 0 (0k)
MSCDEX 27952 (27k) 27952 (27k) 0 (0k)
Free 564304 (551k) 564304 (551k) 0 (0k)

The annoying fact that I've tried about 200 different combinations and have spent
a couple of weeks to get this to work is slowly driving me mad, because I have the
feeling I'm overlooking something, or maybe just don't understand a simple
element yet. It feels like I'm so close and I really want to make it work.

That said, I am pretty much able to play *every other game* in native DOS 6.22
and windows, but the completionist in me just wants to be able to add this game
to my "working library" of old games. It's nothing short of a compulsion, really 😜

Phew... That post turned out to be way longer than I imagined beforehand 😁

Do you have a dos=high,umb at the beginning of your config.sys ? If not, add it and reboot .

darry wrote on 2020-09-05, 23:59:

Chaosbreaker wrote on 2020-09-05, 23:54:

...

Do you have a dos=high,umb at the beginning of your config.sys ? If not, add it and reboot .

I saw a separate dos=umb and later on dos=high in there. Probably not at the beginning, I guess! Will check tomorrow, spent way too much time on this tonight (again), and if I wake up either her or our baby, I'm done for 😁

Note that this forum is for DOS games on *modern* systems. You should ask old hardware and driver/configuration questions in Marvin. Marvin, the Paranoid Android

I think your main problem is that your system has too many (or too big) ROM modules or other hardware that take up most of the low-end 1 megabyte address space. In other words, there aren't really any free upper memory blocks (UMBs) that you could use to load stuff into high memory. Please check how much upper memory MEM reports in total, both with with and without EMS enabled.

When EMM386 is configured with the NOEMS switch, EMS is disabled and DOS/HIMEM can use the entire EMS address space (64k) for additional UMBs. You could try telling EMM386 to use the MDA display address space (I=B000-B7FF), which should give you at least 32k. But I guess that this might just be as far as you can push that particular system.

If you're lucky, you might be able to disable some unneccessary devices in the BIOS setup and free up a bit of address space that way, but I'm not sure about that.

Moved to Marvin -> Software.

There is a virus that is known to stop everything from loading into upper memory. Perhaps you accidentally infected your system in the process of "cramming it full with dos and early windows games"? (If so, it would be very useful for posterity if the particular culprit could be identified.) See for instance What the...??! my UMB blocks have dissapeared! .

You need more UMBs and fewer loaded drivers.

EMS consumes a 64K page frame in upper memory, which wipes out an entire segment into which you would otherwise load modules, and there aren't that many segments to go around.

You don't need SETVER or DISPLAY. If Arena doesn't run from a CD then you don't need VIDE-CDD or MSCDEX.

You should be able to use the mono video area and all of D000 and all of E000 if your machine doesn't have network or SCSI devices with their own BIOS.

Try setting

EMM386.EXE RAM /PE000 I=B000-B7FF I=C800-CFFF I=D000-DFFF

This makes sure the page frame uses the E000 segment and uses typically-available ranges as UMBs: 32k of mono video ram, 32k of video rom, and 64k of adapter rom address space.

Chaosbreaker wrote on 2020-09-06, 00:19:

darry wrote on 2020-09-05, 23:59:

Chaosbreaker wrote on 2020-09-05, 23:54:

...

Do you have a dos=high,umb at the beginning of your config.sys ? If not, add it and reboot .

I saw a separate dos=umb and later on dos=high in there. Probably not at the beginning, I guess! Will check tomorrow, spent way too much time on this tonight (again), and if I wake up either her or our baby, I'm done for 😁

I just realized I did not fully and properly read your post, sorry .

I see now that some stuff is loading into upper memory, so please disregard my recommendation . As others have said, you are likely running out of UMBs .

Most simple way if you use msdos6 type "memmaker" and let the os do it's thing.
If you would like to "help" the OS, then put away the "big ones"

1. Setver is only for compatibility for very old software...I think you don't need it.
2. You don't need display.sys
3. Do you always need the CD/DVD, if not don't load the cdrom-driver and the mscdex.exe
4. If you always need it, use VIDE-CDD.SYS (you use that already) and SHSUCDX from freedos instead of mscdex.

General use:
"Devicehigh=" instead of "device=" in config.sys
and "LH" for "loadHigh" in your autoexec.bat bevor the drivers

Make a copy of "mem /c /p" and let us see what is loaded up and how much is free.

There is also a dependency in the bios. look for an option which describes the "upper-memory" resources.
Some machines can be switched to use a different upper memory management. (Mostly for SCSI-Controllers)

Doc

K1n9_Duk3 wrote on 2020-09-06, 02:16:

I think your main problem is that your system has too many (or too big) ROM modules or other hardware that take up most of the low-end 1 megabyte address space. In other words, there aren't really any free upper memory blocks (UMBs) that you could use to load stuff into high memory. Please check how much upper memory MEM reports in total, both with with and without EMS enabled.

When EMM386 is configured with the NOEMS switch, EMS is disabled and DOS/HIMEM can use the entire EMS address space (64k) for additional UMBs. You could try telling EMM386 to use the MDA display address space (I=B000-B7FF), which should give you at least 32k. But I guess that this might just be as far as you can push that particular system.

If you're lucky, you might be able to disable some unneccessary devices in the BIOS setup and free up a bit of address space that way, but I'm not sure about that.

That sounds interesting, how would I go about telling EMM386 to use this MDA display address space? (Just copying what you've wrote here, because honestly, I have no idea what it specifically means 😜 ) It might just give me the nudge that I need. I've replaced MSCDEX with the SHSCDX, and it gave me a couple of K to play with.

Jorpho wrote on 2020-09-06, 03:38:

There is a virus that is known to stop everything from loading into upper memory. Perhaps you accidentally infected your system in the process of "cramming it full with dos and early windows games"? (If so, it would be very useful for posterity if the particular culprit could be identified.) See for instance What the...??! my UMB blocks have dissapeared! .

I am definitely going to scan the drive for viruses in a bit. My go-to-method for manipulating my dos/win9X machine is to attach the HDD to my modern win10PC through an IDE-to-usb adapter (because it's an 80GB hard drive and modern partitioning tools gave me WAY more options than standard FDISK without or with large HDD support). Weird thing is though, that I've given 386MAX another go, and while ARENA still crashes on boot when I use it (I've had bad luck with installing it, so I decided to just try and load it from config.sys through DEVICE=C:\QMAX\386MAX.SYS with no additional parameters), it *does* enable EMS, and it *does* load other drivers like the CD-ROM in the upper memory. It's almost as if the problem is with EMM386 itself. I don't understand this at all! 😜

JudgeMonroe wrote on 2020-09-06, 04:05:

You need more UMBs and fewer loaded drivers. […]
Show full quote

You need more UMBs and fewer loaded drivers.

EMS consumes a 64K page frame in upper memory, which wipes out an entire segment into which you would otherwise load modules, and there aren't that many segments to go around.

You don't need SETVER or DISPLAY. If Arena doesn't run from a CD then you don't need VIDE-CDD or MSCDEX.

You should be able to use the mono video area and all of D000 and all of E000 if your machine doesn't have network or SCSI devices with their own BIOS.

Try setting

EMM386.EXE RAM /PE000 I=B000-B7FF I=C800-CFFF I=D000-DFFF

This makes sure the page frame uses the E000 segment and uses typically-available ranges as UMBs: 32k of mono video ram, 32k of video rom, and 64k of adapter rom address space.

Thanks for the advice, I ended up REM-ing out setver and display in config.sys, and the display settings in autoexec.bat, it did save me a couple of k. I guess I don't specifically *need* a CDROM to play arena, because there's probably abandonware workarounds for that. I've recently bought a crapload of original retrogames on our local Dutch online marketplace, and the purist in me really wants to make it 'work as designed'. (Then again, Arena is the only game actually giving me problems 😜 )

I'll try configuring EMM386 like you've proposed, but I have absolutely no idea what this'll do. I'm guessing these are specific memory area's/addresses?

It sometimes saddens me that I've been working in IT for 12 years now (I'm 30 ATM) and I know my way around technology post-WINXP, but everything "retro" seems so archaic to me. I really love this era, but it makes me feel so incompetent at a regular basis. 🙁

darry wrote on 2020-09-06, 04:31:

Chaosbreaker wrote on 2020-09-06, 00:19:

darry wrote on 2020-09-05, 23:59:

Do you have a dos=high,umb at the beginning of your config.sys ? If not, add it and reboot .

I saw a separate dos=umb and later on dos=high in there. Probably not at the beginning, I guess! Will check tomorrow, spent way too much time on this tonight (again), and if I wake up either her or our baby, I'm done for 😁

I just realized I did not fully and properly read your post, sorry .

I see now that some stuff is loading into upper memory, so please disregard my recommendation . As others have said, you are likely running out of UMBs .

Haha no worries, it was a pretty lengthy read. And I was so certain that I was going to be able to explain my problem in a short summary, too 😜 I've found that retrostuff is an absolute vacuum when it comes to your free time, and worse, much like an actual black hole, there's simply no escaping it... 😁

dr.zeissler wrote on 2020-09-06, 07:10:

Most simple way if you use msdos6 type "memmaker" and let the os do it's thing. If you would like to "help" the OS, then put awa […]
Show full quote

Most simple way if you use msdos6 type "memmaker" and let the os do it's thing.
If you would like to "help" the OS, then put away the "big ones"

1. Setver is only for compatibility for very old software...I think you don't need it.
2. You don't need display.sys
3. Do you always need the CD/DVD, if not don't load the cdrom-driver and the mscdex.exe
4. If you always need it, use VIDE-CDD.SYS (you use that already) and SHSUCDX from freedos instead of mscdex.

General use:
"Devicehigh=" instead of "device=" in config.sys
and "LH" for "loadHigh" in your autoexec.bat bevor the drivers

Make a copy of "mem /c /p" and let us see what is loaded up and how much is free.

There is also a dependency in the bios. look for an option which describes the "upper-memory" resources.
Some machines can be switched to use a different upper memory management. (Mostly for SCSI-Controllers)

Doc

I have actually started off my 'quest' with using memmaker, but me being the novice I am, I don't actually 'get' what it's doing. When I run memmaker, it'll ask me if my computer uses EMS software, and give me a yes/no prompt. When I say 'no', it'll actually load stuff into the upper memory, but when I say 'yes', it'll simply tell me "MemMaker cannot continue. No upper memory was created".

My config.sys loads all devices (except the ESSAUDIO.SYS audio driver) with DEVICEHIGH, and my autoexec.bat loads everything (again except other than ESSAUDIO.COM) with LH. The sound driver doesn't seem to like being loaded high, but even without using DEVICEHIGH and LH, seems to not take any memory from the conventional memory. I've tested the configuration with other games and the sound works fine, so I'm inclined to leave those specific lines as they are 😜

For my latest experiment, I've initially REM'ed out everything other than himem.sys and emm386.exe (in NOEMS mode) from config.sys, and everything from autoexec.bat. Added CTMouse.exe as a small test to config.sys. Mem /C |MORE output shows the following:

NAME TOTAL CONVENTIONAL UPPER MEMORY
MSDOS 13261 (13K) 13261 (13K) 0 (0k)
HIMEM 1120 (1K) 1120 (1K) 0 (0k)
EMM386 3120 (3K) 3120 (3K) 0 (0K)
COMMAND 2928 (3K) 2928 (3K) 0(0k)
CTMOUSE 3104 (3K) 0 (0K) 3104 (3K)
FREE 689952 (674K) 632800 (618K) 57152 (56K)

As you can see, CTMOUSE is loaded in upper memory, and there's a nice amount of Conventional memory free.

Now, I simply set EMM386 to RAM instead of NOEMS.

Mem /C |MORE output shows the following:

NAME TOTAL CONVENTIONAL UPPER MEMORY
MSDOS 13261 (13K) 13261 (13K) 0 (0k)
HIMEM 1120 (1K) 1120 (1K) 0 (0k)
EMM386 3120 (3K) 3120 (3K) 0 (0K)
COMMAND 2928 (3K) 2928 (3K) 0(0k)
CTMOUSE 3104 (3K) 3104 (3K) 0 (0k)
FREE 624464 (610K) 624464 (610K) 0 (0k)

As you can see, CTMOUSE doesn't load into upper memory, and now that I take a closer gander, there doesn't seem to be any upper memory available at all, which is what I guess some of you have been suggesting (though I don't understand the fine mechanics of how this works).

It still vexes me that while using EMM386 RAM simply doesn't seem to be able to support loading programs in the upper memory, 386MAX does, although unfortunately it doesn't seem to like playing along with ARENA.

Anyways, I want to really thank all of you for your replies, it's awesome to see that the retro-community is so very much alive and kicking.
I'll try the suggestions some of you gave me that I've not tried yet and get back to you with the results!

If memmaker does not work for you there must be something with the machine.
What machine is that and what cards (ISA/PCI/AGP) are already in it.

Try MSD and look for fragmented upper-memory.

dr.zeissler wrote on 2020-09-06, 08:28:

Try MSD and look for fragmented upper-memory.

Sorry, I've edited my post before seeing you've replied already, I didn't expect this community to be as active as it is!
I didn't know what MSD was, until I just typed it in. Can't believe I didn't know this existed. >.<
Will get back to you! 😁

dr.zeissler wrote on 2020-09-06, 08:28:

If memmaker does not work for you there must be something with the machine.
What machine is that and what cards (ISA/PCI/AGP) are already in it.

It's a (relatively modern) Gateway Solo 5300, PIII 600 mhz, with 192MB's of ram (although I've removed 128MB just to see if it's not the amount of memory giving me problems. Didn't work 😜). There's an ESS Maestro 3 (ESS1980 I think?) soundcard in there, and an S3 Savage IX.

dr.zeissler wrote on 2020-09-06, 08:28:

Try MSD and look for fragmented upper-memory.

OK, started MSD and selected the memory. It gives me an overview of the memory blocks in 640K to 1024K, and I guess I kinda see what it's all supposed to mean. This is what I can gather, based on the legend (EMM386 set to RAM):

F000 - FFFF = ROM
E400 - two blocks here show as available, then from the third block to EFFF = possibly available
D400 - E3FF = EMS Page Frame
D000 - D3FF = RAM
C000 - CFFF = ROM
B800 - BFF = RAM
A000 - B7FF = Available

I'm sure you can make more of this than I can, but I think this does show that there's some fragmentation going on right? I mean, RAM and ROM seem 'fragmented' I guess?

Output with EMM386 set to NOEMS for posterity:

F000 - FFFF = ROM
E400 - two blocks as available, then from third block to EFFF = possibly available
Halfway between D400 and D7FF - E3FF = Free UMBs
Three blocks after D400 to a one or two blocks after that = Used UMBs
D000 - D400 and two or three blocks after that = RAM
C000 - CFFF = ROM
B800 - BFFF = RAM
A000 - B7FF = Available

Does nicely seem to illustrate (and confirm) that there's Free UMBs available with EMM386 set to NOEMS, but not with EMM386 set to RAM.

Also used Memory browser, couldn't make head or tails of that. I guess it shows what memory addresses are in use by what program/command? There's multiple of every program, and they seem all over the place. Can't deduce anything right/wrong in there 😣

Setting EMM386 to RAM makes EMS possible. TO access the EMS Memory (your memory above 1MB) a 64KB Block in the upper Memory is USED!
These 64KB are not available for uploading drivers anymore! But I do not use NOEMS Option in every of my systems,
it is simply USELESS. Games that need EMS will not work and Games that only use XMS don't like if EMM386 is loaded.

You should try to move the 64KB EMS PageFrame to another address. Perhaps the rest of the upper blocks are not beeing
used to upload drivers because they don't fit in one piece in the blocks. You should try to move the EMS-Pageframe.

Perhaps using QEMM386 is a better option, it's more versatile and it supports "cpucache on / off" via batch for
speed sensitive games.

JudgeMonroe wrote on 2020-09-06, 04:05:

You need more UMBs and fewer loaded drivers. […]
Show full quote

You need more UMBs and fewer loaded drivers.

EMS consumes a 64K page frame in upper memory, which wipes out an entire segment into which you would otherwise load modules, and there aren't that many segments to go around.

You don't need SETVER or DISPLAY. If Arena doesn't run from a CD then you don't need VIDE-CDD or MSCDEX.

You should be able to use the mono video area and all of D000 and all of E000 if your machine doesn't have network or SCSI devices with their own BIOS.

Try setting

EMM386.EXE RAM /PE000 I=B000-B7FF I=C800-CFFF I=D000-DFFF

This makes sure the page frame uses the E000 segment and uses typically-available ranges as UMBs: 32k of mono video ram, 32k of video rom, and 64k of adapter rom address space.

K1n9_Duk3 wrote on 2020-09-06, 02:16:

I think your main problem is that your system has too many (or too big) ROM modules or other hardware that take up most of the low-end 1 megabyte address space. In other words, there aren't really any free upper memory blocks (UMBs) that you could use to load stuff into high memory. Please check how much upper memory MEM reports in total, both with with and without EMS enabled.

When EMM386 is configured with the NOEMS switch, EMS is disabled and DOS/HIMEM can use the entire EMS address space (64k) for additional UMBs. You could try telling EMM386 to use the MDA display address space (I=B000-B7FF), which should give you at least 32k. But I guess that this might just be as far as you can push that particular system.

If you're lucky, you might be able to disable some unneccessary devices in the BIOS setup and free up a bit of address space that way, but I'm not sure about that.

OH MY GOD. Just reverted my original settings to start from scratch (original autoexec.bat and config.sys). Added the line you guys suggested. Crashed on boot, but that made sense because autoexec was loading the video configuration. REM'ed the video stuff from config.sys and autoexec.bat, kept your EMM386 line intact, AND LO AND BEHOLD:

NAME TOTAL CONVENTIONAL Upper Memory
MSDOS 16221 (16K) 16221 (16K) 0 (0K)
HIMEM 1120 (1K) 1120 (1K) 0 (0K)
EMM386 3120 (3K) 3120 (3K) 0 (0K)
COMMAND 2928 (3K) 2928 (3K) 0 (0K)
CTMOUSE 3104 (3K) 0 (0K) 3104 (3K)
VIDE-CDD 20528 (20K) 0 (0K) 20528 (20K)
MSCDEX 27952 (27K) 0 (0K) 27952 (27K)
Free 703984 (687K) 629808 (615K) 74176 (72K)

This is amazing! This gives me more than enough leeway for some other heavy-hitting conventional-memory games that require EMS, and I still see room for improvement!

Now, for the technical nitty-gritty, I'm not entirely sure I understand what happened, but what I think I can deduce from all of this, is that EMM386 wasn't able to 'locate' all free/available memory locations, and the command parameters simply instructed EMM386 to use a certain memory range that WE know is available "Because I said so!" Am I understanding this correctly? If so, there might be hope for me yet!

I can't tell you guys how incredibly (and probably unproportionally) happy this makes me. Thanks so much for your suggestions, and thanks for having me aboard 😁

dr.zeissler wrote on 2020-09-06, 09:59:

Setting EMM386 to RAM makes EMS possible. TO access the EMS Memory (your memory above 1MB) a 64KB Block in the upper Memory is […]
Show full quote

Setting EMM386 to RAM makes EMS possible. TO access the EMS Memory (your memory above 1MB) a 64KB Block in the upper Memory is USED!
These 64KB are not available for uploading drivers anymore! But I do not use NOEMS Option in every of my systems,
it is simply USELESS. Games that need EMS will not work and Games that only use XMS don't like if EMM386 is loaded.

You should try to move the 64KB EMS PageFrame to another address. Perhaps the rest of the upper blocks are not beeing
used to upload drivers because they don't fit in one piece in the blocks. You should try to move the EMS-Pageframe.

Perhaps using QEMM386 is a better option, it's more versatile and it supports "cpucache on / off" via batch for
speed sensitive games.

I have actually tried QEMM386, but it did not play nice with my computer. Even when installing "express", it would hang on optimization. If I skipped optimization, it would just hang altogether. Shame, because I've heard good things about it.

What does cpucache on/off do? I'm guessing this is for games where the CPU speed would really impact the game and could make it unplayable? (I think I remember GTA1 being unplayably fast if you'd have a faster CPU back in the day)

cpucache.com can set the cpucache on and off bevore starting the game. I use it for some older dosgames like suffer from the tp-bug that is around 200-233Mhz.
I only run qemm386.sys without any parameters, the optimization with stealth e.g. does not work on every machine and my experiances are; if you set it extremer you set the config more games tend to make problems...so it's not getting the most free ram, it's about getting the most games running stable with one or two configs.