what about int 15, service 87?

http://www.drdobbs.com/architecture-and-desig … pc-at/184408072

retardware wrote:

what about int 15, service 87?

http://www.drdobbs.com/architecture-and-desig … pc-at/184408072

doesn't xms driver use that and do some trivial jobs for you?

I'm too lazy to make a table and going into details would take pages of text on each and every of those points - so you get this:

What OS? Because there is DOS and "DOS" aka Windows 95+ command line. You might be able to get away with some tricks on 95/98 but not on NT kernels.
In general, the well-established methods work and are reliable. So real mode XMS/EMS will work on anything 286+ and all compilers, though for Basic and other high-level languages you might need to write custom assembly procedures to link with the program. Basic interpreters - not sure but probably in "forget it" category.

XMS is somewhat slow though as it requires copying the data to a window/buffer below 1MB, and to even do that you need to enter and exit protected mode. Which can be slow on 286 and even on 386 not all BIOSes deal with this efficiently (though HIMEM migh be able to work-around that with LOADALL instruction). True EMS is fast, and so is emulated EMS because it uses 386 paging, but the early standard still requires a transfer window in most cases and the later versions might not be available except on late 386+ / DOS 5+ machines. Also, emulated EMS will put the CPU in V86 mode and also most managers will take a few hundred kBs of RAM (even if mostly above 1MB).

Unreal mode is fun and great on paper. In reality it's difficult to pull off and will have issues on some machines. Forget about any memory managers, XMS including, unless it's physical (and EMS is mostly a 286 thing). In theory you could also do Unreal on a 286 but that'll only give you access to upper 64kB. So, Unreal equals 386 or later CPU.
For Unreal you will need a lot of custom assembly, need to control A20 line mask, and while it's possible to use high-level compilers the code mix will be neither trivial not efficient. Plus to actually use any high memory or 386+ registers you have to add 0x66/0x67 prefixes to all instructions, that bloats and slows down the code.

286 protected mode - a bit faster than XMS in real mode but good luck finding compilers and DOS-extenders and the docs, and then building a reasonably fast dev box that is not a 286 itself. Unless you need to support 286 class CPUs I'd say forget it, or go wild and don't care about compatibility too much. Or prepare to suffer.

386 protected mode - that's probably what you want. Especially since you can just use OpenWatcom with DOS4G/W - or some replacement like PMODE/W. Or you can just code everything in PMODE DOS-extended assembly, I used to do that. C is easier though. It's well travelled path and compatible with memory managers, VCPI/DPMI as well so no big issues (if any at all) with Windows including NT kernels.
There is also DJGPP and HighC and other compilers but I'm only familiar with Watcom and let me put it this way: There is a reason DOS extenders became so popular (instead of every program having it's own way of doing things), and among them DOS4G became a de-facto standard on the PC thanks to games. Watcom C compiler was also fastest 386/486 compiler, even for real mode. Today I guess a modern gcc version that can target i386 might beat it but you won't get any DOS support for gcc.

If this is for-fun personal project then go ahead and start with Unreal. You'll learn a lot and then abandon this idea with a switch to DOS extender. Assembly or C, it's more time spent on doing interesting things than figuring out very obscure PC architectural details that prevent your program from working.

Sorry, I didn't mean to insult, but I hate mis-information, so here's the correction.

Deunan wrote:

Unreal mode is fun and great on paper. In reality it's difficult to pull off and will have issues on some machines. Forget about any memory managers, XMS including, unless it's physical (and EMS is mostly a 286 thing). In theory you could also do Unreal on a 286 but that'll only give you access to upper 64kB.

Unreal mode is *VERY* practical and easy to implement as a small ASM stub that can be linked with 16-bit C real-mode code for the simplicity of real-mode without using DOS extender libraries. Back in the days of embedded x86 programming, C/ASM mixed language programming was almost a must. I had written quite a number of DOS command-line tools utilizing Unreal mode to peek/poke memory for the entire 4GB address space. The only precaution that one should implement in the Unreal mode ASM stub is to check for CR0.PE == 0 before proceeding with changing the data selectors base and limit or just return an error to the C caller, and C caller will print an error message indicating the presence of memory manager.

Unreal mode does work with the presence of XMS from any version of HIMEM.SYS. HIMEM.SYS does not put system in V86 mode. Only 386 memory manager (EMM386, QEMM386, 386MAX etc.) are incompatible with Unreal mode, even with NOEMS.

Unreal mode *CANNOT* work for 286, simply because the design of 286 protected-mode is the land of no-return. This is a well-documented poor engineering decision that failed 286 protected mode in the Intel microprocessor history.

Deunan wrote:

286 protected mode - a bit faster than XMS in real mode but good luck finding compilers and DOS-extenders and the docs, and then building a reasonably fast dev box that is not a 286 itself. Unless you need to support 286 class CPUs I'd say forget it, or go wild and don't care about compatibility too much. Or prepare to suffer.

286 protected-mode is a plague to avoid at all cost. It is not faster than XMS which later used the undocumented LOADALL instruction to enter/return from protected mode. Unless you know that the System BIOS is 100% IBM PC AT compatible, 286 protected-mode will likely fail. That was the reason that early 286 clones were not able to run OS/2 286 because the clones system BIOS did not have the correct CPU INIT handler implementation for 286 to return to real-mode after entering protected-mode, the notorious 286 motto of "switching off the engine to change gears"

I do agree that 386 protected mode with DOS extenders is the best option as this provides a full 32-bit flat mode experience for both code and data, and the DOS extenders library would take care of negotiating with 386 memory managers and interrupt handling as well.

kjliew wrote:

Unreal mode is *VERY* practical and easy to implement as a small ASM stub that can be linked with 16-bit C real-mode code for the simplicity of real-mode without using DOS extender libraries. (...)
Unreal mode does work with the presence of XMS from any version of HIMEM.SYS. HIMEM.SYS does not put system in V86 mode. Only 386 memory manager (EMM386, QEMM386, 386MAX etc.) are incompatible with Unreal mode, even with NOEMS.

I beg to differ. You make a nice Unreal program that works on your machine and then someone tries it on another one, where XMS manager and/or BIOS int 15h will enter protected mode to copy the memory above 1MB and then leave it (perhaps via soft-reset, because why not) and your shadow segment selectors are now back to 64k limits.
So no, it's not practical. It works, mostly, but when it fails it's usually a problem that can't be worked around. Unless it's a for-fun personal project I'd very much advise against using it.

Also it's a pure-DOS only solution as you noticed, will not work with EMM386 or in Windows. The code bloat and slowdown due to prefixes is still there. DOS extenders have exactly none of these issues.

kjliew wrote:

Unreal mode *CANNOT* work for 286, simply because the design of 286 protected-mode is the land of no-return. This is a well-documented poor engineering decision that failed 286 protected mode in the Intel microprocessor history.

I'm sorry, that was a bit of a mental shortcut on my part. What I meant is one can disable A20 mask and then the HMA area becomes available to the program. But it's only 64k and obviously limits what memory managers can be used, if any at all. All of the memory can be accessed with LOADALL but we could argue if that is Unreal mode or not. Certainly it's nothing like 386 with 4G limits.

kjliew wrote:

Unless you know that the System BIOS is 100% IBM PC AT compatible, 286 protected-mode will likely fail.

I have very little experience coding for 286 protected mode, I've only ever made one simple test program to enter and then leave that mode and it worked on my test machine - so I bow to your wisdom. But I do find it curious that you'd recommend Unreal mode on 386 when it has the exact same problem - you run it on a PC with buggy BIOS or OS or memory manager, and it will crash the machine and possibly cause HDD corruption in the process.

Deunan wrote:

You make a nice Unreal program that works on your machine and then someone tries it on another one, where XMS manager and/or BIOS int 15h will enter protected mode to copy the memory above 1MB and then leave it (perhaps via soft-reset, because why not) and your shadow segment selectors are now back to 64k limits. The code bloat and slowdown due to prefixes is still there. DOS extenders have exactly none of these issues.

My friend, please bear in mind that DOS is single thread. Unreal mode is *VERY* practical and reliable for DOS command line system tools. The code that deals with selectors base and limit does it on a per-occurrence basis whenever it needs to access data above 1MB. It is OK for the rest of the code to reset the limit into 64K. The rest are handled in standard 16-bit real-mode C code. The ASM stub to enter Unreal mode is less than 1KB, there is no such things as code bloat in using Unreal mode, where as for DOS extenders the stub itself is much larger. Though I won't say DOS extenders bloated because it makes sense to have that much of code due to other house keeping chores that typical DOS extenders need to provide.

Deunan wrote:

I have very little experience coding for 286 protected mode, I've only ever made one simple test program to enter and then leave that mode and it worked on my test machine

I guess what you did was entering 16-bit protected mode on a 386 or better, instead of a real 286, perhaps using LMSW/SMSW setting the PE bit. On a real 286, the PE bit cannot be cleared with SMSW without going through CPU INIT. So, 286 protected-mode is a failed concept for its intended usage. Back then, IBM worked around this issue with special CPU INIT handler in the BIOS. And no, the correct implementation of Unreal mode has no dependency on BIOS. That's what make it reliable.

I guess we'll have to agree to disagree then. DOS is single-threaded only in theory. I used DOS for many years, I remember what "TSR hell" is and how users would run various programs like accented letters input, screen grabbers / print spoolers, handy calculators, clipboard managers, various networking tools and whatnot. Many of these TSRs use interrupts and monitor keyboard and can run at any moment, and again many of them used various tricks to move the data above 1MB to leave as much free space for the main program as possible.

So unless you fully control your machine, Unreal is not simple. You can't just tell the end-user of a program not to load any memory managers if they have to work in network. The drivers will eat up most of DOS memory in many cases. The TSRs I mentioned might revert to "faster" methods of entering and leaving protected mode to copy data, something they will not do if the CPU is already in V86 mode. That's just examples from the top of my head.

I am fully aware of 286 limitations. All will say that if you claim that Unreal doesn't bloat the code and DOS extenders do then you haven't actally written a lot of said code. Try to make the 16-bit compiler access data that is more than 64kB long without using some really dirty tricks - you can't. Unless you run the code in protected more from the start and pointers are 32-bit so every library call will work properly. Sure if all you need is a simple peek/poke style 4GB access on your own machine for your own program then Unreal seems like a perfect and easy solution. But even that will limit you and anything else becomes much more problematic.

If you need to run on an 8088/8086 then EMS is the only option to go beyond 1MB. If you don't need to run on an 8088 but want to run on a 286, then XMS is probably the best option. If you want to have the widest compatibility you could support both EMS and XMS. These are both essentially bank-switching schemes, and work well in a scenario where you can process the data piece-by-piece without changing banks too often.

DPMI is the easiest to program because you get 32-bit linear addressing and you can have direct access to huge data structures. Besides requiring a 386, it can run into problems if you need to interact with real-mode programs or drivers though.

Doing 32-bit addressing on a 386 from real mode, or messing around with paging yourself is probably not worthwhile unless you're doing a driver that just needs to access memory-mapped I/O above 1MB, or you want to write your own OS.

thanks to all your guys, i still have a few questions that i am not clear on:
1. you can only manually copy data between xms and conventional ram, but not directly address it, allocate it to variables or pointers? that sounds like a temp file on disk.
2. unreal mode doesn't work on 286? any more detailed explanations on it?

Ad 1. You can if you run your code in protected mode - either because the OS natively supports it (Windows NT/95+ 32-bit applications) or via DOS extender and a compatible compiler (like Watcom+DOS4G). You can also, technically, do it in Unreal mode but not in any high-level language. As in, you can't just call, say, strlen in C with a pointer to a string located above 1MB. Because for 16-bit compiler the pointers are 16-bit, and so are all system/C libraries. That's my beef with Unreal mode in a nutshell.

Ad 2. What Unreal does is glitch the segment shadow registers by expanding the limit to 4GB in protected mode and not resetting it to 64kB on return to real mode. That can't be done on 286, it's not a 32-bit chip so there is no way of actually addressing more than 64kB per segment. What you can do is access the 64kB just over 1MB (the so-called HMA) "manually", but I only brought that up as an example (of it being useless). You could use LOADALL286 instruction but (no offense) if you are asking about these details then you don't have enough knowledge about 286/386 CPUs in the first place.

Deunan wrote:

286 protected mode - a bit faster than XMS in real mode but good luck finding compilers and DOS-extenders and the docs

Borland Pascal, and I think some versions of Borland C++ used the RTM.EXE DOS-extender that provided 286 protected mode.

Today I guess a modern gcc version that can target i386 might beat it but you won't get any DOS support for gcc.

On contrary, DJGPP is a DOS port of GCC (and other tools of the GNU toolchain), and last time I checked, it had quite a modern version of GCC.

For assembly I'd recommend NASM. It has clean, improved syntax and is capable of generating instructions with unusual addressing modes so you don't have to add hex prefixes (66h, 65h) manually.

bakemono wrote:

If you need to run on an 8088/8086 then EMS is the only option to go beyond 1MB.

*Ahem* PC/XT emulator PCE will also offer XMS/HMA in 808x mode. 😀

bakemono wrote:

If you don't need to run on an 8088 but want to run on a 286, then XMS is probably the best option. If you want to have the widest compatibility you could support both EMS and XMS. These are both essentially bank-switching schemes, and work well in a scenario where you can process the data piece-by-piece without changing banks too often.

Depending on the point of view, yes. Though EMS perhaps is more elegant in some ways. 😉
It maps memoy and sets pointers, XMS rather stubbornly copies data back and forth (more CPU intensive).
Plus, NEAT-type chipsets had MMU-like logic for EMS built-in. Anyway, EMS 4.0 was likely not the most advanced type of EMS.
AST's EEMS went much deeper, I believe. DESQView had special support for that, I recall.
Speaking of MMUs, there was a special MMU for 286 systems for use with PC-MOS, a sophisticated multi-user/-tasking sytem
that could run DOS applications (->ask your grandpa 😉 ).

Last but not least I would like to add that XMS is not the same as Extended Memory.
The "Extended Memory" INT15h interface could even be emulated on 286 (maybe 808x too ?) systems by using an EMS card.
AST had one named rex.sys. (Re: Is it possible to go over the 640KB barrier on an 8088 using some software ?)

Edit: Edited. Picture added.

Why access ram above 1mb? 640kb ought to be enough for anyone.