DOSBox-X branch

Here you can discuss the development of patches

DOSBox-X branch

Postby TheGreatCodeholio » 2012-3-27 @ 00:42

I figured my port was out of place on another thread about GUS emulation patching, so I thought I'd start this here.

This is a branch of DOSBox 0.74 that I have been working on and off for the past 8 months. I've been modifying a lot in the source to make the emulation more accurate, to fix other parts of the emulation, and to widen the kind of hardware DOSBox emulates. I call it "DOSBox-x" for lack of a better name.

I've been developing it alongside a source code library I've been writing that is both a learning tool and a functional library to talk directly to various hardware on a DOS PC. DOSBox is one of my testing grounds for this code, alongside VirtualBox and some ancient Pentium-100 hardware sitting around my house.

Source: http://jon.nerdgrounds.com/dosbox-x-20120326.tar.gz
Windows binary: http://jon.nerdgrounds.com/dosbox-x-win32-binary-20120326.zip
TheGreatCodeholio
Member
 
Posts: 320
Joined: 2011-8-18 @ 20:15
Location: Seattle, WA

Re: DOSBox-X branch

Postby TheGreatCodeholio » 2012-3-27 @ 00:49

DOSBox-X emulation fixes:

- Accurate PS/2 keyboard emulation, support for scan code sets 1, 2, and 3, emulation and response to LED commands, reset commands, mode changes, and obscure commands

- Gravis Ultrasound panning register fixes. Mainline DOSBox seems to have a buggy implementation that ends up locking all audio to center no matter what value is written. I fixed that code in this version. DOS programs that rely on stereo sound should actually play in stereo now through DOSBox.

- PS/2 AUX port emulation. This enables DOS programs and OSes to work that don't use INT 33h and communicate directly with the PS/2 mouse connected to the keyboard controllers "AUX" port. This code is wired in so that INT 33h emulation is enabled by default, but shut down the instant the "reset" command is sent to AUX.

- CGA snow emulation. If you set machine=cga and enable the option, video memory I/O is slowed down to CGA speeds and writes will trigger "snow", just like an actual CGA.

- Sound Blaster "Goldplay" support. Back in the early 1990's a tracker library was used in some demos that provided music playback to LPT DAC and various other outputs including Sound Blaster. The problem is the library's Sound Blaster support uses a weird hacked DMA transfer mode that is basically setting the DMA buffer length to 1 byte and having the controller loop over that, then writing the byte from an interrupt handler. Mainline DOSBox tends to render this as a low frequency "rumble", while this fixed version detects that case and plays the audio perfectly. This fix is ideal for some demoscene entries in the 1991-1993 time frame.

DOSBox-X emulation additions:

- ISA Plug & Play emulation. This is useful for PnP-aware DOS programs or for running Windows 95 within DOSBox. The implementation provides both BIOS and I/O compatible PnP interfaces to the PnP-aware software.

- Serial mouse emulation. You can enter "serial1=serialmouse" into your dosbox.conf to have DOSBox emulate a Microsoft compatible mouse connected to your serial port of choice. Some software made prior to the introduction of PS/2 mice need this. With Windows 1.0, this option is the only way to get a usable mouse cursor within the graphical interface.

- Higher memory limit. Mainline DOSBox limits system memory to no more than 64MB. This branch removes that limit and allows up to 512MB.

- Finer-grained memory sizes. If you want to relive the days when PCs had less than 640KB of RAM or had odd extended memory sizes like 1630KB, you can use the memsizekb= option to specify memory size in KB instead of MB.

- Refresh rate control. You can set a fixed refresh rate that all video modes are locked to. For example if you are capturing content for television, you can force the refresh rate to 59.94 to avoid framerate conversion issues with your video editing software later.
TheGreatCodeholio
Member
 
Posts: 320
Joined: 2011-8-18 @ 20:15
Location: Seattle, WA

Re: DOSBox-X branch

Postby Great Hierophant » 2012-3-27 @ 16:00

Awesome! Those features very are welcome.

DOSBox supports PS/2 mice, so if you use a serial mouse can you use it in games that support two mice?

The minimum memory that any PC ever had was 16KB. Can you get down that low?

The maximum frame rate supported by Youtube and many other video streaming sites is 30fps, so refresh rate control would be very useful.

TheGreatCodeholio wrote:DOSBox-X emulation fixes:

- Accurate PS/2 keyboard emulation, support for scan code sets 1, 2, and 3, emulation and response to LED commands, reset commands, mode changes, and obscure commands

- Gravis Ultrasound panning register fixes. Mainline DOSBox seems to have a buggy implementation that ends up locking all audio to center no matter what value is written. I fixed that code in this version. DOS programs that rely on stereo sound should actually play in stereo now through DOSBox.

- PS/2 AUX port emulation. This enables DOS programs and OSes to work that don't use INT 33h and communicate directly with the PS/2 mouse connected to the keyboard controllers "AUX" port. This code is wired in so that INT 33h emulation is enabled by default, but shut down the instant the "reset" command is sent to AUX.

- CGA snow emulation. If you set machine=cga and enable the option, video memory I/O is slowed down to CGA speeds and writes will trigger "snow", just like an actual CGA.

- Sound Blaster "Goldplay" support. Back in the early 1990's a tracker library was used in some demos that provided music playback to LPT DAC and various other outputs including Sound Blaster. The problem is the library's Sound Blaster support uses a weird hacked DMA transfer mode that is basically setting the DMA buffer length to 1 byte and having the controller loop over that, then writing the byte from an interrupt handler. Mainline DOSBox tends to render this as a low frequency "rumble", while this fixed version detects that case and plays the audio perfectly. This fix is ideal for some demoscene entries in the 1991-1993 time frame.

DOSBox-X emulation additions:

- ISA Plug & Play emulation. This is useful for PnP-aware DOS programs or for running Windows 95 within DOSBox. The implementation provides both BIOS and I/O compatible PnP interfaces to the PnP-aware software.

- Serial mouse emulation. You can enter "serial1=serialmouse" into your dosbox.conf to have DOSBox emulate a Microsoft compatible mouse connected to your serial port of choice. Some software made prior to the introduction of PS/2 mice need this. With Windows 1.0, this option is the only way to get a usable mouse cursor within the graphical interface.

- Higher memory limit. Mainline DOSBox limits system memory to no more than 64MB. This branch removes that limit and allows up to 512MB.

- Finer-grained memory sizes. If you want to relive the days when PCs had less than 640KB of RAM or had odd extended memory sizes like 1630KB, you can use the memsizekb= option to specify memory size in KB instead of MB.

- Refresh rate control. You can set a fixed refresh rate that all video modes are locked to. For example if you are capturing content for television, you can force the refresh rate to 59.94 to avoid framerate conversion issues with your video editing software later.
User avatar
Great Hierophant
l33t
 
Posts: 2203
Joined: 2003-4-27 @ 08:20

Re: DOSBox-X branch

Postby ovvldc » 2012-3-27 @ 16:54

Cool stuff. Are there any plans by mainline DOSBox developers to take some or all of this cool stuf on board?
ovvldc
Newbie
 
Posts: 41
Joined: 2008-8-17 @ 12:48
Location: NL

Re: DOSBox-X branch

Postby DosFreak » 2012-3-27 @ 17:16

As long as the "cool stuff" benefits gaming then there is no issue.
For the "cool stuff" that doesn't then it won't.

But it isn't as easy as just adding it and being done. Alot of testing would have to be done.
Game Acronym List
DosBox CVS Builds
DosBox Feature Request Thread
DosBox FAQ
PC Game Compatibility List
"People like coldplay and voted for the nazis, you can't trust people."
User avatar
DosFreak
l33t++
 
Posts: 9080
Joined: 2002-6-30 @ 16:35
Location: Your Head

Re: DOSBox-X branch

Postby TheGreatCodeholio » 2012-3-28 @ 05:15

Great Hierophant wrote:DOSBox supports PS/2 mice, so if you use a serial mouse can you use it in games that support two mice?


Well at the moment the mouse input would sent the same movement data to both ports, so there's not much of a point using both even if they did.

I want to point out though DOSBox 0.74 mainline does NOT emulate a PS/2 mouse. It emulates the BIOS calls, the BIOS callback interrupt, and INT 33h mouse driver functions, but if a program were to actually try and talk directly to hardware to get to the mouse, it wouldn't work. That's why the keyboard controller emulation was improved to emulate the AUX port.

I also want to make it clear: the way the code is written now, is that INT 33h emulation is enabled by default. If DOSBox-X sees the program write directly to AUX on the keyboard controller, then the code stops sending mouse events to INT 33h and sends them instead to AUX PS/2 emulation. If the DOS program calls the "reset" function of INT 33h, then mouse events are routed back to INT 33h and not to AUX PS/2 emulation. You may also find that the code is there to emulate the scrollwheel when DOSBox someday handles that event too.

Great Hierophant wrote:The minimum memory that any PC ever had was 16KB. Can you get down that low?


I tried, but below about ~160KB or so (I don't recall exactly) the DOS kernel emulation starts to act funny. I think the initialization code makes some assumptions that need to be addressed to bring it down that low. So currently it keeps the limit at about 192KB or above.

I forgot to mention in the post as well that in this branch I also added emulation of various BIOS INT 15h calls concerning memory maps. This makes it possible to see >= 64MB when you boot an OS from a disk image or hard disk image. I also fixed a bug in HIMEM.SYS XMS emulation that would have confused DOS programs if they used the 'extended' allocate memory block function to allocate 64MB or more. I know someone else on this forum hates me for it, but it brings DOSBox XMS emulation in line with the actual specification and avoids any future problems should we actually find a DOS program that would do allocate that much memory.

Great Hierophant wrote:The maximum frame rate supported by Youtube and many other video streaming sites is 30fps, so refresh rate control would be very useful.


It would eliminate a common source of blurring in DOSBox captures, yes. Be aware though some DOS games and many demoscene entries assume a 70Hz refresh rate and might run extra slow at 30fps.
TheGreatCodeholio
Member
 
Posts: 320
Joined: 2011-8-18 @ 20:15
Location: Seattle, WA

Re: DOSBox-X branch

Postby TheGreatCodeholio » 2012-3-28 @ 05:25

DosFreak wrote:As long as the "cool stuff" benefits gaming then there is no issue.
For the "cool stuff" that doesn't then it won't.

But it isn't as easy as just adding it and being done. Alot of testing would have to be done.


No problem. I maintain this as my own branch for my hobby programming needs. I don't expect all of the features to make it into mainline. I just want it here so that people can try it out just like they can the DOSBox megabuilds.

A lot of what I do to the branch tends to focus on hardware emulation accuracy and completeness rather than getting DOS games to run. I like to use it with a "dos" source code library I've been writing for the past 2-3 years that can be used as an educational tool on how to program the PC hardware directly (using Open Watcom C and NASM assembly language). I plan on releasing this collection as soon as I clean up the code a bit, it demonstrates how to talk to serial & parallel ports, VGA hardware, Sound Blaster and compatible cards, Gravis Ultrasound hardware, the DMA controller, the PIC and timer chips, and various fun things you can find in the BIOS.
TheGreatCodeholio
Member
 
Posts: 320
Joined: 2011-8-18 @ 20:15
Location: Seattle, WA

Re: DOSBox-X branch

Postby mr_bigmouth_502 » 2012-3-28 @ 06:20

the features you've mentioned sound awesome. :D if you found a way to do IDE/ATAPI emulation and simulate different CPUs/clockspeeds as well, then it would be perfect! :D

(no, I don't expect these features to make it into the main version of DosBox, though they would be beneficial for gaming IMO XD)
User avatar
mr_bigmouth_502
Oldbie
 
Posts: 1825
Joined: 2009-5-16 @ 19:43
Location: Canada

Re: DOSBox-X branch

Postby Qbix » 2012-3-28 @ 15:43

Don't bother with IDE emulation. That would be double work as that has been written before.
The tools you are writing sound pretty interesting.
Water flows down the stream
How to ask questions the smart way!
User avatar
Qbix
DOSBox Author
 
Posts: 10134
Joined: 2002-11-27 @ 14:50
Location: Fryslan

Re: DOSBox-X branch

Postby Jorpho » 2012-3-28 @ 18:00

Might the inclusion of CZM's patches be a possibility?
viewtopic.php?t=31689
User avatar
Jorpho
l33t++
 
Posts: 5520
Joined: 2003-2-14 @ 19:50
Location: Canada

Re: DOSBox-X branch

Postby VileRancour » 2012-3-28 @ 20:52

mr_bigmouth_502 wrote:(no, I don't expect these features to make it into the main version of DosBox, though they would be beneficial for gaming IMO XD)


yeah, they definitely would - especially the GUS panning fix, SB DMA hack support, and refresh rate locking.

CGA "snow" could also be useful for testing a few things... (IIRC, I think this only happened with original IBM CGA boards... most popular clone cards eliminated the snow thing).
User avatar
VileRancour
Oldbie
 
Posts: 1401
Joined: 2003-5-14 @ 22:11
Location: 1-01-80 0:00a

Re: DOSBox-X branch

Postby TheGreatCodeholio » 2012-3-31 @ 04:46

Jorpho wrote:Might the inclusion of CZM's patches be a possibility?
viewtopic.php?t=31689


Yes.
TheGreatCodeholio
Member
 
Posts: 320
Joined: 2011-8-18 @ 20:15
Location: Seattle, WA

Re: DOSBox-X branch

Postby TheGreatCodeholio » 2012-3-31 @ 04:59

Qbix wrote:Don't bother with IDE emulation. That would be double work as that has been written before.
The tools you are writing sound pretty interesting.


Too late already started :) I've written low-level C and ASM code to work with IDE controllers before, so I have a fairly accurate idea what I/O and interrupt with them is like. The basic communication is easy, write the registers with the sector/head/etc. and command and wait for the controller to signal completion, wait for interrupt, transfer data as 16-bit reads/writes through the data port. Some other aspects like hard drive DMA are a little more difficult.

The only gotcha with IDE controller programming is that there is a lot of hardware out there with bugs in their IDE controller and quite a bit of variation. I know of one laptop for example, an old P66 laptop, who's IDE controller could hang the entire laptop if my code polled the IDE controller status bits too fast (the fix apparently was to halt and wait for interrupt).

If I have some spare time I might also add floppy controller emulation too.

I'm glad you're interested in the DOS reference code, I hope it serves as an example on how to write DOS programs. I will release the code on this forum when I'm ready.
TheGreatCodeholio
Member
 
Posts: 320
Joined: 2011-8-18 @ 20:15
Location: Seattle, WA

Re: DOSBox-X branch

Postby TheGreatCodeholio » 2012-4-01 @ 01:46

Alright, here's the code.

Two ZIP files and one .tar.gz archive.

http://jon.nerdgrounds.com/dos/1321

dos-rev-00001321-source.tar.gz: Source code and makefiles
dos-rev-00001321-exe.zip: Compiled executables
dos-rev-00001321-build.zip: Compiled OBJ files.

In it's current form, you will need a 32-bit Linux system with NASM, UPX packer, Perl, GNU Bash 4.2, and the Open Watcom compiler installed under /opt/watcom. There is experimental support for auto-generating batch files so that a full compile can be done under DOS/Windows if the equivalent NASM and UPX packers are in your system path, I'm not certain it works properly yet.

You'll see a lot of test code that has been written, including some to deal with running under Windows, or working around DOS extender limits (if you're wondering what "fuck you DOS4GW.EXE" means). A lot of this code has been tested extensively under both DOSBox, VirtualBox, Microsoft Virtual PC, Bochs, and loads of old and new hardware sitting around my place.

Have fun.
TheGreatCodeholio
Member
 
Posts: 320
Joined: 2011-8-18 @ 20:15
Location: Seattle, WA

Re: DOSBox-X branch

Postby TheGreatCodeholio » 2012-4-01 @ 21:33

I thought I'd clarify that the posted source code is NOT an April Fools joke, in case anyone thought it was :)

Enjoy.
TheGreatCodeholio
Member
 
Posts: 320
Joined: 2011-8-18 @ 20:15
Location: Seattle, WA

Re: DOSBox-X branch

Postby DRomero » 2012-4-01 @ 22:36

Hi TheGreatCodeholio, i just registered my self to say thanks! I love your build and i will use for sure your library in my AMSTRAD PC1512.

Keep up the good job! :)
DRomero
Newbie
 
Posts: 2
Joined: 2012-4-01 @ 22:33

Re: DOSBox-X branch

Postby TheGreatCodeholio » 2012-4-01 @ 23:36

DRomero wrote:Hi TheGreatCodeholio, i just registered my self to say thanks! I love your build and i will use for sure your library in my AMSTRAD PC1512.

Keep up the good job! :)


Which code? The DOS library provided, or the DOSBox build I mentioned on this thread?

And speaking of which, I incorporated someone else's patch (on this forum) that added Amstrad 640x200x16 graphics. I verified so far that it works with DeluxePaint II and the Maupiti game. The next release of DOSBox-x will have it.

Finally, Open Watcom is open source but compiling from source on Linux is a total pain in the ass (and it only supports 32-bit hosts). You can get it to compile with a lot of coaxing and some hacking of their wmake files, but it takes a lot of work. Let me know if you want the compiled binary I use (Open Watcom v1.9 32-bit x86).
TheGreatCodeholio
Member
 
Posts: 320
Joined: 2011-8-18 @ 20:15
Location: Seattle, WA

Re: DOSBox-X branch

Postby TheGreatCodeholio » 2012-4-02 @ 01:34

Here is a YouTube video showing the difference the SB DMA hack makes for Goldplay-based demos.

www.youtube.com/watch?v=1SwoeNDte-M
www.youtube.com/watch?v=h_n0ZR2Exac
TheGreatCodeholio
Member
 
Posts: 320
Joined: 2011-8-18 @ 20:15
Location: Seattle, WA

Re: DOSBox-X branch

Postby DRomero » 2012-4-02 @ 22:25

TheGreatCodeholio wrote:
DRomero wrote:Hi TheGreatCodeholio, i just registered my self to say thanks! I love your build and i will use for sure your library in my AMSTRAD PC1512.

Keep up the good job! :)


Which code? The DOS library provided, or the DOSBox build I mentioned on this thread?



Both :) The custom DOSBox build for my current PC and the DOS libraries for playing/learning on my old computers (just for fun, nothing serious).
DRomero
Newbie
 
Posts: 2
Joined: 2012-4-01 @ 22:33

DOS library build 1392

Postby TheGreatCodeholio » 2012-4-18 @ 09:46

I am releasing the latest revision (build 1392) of the DOS source code library. This fixes some issues I found while testing the source code against some new scenarios and while filming the code running on various hardware (I will be releasing a video when I have time on the 8042 keyboard controller and how it works, and what the DOS source code library does to interact with it).

http://jon.nerdgrounds.com/dosbox-x/1392/

Additions to the library:
- DOS "flavor" detection. The library is able to detect MS-DOS vs FreeDOS
- If FreeDOS, the code is able to obtain the kernel revision and revision string
- VGA library now contains detection for Amstrad 640x200x16 graphics mode and corresponding test program.

Bug fixes: 8042 keyboard library/test program
- The PS/2 mouse main menu no longer reprints the menu text on EVERY keypress. But if it scrolls off-screen, you can force a reprint by pressing "space"
- Exiting the PS/2 mouse menu now forces the 8042 to drain it's output. This resolves issues with an unresponsive keyboard when exiting the PS/2 mouse menu.

Bug fixes: DOS library
- Fixed 16-bit builds not detecting HIMEM.SYS when HIMEM.SYS is obviously loaded in memory.
- DOSBox and VirtualBox detection code seems to cause mystery GPF fault and DOS32a brainfuck when 32-bit builds are run under FreeDOS 1.0 (kernel version 0.0.36), code has been updated NOT to scan if that particular FreeDOS kernel is detected. FreeDOS 1.1 doesn't have this problem.

Bug fixes: Sound Blaster library
- The latest changes to the DOS library made building the 16-bit "compact" memory model version of the TEST program impossible (more than 64KB of code), the makefile has been modified to not compile the dos86c build.

Bug fixes: VGA library
- Direct linking within the makefile caused Watcom to produce a 386 32-bit binary that did NOT link properly to the C runtime and was effectively unusable. Rewritten to use wlink instead, like all other projects. FIXED.

Cleanup:
- All make files have been cleaned up and dependencies rewritten in a cleaner manner.

Additional code checked:
- Under "misc" the hexmem program (a program for browsing the contents of your computer's memory) has been updated to use the LLMEM library. This means that if you have a recent processor and your motherboard supports 4GB of RAM or more, you can look at memory at or above the 4GB memory boundary from DOS! The code has also been updated to enforce physical memory limits reported by CPUID to avoid crashes, as some CPUs (especially Intel Atom CPUs) will trigger a fault if you attempt to read beyond the limit. (obviously, you must be running in pure DOS mode with or without EMM386.EXE as such techniques are invalid within a Windows DOS box).

Have fun!
TheGreatCodeholio
Member
 
Posts: 320
Joined: 2011-8-18 @ 20:15
Location: Seattle, WA

Next

Return to DOSBox Patches

Who is online

Users browsing this forum: No registered users and 0 guests