OK, that suggestion (to add src/cpu/core_nomal_8086cpp) worked, and I was able to build DOSBox-X, and I figured out how to create a diff file with the clipboard paste code (which I didn't describe adequately in my previous message; it requires changes to keyboard.h and keyboard.cpp also).

Anyway, I've attached a diff file that shows the changes. I hope these might be useful as a starting point.

Again, this is NOT my code; it comes from the anonymous author of the dbDOS fork.

I merged the clipboard paste code.

I also need to explain one of my latest changes, the "sample accurate" mode of the DOSBox-X mixer. What that code does is it drives the mixer code to render one sample at a time at the output sample rate, rather than in 1ms blocks. The reason for this change is to support older DOS games that modulate Tandy or OPL chipset registers rapidly to do tricks like digitized speech. The 1ms block rendering is not high resolution enough to allow that trickery to work.

The only damage from the addition is that enabling "sample accurate" mode causes PC speaker and Sound Blaster direct DAC (non-DMA) output to go silent. I'll look into why this happens when I have some more free time. Until then, disable "sample accurate" mode if you need those two modes of audio output.

This fixes:

- The digitized music channel in the MS-DOS Tandy port of "Skate or Die". Enabling sample accurate mode makes the guitar clearer and much less muffled.
- DOS games that support Adlib "digitized speech" output. What was once buzzy noise becomes clearer when you enable the sample accurate mode.

Jorpho wrote:

TheGreatCodeholio wrote:

And despite that I just found out a recent 2015 demo "8088mph" does some really bizarre things to CGA hardware that drives DOSBox's CGA emulation absolutely nuts.

I reckon trying to emulate that one is mostly a lost cause. I have the impression that you'd need more cycle-accurate emulation than you would ever need for anything else, and that the result would only run very, very slowly. (Whatever tiny handful of programs are left written for the 5150 that people might still want to run, I can't imagine any of them having the same stringent hardware requirements, i.e. nothing but a 5150.) But that's just my feeling.

8088 MPH might be 'one of a kind' in that respect... at least, so far...
However, I don't think it's impossible to do cycle-exact emulation of an 8088+CGA system in realtime.
CPU-wise it's about as slow as a C64, and the graphics/audio hardware is much more primitive. And cycle-exact realtime C64 emulators have been around for years.
Even classic Amiga is emulated cycle-exact in UAE, without a problem in realtime on a modern system.

I think the PC emulation world needs an attitude-change, and I am glad TheGreatCodeholio is doing things to make PC emulation more accurate. It's just crazy that all old systems have very accurate emulators, except for the PC.
Sure, you won't need cycle-exact emulation for most games or even demos... But if you DO need one, there isn't anything available at this point.

Besides, DOSBox is actually very FAR from cycle-exact... Take for example the Tandy/PCjr fix that TheGreatCodeholio has just done. Games/demos that tried to play samples wouldn't even work until now, because DOSBox made no effort at time-accurate sound emulation whatsoever.
Even if 8088 MPH may never run on DOSBox, I welcome any improvements in its accuracy.
And I will try to help and assist where I can, by providing sample code, testing on real hardware, and investigating the DOSBox emulation code.

Scali wrote:

However, I don't think it's impossible to do cycle-exact emulation of an 8088+CGA system in realtime.
CPU-wise it's about as slow as a C64, and the graphics/audio hardware is much more primitive. And cycle-exact realtime C64 emulators have been around for years.
Even classic Amiga is emulated cycle-exact in UAE, without a problem in realtime on a modern system.

Are they really that "exact", though? I keep thinking of BSNES/Higan , which aims for cycle-exact emulation of a much more primitive system and is much slower.

I think the PC emulation world needs an attitude-change, and I am glad TheGreatCodeholio is doing things to make PC emulation more accurate. It's just crazy that all old systems have very accurate emulators, except for the PC.
Sure, you won't need cycle-exact emulation for most games or even demos... But if you DO need one, there isn't anything available at this point.

Besides, DOSBox is actually very FAR from cycle-exact...

Indeed, something like Bochs or MAME would already be much closer in that regard, I would think.

Jorpho wrote:

Are they really that "exact", though? I keep thinking of BSNES/Higan , which aims for cycle-exact emulation of a much more primitive system and is much slower.

Yes, especially on C64, a lot of graphics and sound effects require cycle-exact modification of certain registers.
So in order for these effects to work in an emulator (which they do), the emulator needs to keep track of the states of the VIC-II, SID and 6510 on a per-cycle basis.

On Amiga, the copper works by waiting for the CRT to reach a certain raster position. Again, this only works when you have very exact emulation.

Last time I looked, DOSBox only kept track of the CRT on a per-scanline basis, and only in CGA/EGA/VGAONLY configurations, not the default config.
Doing rastersplits simply isn't even possible at all, cycle-exact or not.
'Cycle-exact' is not all that relevant on PC though, since most clones are not cycle-exact to eachother anyway (for 8088 MPH we specifically chose the IBM 5150/5155/5160 family, they are all 100% cycle-exact to eachother, and there are clones out there that are 100% cycle-exact to these machines as well, probably by far the largest group of cycle-exact PCs). But at least the emulation should behave properly on a time-based scale... At the least it should be able to behave properly up to the PIT resolution, since time-critical effects on PC are often based on that (as the Tandy/Adlib sample playing fix demonstrates), and this gives consistent results across most clones.

Last time I looked, DOSBox only kept track of the CRT on a per-scanline basis, and only in CGA/EGA/VGAONLY configurations, not the default config.
Doing rastersplits simply isn't even possible at all, cycle-exact or not

I suppose the last sentence only applies to the default configuration, because mixed screen modes work quite well in CGA/Tandy/PCjr mode here.

NewRisingSun wrote:

Last time I looked, DOSBox only kept track of the CRT on a per-scanline basis, and only in CGA/EGA/VGAONLY configurations, not the default config.
Doing rastersplits simply isn't even possible at all, cycle-exact or not

I suppose the last sentence only applies to the default configuration, because mixed screen modes work quite well in CGA/Tandy/PCjr mode here.

That would work with a per-scanline algo.
I am talking about rastersplits, as in per-column, so processing the video registers multiple times per scanline.
Like this, from around 2:00: https://youtu.be/Am3sf9Ak-Vc

Ah, now I understand. Given the variety of I/O delays in EGA and VGA cards, I wonder if such a thing ever was feasible on real PC hardware.

TheGreatCodeHolio wrote:

So perhaps I should add test cases to DOSLIB that allows you to see what happens when DMA is directed at VGA memory vs system memory

In DOSBox-TC (DOSBox supporting copy-protected Transcopy-format images), I use mem_writeb rather than phys_writeb in dma.cpp if (and only if) the address is video memory. It's a bit hackish but seems to work so far.

NewRisingSun wrote:

Ah, now I understand. Given the variety of I/O delays in EGA and VGA cards, I wonder if such a thing ever was feasible on real PC hardware.

On EGA/VGA it's pretty much impossible to synchronize such effects, because the videocard runs asynchronous to the rest of the system, from its own crystal.
On CGA/PCjr/Tandy however, the video chip is clocked to the system clock, so you can write cycle-exact code based on the PIT or the CPU itself, which is something we also exploited to some extent in 8088 MPH.
As long as you target a specific CPU and chipset, it is possible to do raster splits like on the C64 that way.

Anyway, an emulator should work the same as the real hardware, so even if you can't control this effect perfectly on most PC-hardware, the emulator should show changes to video registers at the moment they happen.

Scali wrote:

On EGA/VGA it's pretty much impossible to synchronize such effects, because the videocard runs asynchronous to the rest of the s […]
Show full quote

NewRisingSun wrote:

Ah, now I understand. Given the variety of I/O delays in EGA and VGA cards, I wonder if such a thing ever was feasible on real PC hardware.

On EGA/VGA it's pretty much impossible to synchronize such effects, because the videocard runs asynchronous to the rest of the system, from its own crystal.
On CGA/PCjr/Tandy however, the video chip is clocked to the system clock, so you can write cycle-exact code based on the PIT or the CPU itself, which is something we also exploited to some extent in 8088 MPH.
As long as you target a specific CPU and chipset, it is possible to do raster splits like on the C64 that way.

Anyway, an emulator should work the same as the real hardware, so even if you can't control this effect perfectly on most PC-hardware, the emulator should show changes to video registers at the moment they happen.

I've had that in mind with DOSBox-X for awhile, the trick is how to update the VGA/EGA/CGA emulation to render with higher precision than one scanline. If I'm correct about this, CGA/EGA/VGA have a dot clock to drive pixel output, and then data is fetched in "character clocks" which are either 8 or 9 pixels wide? So if DOSBox were to render one character clock at a time at a high rate instead of once a scanline, that would better represent real hardware. A quick naive implementation of this would call that function 14318181Hz / 8 = 1789772 times a second to spit out 8 pixels per call for CGA, and double that for some VGA modes. Probably a bigger drag on emulation speed than the 48KHz sample accurate mode I added.

So I've been thinking of an implementation that withholds rasterization until an event happens that affects the raster, whether that's just the start of a new VGA frame or any change to DAC, AC, and some CRTC registers happen, or if VGA memory writes happen within a small window around the raster position. Then at the event, the emulation time is used to determine where the raster is, and scanlines right up to that point are rendered before applying the change to VGA state. For most DOS games that would mean the VGA engine is completely idle until the frame completes or the DOS game happens to draw where the raster is rendering which you would see as a tearline anyway. For DOS demos that use copper like effects and other trickery, the implementation would automatically work harder to render what actual hardware would do.

TheGreatCodeholio wrote:

So I've been thinking of an implementation that withholds rasterization until an event happens that affects the raster, whether that's just the start of a new VGA frame or any change to DAC, AC, and some CRTC registers happen, or if VGA memory writes happen within a small window around the raster position. Then at the event, the emulation time is used to determine where the raster is, and scanlines right up to that point are rendered before applying the change to VGA state. For most DOS games that would mean the VGA engine is completely idle until the frame completes or the DOS game happens to draw where the raster is rendering which you would see as a tearline anyway. For DOS demos that use copper like effects and other trickery, the implementation would automatically work harder to render what actual hardware would do.

Yes, a sort of lazy evaluation, I guess that would work.
Perhaps it may help to inspect the code of Vice or UAE to see how they solved it, but I assume it would be something like this.
I guess if you also introduce events for display_enable, you'd get an event at the start and end of every scanline. Not only will the emulator be emulating hardware accurately, you can also render the image on a per-scanline basis then, if nothing else happens, raster-wise.

What you could do, if you have such an emulation system in place, is to add a 'debug raster'. Vice has this, which allows you to see the whole scanline, even the part that is normally hidden in the hsync-area of the screen. This allows you to see EXACTLY where your raster effects occur, when you're trying to hide things in the border.

Also, speaking of accurate emulation... One thing that has always bothered me is that there is no border in DOSBox. Real CGA/EGA/VGA do have a border, and they have the ability to change its colour (games like Wolf3d and Doom colour the border red when you get hit). Rasterbars actually stretch across the whole width of the screen, not just the 'pixel area'.
That would be a nice improvement as well.

Scali wrote:

Also, speaking of accurate emulation... One thing that has always bothered me is that there is no border in DOSBox. Real CGA/EGA/VGA do have a border, and they have the ability to change its colour (games like Wolf3d and Doom colour the border red when you get hit). Rasterbars actually stretch across the whole width of the screen, not just the 'pixel area'.
That would be a nice improvement as well.

That's also what I had in mind! I always thought that what DOSBox was missing (and what modern LCDs and TVs are hiding from you on VGA inputs) are the overscan border that you used to be able to see on VGA monitors. Also that DOSBox's focus on the active picture area isn't very helpful when you're trying to write and test custom modesetting code. It doesn't show you how a VGA monitor would position the active area based on the blanking parameters you programmed. So you'd think everything was fine until on real hardware your VGA monitor shows the picture halfway off the monitor!

I also thought that if the overscan+blanking areas were rendered by VGA emulation that the AVI capture would better represent the overall raster and I wouldn't have to fake the blanking area and underscan effect with FFMPEG's crop/expand filters when transcoding to MP4.

Actually Scali that's an interesting point. As far as I know the raster bars can't extend into the blanking area of the VGA (or maybe, some SVGA chipsets allow that?). I'm not so sure about CGA. But since the DOS program can reprogram those CRTC parameters anyway there's no reason I know of that you can't just extend the overscan border all the way out to the point that it sits right against the h/v retrace area, effectively removing the blanking area.

TheGreatCodeholio wrote:

That's also what I had in mind! I always thought that what DOSBox was missing (and what modern LCDs and TVs are hiding from you on VGA inputs) are the overscan border that you used to be able to see on VGA monitors. Also that DOSBox's focus on the active picture area isn't very helpful when you're trying to write and test custom modesetting code. It doesn't show you how a VGA monitor would position the active area based on the blanking parameters you programmed. So you'd think everything was fine until on real hardware your VGA monitor shows the picture halfway off the monitor!

What's worse, DOSBox does not actually emulate the full screen timing. It seems to just grab the effective width/height of the screen area, and that's it.
I encountered this a few days ago, when I tried to run this prod: http://www.pouet.net/prod.php?which=65292
Apparently it was written against DOSBox, and never tested on real hardware.
The code to set up the 80x50 and 80x100 tweaked textmodes was really sloppy, and did not program the vsync area properly. DOSBox didn't care, but this is what happened on real hardware: https://youtu.be/E1B8yO3kJ-U

Because they only reprogrammed the number of visible scanlines, the vsync area just shifted upwards, and your frame was just shortened, causing multiple frames to appear inside a single image on the monitor.

Another, unrelated, issue here is that they also used the AT-based CMOS timer and second interrupt handler.
DOSBox doesn't care about that. It would be nice to have more control over what hardware your configuration does and doesn't have.
An 8088-class machine will not have such AT-hardware, so you should be able to disable it in your configuration.

TheGreatCodeholio wrote:

Actually Scali that's an interesting point. As far as I know the raster bars can't extend into the blanking area of the VGA (or maybe, some SVGA chipsets allow that?). I'm not so sure about CGA.

Well, I'm not entirely sure what happens on RGB interfaces, since they have separate lines for syncing and for colour.
So I think in theory they could just leave the background colour on at all times.
I'm quite sure VGA doesn't do this though, from what I recall on CRT screens, the VGA border was only a few pixels wide, and if you used the dials on your monitor to shrink the image, you'd get black around it.
CGA RGBI extends the border much further, basically full overscan. EGA probably does the same, since it uses the same RGBI interface.
I do see black if I turn my screen all the way to the right, but I'm not sure if that's because the RGBI is turned off for a small part of the screen, or if it's just because my screen is so far out of whack that the signal simply doesn't reach that part of the CRT anymore.

With CGA composite, there is just one signal, so the colour has to be turned off during horizontal blank, to sync the screen, as far as I know.
On a C64 you'd get the same thing, but the debug view in Vice is just that: debug. It just visualizes the full width of the scanlines, so you can see what cycle-timed code does even in those places, even though you probably could never get that part on screen from a real C64.

TheGreatCodeholio wrote:

But since the DOS program can reprogram those CRTC parameters anyway there's no reason I know of that you can't just extend the overscan border all the way out to the point that it sits right against the h/v retrace area, effectively removing the blanking area.

Hum yes, that is an interesting point. I wonder if reprogramming the VGA registers changes the border size, and if so, how does it respond?
That's something that would need to be tested on real hardware.
I know on CGA that if you use a smaller window tweakmode, the background colour still extends to full overscan.

Proper CRTC border emulation is necessary for "Exploring the IBM PC" and the Tandy mode of Impossible Mission II, as they modify CRTC values for scrolling-like effects.

I can confirm on an S3 Virge that it's possible to extend the overscan border out and replace the blanking area.

The VGA registers define active picture area, blanking start/end, and when retrace starts and ends. For any part of the picture NOT in the blanking area, the VGA renders active picture or overscan color. Overscan color is rendered for any area beyond the "display end" row/char count. If you reprogram the blanking start/end values to blank ONLY during retrace, then you extend the overscan color all the way out to the borders of the VGA raster. Not without some issues, of course. You'll see the right edge of the raster on a VGA monitor where the beam begins to move back to the left as a vertical line that's slightly brighter than the rest of the overscan border. You might also notice that your VGA BIOS programs the VGA raster so that it's vertically "lopsided", at least on my S3, there's more non-retrace scanlines on the top than on the bottom.

Also on the S3, setting the blanking start/end values beyond the total count means that no blanking happens whatsoever. This is not necessarily a good thing since that also means the RGB output is still emitting the overscan border during horizontal/vertical retrace which would show up as extra lines on older monitors and as a darkened picture on newer monitors that apparently use hsync to measure analog RGB "0V reference" that they use during the active raster.

Of course, if you want, you can also extend the active display area to the edges the same way. Perhaps to turn 320x200x256 mode into 352x220 or something like that without changing the horizontal and vertical scan rate.

Take a look: http://www.osdever.net/FreeVGA/vga/crtcreg.htm

As far as I know, CGA does not have the same level of control over the raster.

TheGreatCodeholio wrote:

I can confirm on an S3 Virge that it's possible to extend the overscan border out and replace the blanking area.

Ah, that's interesting, so you actually have control over the size of the border.
CGA doesn't have that.

TheGreatCodeholio wrote:

Also on the S3, setting the blanking start/end values beyond the total count means that no blanking happens whatsoever. This is not necessarily a good thing since that also means the RGB output is still emitting the overscan border during horizontal/vertical retrace which would show up as extra lines on older monitors and as a darkened picture on newer monitors that apparently use hsync to measure analog RGB "0V reference" that they use during the active raster.

That's interesting, I hadn't thought of that yet. CGA is digital, so there are no reference signals required.
Perhaps they added these border limits to VGA for exactly this reason: to make sure the hsync/vsync areas could be used as 'black' reference.
In NTSC/PAL signals, the definition of the signal encodes these reference-periods explicitly.

That does make me wonder how Amiga does it exactly. The border is 'full overscan', and the output is analog RGB, similar to VGA signaling.

Anyway, this means the borders of VGA could be emulated quite accurately.
With CGA/EGA I guess you could just derive the actual size from maximum overscan of an NTSC signal: A field in NTSC is 720x240. CGA/EGA actually output 640x200 pixels inside that area. You can derive the position of this 640x200 area within the full 720x240 frame from the CRTC values, and there's your border 😀
Would be nice if CGADEMO actually looked the same as on a real monitor: the rasterbars extending to the full width of the screen, not just the active pixel display area.

Edit: if you want to go for proper emulation of NTSC as well, eg for the 1024-colour mode in 8088 MPH, it definitely makes sense to emulate the position of the active area within the total frame, as well as the background colour, at least on a per-scanline basis.
Namely, the start of a scanline, and the NTSC colorburst signal are always at a fixed place. The pixel/background colour data actually modulate this colorburst signal, so their relative position determine the phase, which determines the hue shift you get.
Being able to position the pixel data at the proper place in the total NTSC waveform is what you need to accurately decode all the possible colour artifacts.
Reenigne already has a good NTSC decoding routine for that. The other problem that needs to be solved is that DOSBox currently works with just 8-bit indexed internal buffers, where you'd need truecolour for this.

Scali wrote:

With CGA/EGA I guess you could just derive the actual size from maximum overscan of an NTSC signal: A field in NTSC is 720x240. CGA/EGA actually output 640x200 pixels inside that area. You can derive the position of this 640x200 area within the full 720x240 frame from the CRTC values, and there's your border 😀

You're comparing apples and oranges there... When people speak about NTSC having 720 pixels horizontally, they're talking about pixels that are 1/858 of a scanline wide (13.5MHz), but CGA pixels are 14.318MHz (1/910 of a standard NTSC scanline, 1/912 of a CGA scanline).

With the decoders I've written, I've used borders of 10% on each side, giving 768x240 which is a pretty close match to image area you've mentioned (720*910/858 ~= 764). It'd be a nice thing for emulators to have image size and position controls like real monitors as well, but I doubt such controls would be accepted into DOSBox SVN.

Scali wrote:

Reenigne already has a good NTSC decoding routine for that.

And I'm planning to adapt it to work with DOSBox Real Soon Now (like, within the next week or two). I'd quite like to have a go at proper CRTC emulation as well (at least enough to get 8088 MPH to work properly).

Scali wrote:

The other problem that needs to be solved is that DOSBox currently works with just 8-bit indexed internal buffers, where you'd need truecolour for this.

I had a brief look at the DOSBox SVN source and it looks like all the infrastructure for this is already there (for SVGA modes). So this shouldn't be a big problem.

reengine wrote:

It'd be a nice thing for emulators to have image size and position controls like real monitors as well, but I doubt such controls would be accepted into DOSBox SVN.

Please take a look at at old post of mine for suggestions on dealing with overscan borders on non-4:3 displays.

NewRisingSun wrote:

reengine wrote:

It'd be a nice thing for emulators to have image size and position controls like real monitors as well, but I doubt such controls would be accepted into DOSBox SVN.

Please take a look at at old post of mine for suggestions on dealing with overscan borders on non-4:3 displays.

Ah right... I was only thinking about windowed-mode.
I think the full border should be visible in fullscreen mode as well though, so not just on the sides, but also the top and bottom.
And I agree with reenigne, having some controls to control the size and centering in horizontal and vertical direction would be nice, since that is how a real CRT works as well. There is no 'fixed' border size, you'd adjust it to your personal taste. In fact, in the early multisync years, you often had to adjust the screen for each different resolution, as there was no single setting where they were all equally centered, especially horizontally.