truth5678 wrote:

Please do -- it sounds like an interesting problem. Do you think the error is related to the the demo's loading of datafiles via INT 21h, or that dosbox loads the data to memory incorrectly? It seems like 0.72, 0.74, and SVN all have the same location for the error, but report it differently in terms of its origin.

Does the program run in real mode only?

It doesn't seem to really matter what mode the demo is run in, v86 or real mode, the crash is the same.

I see, I was merely guessing based on the error and the place in dosbox code where the error is logged. I wonder if a debugger with breakpoints would help (in core=full or normal)? Also, does pcem have the same error?

Edit: I'll recharge overnight, but I'll work on this further tomorrow! 😀 I'd like to try pcem to verify. I wonder where we could log for the source of the error, I'll continue to think about logging in dosbox-x.

The youtube video of the demo is here: https://www.youtube.com/watch?v=rFv7mHTf0nA. I think the crash was typically between 6:15 and 6:40 (according to the time on that recording). And another description of the scene here: http://en.wikipedia.org/wiki/Second_Reality#Vector_Balls. It is coincidentally involving a change of scene, presumably a loading of a datafile? (as you suggested?)

I just realized that that scene matches your description of 3d objects rotating as a tornado! It's definitely a candidate for looking at their source code and trying to remove that scene, or something similar.

Edit: the wikipedia page is more exact on the presumed cause of the crash during "vector balls":
"Because of a bug, this part will crash if the demo is installed in a directory with the complete path length exceeding 30 characters."

I should have previously realized that your insight was based on the path length problem which you fully tested? However, this must occur on real hardware with dos, and on pcem? Thinking more about it, it sounds like that you had already isolated the problem as a section of code associated with a hook on int21h to load files. We could start there and patch their source code?? 😀

Edit2: more from wikipedia: "The demo runs best on an Intel 80486 PC with a Gravis Ultrasound or a Sound Blaster Pro (or register-compatible clone). As the demo had in the original released version a slow down bug, a patch was released later."

And: "DOSBox is capable of emulating the exotic video modes and the Gravis Ultrasound preferred by Second Reality, and can be configured to the 33 MHz recommended on the demo's configuration screen for optimal viewing."

And code: https://github.com/mtuomi/SecondReality/.

Edit3: U2.ASM in the directory MAIN describes "MiniVectorBalls (PSI)" (MINVBALL.EXE); I think this must be the vector ball scene (given it's order in the list of scenes since other scenes have the word "vector" in it). MINVBALL.EXE is also mentioned in PACKFINA.INC and PACKING.INC.

I would think we could remove the line ";txt15 db 'MiniVectorBalls (PSI) ',0" from U2.ASM and recompile? Is it possible that the error is in MINVBALL.EXE?

Just one more redundancy in sdlmain.cpp. 😀

1void VFILE_Shutdown(void);
2void PROGRAMS_Shutdown(void);
3...
4void DOS_ShutdownDrives();
5void VFILE_Shutdown(void);

Also, in dosbox.cpp:

1static Bit32u			Ticks = 0;

It's not used in that file, but this appears in dos_files.cpp: Bit32u ticks = mem_readd(BIOS_TIMER);

I could check if it occurs in other locations, too.

And in dosbox.cpp:

1@@ -1007,7 +1897,7 @@ void DOSBOX_Init(void) {
2 		Pstring = secprop->Add_string("mode",Property::Changeable::WhenIdle,"ps2");
3+		if (i == 0) Pstring->Set_help(
4-		Pstring->Set_help(
5 				"Floppy controller mode. What the controller acts like.\n"
6 				"  ps2                          PS/2 mode (most common)\n"
7 				"  ps2_model30                  PS/2 model 30\n"
8@@ -1016,7 +1906,7 @@ void DOSBOX_Init(void) {
9 		Pstring = secprop->Add_string("chip",Property::Changeable::WhenIdle,"82077aa");
10+		if (i == 0) Pstring->Set_help(
11-		Pstring->Set_help(
12 				"Floppy controller chipset\n"
13 				"  82077aa                      Intel 82077AA chipset\n"
14 				"  82072                        Intel 82072 chipset\n"

truth5678 wrote:

Just one more redundancy in sdlmain.cpp. :) […]
Show full quote

Just one more redundancy in sdlmain.cpp. 😀

Copy code to clipboard

1void VFILE_Shutdown(void);
2void PROGRAMS_Shutdown(void);
3...
4void DOS_ShutdownDrives();
5void VFILE_Shutdown(void);

Also, in dosbox.cpp:

Copy code to clipboard

1static Bit32u			Ticks = 0;

It's not used in that file, but this appears in dos_files.cpp: Bit32u ticks = mem_readd(BIOS_TIMER);

I could check if it occurs in other locations, too.

And in dosbox.cpp:

Copy code to clipboard

1@@ -1007,7 +1897,7 @@ void DOSBOX_Init(void) {
2 		Pstring = secprop->Add_string("mode",Property::Changeable::WhenIdle,"ps2");
3+		if (i == 0) Pstring->Set_help(
4-		Pstring->Set_help(
5 				"Floppy controller mode. What the controller acts like.\n"
6 				"  ps2                          PS/2 mode (most common)\n"
7 				"  ps2_model30                  PS/2 model 30\n"
8@@ -1016,7 +1906,7 @@ void DOSBOX_Init(void) {
9 		Pstring = secprop->Add_string("chip",Property::Changeable::WhenIdle,"82077aa");
10+		if (i == 0) Pstring->Set_help(
11-		Pstring->Set_help(
12 				"Floppy controller chipset\n"
13 				"  82077aa                      Intel 82077AA chipset\n"
14 				"  82072                        Intel 82072 chipset\n"

For your last correction, I don't want to bloat other people's dosbox.conf files with 8 copies of the FDC help text, that's why it only sets the help text for the first FDC interface. Same reason the IDE code only sets help text for the primary interface section.

truth5678 wrote:

The youtube video of the demo is here: https://www.youtube.com/watch?v=rFv7mHTf0nA. I think the crash was typically between 6:1 […]
Show full quote

The youtube video of the demo is here: https://www.youtube.com/watch?v=rFv7mHTf0nA. I think the crash was typically between 6:15 and 6:40 (according to the time on that recording). And another description of the scene here: http://en.wikipedia.org/wiki/Second_Reality#Vector_Balls. It is coincidentally involving a change of scene, presumably a loading of a datafile? (as you suggested?)

I just realized that that scene matches your description of 3d objects rotating as a tornado! It's definitely a candidate for looking at their source code and trying to remove that scene, or something similar.

Edit: the wikipedia page is more exact on the presumed cause of the crash during "vector balls":
"Because of a bug, this part will crash if the demo is installed in a directory with the complete path length exceeding 30 characters."

I should have previously realized that your insight was based on the path length problem which you fully tested? However, this must occur on real hardware with dos, and on pcem? Thinking more about it, it sounds like that you had already isolated the problem as a section of code associated with a hook on int21h to load files. We could start there and patch their source code?? 😀

Edit2: more from wikipedia: "The demo runs best on an Intel 80486 PC with a Gravis Ultrasound or a Sound Blaster Pro (or register-compatible clone). As the demo had in the original released version a slow down bug, a patch was released later."

And: "DOSBox is capable of emulating the exotic video modes and the Gravis Ultrasound preferred by Second Reality, and can be configured to the 33 MHz recommended on the demo's configuration screen for optimal viewing."

And code: https://github.com/mtuomi/SecondReality/.

Edit3: U2.ASM in the directory MAIN describes "MiniVectorBalls (PSI)" (MINVBALL.EXE); I think this must be the vector ball scene (given it's order in the list of scenes since other scenes have the word "vector" in it). MINVBALL.EXE is also mentioned in PACKFINA.INC and PACKING.INC.

I would think we could remove the line ";txt15 db 'MiniVectorBalls (PSI) ',0" from U2.ASM and recompile? Is it possible that the error is in MINVBALL.EXE?

I was guessing on the memmove/memcpy thing. A quick look last night didn't reveal any obvious places I could see where memcpy() was used for overlapping memory copy. But the pattern seems to hold true in DOSBox. I hadn't thought of trying it in other emulators. I'll see if the same pattern happens if I boot into MS-DOS or if it happens in VirtualBox and QEMU.

The suggestion is to add "if (i == 0) " to both lines to avoid excess lines. The "+" is the addition, originally it was missing the if statement. It's confusing to see because I wrote the +/- in reverse order (please take a glance at the code, if you have time). 😀

TheGreatCodeholio wrote:

I'll see if the same pattern happens if I boot into MS-DOS or if it happens in VirtualBox and QEMU.

I was thinking the same thing today about booting dos, even 95 pmode dos!

I meant within dosbox-x, not real dos.

Also, is this ok? Would it be sufficent to unset vm_fake_io variables in cpu.cpp and let callback.cpp define their values (mainline dosbox option off)?

1Bitu vm86_fake_io_seg = 0xF000; /* unused area in BIOS for IO instruction */
2Bitu vm86_fake_io_off = 0x0700;
3Bitu vm86_fake_io_offs[3*2]={0}; /* offsets from base off because of dynamic core cache */
4void init_vm86_fake_io() {
5Bitu phys = (vm86_fake_io_seg << 4) + vm86_fake_io_off;
6Bitu wo = 0;
7if (vm86_fake_io_offs[0] != 0)
8return;
9/* read */
10vm86_fake_io_offs[0] = vm86_fake_io_off + wo;

Sorry for the delay. I tested the Second Reality demo in dosbox-x (actually a dosbox-x build with the vanilla dosbox s3 video) by booting a windows 95 image inside dosbox-x, exiting to DOS 7.0, and ran the demo from there. The demo worked in this setting while sound was off! I didn't test further, but I definitely passed the "Mini Vector Balls" scene. In fact, it ran very fluidly and my impression is that it ran better than in dosbox-x without a boot image (the part that previously worked anyways).

I could guess that there is missing emulation from vanilla dosbox(-x) which caused the previous crash. Also, this would prove that real DOS should run the demo without problem. Another guess is that this is related to the path problem in dosbox (the wikipedia author must have only tested inside dosbox); and that the 95 boot image is able to properly handle the directory paths. In any case, this means that dosbox-x has no bug related to the demo, but instead it's a case of incomplete emulation that is made complete with a dos image, or a 95 image (almost the same thing!). 😀

An interesting side note is that I didn't load anything in autoexec.bat or config.sys. So, EMS must have been entirely disabled.

Edit: further information on demo is reported here: Future Crew's "Second Reality" Demo won't run properly under DOSBox 0.7x

The vectorball code appears to use an uninitialized variable on the stack. The stack also contains the full pathname to 2ndfix.exe so that changing the path the initial garbage value for the variable also changes. The code works if you're lucky. If I set the variable to 0 under dosbox debugger, the vectorball effect works regardless of the path.

-and-

DOS stack changes may be because of IRQs or routines that use more stack space than dosbox.

However, since a dos image in dosbox bypasses the problem, and assuming that it isn't an immediate timing issue, then booting from the dos image may provide larger stack space somewhere? Seems like the dosbox debugger could yield further insight.

And a post by ripsaw may have relevance: [DemoScene]Bugs with some demos.

Edit2: tested ripsaw's patch with dosbox but didn't use a boot image, only emulated dos. The patch has an effect. Now it doesn't crash in the same way, instead the demo video freezes and the last second of audio repeats. However, the patch yields direction, as the change is in dos_execute.cpp.

I also didn't yet run with default dosbox configuration settings, as the above post recommends. However, tried 1) core=normal, cycles=15000 and 2) no sound in demo, core=dynamic,cycles=max,ems=false with same result. This reduces the likelihood of EMS emulation or Audio having a role. I think the debugger and the "stack space" idea should provide a test, at least the debugger result could be reproduced as reported in the above referenced Vogons thread. This could be compared to a debugger result where a dos boot image is used (or better yet using dos on real hardware).

Edit3: reproduced issue with dosbox 0.65, but the only difference is the audio will still play. Regressed dos_execute.cpp (at least most of it) in current SVN without effect.

Have an updated result for running the Second Reality demo.

I finally ran "2NDFIX.EXE", an update for Second Reality because of a timing bug in faster PCs. Running this updated executable runs the demo and I was able to bypass the "Vector Balls" scene, along with uninterrupted sound output. I confirmed this result with and without ripsaw's above "demo patch".

There is one remaining issue, however. When the "Vector Balls" scene starts, the video goes black. The audio continues to play. Then, when the next scene loads, the video returns and all is back to normal.

Edit: above remaining issue now solved, but the solution was outlined in the thread referenced above (I went back and read their posts carefully). The Second Reality directory cannot be too short in name (as Qbix discussed), so I renamed the directory from /SR to /REAL, as suggested. This allowed the "Vector Balls" scene to work without issue and then the next scene loads/plays as expected.

This confirms the directory path problem and that the demo is accessing the dos emulation in an unexpected way; implying that the dos emulation is missing something that is present in real DOS where booting it within dosbox.

This agrees with your analysis posted earlier, so my post may be redundant, but I wanted to confirm that the demo will play. As you said, there may be a simple workaround, but it seems debugger output with a comparison between real DOS and emulated DOS may show the problem lines and suggest a solution. 😀

Edit2: perhaps following DS:DX in the dosbox debugger will show differences in the directory path between real DOS and the emulated DOS?

I read through U2.ASM and U2A.ASM to find the file functions, so ran a breakpoint on irq 21 ah=3D, open file. Used a path which is known to fail, C:\SR\, confirmed that program execution stopped on the "open file" command for the problem spot, DS:DX showed MINVBALL.EXE file name, but I missed the instructions afterwards since "CPU:illegal unhandled irq called 6" printed many times to the debugger screen. Earlier in the debugging, found "C:\SR\SECOND.EXE" at 9E36:19D9. Also, there were a number of memory allocation commands before a file opens. The debugger seems powerful, but I wonder if searching for path names will reveal the problem? It presumably loads the MINVBALL.EXE program and starts that part of the demo, but then it has the crash given the directory path is in a given configuration (as you tested and showed). Probably better to log the instructions in the debugger and make a difference file between emulated DOS and real DOS?

Sorry if this is a question related to an outdated version of dosbox-x, but what does win95 do to communicate with the ide controller? Which ide commands? Because I'm getting "Unit number 00 going through real mode drivers." on win95 pc98 on xnp2 (I know this is an entire different architecture and won't be supported on dosbox-x, but my point is what does win95 actually do with the controller to enable 32-bit disk access).
Note: the pc98 implementation of the ide controller is almost the same except for the irq number which is only 1 instead of 2 (irq9 on pc98) and the i/o addresses differ, the rest is the same as the PC/AT (mainly the ata and atapi commands).

SA1988 wrote:

Sorry if this is a question related to an outdated version of dosbox-x, but what does win95 do to communicate with the ide controller? Which ide commands? Because I'm getting "Unit number 00 going through real mode drivers." on win95 pc98 on xnp2 (I know this is an entire different architecture and won't be supported on dosbox-x, but my point is what does win95 actually do with the controller to enable 32-bit disk access).
Note: the pc98 implementation of the ide controller is almost the same except for the irq number which is only 1 instead of 2 (irq9 on pc98) and the i/o addresses differ, the rest is the same as the PC/AT (mainly the ata and atapi commands).

There are two ways to get Windows 95 to enable it's 32-bit IDE driver.

One way is to inform Windows 95 through the ISA Plug & Play BIOS. Windows 95 seems to trust the BIOS to the extent that you can give it any weird combination of IRQ and IO port and it will treat it like an IDE controller. So enable PNP bios emulation and tell the IDE controller to announce itself through that.

If you don't do the PnP BIOS method, then you will need to go into the IDE controller section and set int13fakev86io=true. Windows 95's non-PnP IDE driver likes to autodetect what the BIOS is using by executing INT 13h AH=0x02 (read sector) in virtual 8086 mode with the IDE I/O ports trapped in a way that it can take note which I/O ports were touched. If it can't detect the IDE controller this way then it refuses to talk to the IDE controller. This is why DOSBox-X has code to deliberately cause I/O trap GPF faults where Windows 95 can see it to help this detection method work.

The problem with the vectorballs in the Second Reality demo is an uninitialized variable as described here.

Comparing the machine code running in the DOSBox debugger to the demo's source code, it becomes clear that the "j" variable in DOTS/MAIN.C is the culprit.

Probably should have been initialized to zero here, but was not:

1-   int a,b,c,d,i,j,mode;
2+   int a,b,c,d,i,j=0,mode;

The modulus keeps it in range, but the initial value can be problematic:

1            i=dottaul[j];
2            j++; j%=dotnum;

If this array index value is initially inside the array then nothing particularly bad happens; but if it's initially outside the array then the result can vary from nothing of consequence to a crash. Uninitialized behavior is usually considered arbitrary; but many things can influence the initial value sitting on the stack, so workarounds are possible.

I made a fix program that targets the issue by setting the initial value of the variable to zero -- no messing with paths needed. Run the program where SECOND.EXE is located to start the demo, including any command line parameters to be passed along. Assembler source included.

Edit: slight enhancement of the fix program to instead run the speed patch (2NDFIX.EXE) if it exists.

Thank you again, ripsaw! I'm in awe of your work, if I can call it that, it seems more like magic to me. 😀

Is it worthwhile to condense this part?

1	case M_TEXT:
2		vga.draw.address_add=vga.config.scan_len*(2<<vga.config.addr_shift);
3		break;
4	case M_CGA2:
5		vga.draw.address_add=vga.config.scan_len*(2<<vga.config.addr_shift);
6		break;
7	case M_CGA4:
8	case M_CGA16:
9	case M_AMSTRAD:	// Next line.
10		vga.draw.address_add=vga.config.scan_len*(2<<vga.config.addr_shift);
11		break;

to:

1	case M_TEXT:
2	case M_CGA2:
3	case M_CGA4:
4	case M_CGA16:
5	case M_AMSTRAD:	// Next line.
6		vga.draw.address_add=vga.config.scan_len*(2<<vga.config.addr_shift);
7		break;

Can someone tell me if the FDC-primary controller emulation is ready for 32-bit DA yet? For some reason I can't get the "int13fakev86io" to work yet in windows 9x. (This was in DOSBox-X 12-24-2014 code, tested with a regular 1.44MB image)

I don't think it's complete. I did confirm this by testing, too. In fact, ykhwong's newest build has additional floppy emulation code which is not in the latest release version of dosbox-x. The additions are documented at the dosbox-x github.

Also, you may search for a DOS utility named "fdc_test.zip". This archive has a program to test the FDC functionality in DOS. It seems to work ok in dosbox-x and the logs should show the test running, but it supports that the floppy emulation is still a work in progress.

[Ykhwong's] build from this January has an issue with the GOG versions of Secret Agent and Crystal Caves - when the screen scrolls horizontally, the UI elements at the bottom stutter/jump around.

Confirmed this issue. However, it doesn't occur in dosbox-x (July 19th build), so it must be a commit after that date. Also, the former solution is here to the shaking HUD problem: Crystal Caves Hud Shaking. So, assuming the current issue is related, then it should be a change to vga_draw.cpp, particularly in a commit that occurred after July 19th.

Edit: isolated the issue to this commit: https://github.com/joncampbell123/dosbox-x/co … f6d8174d2ba07b4. Removing that commit prevents the above issue. 😀

truth5678 wrote:

I presume that VGA chained mode was not clearly specified by a "standards" committee.

In the early days, IBM designed all these video standards. MDA, CGA, EGA and VGA were proprietary standards.
It's just that other companies stepped in, reverse-engineered IBM's designs and started marketing clones, and adding extra features.
You could argue that any card that doesn't work exactly like a real IBM in all cases is not 100% compatible.

I have actually tried to play with chained/unchained mode myself... I wanted to see if I could get linear access while still being able to access more than 64K of memory.
Since I didn't use an ET4000 (I used a CL5426), I saw the same thing: the 64K of memory is spread out all over the 256K, and switching modes will basically trash your graphics.
Under DirectDraw I can do this trick on that hardware though: I can have a primary surface of linear 320x200 256 colours, and have a secondary surface in videomemory as well, and use hardware pageflipping between them. In DOS that would probably require hardware-specific register tweaking.