Hmmm.... Just discovered something weird trying to boot Windows NT 4.0 to verify CPU behviour to be working properly still.

Somehow it reaches what seems to be opcode 84h (TEST r/m8,r8), but fails to fetch the modr/m parameters for some weird reason?
Looking at the fetching precalcs, it looks like the settings aren't properly retrieved from the instruction parameter lookup table, if at all (perhaps it points to another opcode, so never fetched properly)?
Edit: Hmmm... Now after some more testing (Windows 95 again), I see it failing on a opcode seemingly page faulting and failing somehow (proceeding to execute an invalid instruction).
Edit: After improving that, now Windows 95 fails to boot, telling there's not enough memory to load the registry?

Modified immediate parameter reading faulting/aborting(including paging locks) to check for active instruction by checking the parameter passed to the immediate read function(indicating checking(0)/loading from instruction(1) or main fetching routine(3, for 286+ CPUs only)) instead of directly checking the CPU's fetching/EU execution status (which is reset on any fault or execution phase change).

Also fixed resetting EIP for faults not to manually just load the CS selector, to keep CS selector/descriptor relationship consistent.

Now Hiren's BootCD 7.8 changes once again, now trying to execute an invalid LXS (LDS,LES etc.) instruction with register parameter, in real mode?

Just improved two things on the x86 segment descriptor handling:
- Make RSM properly set the CS segment descriptor mode to real or protected-mode/V86 mode on Pentium and up (applying the CS real mode limit behaviour on Pentium properly).
- Make segment descriptor read/write/execute permission precalcs point directly into the global precalcs lookup table, instead of copying all 256 bytes over, for increased performance (as the data in the table never changes), thus reducing the 256-byte memcpy to a simple 64-bit or 32-bit assignment (depending on the executable address size being 64-bit or 32-bit).

Just tried some older stuff I was testing with in the past and tried booting MS-DOS 6.22 with QEMM 7.04 again.
It, after fixing some CPU committing to behave more correctly (globally after the instruction instead of while parsing it), it crashes, saying it 'Cannot load because there is not enough memory.', which is interesting at least. At least it doesn't crash like it did before.

I made a log of it running the protected mode part of the program. Looking at what it's doing, I don't see anything weird?
Perhaps the issue itself is inside V86 mode that's causing this?
Edit: Looking into good old Hiren's BootCD 7.8 I see the same booting it. Instead of properly starting to extract files, it will spew out some garbage now and execute an invalid GRP5 /7 opcode (FF /7), throwing it into an infinite loop.
Edit: Wfw 3.11 otoh now is crashing immediately upon boot, soon after displaying the loading screen.

Hmmm... Seeing as the testsuite (test386.asm) doesn't seem to give any errors and MS-DOS still boots, but anything on top of it (Windows, memory managers etc.) keep 'running out of available RAM', environment space etc. (even though 128MB is installed) there might be something else going wrong in the CPU emulation somehow?

OK. While finding this bug, I've improved the settings menu text and address inputs a bit.
It now properly supports the valid numpad keys too.
So on an address input (for breakpoints etc.) it adds numeric keys and kpen (keypad enter for confirmation, just like normal enter) key.
And on text inputs (filenames etc.) it supports -, + and . keypad keys as well, ignoring shift status (to prevent del and incorrect _ behaviour on those keys).
That will improve input support a bit when inputting numbers into the text fields.

Improved debugger support for two new types of breakpoints:
- Adding an U qualifier to the end of the breakpoint setting (after processor modes or ignoring of them) causes it to apply to SMM mode only (if not supplied it applies to non-SMM mode only).
- Adding an ! qualifier to the end of the breakpoint setting for disable it.

So now SMM can be filtered (or SMM being ignored otherwise), with breakpoints also allowed to be saved disabled (saving them for easy re-enabling once needed).

Edit: Hmmm... On the 8088 in cycle-accurate mode I see it executing, but somehow not timing correctly it seems? RETF completes in 1 cycle, which it shouldn't.
Edit: Fixed the CPU new instructions to be guaranteed proper state reset to execute all phases of an instruction, but it still crashes somehow, doesn't even POST on 8088 XT anymore (in cycle-accurate mode)?

OK. As a little side project, I've modified the disk image support a bit:
- sfdimg and normal static disk images now support having a global setting for different sector sizes (still defaulting to 512 bytes, but the data in the header is now effective and changable in new disk images). Software using custom values in said field will now change sector read/write behaviour, as the field is actually used instead of ignored (it used to be assumed to be set to 512 even if it wasn't). If using disk images with non-512 byte sectors on older versions of UniPCemu, they will be written as 512 byte sectors instead, so take caution on using these on newer versions.
Simply said, don't use sfdimg disks with non-512 byte sectors on older versions of UniPCemu, as it will write the data incorrectly on those (although by default it's already set to 512 on creating those disk images).
- Implemented FDI/HDI disk image support on the floppy and ATA hard drive controllers. Non-512 byte sectors will be handled accordingly now with the new support of these formats (as well as on the other formats that didn't support those, like static and dynamic disk images mentioned above).
- Implemented mounting of FDI/HDI disk images. FDI disk images are mounted on floppy disks (filtered actually) and HDI disk images on hard disks. The backend does actually support both on both disk types (when manually editing the SETTINGS.INI file), but the settings menu's own mounting option will only show the fdi in the list of floppy disk drives and hdi on the list of hard disk drives to make management and seperation easier. Both are handled in roughly the same way on the backend, so you can actually rename the files between the two extensions to change from hard disk and floppy disk and vise versa (the file format of both disk types is identical, minus a floppy unused field that isn't emulated, so both types of drives will mount the disk image is requested to). The floppy or hard disk drive controller itself might have problems doing that though (due to interface limitations of sector addressing and LBA addressing support).

The 808x still doesn't boot right now, so that's still to be fixed first for a new release to happen.

The i440fx running MS-DOS 6.22 also doesn't boot now. It displays "Starting MS-DOS..." and then starting executing some LDS infinite loop (checking for non-zero values, which never happen).
Edit: A little bugfix later (on REP instruction issues), I see (once a MS-DOS 6.22 floppy disk is inserted into the floppy drive), MS-DOS 6.22 properly booting from the floppy drive.
So the OS itself is working properly now?
So does hard disk file content listing using the dir-command.
So could that one REP instruction issue have been the main issue causing the crashes?....
At least booting from floppy works correctly it seems (MS-DOS 6.22 at least).
Booting the XT machine gets the BIOS stuck in what seems to be an endless memory validation loop? (Generic Super PC/Turbo XT BIOS 3.0 I think).

On the XT, I see the following BIOS ROM points being reached so far:
e0dc: put zeroes in memory until BX=A000 (end of 640K memory space).
Reaches E0DE.
F92D: Fill memory with 5555.
F933: Check against 5555.
F93F: Fill with AAAA.
F945: Check against AAAA.
F953: Fill with FFFF.
F959: Check against FFFF.
F965: Fill with 0000.
F96B: Check against 0000.
F971: Completed.
e0f4: Reached some kind of check?
E0F8: Memory size 640KB detected valid.
Gets to E100.
Setup stack at E111.
Some routine at E5C0 reached (AH=1, ES=40, DS=F000)?

OK. Ran my latest version of baresifter (within my own repostory, waiting to be merged into the official baresifter by blitz) with all filters disabled.
Somehow, it immediately errors out when handling an interrupt (?) from user mode apparently, when fetching the first instruction of the fault handler, it's fetching from a weird location somehow?
It was the user-mode "{ 4, { 0x66, 0xE9, 0x00, 0x00 } }, // jmp 0x4" testcase.
From what I can see, the JMP itself runs fine, but the fault handler starting somehow messes up, somehow setting EIP to an invalid value? It (EIP) seems to be only 16 bits wide somehow?
Edit: Huh? It looks like after that instruction, the CPU breakpoint isn't triggered correctly and the next instruction doesn't start properly for some reason?

Edit: Fixed potential issues with instruction handling not finishing properly when the fault handler for single-step interrupt handling terminates (properly detecting it's use and finishing the instruction).
Edit: Fixing that, it still reports "Instrution length: b0rken!" for some weird reason?
Edit: OK. Now MS-DOS 6.22 doesn't boot anymore, even from floppy?
Edit: Reverting to the single step breakpoint handling bugfix doesn't fix the issue, but booting the floppy disk version of MS-DOS 6.22 does somehow?
Edit: And with the last commit of non-repeating instructions committing CS/EIP to be returned to properly, that's fixed as well.
The floppy boot also crashes in the same way as the hard disk version if it's committing like a normal 'finished' instruction (setting a return point of hardware interrupts to be incorrectly past the REP instruction instead of before it, thus not interfering with the special REP handling and other repeated instruction breakpoints).
Edit: And floppy MS-DOS 6.22 is still booting now at least.
scandisk, otoh, crashes?
Edit: OK. Trying again to run the old 80186 testsuite ROMs (from https://forum.osdev.org/viewtopic.php?f=13&t=23739&start=15), it immediately faults on the 186 testsuites executed on the 8088 CPU. So somehow, something very wrong is in the CPU core somehow.
Edit: Trying the entire testsuite, during the jmp001 bin file execution it goes wrong immediately after the JS jump to a MOV [0],1234h instruction. Somehow, it's skipping the mov instruction entirely it seems (nothing is executed) an then immediately executes the folowing '1234h' instruction instead? So the MOV instruction is fetched, but instead of going to execution phase (and fetching the 1234h parameters) it starts executing the next instruction instead, parsing the immediate 1234h value as a new instruction! Or at least that's what I can see from the debugger log that's now properly implemented for those test ROMs (although you can't break execution, only terminate if it's hanging by closing the app).
Edit: More bugfixes. Now only the BCD, shifts and MUL tests report errors on the (upper FLAGS 4 bits stuck cleared modified 80186) latest commit. Though logically they shouldn't error out like that (from what I can see mostly on Arithmetic carry flag results somehow I think)? Did those changed vs 386+ CPUs?

While testing the new functionality, I found something very weird: the logging vsnprintf call was crashing on an invalid location!

Eventually found out that it was being called with a %u parameter (for file size, in this case a 64-bit INT containing 0x600000000000, being the file size (low 32-bits) and file start location (high 32 bits) of mapping).
But since the parameter after it was a %s (for a filename input, the next variables were all displaced by 32-bits in memory (used by the parser)).
Long story short, eventually it tries to use 6000h as the string input once it reaches that.
Always remember to check your vsnprintf parameters against actual variable sizes (if they mismatch, trouble occurs because of the stack!).

I did found out by watching the function run and looking how far it's been stored inside the resulting string container. The function was doing it's job as expected, but because the stack wasn't setup properly with the parameters, the crash occured.

Also running The Troubleshooter's CPU tests, all succeed (but the 'String instructions') and the HDD tests (for the controller test and some seeking tests, although those shouldn't happen?). HDD reads succeed properly though? PC Doctor doesn't report any CPU problems at all?

Just tried booting off the hard disk image again. It seems to boot just fine?

So if The Troubleshooter is reporting errors there, it's probably something else causing it? Perhaps some kind of check or write instruction failing?

Some ET3000 improvements are added now. Besides the logging crash (of option ROM addresses) bugfixes, the ET3000 now supports 8 simultaneous fonts (interestingly enough, it's the same as the VGA base memory map with bits 4-5 cleared. And the mapping of the top 2 bits effectively replaces the register select bits (from VGA) and thus mapping the 8 VGA character sets into direct use). Although software would need to disable the background attribute (blinking disable) using normal VGA functionality (nothing but the character map is affected by it right now. All font/back color handling stays unmodified right now.

Hmmm... Things seem to run fine just getting to the boot menu of MS-DOS 6.22 on the hard drive. Somewhere after that it manages to mess up the BIOS ROM it seems (seeing as the BIOS ROM flash itself seems to go fine the first boot)? But after reaching past the MS-DOS boot menu(Startup menu), it manages to mess up the BIOS flash ROM somehow (seeing as the hard disk image is unchanged when the issue occurs)?
Edit: Reboot during boot menu didn't change behavior. Going to a submenu and returning to the main menu somehow did seem to cause the BIOS ROM boot issue (so it's corrupted?)?

OK. Added a testsuite for checking the behaviour of string instructions (zero flag, with and without prefix, CX and memory results).
It's basically a more complete version of the checks done by test386.asm (although written myself) and able to compile as both a .COM MS-DOS program and as a BIOS ROM.

In dosbox 0.74-3 it gives the expected result ("No errors with string instructions.") and at least tries to terminate the app cleanly (using INT 21h AH=4C function call, followed by INT 20h and a simple RET). The caller(MS-DOS) FLAGS register is also pushed and restored to/from the stack on program entry and before said MS-DOS exit function call.
But for some reason, Dosbox becomes unresponsive? I don't know if the issue with Dosbox itself or my app though.

Now I'd just need to run it inside my own emulator. One nice thing is that if ran as a BIOS ROM (when compiled with the flag enabled) it will post it's results on the POST diagnostics port (default 80h, configurable) instead, when 'terminating the app' by hanging the CPU (CLI followed by HLT, just like other test ROMs do).

Oddly enough, when changing the setting to use 80386 opcodes (and adding a 32-bit version of the same tests, with 32-bit instructions on 16-bit address size), Dosbox doesn't give any results anymore somehow, perhaps crashing for some unknown reason? The code should still be functional afaik? Unfortunately with Dosbox, you can't inspect the POST card status, if it even has one, can you?
Perhaps an issue with compiling for 32-bit somehow? Or specifically the 32-bit operand size versions of the opcodes and data?

The code can be found at UniPCemu's repository within the UniPCemu/assembly subfolder, under the name teststring.asm (can be compiled with nasm using the supplied makefile, compiling to the same folder as the UniPCemu executable relative path (within ../../projects_build/UniPCemu)).
Edit: I think I found the issue. The startup routine creates a data segment out of a part of the payload of the executable, then takes the start of it, rounding the segment selector to use up towards it's first 16-byte alignment (to keep the data inside aligned as much as possible, which is why the segment is made larger by 32 bytes just to make sure there's enough room for all the data.
The program saves the stack pointer of the caller, together with the DS and ES segments (the code segment doesn't change) and then replaces the data segments with it's own (except the stack segment, which is stored and restored after execution, but otherwise used as before). The data and extra segments however are overwritten to point to the aligned segment address inside it's data block that's directly after the executable code (after the text constants).

Having fixed the global routine to perform it's job properly, using the data segment that's required instead of the segment that's incorrectly located at 0000:0000 in memory (where the IVT and a whole bunch of things like MS-DOS itself lives! 😖 ), it now properly writes and reads the data from the blocks it creates within said memory area (at offset 0, 1000h and 2000h if needed).
The program properly finishes executing on the Dosbox emulators without any visible issues, so it seems to be operating properly.
Of course the BIOS version strips away much of the DOS functionality, replacing it with simply I/O to the diagnostic port (configured in the configuration section, much like how test386.asm does it, except not in a seperate file).
It also has an option to add 80386 support, which really only turns the instruction set to a 386-compatible one (instead of 8086-compatible) and adds the 32-bit operand size instruction tests that work in the same way as the 16-bit and 8-bit ones (those should be theoretically unmodified from the 8086 version, unless the compiler does weird things with it (I know it converts some conditional jumps to the error handler to a "jump-if-to-not-take; jump to error handler; label to not an error" (which takes up no space, being the next instruction of the testsuite of course), which is factually just a 2-instruction combination it somehow creates itself (it's just a conditional jump inside the source code, but not in the executable and assembly listing (if you look at the instruction bytes that is, with knowledge of how x86 instructions encode you can clearly see a conditional jump to past the jump to the error handler if you do the math (the conditional jump immediate byte added to the unconditional jump instruction address that follows after it))).

So I now have a pretty good testsuite I think (although it's roughly the same as test386.asm, just checking more cases with REP and without REP and the zero flag operating correctly in multiple cases (zero flags (being set, cleared or left alone all being tested), repe/ repne/rep all operating correctly for every instruction it applies to)).
This would probably help me figure out what happens to the string instructions inside UniPCemu, if there's anything going wrong with them at all in the first case (and the issue isn't something entirely different).

Just improved the IBM XT PPI a bit. It now tries to emulate the command register a bit (writing flags to register C's outputs when instructed).
I've also implemented the IBM XT PPI output/input ports for ports A and B (port A in input mode writes won't write to the POST diagnostics for example, and reading port A while in output mode will give all ones (since it's in output mode, it can't be read afaik).

Anyway, with my new string instruction test application (and since MS-DOS is still booting properly somewhat at least), I can at least verify if what the testsuites are saying (that the string instructions are malfunctioning) is actually true (and it would identify what instruction case is failing) or if they are actually executing without issues and the issues are with another instruction instead (that's used by the diagnostics software).

Edit: Hmmmm... I can't seem to get UniPCemu to boot MS-DOS 6.22 past the boot menu anymore somehow? It reaches an undefined opcode right now, no matter what option in the boot menu is used?
Edit: Booting from a floppy disk still works properly...
Could it be that the hard disk controller is not working properly somehow?
Running my own teststring program to verify operation of the string instructions properly returns that there are "No errors with string instructions.".
So the actual string instructions are working without issues (this confirms that the prefixes, zero flag and string instructions in all of their combinations (with and without REP(E/NE) and using different zero flags as input and CX counts) are operating properly.

A bit more progress towards the DS=6FF0 loop at 8e81:00000df5 issue.

Managed to create a log of what happens until the issue is reached:

When DS becomes 6FF0, the main issue occurs.
Anyone can see exactly what's going wrong?
Edit: Hmmm... Just before the hang loop, the value loaded by DS:SI is a NULL pointer (pointing to 0000:0000, so the IVT, which probably shouldn't happen? It's then loading said contents from 0000:0019 due to the displacement used, causing the hang loop between 0 and 6FF0.

Looking at where that gets loaded, it's at:

1BIU T1 I		RealRAM(p):0008f614=8e(�); RAM(p):0008f614=8e(�); Physical(p):0008f614=8e(�); Paged(p):0008f614=8e(�); Normal(p):00000e04=8e(�); RealRAM(p):0008f615=d8(�); RAM(p):0008f615=d8(�); Physical(p):0008f615=d8(�); Paged(p):0008f615=d8(�); Normal(p):00000e05=d8(�)
2BIU T1 -	8e81:00000df7 C5 74 19 lds si,word ds:[si+19]	RealRAM(r):0001c763=00( ); RAM(r):0001c763=00( ); Physical(r):0001c763=00( ); Paged(r):0001c763=00( ); RealRAM(r):0001c764=00( ); RAM(r):0001c764=00( ); Physical(r):0001c764=00( ); Paged(r):0001c764=00( ); RealRAM(r):0001c765=00( ); RAM(r):0001c765=00( ); Physical(r):0001c765=00( ); Paged(r):0001c765=00( ); RealRAM(r):0001c766=00( ); RAM(r):0001c766=00( ); Physical(r):0001c766=00( ); Paged(r):0001c766=00( )
3Registers:
4EAX: 00003a01 EBX: 0000ef6f ECX: 0000008d EDX: 00020008
5ESP: 0000052e EBP: 00000002 ESI: 0000c00a EDI: 0005005a
6CS: 8e81 DS: 1074 ES: 7cd1 FS: 0000 GS: 0000 SS: 8e81 TR: 0000 LDTR: 0000
7EIP: 00000df7 EFLAGS: 00000202
8CR0: 00000010 CR2: 00000000 CR3: 00000000 CR4: 00000000
9DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
10DR6: 00000000 DR7: 00000000
11GDTR: 00000008f466ffff IDTR: 00000000000003ff
12CS descriptor: 00009308E810FFFF
13SS descriptor: 00009308E810FFFF
14DS descriptor: 000093010740FFFF
15ES descriptor: 00009307CD10FFFF
16FS descriptor: 000093000000FFFF
17GS descriptor: 000093000000FFFF
18TR descriptor: 00008B000000FFFF
19LDTR descriptor: 000082000000FFFF

Hmmm... Reverting the HDD emulation to the old of the last release (2023/04/11 18:33), except the PCI changes, power management triggers and active drive interrupts bugfix, seems to fix the MS-DOS 6.22 booting from the hard disk, except that it seems to somehow spam what looks like the roughly the first 32 or so ASCII characters on the screen on some places before displaying the boot menu (with sometimes the 4th characters (out of every 4 characters) different)?

Hmmm... Just ran PC-Check. All core processor checks pass? So the CPU is probably running fine according to it?

Just improved the SDL_net compatiblity on all versions (SDL1/2_net using the thread system to perform asynchronous connects) to support asynchronous connects.
This prevents the main (emulation) thread from hanging on itself while it's connecting to a host using the emulated Hayes modem. It will simply keep parsing the command handling (waiting for a result) until the connection finishes or aborts, unless aborted manually (by destroying the connection, like using ATH, although it won't accept new commands while performing a dial out yet).
The modem connection receiver is also adjusted to ring when it detects an incoming connection (not accepted yet) on SDL3 (this should be working properly now on latest SDL3, although untested due to main CPU/hardware issues preventing running the diagnostic software (Telemate in this case)), with accepting performing the actual accepting of the incoming connection (just like with a real modem).
Edit: And fixed SDL3 subsystem initialization. It was checking on the result being <0 for SDL_Init/SDL_initSubsystem while it's a simple SDL_TRUE(1)/SDL_FALSE(0) now, SDL_true on success ofc.
That fixes the main subsystem initialization, as well as the loading of SDL3_net (which requires it to load afaik).
SDL_net used to require the timer and events subsystems to be running to work. Although the timer subsystem doesn't exist anymore on the latest SDL3 version it seem (or is automatic)?