This is what happens with the POST 10h part(full log of all memory and registers in the old format(the common log format doesn't have any of those yet)):

Edit: Looking at the debugger, somehow ES is cleared after loading it with selector 0x18(16-bit protected mode segment with 1MB limit)? That causes the memory access after it to detect a NULL ES access, which causes a #GP fault?

Edit: Hmmm... Strange: The loading of DS seems to clear the ES selector?
Edit: Found the bug: Loading 32-bit registers into a segment register caused the 32-bit write routine to write to the segment register and immediately after write a 32-bit value to the 16-bit segment register pointer(which was typecased to a 32-bit pointer for easy compatibility). Thus it was writing zeroes to the next segment register(or CPU data following it itself), causing bugs(NULL checks being triggered on data segment registers when being used in instructions). Fixing this makes it continue to the next steps.

Edit: After fixing one more bug(16-bit segment register write is always 16-bit, never using 32-bit registers or memory), it now proceeds to step 12h, which fails. Apparently the BSF instruction is incorrect?

Current implementation:

1void CPU80386_OP0FBC_16() {
2	word temp;
3	modrm_generateInstructionTEXT("BSF",16,0,PARAM_MODRM12);
4	if (modrm_check16(&params,1,1)) return;
5	if (modrm_check16(&params,1,0)) return;
6	if (CPU8086_instructionstepreadmodrmw(0,&instructionbufferw,1)) return; //Read src!
7	if (instructionbufferw==0) //Nothing?
8	{
9		FLAGW_ZF(1); //Set zero flag!
10		BST_cnt = 0; //No count!
11		CPU_apply286cycles(); /* Apply cycles */
12	}
13	else
14	{
15		if (CPU8086_instructionstepreadmodrmw(2,&instructionbufferw2,0)) return; //Read dest!
16		if (CPU[activeCPU].instructionstep==0) //Executing?
17		{
18			FLAGW_ZF(0);
19			temp = 0;
20			BST_cnt = 0; //Init counter!
21			for (;(((instructionbufferw>>temp)&1)==0) && (temp<16);) //Still searching?
22			{
23				++temp;
24				instructionbufferw2 = temp;
25				++BST_cnt; //Increase counter!
26			}
27			++BST_cnt; //Increase counter!
28			++CPU[activeCPU].instructionstep;
29			CPU_apply286cycles(); /* Apply cycles */
30			if (modrm_ismemory(params)) return; //Delay when running!
31		}
32		if (CPU8086_instructionstepwritemodrmw(4,instructionbufferw2,0,0)) return; //Write the result!
33	}
34} //BSF /r r16,r/m16
35void CPU80386_OP0FBC_32() {
36	uint_32 temp;
37	modrm_generateInstructionTEXT("BSF",32,0,PARAM_MODRM12);
38	if (modrm_check32(&params,1,1)) return;
39	if (modrm_check32(&params,1,0)) return;
40	if (CPU80386_instructionstepreadmodrmdw(0,&instructionbufferd,1)) return; //Read src!
41	if (instructionbufferd==0) //Nothing?
42	{
43		FLAGW_ZF(1); //Set zero flag!
44		BST_cnt = 0; //No count!
45		CPU_apply286cycles(); /* Apply cycles */
46	}
47	else
48	{
49		if (CPU80386_instructionstepreadmodrmdw(2,&instructionbufferd2,0)) return; //Read dest!
50		if (CPU[activeCPU].instructionstep==0) //Executing?
51		{
52			FLAGW_ZF(0);
53			temp = 0;
54			BST_cnt = 0; //Init counter!
55			for (;(((instructionbufferd>>temp)&1)==0) && (temp<32);) //Still searching?
56			{
57				++temp;
58				instructionbufferd2 = temp;
59				++BST_cnt; //Increase counter!
60			}

61			++BST_cnt; //Increase counter!
62			++CPU[activeCPU].instructionstep;
63			CPU_apply286cycles(); /* Apply cycles */
64			if (modrm_ismemory(params)) return; //Delay when running!
65		}
66		if (CPU80386_instructionstepwritemodrmdw(4,instructionbufferd2,0)) return; //Write the result!
67	}
68} //BSF /r r32,r/m32

'

Edit: There were some bugs in the way the data was read/written, as well as the results being incorrect. After fixing this, the test now passes and moves on to the next test(Bit Test instructions), which fail also.

Edit: Having fixed those, it now continues on to the Near and Far call in protected mode tests(POST 16h). That one HLTs when it returns from a CALLF?

Edit: It seems to be HLTing after a 16-bit RETF at 0010:19DA?

The RETF after the "o16 call word %1:%%farfn16" at https://github.com/barotto/test386.asm/blob/m … /call_m.asm#L65 seems to go wrong for some unknown reason? The offset is written, but the segment is zero?

Edit: There was a problem with 16-bit stack pushes on 32-bit operand sizes. This has now been fixed, so the CALLF test passes.

Now it tries the ARPL instruction. The first one succeeds, but the second one fails(the one on a memory operand).

Note that ARPL won't write to memory unless it has to alter the value. If the page were write-protected, it would be a test against always-write (cmpxchg does the same).
That would be a good test. I broke QEMU many years ago like that.

My emulator reached the arithlogic tests on the first attempt. I'm happy about that.

hottobar wrote:

peterferrie wrote:

This tester needs some of the corner cases. I shall make a PR for them. 😀

This would be awesome!
There are a lot of tests still missing...

I opened an issue rather than forking the code because I haven't spent the time yet to understand how to add new tests by myself.
I hope that's acceptable.
It might become necessary at some point to know which CPU family is being targeted. Consider the prefetch-queue flush in the Pentium, or the rep string-overwrite flush in the Pentium 4.
These were major changes that broke a set of anti-debugging code at the time.
I'm assuming that the tester is targeting a 486 at most (considering the SETxx tests). Is that so?

I've managed to fix those bugs. Now it's continuing to the 0xEE tests, which fails somewhere in there(quite a big block of code to find a single bug?). A direct COM/LPT to log output isn't supported, but the Bochs 0xE9 hack port is available? The E9 port+EA port extensions can be used to control some emulator functionality(like changing the log filename). The enable string to send can be read from port EA, with 0xFF as a NULL-string ending. Sending it to port E9 with a line break enables port 0xEA until 0xEA receives a termination command.

More information about this E9 hack extension can be found at: https://bitbucket.org/superfury/unipcemu/wiki/Debugging

I wrote a little Pascal(Turbo Pascal 6.0) unit to interface with the protocol and port: https://bitbucket.org/superfury/unipcemu/src/ … U.PAS?at=master

A little program that enables 640x480x256 and draws with enabled VGA logging through the E9 hack extension(using the above unit): https://bitbucket.org/superfury/unipcemu/src/ … S.PAS?at=master

peterferrie wrote:

Note that ARPL won't write to memory unless it has to alter the value. If the page were write-protected, it would be a test against always-write (cmpxchg does the same).
That would be a good test. I broke QEMU many years ago like that.

I'll add this to the tests!

peterferrie wrote:

I opened an issue rather than forking the code because I haven't spent the time yet to understand how to add new tests by myself.
I hope that's acceptable.

Sure, everything helps. I'll put your suggestions into code.

peterferrie wrote:

It might become necessary at some point to know which CPU family is being targeted. Consider the prefetch-queue flush in the Pentium, or the rep string-overwrite flush in the Pentium 4.
These were major changes that broke a set of anti-debugging code at the time.
I'm assuming that the tester is targeting a 486 at most (considering the SETxx tests). Is that so?

Yes, 486 at most, even though it does not tests any 486 specific instruction at the moment.

So perhaps an option for 486-specific tests, defaulting to 386 compatibility?

superfury wrote:

Bochs 0xE9 hack port
More information about this E9 hack extension can be found at: https://bitbucket.org/superfury/unipcemu/wiki/Debugging

Interesting. I'll definitely consider this.

superfury wrote:

So perhaps an option for 486-specific tests, defaulting to 386 compatibility?

Yes, a configurable equ to enable CPU specific tests.

One thing to note about UniPCemu's port E9 hack: Bochs seems to print to it's console every single byte written to that port, while UniPCemu only logs a line after it receives any line ending character(either CR, LF or CRLF or LFCR. Repeating any of those causes multiple newlines accordingly, independant on the type of newline proceeding it. So CRLFLF or LFCRCR, CRCR and LFLF will count as two newlines. It allows for easy cross-platform compatibility.).

1OPTINLINE static void debugger_writecharacter(byte c) //Write a character to the debugger!
2{
3	if (__HW_DISABLED) return; //Abort!
4	if ((c=='\n') || (c=='\r')) //Newline character?
5	{
6		//we count \n, \r, \n\r and \r\n as the same: newline!
7		if (!softdebugger.newline) //First newline character?
8		{
9			debugger_flush(); //Flush!
10			softdebugger.newline = c; //Detect for further newlines!
11		}
12		else //Second newline+?
13		{
14			if (softdebugger.newline==c) //Same newline as before?
15			{
16				debugger_flush(); //Flush!
17				//Continue counting newlines!
18			}
19			else //No newline, clear the newline flag!
20			{
21				softdebugger.newline = 0; //Not a newline anymore!
22			}
23		}
24	}
25	else //Normal character?
26	{
27		sprintf(softdebugger.writtendata,"%s%c",softdebugger.writtendata,c); //Add to the debugged data!
28	}
29}

Of course, the remaining line(if any) is flushed when the emulator terminates by closing it.

The line-based flushing to the specified/default log file is a bit different from the Bochs E9 hack, though: the Bochs E9 hack seems to write every and any character to it's output, so it doesn't care about line endings afaik. So writing "ABCD" on Bochs might log immediately, while UniPCemu only logs when it receives a line break character in either \n\r, \r\n, \r or \n format(which it autodetects using the above algorithm). Also, empty lines aren't logged(they're ignored).

This is POST EE currently executing(hanging?), possible invalid code:

Can anyone see what's going wrong? I can't seem to find the executed code anywhere in the asm files?

Some simple Port E9 hack support(untested, but compiling with current masm):

1	
2	; Port E9 hack support, with autodetection
3	push   eax ;Backup for autodetection
4	in     al, 0xE9
5	cmp    al, 0xE9
6	jnz    .noE9hack
7	pop eax ; Restore register
8	out    0xE9, al
9	jmp    .finishE9hack
10	.noE9hack:
11	pop    eax ; Restore register
12	.finishE9hack:

Insert at line 40 of print_p.asm.

Edit: Tested and logging properly.

superfury wrote:

This is POST EE currently executing(hanging?), possible invalid code:

debugger_UniPCemu_20171022_1757.zip

Can anyone see what's going wrong? I can't seem to find the executed code anywhere in the asm files?

Can't say by reading the log.
Can you post your lst file and the ROM output?

superfury wrote:

I can't seem to find the executed code anywhere in the asm files?

The lst file is your helping hand here. In there you will see the address and instruction combos so you can match that with your log and see where you started executing in the woods.

Just pulled the latest changes: the lea tests(post E) seems to fail now? It didn't crash on the last build(ran within 0xEE before the latest commits)?

Looking at the debugger, the CPU seems to be idling during a JNZ rel32 instruction. Why? Because the BIU and EU are infinitely waiting for each other: The BIU Prefetch waits for the EU to load the TLB entry for the current byte address to continue fetching. The EU on the other hand, refuses to read the Prefetch and check protection rights(and load the TLB while it's at it) until it has enough bytes(4 to be exact) loaded in the prefetch buffer. So both start idle spinning, waiting for each other. That happens at EIP=0x1000.

Edit: Luckily, the solution is simple: check the entire word or dword to be gotten from the PIQ beforehand. That will fault properly and prevent the BIU/EU from locking each other up. Now it once again resumes to POST EEh.

Looking at the 0xEE output, the last line starts with "IMUL32 D EAX=FFFFFFFF EDX=FFFFFFFF PS=0000", which only appears once in the reference file. So the problem happens after returning from logging, but before the "DIVDL B EAX=00000000 EDX=00000000 PS=0000 #DE EAX=00000000 EDX=00000000 PS=0000" line, according to the reference(also, the imuld instruction seems to have output problems?)?

Edit: So, the problem is at EIP 2BF5+?

Edit: Also, a little optimization of the E9 hack code:

1	; Port E9 hack support, with autodetection
2	push   eax ;Backup for autodetection
3	in     al, 0xE9
4	cmp    al, 0xE9
5	pop    eax ; Restore register
6	jnz    .noE9hack
7	out    0xE9, al
8	.noE9hack:

superfury wrote:

Looking at the 0xEE output, the last line starts with "IMUL32 D EAX=FFFFFFFF EDX=FFFFFFFF PS=0000", which only appears once in the reference file. So the problem happens after returning from logging, but before the "DIVDL B EAX=00000000 EDX=00000000 PS=0000 #DE EAX=00000000 EDX=00000000 PS=0000" line, according to the reference(also, the imuld instruction seems to have output problems?)?

Maybe a problem with the handling of the #DE exception?

Strange: I only see UniPCemu emitting one 32-bit DIV instruction, which isn't overflowing or divide by 0? 0x245af920e27415a5 / 0x88776655 = 0x44332211 remainder 0.

After the final IMUL32 D entry, it ends up at a interrupt-disabled HLT instruction? No DIV instruction is executed at all.

To generate your EE list format from the portE9.log, use:

1sed -i 's/^[0-9]\+:[0-9]\+:[0-9]\+:[0-9]\+\.[0-9]\+: //g' porte9.log

superfury wrote:

Strange: I only see UniPCemu emitting one 32-bit DIV instruction, which isn't overflowing or divide by 0? 0x245af920e27415a5 / 0 […]
Show full quote

Strange: I only see UniPCemu emitting one 32-bit DIV instruction, which isn't overflowing or divide by 0? 0x245af920e27415a5 / 0x88776655 = 0x44332211 remainder 0.

After the final IMUL32 D entry, it ends up at a interrupt-disabled HLT instruction? No DIV instruction is executed at all.

test386_executing_50171023_0204.7z

To generate your EE list format from the portE9.log, use:

Copy code to clipboard

1sed -i 's/^[0-9]\+:[0-9]\+:[0-9]\+:[0-9]\+\.[0-9]\+: //g' porte9.log

You really should include the LST file with the logs, otherwise debugging is difficult.
Anyway, the retd at line 16111046 transfer the execution back at start (F000:0000) for an unknown reason, and then there's a #GP shortly after. The #GP handler then correctly halts the CPU.
I'd start investigating on why the CPU is reset at line 16111046.

The list file is unchanging, though(only if source code is changed in the testsuite, which is only the base constants and the additional logging code):

The divide-by-zero test is causing some stack problem for you.

0010:0000C20C C7 04 24 14 C2 00 00 mov dword ss:[esp],0000c214
0010:0000C213 CF iretd
0010:0000C214 C3 retd
0010:00000000 74 65 je 00000067

The retd is returning to the wrong location. The expected return address is 0010:00002941.