It's probably an FPU detection routine.
Even if a system has no FPU, the FPU instructions will still be decoded by the main CPU. So your emulator should accept them and interpet them as NOPs.
(The way to detect an FPU is usually something like executing fnstsw [sp] and then checking with the CPU to see if the FPU actually did something, or if [sp] was unaltered).

Currently all FPU opcodes raise an exception, except for FNINIT, FNSTSW and FNSTCW:

1void FPU8087_OPDBE3(){debugger_setcommand("<UNKOP8087: FNINIT>");}
2
3void FPU8087_OPDB()
4{byte subOP = CPU_readOP(); CPUPROT1 word oldCS = REG_CS; word oldIP = REG_IP; if (subOP==0xE3){FPU8087_OPDBE3();} else{REG_CS = oldCS; REG_IP = oldIP; FPU8087_noCOOP();} CPUPROT2 }
5void FPU8087_OPDFE0(){debugger_setcommand("<UNKOP8087: FNINIT>");}
6void FPU8087_OPDF(){CPUPROT1 byte subOP = CPU_readOP(); CPUPROT1 word oldCS = REG_CS; word oldIP = REG_IP; if (subOP==0xE0){FPU8087_OPDFE0();} else {REG_CS = oldCS; REG_IP = oldIP; FPU8087_noCOOP();} CPUPROT2 CPUPROT2 }
7void FPU8087_OPDDslash7(){debugger_setcommand("<UNKOP8087: FNSTSW>");}
8void FPU8087_OPDD(){word oldCS; word oldIP; oldCS = REG_CS; oldIP = REG_IP; modrm_readparams(&params,1,0); CPUPROT1 if (MODRM_REG(params.modrm)==7){FPU8087_OPDDslash7();}else {REG_CS = oldCS; REG_IP = oldIP; FPU8087_noCOOP();} CPUPROT2}
9void FPU8087_OPD9slash7(){debugger_setcommand("<UNKOP8087: FNSTCW>");}
10void FPU8087_OPD9(){word oldCS; word oldIP; oldCS = REG_CS; oldIP = REG_IP; modrm_readparams(&params,1,0); CPUPROT1 if (MODRM_REG(params.modrm)==7){FPU8087_OPD9slash7();} else {REG_CS = oldCS; REG_IP = oldIP; FPU8087_noCOOP();} CPUPROT2}
11void FPU8087_noCOOP(){debugger_setcommand("<No COprocessor OPcodes implemented!>"); CPU_resetOP(); CPU_COOP_notavailable();}

FPU8087_noCOOP throws exception #7 (also executed for opcodes D8, DA, DC and DE).

A real 80186 controls whether int 7 is triggered when an FPU opcode is used by setting bit 15 of the relocation register. Your CPU emulation is lacking all of the 80186 peripherals (which is fine unless you really want to emulate machines that have an 80186 none of which are PC compatible) so it's more like a V20/V30 which lacks any FPU not present/int 7 support like the 8086. Basically only 286+ should do that when the appropriate flags are set in the MSW.

Do programs compiled with TP6 run?
IIRC TP6 has an option to generate code for 8087 or emulation.
The actual code generated for floating point operations consists of some specific interrupt instructions. When emulation is chosen, the interrupt routine calls the emulator. When 8087 is chosen, the interrupt routine will modify the interrupt instruction to the appropriate 8087 instruction (and possible a NOP) so the interrupt routine only needs to be called one time.
I don't know if any of this applies to your problem, though.

It tries to execute the coop instruction without FPU and with only the BIOS int 7 handler installed(which simply confirms as done to the PIC and IRETs back to the faulting instruction, causing an exception in a infinite loop).

This happens when running TP6's TURBO.EXE.

Int 7 is a 286+ thing. Do not trigger it for 8086-class systems (there are no PC-compatible 80186-systems because the CPU's built-in features simply aren't 100% compatible with PC/XT hardware).
An 8088/8086 will silently ignore FPU instructions if no FPU is present. Which is why a compiler will explicitly emit int-calls to invoke an emulator before FPU opcodes. You can't transparently emulate an FPU on these systems. On 286+ it can be done via the int 7 handler.
As I already said: decode them and treat them as nops. Also, RTFM 😀
Here is some background info on the difference between 8087 and 80287:
https://web.archive.org/web/20150503181815/ht … handling_on_x87

Turbo Pascal 6.0 now runs. It seems to have had something to do with incomplete Lo-fi 2MB EMS board emulation causing some trouble. It even compiles simple files (most examples complain about missing .TPU files when simply compiling).

Scali wrote:

An 8088/8086 will silently ignore FPU instructions if no FPU is present. Which is why a compiler will explicitly emit int-calls to invoke an emulator before FPU opcodes. You can't transparently emulate an FPU on these systems. On 286+ it can be done via the int 7 handler.

So that's why! I was just reporting my experience from analyzing the code TP produced, I found it somewhat strange at the time. My guess at the time was that the INT instruction was executed faster than a trap instruction for an unavailable coprocessor would have been. The funny thing is that the generated code stayed the same even if I choose the settings for 8087 code generation and no emulation. In that case the compiler merely left out the full emulation library. I suppose that made it easier for the compiler developers...

idspispopd wrote:

The funny thing is that the generated code stayed the same even if I choose the settings for 8087 code generation and no emulation. In that case the compiler merely left out the full emulation library. I suppose that made it easier for the compiler developers...

Yes I noticed that too. Perhaps they deliberately did that so that you could still load an emulator TSR and run the code. This way the emulator would not have to be compiled into every single program.
The interesting part is that the debugger will disassemble the int-calls silently (just like the fwait instructions), so you have to look at the opcode bytes to see how the FPU instructions are actually encoded.
I noticed when stepping through some code, that the FPU instructions were really long. But that was because they first had two bytes of int prefixed, then a third byte for the fwait, and finally the actual FPU instruction.