First post, by earlz
Hi, so I have much more specialized VM than most projects here. Basically, we're making an embedded VM for use to run untrusted code in a specialized environment. The end result of this is that we don't need a full PC emulator. No hardware to emulate (since the environment is incredibly restricted), and also no ring0, gdt, idt, etc. We can make very broad assumptions, like a flat address space, no segment register usage, no "kernel" code that uses TSS and other system features.. oh, and also it's always in 32-bit mode. No need to worry about real mode (though we still implement address/operand size prefixes of course). Basically the intent is that you can compile a normalish program in C or whatever, and run it in this VM and get some output
So, the cool thing about this is that there are a ton of complex things we get to skip out on. However, the bad thing about this is that we can't just pop in Windows 95 and ensure that the opcodes are correct by it not crashing... Right now we are almost up to par with 386 support, with our eventual goal being 686 (though potentially skipping out on FPU support).
However, while making a toolchain for this custom environment we ran into several bugs... Fixed the easy ones, and now sometime after ~200 opcodes are executed it goes into bad logic and jumps to an invalid address. The invalid address comes from a load from the stack that is set significantly earlier.... so it's not trivial to figure out why the invalid address was loaded..
So, we're looking to basically implement a test suite of sorts that validates that the opcodes are implemented correctly. We just now designed a test suite template system that makes this somewhat easy.. but testing every single instruction in the i386 ISA is no weekend project, even when excluding the ring 0 opcodes.
I found this forum while googling for some issues around the AF flag, and figured I would ask this question here since there is a lot of interesting discussion happening (and this is literally the only x86 emulator community I've found) . Is there a test suite that is useable for testing single instructions without needing to execute some stub OS that our VM would not support? And what is the best route for detecting bugs in the opcode implementations other than making sure to have some automated testing? In addition, ensuring operation is correct is paramount. Once this is deployed, it is incredibly difficult to fix bugs for various reasons.
VM source code: https://github.com/qtumproject/x86lib
Example test code: https://github.com/qtumproject/x86lib/blob/ma … s/mov_tests.cpp