VOGONS


Common log format

Topic actions

Reply 60 of 90, by superfury

User metadata
Rank l33t++
Rank
l33t++

Yes, it's the address that goes through Paging translation unit to RAM/hardware after all(linear->physical translation layer logging). Currently Paged accesses are usually prefixed(reads) or suffixed(writes) with the RAM layer(or none when accessing hardware, since hardware doesn't log anything, they just process and result without logging atm, due to various hardware(VGA, BIOS flash ROM) responding to it). I/O port(BUS accesses) aren't logged yet.

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 61 of 90, by vladstamate

User metadata
Rank Oldbie
Rank
Oldbie

Questions about debug registers. You display DR0,1,2,3 then 6 followed by 5&7.

The only valid registers are 0 through 3 and then 6 and 7. Can we display them like this instead:

DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
DR6: 00000000 DR7: 00000000

In other words drop 5&.

or we could display them all (are 4 and 5 used on 486+ ?)

DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
DR4: xxxxxxxx DR5: xxxxxxxx DR6: 00000000 DR7: 00000000

YouTube channel: https://www.youtube.com/channel/UC7HbC_nq8t1S9l7qGYL0mTA
Collection: http://www.digiloguemuseum.com/index.html
Emulator: https://sites.google.com/site/capex86/
Raytracer: https://sites.google.com/site/opaqueraytracer/

Reply 62 of 90, by superfury

User metadata
Rank l33t++
Rank
l33t++

Well, that's the odd one: registers 5&7 map to the same register(DR7), or fault depending on DE in CR4. DR4 doesn't exist depending on DE bit and CPU?

http://www.felixcloutier.com/x86/MOV-2.html
https://stackoverflow.com/questions/21901742/ … r5-are-not-used

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 63 of 90, by vladstamate

User metadata
Rank Oldbie
Rank
Oldbie

Well, CR4 only "exists" in 486+.

It goes like this:

386:

CR0-CR3 - OK
CR4-CR7 - #UD
DR0-DR3 and DR6 DR7 - OK
DR 4 and 5 - #UD

486:

CR0-CR4 - OK
CR5-CR7 - #UD
DR0-DR3 and DR6 DR7 - OK
DR 4 and 5 - #UD unless CR4 has DE bit disabled in which case they are DR6 and DR7

So maybe we should display 8 CR and 8 DR registers and have xxxxxxxx for anything reserved and actual value if they are used or have alias.

YouTube channel: https://www.youtube.com/channel/UC7HbC_nq8t1S9l7qGYL0mTA
Collection: http://www.digiloguemuseum.com/index.html
Emulator: https://sites.google.com/site/capex86/
Raytracer: https://sites.google.com/site/opaqueraytracer/

Reply 64 of 90, by superfury

User metadata
Rank l33t++
Rank
l33t++

It seems to be almost correct and matching UniPCemu's behaviour, looking at my ModR/M decoder: CR1 doesn't exist in any CPU(At least until Pentium)?

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 65 of 90, by vladstamate

User metadata
Rank Oldbie
Rank
Oldbie

Ok, well in that case how about we do not display what is not present (stuff that generates #UD). I can change my code too

So for 386 display:

CR0, CR2, CR3
DR0, DR1, DR2, DR3, DR6, DR7

and for 486 display:

CR0, CR2, CR3, CR4
DR0, DR1, DR2, DR3, DR4, DR5, DR6, DR7

Now we need DR4 and 5 because they do have contents. I'll try to find 486 documents, it is possible back then they were separate but in recent architecture they just alias.

YouTube channel: https://www.youtube.com/channel/UC7HbC_nq8t1S9l7qGYL0mTA
Collection: http://www.digiloguemuseum.com/index.html
Emulator: https://sites.google.com/site/capex86/
Raytracer: https://sites.google.com/site/opaqueraytracer/

Reply 66 of 90, by superfury

User metadata
Rank l33t++
Rank
l33t++

Don't DR4&5 both either redirect to DR6&7 or both fault, depending on DE in CR4(Pentium+). DE set makes it fault, while cleared gives 80386-compatible redirection. Essentially always 'cleared' on pre-Pentium, so DR4&5 don't exist anyway(no way to actually access them due to #UD). So it's just CR0, CR2, CR3, DR0-3, DR6 and DR7. The other two(DR4&5) are untracable without die analysis(since it's unaccessable through instructions). Does it make sense to log an register that might not even exist, uncheckable if it does and unreachable by any instruction?

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 67 of 90, by vladstamate

User metadata
Rank Oldbie
Rank
Oldbie
superfury wrote:

Does it make sense to log an register that might not even exist, uncheckable if it does and unreachable by any instruction?

You are right. Lets drop DR4 and DR5. However CR4 does exist from 486 upwards, not from Pentium upwards.

YouTube channel: https://www.youtube.com/channel/UC7HbC_nq8t1S9l7qGYL0mTA
Collection: http://www.digiloguemuseum.com/index.html
Emulator: https://sites.google.com/site/capex86/
Raytracer: https://sites.google.com/site/opaqueraytracer/

Reply 68 of 90, by superfury

User metadata
Rank l33t++
Rank
l33t++

CR4 seems to exist on some later 80486 models(e.g. 80486-DX2) with PSE. As far as I can see, the plain 80486(SX/DX) doesn't have it? I'm emulating a plain 80486SX/DX, so no CR4 exists?

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 69 of 90, by vladstamate

User metadata
Rank Oldbie
Rank
Oldbie

I am putting all this in writing, so that we have the specification somewhere:

https://sites.google.com/site/capex86/common

Feel free to comment if I made any mistakes.

YouTube channel: https://www.youtube.com/channel/UC7HbC_nq8t1S9l7qGYL0mTA
Collection: http://www.digiloguemuseum.com/index.html
Emulator: https://sites.google.com/site/capex86/
Raytracer: https://sites.google.com/site/opaqueraytracer/

Reply 70 of 90, by superfury

User metadata
Rank l33t++
Rank
l33t++

I've modified the logging as follows now:
CR5-7 are removed from logging, only CR4 is left on that line(only with 80486+).
DR4-5 are ommitted from logging, only DR6 and DR7 are logged(no 5&7 anymore).

I see that on your website the DR5&7 is still mentioned(which should be DR7 only now).

I still haven't had any response on the matter of memory transactions? Any idea about it? I've already given two transaction methods I've implemented in UniPCemu(the latter one is currently active in my current commits), any input on how to log those?

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 71 of 90, by vladstamate

User metadata
Rank Oldbie
Rank
Oldbie
superfury wrote:

I've modified the logging as follows now:
CR5-7 are removed from logging, only CR4 is left on that line(only with 80486+).
DR4-5 are ommitted from logging, only DR6 and DR7 are logged(no 5&7 anymore).

Perfect!

superfury wrote:

I see that on your website the DR5&7 is still mentioned(which should be DR7 only now).

Nice spotted. I fixed it now.

superfury wrote:

I still haven't had any response on the matter of memory transactions? Any idea about it? I've already given two transaction methods I've implemented in UniPCemu(the latter one is currently active in my current commits), any input on how to log those?

Yes, give me some time to think of it. I've been busy implementing the register log and fix some 386 issues. I know it is important we figure this out as I've already reserved space on the webpage/doc for it, as you saw.

YouTube channel: https://www.youtube.com/channel/UC7HbC_nq8t1S9l7qGYL0mTA
Collection: http://www.digiloguemuseum.com/index.html
Emulator: https://sites.google.com/site/capex86/
Raytracer: https://sites.google.com/site/opaqueraytracer/

Reply 72 of 90, by vladstamate

User metadata
Rank Oldbie
Rank
Oldbie

BTW I put the spec on the OP, so it is easier to see.

YouTube channel: https://www.youtube.com/channel/UC7HbC_nq8t1S9l7qGYL0mTA
Collection: http://www.digiloguemuseum.com/index.html
Emulator: https://sites.google.com/site/capex86/
Raytracer: https://sites.google.com/site/opaqueraytracer/

Reply 73 of 90, by superfury

User metadata
Rank l33t++
Rank
l33t++

You still forgot to update one thing: the DR5&7 on the full log example, which is supposed to be DR7.

Edit: We also still need to do something about the memory access logging, as it's essential to things like instructions that have memory operands(their input and output has effect on execution after all, in the same way the registers do(although it may be at different times, as registers are more frequently used due to speed)).

Any ideas about logging those? I think the latter format is better, as it's more compact(due to shorter notation and single lines for each memory access) while still logging every data required.

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 74 of 90, by superfury

User metadata
Rank l33t++
Rank
l33t++

One little thing to note: the memory accesses are logged as a tab after the executing cycle: this can be either between instructions(in the common log format) as well as after the instruction line itself(following the instruction disassembly(e.g. "call 00009190<tab>memoryaccess")).

For example(from the BIOS log failing to boot a floppy disk):

f000:0000909e E8 EF 00 call 00009190	Physical(r):000F9190=33(3); Paged(r):000F9190=33(3); Physical(r):000F9191=DB(Û); Paged(r):000F9191=DB(Û); Physical(r):000F9192=8A(Š); Paged(r):000F9192=8A(Š); Physical(r):000F9193=5E(^); Paged(r):000F9193=5E(^)

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 75 of 90, by vladstamate

User metadata
Rank Oldbie
Rank
Oldbie
superfury wrote:
[…]
Show full quote
f000:0000909e E8 EF 00 call 00009190	Physical(r):000F9190=33(3); Paged(r):000F9190=33(3); Physical(r):000F9191=DB(Û); Paged(r):000F9191=DB(Û); Physical(r):000F9192=8A(Š); Paged(r):000F9192=8A(Š); Physical(r):000F9193=5E(^); Paged(r):000F9193=5E(^)

Questions:

1) Does (r) mean a byte read?
2) What do you print for non-ASCII characters? Like those with code < 32 ?
3) (not important to the format but just curious): who issues those reads? The call instruction? CALL should issue writes to the stack, where are they?

YouTube channel: https://www.youtube.com/channel/UC7HbC_nq8t1S9l7qGYL0mTA
Collection: http://www.digiloguemuseum.com/index.html
Emulator: https://sites.google.com/site/capex86/
Raytracer: https://sites.google.com/site/opaqueraytracer/

Reply 76 of 90, by superfury

User metadata
Rank l33t++
Rank
l33t++
vladstamate wrote:
superfury wrote:
[…]
Show full quote
f000:0000909e E8 EF 00 call 00009190	Physical(r):000F9190=33(3); Paged(r):000F9190=33(3); Physical(r):000F9191=DB(Û); Paged(r):000F9191=DB(Û); Physical(r):000F9192=8A(Š); Paged(r):000F9192=8A(Š); Physical(r):000F9193=5E(^); Paged(r):000F9193=5E(^)

Questions:

1) Does (r) mean a byte read? Yes. Otherwise it logs (w) for writes.
2) What do you print for non-ASCII characters? Like those with code < 32 ? NULL, 0xD and 0xA are patched to space to prevent invalid logging. All others are logged as-is(%c).
3) (not important to the format but just curious): who issues those reads? The call instruction? CALL should issue writes to the stack, where are they? All logged reads/writes are by the CPU. So it's the executing instruction as well as Paging table lookups, descriptor fetches/writeback(e.g. busy task bit) and prefetching unit fetches. The previous reads/writes are on the previous lines. The access after the instruction is the final memory access by the instruction on that cycle. Although, usually this isn't with the instruction, as the BIU finishes the access BEFORE the instruction is finished(executed stays 1, prompting the debugger to start logging initial register state etc. that very cycle. So it usually only happens on that cycle when it's an access that's not emulated through the BIU(like a memory access without using the BIU, like loading/storing descriptors and Paging-related tables atm.)

In this case it's probably a prefetching PIQ dword being fetched.

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 77 of 90, by superfury

User metadata
Rank l33t++
Rank
l33t++

I've been thinking, isn't prefetch a special kind of read(it reads into the prefetch instead of normal transactions, which read for the application to process, instead of opcodes)? Shall I modify the (r) to become (p) for those fetches? That way, you can easily see the normal transactions from the prefetch ones?

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 78 of 90, by vladstamate

User metadata
Rank Oldbie
Rank
Oldbie

Prefetch is asynchronous with the instruction execution though. I would not log them into anything related to an instruction.

YouTube channel: https://www.youtube.com/channel/UC7HbC_nq8t1S9l7qGYL0mTA
Collection: http://www.digiloguemuseum.com/index.html
Emulator: https://sites.google.com/site/capex86/
Raytracer: https://sites.google.com/site/opaqueraytracer/

Reply 79 of 90, by superfury

User metadata
Rank l33t++
Rank
l33t++

But they are related to the instruction? It's the logging of BIU fetches after all in both the (r) and (p) cases, just into a different buffer(either buffered for the EU to read as data(e.g. data from a memory operand, like in mov ax,[1324]) or the opcode bytes(which are added to the prefetch buffer instead)). Both are BIU reads that are transferred at cycle T3-T4. The only difference is the destination?

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io