It crashes in the dynamic linker helper that's why it works the second time. I found a nice page describing the process: http://timetobleed.com/dynamic-linking-elf-vs-mach-o

When the PLT already contains the cos() address (when it was loaded with a previous run with normal core) there is no problem. If the library is not loaded, then the dyld_stub_binder() is called. It crashes at the first xmm0 instruction so I assume the error is that the stack is not 16-bit aligned at that point.

Why is 0x08 added to rsp in gen_call_function_setup()?

gen_call_function_setup() is not called for cos only gen_call_function_raw(). So the stack is never aligned?

Replaced gen_call_function_raw() with gen_call_function_setup() and it works the first time as well, so it is the stack alignment 😎. Although it is quite a lot slower...I guess the real solution would be to see which calls really need aligned stack and fix only those 😁

Will have a look at that, possibly only floating point stuff needs alignment which may hit that function (and the _raw is too shortcut for that...)

Thanks for figuring that out 😀

Could try that:

1static void INLINE gen_call_function_raw(void * func) {
2	cache_addb(0x48);
3	cache_addw(0xec83);
4	cache_addb(0x08);	// sub rsp,0x08
5
6	cache_addb(0x48);
7	cache_addb(0xb8);	// mov reg,imm64
8	cache_addq((Bit64u)func);
9	cache_addw(0xd0ff);
10
11	cache_addb(0x48);
12	cache_addw(0xc483);
13	cache_addb(0x08);	// add rsp,0x08
14}

I have already returned my OS X testing platform, perhaps Dominus can test these changes.

Ok, tested it just now.
I assumed the changes by wd need to go on top of the diff by gulikoza (and at risc_x64.h line 336).
With this core dynamic still crashes at the PCPbench. Also crashes if the Pcpbench first ran in normal core and I then switched to dynamic.

Same location of the crash (due to cos function arguments not aligned)?

If you can trace the generated code, you could check if i've messed the encoding...

Don't know how to trace this. Sorry.

I was able to steal the macbook for the weekend again 😁

With wd's changes, PCPBENCH works fine here. Dominus, are you sure you changed the gen_call_function_raw() (as that one is at line 378 here (although yes, my risc_x64.h is probably modified))? The gen_call_function_setup() should be left as it is (line 343 here). And yes, these changes should go after applying my patch from the previous page.

edit: nvm, I probably moved gen_call_function_raw, but still it should work. Let me try again with clean risc_x64.h

Yes, please let me know which changes are correct. I'm getting confused 😉

Ok...checked out risc_x64 from svn again. Touched the file (so the modifications get picked up by make). Ran dosbox - crash before prompt. Applied my patch. Crash just after running PCPBENCH. Pasted wd's code on line 336. PCPBENCH works. 😀

There's another problem of course 😠
I had CreateCacheBlock set to 1...with default 32, PCPBENCH no longer works...

Ok so your patch (fixed memory addressing) + the stack adjustment i've
posted + small cache blocks makes this work? Nice.

Does it work with cache block entries:=32 and the _raw replaced by the other
function setup routine? The logic behind my small change was just to align
the stack to 16byte boundaries always since that seems an ABI requirement,
but i'll check where we call that (though it doesn't sound like it'd be related
to the max cache block entries).

Do you know where this code comes from?

10x11e04cd8c:	and    rsp,0xfffffffffffffff0
20x11e04cd90:	add    rsp,0x8
30x11e04cd94:	push   rax
40x11e04cd95:	mov    rax,0x10000c340
50x11e04cd9f:	call   rax
60x11e04cda1:	pop    rsp
70x11e04cda2:	or     al,al
80x11e04cda4:	jne    0x11e04d0b2
90x11e04cdaa:	mov    di,0x44a
100x11e04cdae:	mov    WORD PTR [rip-0x1dde28a9],di        # 0x10026a50c <cpu_regs+12>
110x11e04cdb5:	mov    di,WORD PTR [rip-0x1dde28b0]        # 0x10026a50c <cpu_regs+12>
120x11e04cdbc:	mov    si,0xf
130x11e04cdc0:	mov    eax,edi
140x11e04cdc2:	add    eax,esi
150x11e04cdc4:	jmp    0x11e04cdcc
160x11e04cdc6:	nop    
170x11e04cdc7:	nop    
180x11e04cdc8:	nop    
190x11e04cdc9:	nop    
200x11e04cdca:	nop    
210x11e04cdcb:	nop    
220x11e04cdcc:	add    BYTE PTR [rax],al
230x11e04cdce:	call   rax
240x11e04cdd0:	add    rsp,0x8
250x11e04cdd4:	mov    WORD PTR [rip-0x1dde28cf],ax        # 0x10026a50c <cpu_regs+12>
260x11e04cddb:	mov    di,WORD PTR [rip-0x1dde28d6]        # 0x10026a50c <cpu_regs+12>
270x11e04cde2:	mov    si,0xf0
280x11e04cde6:	mov    eax,edi
290x11e04cde8:	and    eax,esi

It crashes at 0x11e04cdcc. 0x11e04cdce looks like the second part of the gen_call_function_raw(), but add BYTE PTR [rax], al?? Memory at that address is:

1(gdb) x/24xb 0x11e04cdc4
20x11e04cdc4:	0xeb	0x06	0x90	0x90	0x90	0x90	0x90	0x90
30x11e04cdcc:	0x00	0x00	0xff	0xd0	0x48	0x83	0xc4	0x08
40x11e04cdd4:	0x66	0x89	0x05	0x31	0xd7	0x21	0xe2	0x66

The last translated opcode (if my logging is correct) is 0x75.

Thanks, so the cache blocks is the difference why it crashed for me...

I guess those NOPs come from gen_fill_function_ptr() but they no longer correctly overwrite the function call. Disabling DRC_FLAGS_INVALIDATION fixes the crash.

I guess those NOPs come from gen_fill_function_ptr() but they no longer correctly overwrite the function call.

Wow, yeah, sorry, had a hard time remembering how advanced that thing is 😉

We have 8 bytes in addition now, so the blocks in gen_fill_function_ptr look like

1			*(Bit32u*)(pos+0)=0xf001f889;	// mov eax,edi; add eax,esi
2			*(Bit32u*)(pos+4)=0x90900eeb;	// skip
3			*(Bit32u*)(pos+8)=0x90909090;
4			*(Bit32u*)(pos+12)=0x90909090;
5			*(Bit32u*)(pos+16)=0x90909090;

I'll see about merging these things into current sources unless somebody is keen on doing that...

There's a problem with default case (pos+2) I guess. This becomes pos+6 when _raw is called, but remains (?) pos+2 when _setup is called. I'm not sure I can find all cases when pos+6 is required. I've added -4 to cache.pos in _setup in the following patch. Feel free to change that (or put something in the comment) 😀

The following patch has all the changes merged and seems to work (MAC OSX at least, I'll try to test linux later as well)

Actually no idea when the default is used at all, i'll check that, thanks.