VOGONS


Getting UniPCemu up and running

Topic actions

Reply 20 of 29, by superfury

User metadata
Rank l33t
Rank
l33t
mr.cat wrote on 2021-03-25, 15:56:
Sounds good :D My "64-bit black screen problem" can be considered solved. I just tried building UniPCemu on a different host and […]
Show full quote

Sounds good 😁
My "64-bit black screen problem" can be considered solved. I just tried building UniPCemu on a different host and it works there just fine.
So I'll have to do some comparison to see what that was about. Could be hw related (Intel vs. AMD etc.), but we'll see.
I'll report back here, if I manage to find out the root cause.

Btw new Tseng card coming up 😁
EXCELGRAPH - ISA Video Display Controller (ET4000/W32i)

Maybe ALEKS has some hints about test software for you?

Hw related depends. When you enable the cpu speed percentage display with the non-working hardware case (setting the setting to 1 and starting the emulator), does the cpu percentage display itself in the top-left corner during the 'black screen'? If so, output is working and the app is malfunctioning at some emulated hardware(CPU, video card etc.).
If the entire app display (save for the title bar and borders) is black, it's indeed either a bug in SDL2 or the app itself. It should always start with displaying some yellow text("Press SELECT to run BIOS SETUP") in the top left corner when it starts. If it doesn't do that, but the normal text at the bottom right displays, it's an emulator startup bug (it's started by the first 10-ish instructions ran on the emulated CPU, using a emulator escape instruction sequence(using some specific port OUT instructions to a specific I/O port)). If that's failing, it will simply run and do nothing(no valid executable code is executing). In that case you'll only see the buttons and keyboard LED display in the bottom right of the window.

If it's actually lighting the floppy LED(A or B), there's probably an emulator hardware(display) or CPU emulation issue at fault.
Pure EGA emulation is still known to be faulty(I don't know yet what's wrong with it atm).
It could also mean a wrong/faulty ROM or missing ROM with the last case (booting but no display).

The very code that displays the yellow text also loads the ROMs after it disappears and resets all emulator components to start using them in a clean system(both CPU and hardware).

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows and PSP on itch.io

Reply 21 of 29, by mr.cat

User metadata
Rank Member
Rank
Member

Interesting! Thx for the explanation. It's some kind of bug *somewhere*, but the hunt is still on...

The CPU speed indicator (and the startup yellow text, and the configuration menus) do all work.
They are visible in the screenshot (in that previous post).

The reason why it seems hw related to me is that the whole UniPCemu directory was simply copied to another host, and it works there.
That includes the UniPCemu binary and all the related files such as configuration and ROMs. So the problem is definitely related to the host setup somehow.
The hosts are both using the same distro with the same kernel/gcc/library versions. There is some difference in packages and other configuration, so it's not a 100% clone.

However, I managed to replicate the problem just now on another host and that leads me to believe that the issue is with the number of threads.
At a glance the only obvious thing in common with these non-working hosts is that they both have large number of threads.
That would also neatly explain the problems with the win64 version.

So to replicate, just try to run it on a host with 8 cores or more. Or maybe it's just the "two threads per core" that triggers it.

EDIT: ...or then again, maybe not. I fooled around with qemu a bit and got somewhat baffling results: If I run UniPCemu in a qemu guest, it works 😁
So even when it doesn't work in the host...just launch it in qemu and there it runs ok (tested with 8 cores and that was ok too).
EDIT2: The problem shows itself in qemu IF it is run on a non-working host and the "-cpu host" parameter is used.

I also tried "taskset -c 0" to see if that helps any (since it's supposed to limit the number of cpu cores used), but it didn't seem to have any effect.
Here's the output of "info threads":

(gdb) info threads
Id Target Id Frame
* 1 Thread 0x7ffff6df77c0 (LWP 88574) "UniPCemu" 0x0000555555566950 in ?? ()
2 Thread 0x7fffe3cdf700 (LWP 88578) "PulseHotplug" 0x00007ffff7c15bf6 in __ppoll (fds=0x5555570d4880, nfds=3, timeout=<optimized out>, sigmask=0x0) at ../sysdeps/unix/sysv/linux/ppoll.c:44
3 Thread 0x7fffe3c3e700 (LWP 88579) "SDLAudioP1" 0x00007ffff7c15bf6 in __ppoll (fds=0x555557183b00, nfds=3, timeout=<optimized out>, sigmask=0x0) at ../sysdeps/unix/sysv/linux/ppoll.c:44
4 Thread 0x7fffe343d700 (LWP 88580) "SDLAudioC2" 0x00007ffff7c15bf6 in __ppoll (fds=0x55555717f230, nfds=3, timeout=<optimized out>, sigmask=0x0) at ../sysdeps/unix/sysv/linux/ppoll.c:44
5 Thread 0x7fffe2c3c700 (LWP 88581) "UniPCemu_Timing" 0x00007ffff7be03bf in __GI___clock_nanosleep (clock_id=clock_id@entry=0, flags=flags@entry=0, req=0x7fffe2c3bca0, rem=0x0)
at ../sysdeps/unix/sysv/linux/clock_nanosleep.c:78

The above trace is with the release binary, see the attachment below for a session log with the debug one.
In that one, I ran "info threads" twice. The latter one is taken in a situation where the green A appears. Not that it makes much difference.
I also have a core file, but it's too big for attachment (even compressed).

Attachments

Last edited by mr.cat on 2021-03-27, 12:29. Edited 4 times in total.

Reply 22 of 29, by superfury

User metadata
Rank l33t
Rank
l33t

Hmmm... UniPCemu just uses multithreading sporadically.

There are essentially only up to 3 threads at a time used (off the top of my head):
- 1 thread for the Settings menu and the startup one(they are the same handler in the same thread, just with different parameters). These are named UniPCemu_BIOSMenu(Normal trigger by user) and UniPCemu_POST(triggered by internal BIOS int19h handler and when starting the emulator).
- 1 thread for the debugger(during breakpoints). It's named UniPCemu_debugger.
- 1 thread for general timing (UniPCemu_Timing). That's for the framerate, PSP keyboard swapping(L trigger PSP input) and MIDI active sense on the emulated MIDI synth.

The first 2 are terminated when not displaying anymore. The debugger one is restarted for every breakpoint triggerred. The timer thread runs when the emulator is running or paused.

SDL_CreateThread is called repeatedly by the thread module if it fails to give a thread, though(and returns NULL).

Edit: During normal runtime on Linux, there's only 5 threads active:
- (dbg)UniPCemu: the main thread itself.
- PulseHotplug: Linux provided Pulse library?
- SDLAudioP1: SDL2 audio rendering thread
- SDLAudioC2: SDL2 audio recording thread
- UniPCemu_Timing: the timer thread of UniPCemu

When you run gdb, then "run" then "info threads", what does it report?

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows and PSP on itch.io

Reply 23 of 29, by superfury

User metadata
Rank l33t
Rank
l33t

Ran the test software from ALEKS within UniPCemu's ET4000/W32... It after the final test that he posted reported something weird in UniPCemu's emulation:

Error reg:SEQ   #07
Error reg:ATT #06
Error reg:ATT #07

Then at the bottom: "Hit any key to continue", but it's unresponsive, not responding to keyboard input?

Edit: The BIOS still responds, but the app is 'hanging' because it's executing an invalid 0xFFFF opcode?
Edit: Hmmm... It's in the video BIOS at C000:700A.
Edit: So somehow the BIOS is setup for the lower memory aperture BIOS ROM only, while requiring it to be present?
Edit: Just fixed the Tseng registers read-protection a bit.

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows and PSP on itch.io

Reply 24 of 29, by mr.cat

User metadata
Rank Member
Rank
Member

EDIT:
I managed to reproduce the problem in qemu by running it on a non-working host and using "-cpu host". So UniPCemu or some component doesn't like AMD FX or old Xeons?
It doesn't seem to be related to number of threads or cores (tested with only one core, and the problem shows itself if -cpu host is used).

EDIT2:
Yep - it certainly looks like it's the cpu type that triggers this black screen problem.
To replicate, use "-cpu Nehalem" or "-cpu Penryn" for qemu guest.

EDIT3:
OK I compared the cpu flags of "Nehalem" and "core2duo" qemu cpu types and found the cpu flag that causes the problem to show itself in qemu: sse4.1
The popcnt flag is also present in Nehalem and missing in core2duo, but it doesn't seem to have the same effect.

Here's how the "black screen situation" looks like in kcachegrind (callgrind output is also attached, see the .7z file).
(Only the most called functions are shown here I think.)

Attachments

Last edited by mr.cat on 2021-03-27, 17:04. Edited 3 times in total.

Reply 25 of 29, by superfury

User metadata
Rank l33t
Rank
l33t

Speaking of black screens, I've just added a new feature and setting that allows setting the black level to 7.5 IRE on all video cards. It only acts on the active display area, but in the case of the CGA/MDA adapters it acts on the entire display (since they process the display entirely a scanline at a time due to the NTSC mode being supported).
And since the difference of active display vs overscan and blanking only applies to EGA adapters and up, only on said newer cards is the difference visible (mainly seen in the contrast between the active display/overscan area and the black area around it).
Since the total level of NTSC seems to be using an IRE of 7.5 for black and an IRE of 100 for pure white, I've taken it as meaning that the color range in the red/green/blue channels are from 0-255 translated using a 7.5-100 scale to their correct 0-255 r/g/b channels? So 7.5=0, 255=255 and everything in between is in equal steps?
At least, that's how I've implemented it right now.
One nice thing is that it also allows you to view the exact active display area during testing of the video card or apps (since you can see where the active display ends). It also can add (if set to 7.5 IRE) some realness to the emulated display.

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows and PSP on itch.io

Reply 26 of 29, by mr.cat

User metadata
Rank Member
Rank
Member

Hi, I did some further testing on that "Linux x64/sse4.1 black screen bug" I mentioned earlier and found out that this bug is glibc-related:
UniPCemu works just fine on the musl-based Void Linux, for example.

Here's a workaround for glibc-based distros (needs glibc >= 2.26 I think). Add this env variable in front of your UniPCemu command line:

GLIBC_TUNABLES=glibc.cpu.hwcaps=-SSE4_1

The trick works for the win64 binaries run in Wine too.

EDIT2:
OK I have found the root cause for this problem. In commonemuframework/emu/gpu/gpu_sdl.c we have this function:

//Returns 1 on not equal, 0 on equal!
OPTINLINE byte memdiff(void *start, void *value, uint_32 size)
{
return memcmp(start,value,((size_t)size)<<2); //Simply convert memory difference to 0-1!
}

Looks like the return value of memcmp() is not what is expected here. Simply changing the type from byte to word fixes the black screen problem.
The function return values will still be way off (so won't be just 0/1) but I'm not sure if it matters any.

Reply 27 of 29, by superfury

User metadata
Rank l33t
Rank
l33t
mr.cat wrote on 2021-04-01, 20:41:
Hi, I did some further testing on that "Linux x64/sse4.1 black screen bug" I mentioned earlier and found out that this bug is gl […]
Show full quote

Hi, I did some further testing on that "Linux x64/sse4.1 black screen bug" I mentioned earlier and found out that this bug is glibc-related:
UniPCemu works just fine on the musl-based Void Linux, for example.

Here's a workaround for glibc-based distros (needs glibc >= 2.26 I think). Add this env variable in front of your UniPCemu command line:

GLIBC_TUNABLES=glibc.cpu.hwcaps=-SSE4_1

The trick works for the win64 binaries run in Wine too.

EDIT2:
OK I have found the root cause for this problem. In commonemuframework/emu/gpu/gpu_sdl.c we have this function:

//Returns 1 on not equal, 0 on equal!
OPTINLINE byte memdiff(void *start, void *value, uint_32 size)
{
return memcmp(start,value,((size_t)size)<<2); //Simply convert memory difference to 0-1!
}

Looks like the return value of memcmp() is not what is expected here. Simply changing the type from byte to word fixes the black screen problem.
The function return values will still be way off (so won't be just 0/1) but I'm not sure if it matters any.

I've updated the file to use a comparison of the result against 0 (!=0) and a cast to byte just to be sure (I assume the result of a comparision (!= operator) is always 0 or 1).
That should make it work correctly.
Edit: Another slight update to make it use double negation instead (!!memcmp(...)).

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows and PSP on itch.io

Reply 29 of 29, by superfury

User metadata
Rank l33t
Rank
l33t
mr.cat wrote on 2021-04-06, 17:08:

Thanks, that double neg one seems to work. 👍

OK. As for why that was failing, the memdiff function itself is used to detect whether or not block copies between video surfaces are to be made and if the destination is supposed to be marked dirty for rendering on their respective destinations etc.
If such a dirty check fails(in this case it was due to size overflow(library to output) and truncation of it's result to an invalid value), the detection misbehaves and the caller thinks nothing is changed to render, thus ommitting the rendering of the graphics row it's transferring and not marking the surface dirty.
And this trinkles down to the surfaces said surface is used in and blitted onto(due to the dirty bit not being set). So if the surface being rendered from doesn't have it's dirty bit set, the app won't render it onto other surfaces(as it assumes they all are up-to-date). And eventually, since the main SDL surface (gotten from SDL when creating the window) doesn't have it's dirty bit set, the emulator doesn't blit the surface to the windows output(thus stays it's initial output: fully black), because why waste time rendering something that's not required to render?
This is only enforced (setting the main surface dirty bit) when it needs to be redrawn(due to the SDL event signalling that the display needs to be redrawn, like for example another app has drawn over it and it needs to be redrawn when moving it away again(leaving invalid pixels on Windows where the other window overlapped for example)).

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows and PSP on itch.io