VOGONS


Reply 480 of 591, by superfury

User metadata
Rank l33t++
Rank
l33t++

OK. With the latest changes to the way it handles the ARP packets, it can see almost any computer on the host network and internet.
The only weird things still is that sending an ARP from the host to itself (it's host IP address, where the server itself is located), the host doesn't seem to reply within the allotted time (0.5 second).
So I've modified the ARP replies to also give results for the host's IP address besides the client if a binding of a ARP IP address and filtering of an IP address for receiving is used.
So the host now sends an ARP onto the network, which it replies to itself, finally the app sees/receives it's own reply on the network and starts using the MAC address like any other IP address on the network for sending IP datagrams to (it repeats said process once it's evicted from the cache by another IP address or 30 seconds elapse to cause the cache to be automatically flushed).
But somehow, it still can't ping the server's IP address or the other way around (the server somehow can't ping it either)?

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 481 of 591, by superfury

User metadata
Rank l33t++
Rank
l33t++

Just improved the IP address assigning to use ARP to try and detect IP addresses used on the network before considering an IP address for the client used.
So no more collisions if a IP address is used by another device on the network.
Also fixed the ARP replies to be working properly when the same client is requesting it or another IP address to be resolved.
Edit: Added a few more code fixes for properly compiling the app again on platforms with the new functions.

Still need to verify it's working properly though. I'll edit this post again if it's working.

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 482 of 591, by superfury

User metadata
Rank l33t++
Rank
l33t++

OK. After lots of debugging and checking the different results I finally found out why the collision detection wasn't working properly for the IP addresses.
The cause was pretty simple: when the client was authenticating in the simple text-mode tests I was doing (for quick verification), it was checking the current stage of the login process it was at.
It saw it was still at the protocol input stage (the final stage before DHCP(unimplemented) and the modes where it has been assigned an IP address. So when it checked the current stage against being in the Open state (which was simply to be past the protocol input stage or DHCP stage), it thought it wasn't parsing ARP yet (since the user wasn't authenticated with an IP address assigned yet).
So I then modified it to:
1. Handle the receiving of the ARP response (for collision detection) when still at the protocol stage, as long as it's already finished authenticating (a newly implemented finished substage which is used to perform the ARP lookups that are pending results from the network) and an ARP request is waiting for a response.
2. Keep the ARP to only give responses to requests when past the authentication phase and DHCP phases(DHCP being unused atm). So during the authenticating against the network itself to check for IP collisions, case 1 happens but case 2 doesn't(as the IP address isn't assigned yet, it's still scanning for usable addresses after all).

I also modified the pcap thread to also check other clients before discarding the incoming ARP packets. So if multiple clients respond to the same IP address, such a collision will also happen (although that shouldn't normally happen, as the checks before ARP is performed to determine a valid IP address already has checked against that and aborted the IP address validity check if it's already allocated on the same server).
So checks will only happen to validate IP addresses that aren't allocated by the server to also be free on other servers, if multiple packet servers are used on the same network.

Just ran 4 clients with the same packet server setting (although using different listen ports for the server to use of course).
Running them after each other, providing a login of the same range, IP address allocation accross the 4 servers work as intended: they will verify the IP addresses they haven't used in the account's IP address range against each other, counting the addresses as used if either they own it themselves or another client responds to the ARP request with a proper ARP reply.

So on the latest commits, you should be able to run the packet server with address ranging overlapping other devices in the network, provided they respond to ARP requests properly.

The next step, of course, is checking if the cheks with the PPP IPCP protocol run properly as well.
Although I can probably assume those run the same (it's using the same ARP mechanism after all, just at the packets stage (since it's in PPP mode, which requires it to be in the packets stage to run)).
Edit: OK. IP on PPP connects properly it looks like (on NT 4.0 workstation). Didn't try a second client yet though.
ipconfig does say the Default gateway is at the same address as the IP address assigned, but that might just be some oddity of the dial-up connection?

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 483 of 591, by superfury

User metadata
Rank l33t++
Rank
l33t++

So far the second client got the wrong IP address, because it thought the first client's IP address wasn't in PPP open state (thus not allocated the address).
Now, with the latest changes, it should actually detect such an address used properly and use the next IP address (if available). Otherwise, it will answer a NAK with IP address 0.0.0.0, which means no IP address is present (in this case, it can't provide one since it's ran out of available addresses).
The same will apply for static IP addresses (addresses not within the auto-allocating range). It will simply not detect those within it's own IP addresses and will double check against the local network using ARP in that case.

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 484 of 591, by superfury

User metadata
Rank l33t++
Rank
l33t++

OK. After some more testing, I've finally managed to get the IP address allocation working properly.
Now, the autoconfiguration seems to work properly.

The only weird thing is that somehow the host that's running the packet server somehow doesn't respond to ARP requests?
Edit: Just added a check when an ARP request for the host itself times out. In that case, the known local MAC address of the server is used for it instead, causing it to be properly detected.

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 485 of 591, by superfury

User metadata
Rank l33t++
Rank
l33t++

Hmmm... The host IP still doesn't connect to the clients on the packet server?

ARP seems to succeed though?
The connected clients can reach the other hosts on the network.

Now, with the latest fixes, somehow it can't reach the external network anymore. Like for example pinging 8.8.8.8, I see it's sending the packet on the host network (using Microsoft Network Monitor 3.4), but it doesn't seem to get a reply back (no network activity with a response)? The same for DNS lookups (same IP)?

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 486 of 591, by superfury

User metadata
Rank l33t++
Rank
l33t++

OK. Just added a bit of extra functionality to the server. After having allocated an IP and having validated it to be usable(not allocated on the host network already), it will send one final ARP request without waiting for the answer(it isn't needed for the client after all, probably). That way, the other hosts(including the default gateway that was so far failing to do ARP on the first incoming packet from outside the host network) might work because it'll see a request with a proper reply, setting it's own ARP cache.

Until now (latest behaviour still unverified though) my router won't perform the ARP request for the newly registered static IP address, although other clients on the host network do seem to do this properly.
I noticed that the server version (current release version to be exact) without the new ARP-without-reply IP allocation method do actually cause the router to do a proper ARP request, while the new method don't cause it to perform it (perhaps because it thinks the IP address is unallocated anyways because of the ARP request to check if a IP address already exists is timing out?).

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 487 of 591, by superfury

User metadata
Rank l33t++
Rank
l33t++

Just improved the packet server's LCP/PAP/IPXCP/IPCP sending packets only once every second.
Also managed to find a bug where tick timing was doubled in many cases due to the function being called twice (once for the first packet type, once for the second packet type).

So now with those timings fixed, the packet server won't be spamming it's request packets anymore.
It also fixes timing for the IPXCP negotiation to be the proper timing.
IP negotiation isn't affected, as it's done by using the real time of the host OS.

Now the negotiation for both in an optimal case is reduced to just about 25 incoming packets on the client side (and roughly the same for the outgoing packets). Previously this was many hundreds (300+) incoming packets due to the packet server spamming packets (although the client was a slow one running at about 20% realtime speed while the packet server was running realtime).

One nice thing about the latest comits is that it will run much faster (up to realtime depending on the performed action) when playing audio and when running without any required motherboard BIOS ROMs loaded.

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 488 of 591, by superfury

User metadata
Rank l33t++
Rank
l33t++

OK. Managed to get the packet server clients back onto the network with ARP working properly at least using a combination of the old (working) method and the new ARP-based automatic IP address allocation method.
It will now perform ARP lookups during IP allocation using the host's IP address, perform ARP request as if requested by the host and by the default gateway to make their caches work properly. Then, once the client is fully connected, it will use the client's own IP address as the source of the ARP requests (which is the old method of ARP resolving).

This seems to satisfy the different machines on the network I'm using and allows them to see each other at least.

The only strange thing I still see is that Windows 95 doesn't seem to reply to ICMP ping requests, but that might be a Windows 95 issue, not a packet server issue (as stuff like other machines on the network and machines outside of the network can be reached without issues now).

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 489 of 591, by superfury

User metadata
Rank l33t++
Rank
l33t++

Eventually just changed the local MAC address of the users to be accepting two packets:
- It recognises it's own client number to use an MAC address on the network that's not the host.
- It recognises incoming packets on the host's address for the local network instead.
- It sends packets to any packet outside the subnet for the client to either the router(outside the host network) or ARP for it(inside the host network but outside the client's network).
- It sends packets for the local subnet(for the client) to the host's MAC address instead. The filtering on the packets also accept the host MAC address, thus the packets for those arrive on the host's MAC address with the client's IP address (which is ignored by the host and accepted by the server).

That way, both the local network inside the server works properly, the network on the host is reachable and the default gateway functions properly.

The only weird thing left is the whole case with the host itself being unreachable, but that is still an unknown problem.

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 490 of 591, by superfury

User metadata
Rank l33t++
Rank
l33t++

With the latest commits, now the client can have it's own MAC address range instead of the host's MAC address.
So each client gets a MAC address assigned from a list of addresses (the amount of clients is the count, the base address being setup in the settings file). Although the base address still defaults to the host's MAC address (for easy compatibility), it can now be properly overridden to another address using a new setting in the settings file.

So each client gets it's own MAC address (determined when the server starts) and IP addresses are allocated using a combination of ARP to detect collisions and the server's own IP addresses for active clients to determine which addresses are used.

Strangely enough, even though the IP addresses nor MAC addresses seem to collide with the host's addresses or other addresses in the network, the connections to the servers outside the network seem to continue failing for some reason?

Although it might just be an Internet Explorer issue with connecting to vogons.org (what I'm testing it with to quickly verify it's behaviour)? I see communication between the two hosts using Microsoft Network Monitor 3.4, but eventually it just stops sending or receiving packets?
The router itself still shows both the host and client with their correct IP and MAC addresses in it's DHCP and IP tables.
Although I can't directly interrogate the ARP cache of the router, since it's made unviewable by the ISP.
Internet Explorer itself just seems to hang on Windows 95?

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 491 of 591, by superfury

User metadata
Rank l33t++
Rank
l33t++

Just changed the IP address announcing to perform a gratuitous ARP towards the IP address of the router.
Now the router seems to continue seeing the connected client.

When the client disconnects or releases the IP address (using PPP, closing the IPCP or LCP layer or providing invalid credentials during PAP authentication), the server will send a gratuitous ARP with a 0.0.0.0 IP address towards the IP address of the router.
That seems to cause the router to remove it's ARP entry, at least after a little bit of time. Otherwise, it looks like the entry stays in the router, so it'll remember the IP being used.

The static IP address seems to work properly? Although some websites might get fed up with the slow downloads speeds and abort. Also, IPv4 is required for the websites (obviously). And Windows 95 doesn't support HTTPS either, so that's also not working (although since it's normal IPv4, it might work if the OS supports it).
Edit: Just fixed the local IPv4 subnet (on the host MAC address, which is an exception to the usual MAC filtering) to work again.

Also, if the IP address for a client is disabled, it's now ignoring all packets that arrive at it's interface.

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 492 of 591, by superfury

User metadata
Rank l33t++
Rank
l33t++

OK. So I've just added the support for the local network broadcast IP address to the server (based on the client's IP address (when assigned) and subnet mask).

The subnet mask defaults to all 1s. The setting in SETTINGS.INI can override this value. That's what the Windows 95 client uses, as it doesn't specify said option in it's PPP settings during the IPCP phase of authentication.
Although I have the settings set to 255.255.255.192 for a smaller IP address space within my own network to work with internally, since it's shared with the host network.
Now, the IP addresses assigned also fall into this range, but since Windows 95 doesn't specify the subnet mask in it's settings, the server doesn't use said setting and tries to use the server-provided setting instead.
So that causes the server to not correctly route the packets to itself for the local IP broadcasts that Windows 95 is using to reach the other clients.

When this is done correctly as Windows expects (a subnet mask of 255.255.255.0), the server correctly routes said packets to the internal network, reaching the other Windows 95 clients. In that case both clients can reach each other (although weirdly enough not themselves using the \\computername addressing?).

The Windows 10 host can actually see the clients, but cannot reach them?
Edit: Just having implemented the same logic for sending and IP filtering when receiving for the host IP and subnet address, the Windows 95 clients seem to properly communicate with each other at least (minus them not being able to address themselves on the network for some weird reason?).
The Windows 10 host can see the two machines on the network, but can't reach them for some reason?

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 493 of 591, by superfury

User metadata
Rank l33t++
Rank
l33t++

Just modified the direct serial port connection to be able to use all modem modes. So then you can use said emulated modem as an interface on a real COM port (although the CD and RING signals are missing).

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 494 of 591, by superfury

User metadata
Rank l33t++
Rank
l33t++

Just fixed the packet server to properly shut down and start up together as needed, with the client always having a reserved TCP connection entry for dialing out. So the packet server can use the full range of available connections, while also being able to dial out to itself or another TCP port.
Previously, this was an either/or scenario, where the client couldn't dial out or receive anything when the packet server was running on itself.
Now, it can act as both the server and client (although in that case it cannot receive any calls on the emulated modem, as the packet server overrides that functionality).
So basically, the server with it's multiple connections(up to 256 to be exact) replaces the incoming line on the modem, instead of both incoming and outgoing calls as is the case in current releases.

Simply said, you can now dial out to the packet server that's running on the same client.
The only exception is if the packet server isn't running and is replaced with a direct serial connection (using the directserial setting). In that case, the entire modem is replaced (both incoming and outgoing) and disabled on the COM2 emulated port (it's connected, but not emulated, so the COM port exists, but nothing is connected to it's connector and reponding (essentially a black hole or floating lines on it's pins, as would be the case on a real UART)).

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 495 of 591, by superfury

User metadata
Rank l33t++
Rank
l33t++

Just fixed the i450gx settings to be properly available and responding again in the settings menu.

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 496 of 591, by superfury

User metadata
Rank l33t++
Rank
l33t++

The packet server on it's current commit now properly supports the CHAP authentication protocol (no MS-CHAP or MS-CHAPv2 though).

Although I didn't find any client that's using the CHAP authentication from server to client (only client to server authentication). Windows 9x doesn't seem to like server-to-client authentication at least.

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 497 of 591, by superfury

User metadata
Rank l33t++
Rank
l33t++

Just added a new little feature to the passthrough mode of the serial connection. It can now (if specified in the settings), respond to DTR being lowered by hanging up the serial connection if the connection is setup to be connecting to a TCP port using the phonebook when it's raised.
So by default it will behave as earlier (it will keep the connection live until raised again), but if enabled in the settings, it can now also disconnect the TCP connection to the connected client when DTR is lowered.

This can be used, for example, to make it behave like a dial-up server for a single modem (where DTR being raised connects a new server instance and it beign lowered disconnects for a new connection to be started when it's raised again).
Of course the configuration of any external modem or serial line is still required, but not needing any external logic anymore.
Edit: Just added some extra functionality to the nullmodem cable setting, where it can connect using the DTR line to connect(and optionally disconnect) without any line signals being added to the TCP data stream.

So now it should be possible to easily configure two UniPCemu instances to do things like answer an serial port or modem on said serial port directly, with one instance setting up an TCP connection from the serial port(or modem) when it raises DTR, with some other program on a TCP port(without line signalling) or UniPCemu's server build answering said request from the serial line based on the DTR incoming signal.
Of course such a physical modem would need to have been setup to autoanswer the incoming call and raise DTR accordingly(raising it when answering and lowering it when the other side hangs up).
It can in that way run as an actual ISP dial-up server (although for one client for each used serial modem) with it's full packet server support on it.

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 498 of 591, by superfury

User metadata
Rank l33t++
Rank
l33t++

OK. Trying Debian Jessie again on UniPCemu's latest build reveals something interesting.
It keeps trying to execute a TEST UNIT READY command on a drive without a disc inserted, causing it to spew out DRDY ERR messages, followed by "soft resetting link", configurations for MWDMA2, EH complete, then again:

ata2.00: cmd a0/00:00:00:08:00/00:00:00:00:00/a0 tag 0 pio 16392 in
res 41/40:03:00:00:00/00:00:00:00:00/a0 Emask 0x3 (HSM violation)

Anyone knows why it keeps trying to do that infinitely it seems?

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 499 of 591, by x64lover

User metadata
Rank Newbie
Rank
Newbie

Unrelated to your recent post here but still related to the thread: The "x86/emulator.c" file in https://github.com/kvm-unit-tests/kvm-unit-tests might be useful for testing your emulator insturctions and find hidden bugs.

I will have to look at the SATA thing later.