I'm impressed and surprised the cpu emulator is only 3000 lines of code. Once you start coding for performance the lines of code start to explode. For example, lazy flags and caching the decoded instructions can add a lot of complexity, but really improve performance.

When you get to the point of improving performance, I'd recommend reading this article on branch prediction.

http://www.emulators.com/docs/nx25_nostradamus.htm

Thanks for the link, I'll read through that tonight.

Yeah, the CPU emulator is pretty unoptimized. It's actually taken right from an emulator I wrote 10 years ago with only some minor modifications, and that old PC emulator had a slew of poor design decisions and poor code all around so I decided to write this new one. Only the CPU and a few other small parts were ripped from the old one. I was going to re-examine the entire structure of the CPU once I start adding 286 and 386 support, I thought that would be a good time to do it. I'd also like to add proper cycle counting at that point.

As it stands, it's still more than fast enough since it's only trying to emulate an 80's era PC. When running unthrottled on a modern desktop, it can easily reach high-end 486 or even much better speeds depending on what code is being emulated.

This was actually my first CPU emulator! I've done a few since then. (6502, MIPS, 8080, PIC)

Hi, good work! :)

Edit: I wonder if it can run Desqview or Geoworks Ensemble, too. I'll have to check..

Thanks! I looked at your link. This does allow you to use base memory all the way up to the beginning of CGA memory with these command line parameters: -mem 736 -video cga

But you'll have to build build a custom binary of the Super/Turbo XT BIOS to access it all, which is very easy to do.

Unfortunately no x87 support or EMS yet, but EMS is something I've already spent a little time looking into! I'll be adding that.

I think I ran Geoworks on it early in development? I might be wrong. Let me know what you find! 😀

DOS Harddisk images don't work yet, right?

I could run a few PC Booters, everything looks fine.

Glad to have another PC emulator on the scene.

Neville wrote on 2021-02-19, 14:09:

Glad to have another PC emulator on the scene.

Exactly my thoughts!
I like the idea that XTulator comes close to the real world scenario of back then, when XT users made most of their XTs.
Enlarged memory, 80186 instructions set.. Just named a few. 😀

nice to see another pc emulator!

because it is still at an early stage, it might still be possible to include a good debugger interface like we discussed here Emulators with integrated debugger??

There are other project that focus on accuracy, supporting as many machines as possible and so on ... perhaps you could focus on emulation of some generic bog standard machines and compatibility and a good debugger? That would be unique compared to other projects.

It would be nice too, if the machine could mount directories form the host like dosbox does?

TheMechanist wrote on 2021-03-11, 13:47:

nice to see another pc emulator! […]
Show full quote

nice to see another pc emulator!

because it is still at an early stage, it might still be possible to include a good debugger interface like we discussed here Emulators with integrated debugger??

There are other project that focus on accuracy, supporting as many machines as possible and so on ... perhaps you could focus on emulation of some generic bog standard machines and compatibility and a good debugger? That would be unique compared to other projects.

It would be nice too, if the machine could mount directories form the host like dosbox does?

I don't think that host directory mounting would be possible using plain XT/AT hardware? Mostly due to host(directory entries, subdirectiries, physical file system), so file-based, being completely different than the guest(which expects raw filesystem mounts, in 512 byte sector format).
Perhaps the closest you could get would be some networking filesystem service to run within the app(like Virtualbox does with it's network mounts)?
Dosbox seems to archieve this by REPLACING MS-DOS using it's own file-based handling(converting all int 21h calls to real file calls internally).
With such a virtual machine running MS-DOS, without replacing MS-DOS itself, some normal network mounting service(supporting SUBST etc.) would theoretically be the only option? Perhaps some networking mount points through a network card or serial modem through a protocol like SMB that MS-DOS drivers support?

UniPCemu running Windows 95 could connect to the LAN using UniPCemu's SLIP server. It even went far enough to show itself on Windows 10's network devices(although slow and mostly inaccessable)! Though it's failing to dial out to the server(serial modem commands are wrong(dualing ATDT; instead of ATDT0), it was actually able to identify itself to the modern machine(using NT network logon).
So using the dial-up service to serve a virtual disk over the serial modem connection should be possible, given the right drivers for the SLIP connection.

Edit: Perhaps the emulator with dial-up modem, dialling out to UniPCemu's SLIP server, using the Microsoft LAN Client(http://www.jacco2.dds.nl/samba/dos.html#lanman) with NetBIOS over TCP/IP(it's a TCP/IP connection after all using SLIP), then all you still need is a TCP/IP stack and you're golden with subst for the actual drive? Never tried it yet, though.

I did manage Arachne's miniterm dialing out and slip8250.com to setup the serial passthrough. Now only TCP/IP and some tools to mount?

Always nice to have options. Looking quite good already I see 😁
That 3000 lines danoon mentioned might just be the differentiating factor for this project! Simplicity is a feature too.
All you would need then is to write an accompanying book à la Xv6...
But yeah, I can well understand the idea about slowly working your way to a 386/486, if only to open up the floodgates for MOAR APPS.

I saw no mention about debugger in XTulator's future plans, but I guess there are more pressing matters to be dealt with first...like the GUI...
Perhaps the codebase is still sufficiently small that a debugger isn't really needed yet.

There's also that thing about new features:
As soon as you implement them, from then on there's always some whiny user asking for support for it...

Here's a previous thread discussing file transfer:
Injecting files into a running PCem/86Box machine

I know that it's a problem for generic pc emulators unlike dosbox ... a quick & dirty solution (just an idea, not well thought out) could be - in case msdos is used as guest - to provide a special (unused) interrupt by the emulator that could be used from inside the guest and write a msdos TSR that hooks int 21h / 13h (long time ago on the dark side of coding I worked with such tricks 😀, so when e.g. a special drive is accessed the emulator delivers files from the host ... this solution does not necessarily have to work in all situations/programs, it would be enough, if you could work on command line ... or you provide a special dir, copy etc. command just for working with host files ..

But this was just an side issue 😉 .. would forget it, when I had a nice debugger 😀

maybe too complicated .. why can't the emulator provide the mapping through the img handling itself? it shows the guest a hdd drive with FAT filesystem but fills it instantly with files (and converts to FAT specs if necessary) from a host directory? .. that would be transparent for the guest ...

That would be difficult (thinking logically), as software like MS-DOS likes to cache all disk accesses into it's memory, causing corrupted data to appear. The other way around is the same(the host OS also caches all kinds of structures on said disk, causing the same corruption issue).

ok, didn't know, that msdos caches ... but when I said "img handling" I didn't mean something like VileR's file injector (nice tool btw.) working with real images, but that there is just no image but the "guest-host-file-transfer", shown on guest side as FAT HDD and on host side just reading the directory ..

That's logically impossible, since there's always an OS required to interpret the difference in filesystem structures(which cannot be used for host disks or paths, as they have no file system of their own).

Imagine an OS dealing in FAT/FAT32 data and the OS dealing in NTFS. Then the OS would write FAT data on the NTFS partition, causing mayhem.
The same kind of issue when caches are involved, as MS-DOS has a different idea of data on disk than the host OS does.

The only thing that you can theoretically do is make sure it uses a filesystem that cannot be cached(like a network filesystem), but that requires drivers for it(like LANMAN for SMB under MS-DOS etc.).

So far managed to install LANMAN in UniPCemu, but the configuration for SLIP will have to wait till later(don't have the time atm).
Installation succeeded, just needs configuration still for SLIP over the serial modem(it should be theoretically possible with arachne's miniterm setup correctly(already is in my case) combined with SLIP8250.COM for the packet driver)?

Although atm SLIP8250.COM hangs with the latest UniPCemu code for some weird reason, still unknown why (because of the changed serial port hardware perhaps?)... The WIndows 95 driver has issues since then as well(sending ATDT; instead of ATDT0 to dial, which is an incorrect phone number).
It's current configured like this:

1slip8250 0x60 -h slip 3 0x2f8 115200

Perhaps an hardware emulation change issue (since last build of december (2020/12/16 08:35))? I've just added some break line emulation in that?

superfury wrote on 2021-03-11, 20:37:

That's logically impossible, since there's always an OS required to interpret the difference in filesystem structures(which cannot be used for host disks or paths, as they have no file system of their own).

Imagine an OS dealing in FAT/FAT32 data and the OS dealing in NTFS. Then the OS would write FAT data on the NTFS partition, causing mayhem.
The same kind of issue when caches are involved, as MS-DOS has a different idea of data on disk than the host OS does.

maybe the discussion should be moved to another thread 😉 we are going off topic ..

... I guess you misunderstood my idea, I don't want a hdd mapping .. it's about a virtual FAT image which is residing in memory of the emulator (host) (comparable to RAM Disks back in the day) ... but more like an array/dictionary, holding information about how and where files from host are mapped to this virtual internal FAT file system ... if the guest accesses the image (msdos e.g. low level via int 13h) the emulator looks, which data have to be delivered ... since FAT file system isn't very complicated it shouldn't be to hard to find the right file data when sector xy is accessed through guest ... the emulator uses the usual host os api to access the host files ...

Because I have no clue about how the image handling in an emulator is done, it's just an idea .. back in the days I played around in msdos with swap files (to extend memory) and RAM disks but the days are gone 😀

TheMechanist wrote on 2021-03-12, 06:25:

maybe the discussion should be moved to another thread ;) we are going off topic .. […]
Show full quote

superfury wrote on 2021-03-11, 20:37:

That's logically impossible, since there's always an OS required to interpret the difference in filesystem structures(which cannot be used for host disks or paths, as they have no file system of their own).

Imagine an OS dealing in FAT/FAT32 data and the OS dealing in NTFS. Then the OS would write FAT data on the NTFS partition, causing mayhem.
The same kind of issue when caches are involved, as MS-DOS has a different idea of data on disk than the host OS does.

maybe the discussion should be moved to another thread 😉 we are going off topic ..

... I guess you misunderstood my idea, I don't want a hdd mapping .. it's about a virtual FAT image which is residing in memory of the emulator (host) (comparable to RAM Disks back in the day) ... but more like an array/dictionary, holding information about how and where files from host are mapped to this virtual internal FAT file system ... if the guest accesses the image (msdos e.g. low level via int 13h) the emulator looks, which data have to be delivered ... since FAT file system isn't very complicated it shouldn't be to hard to find the right file data when sector xy is accessed through guest ... the emulator uses the usual host os api to access the host files ...

Because I have no clue about how the image handling in an emulator is done, it's just an idea .. back in the days I played around in msdos with swap files (to extend memory) and RAM disks but the days are gone 😀

Like I said, it's the MS-DOS system that caches the entire FAT that the emulator would provide into it's own data structures, just like normal hard disks.
Say that you're writing a file. It will update it's own idea of the FAT, write changes to disk(which can be interpreted as, 'I might want to write data to a/some file(s) at the same time', since more data writes can't be guaranteed). The main issue is when the host OS also changes said file in the meantime. Since the 'RAM disk' or virtual HDD has no way of tracking opened files(it's a raw medium after all), the host can change stuff in the meantime. Thus generating a sort of race condition, both writing over each other;s data.
Even if you would update the RAM disk's FAT to reflect those changes, MS-DOS would fully ignore them since it will only use it's own copy of the FAT only loaded from the disk during system boot or driver load(when mounting the medium). So it will simply corrupt the entire FAT that the emulator provides, including any affected files.
Edit: Ever wondered why the virtual directory mounts of Dosbox won't work when using the boot command for full system emulation? That might be the very reason: MS-DOS HDD caching. The same can be found with the command that's related. Part of it is what Dosbox also provides: the rescan command. So automatic client flush detection probably also won't work, otherwise said command wouldn't exist (only usable while overriding int 21h through Dosbox's own MS-DOS handler).

I see the problems .. but as mentioned earlier, it doesn't need to be a feature for daily usage 😉 ... just a short & simple way to copy files from host to guest, so e.g. emulator could block write access from guest ... and a "handle with care" warning, not to work on the drive when it's accessed by the host/guest ... it's from a developers/testers point of view who wants a simple way to copy files to guest (see VileR image file injector) ... not every scenario/program must work flawlessly and it wouldn't even make any sense to put too much effort in it since ms dos times are gone ...

but coming back to the main concern - a neat GUI debugger interface would be nice 😀

TheMechanist wrote on 2021-03-12, 15:20:

... just a short & simple way to copy files from host to guest, so e.g. emulator could block write access from guest ...

If you haven't looked at QEMU, then perhaps its scary story of "very user unfriendliness" had fogged your thoughts ... 🤣
The concept of simple block layer for virtual FAT that reflects the content of host into guest had long been available from QEMU since version 1.0, more than 10 years ago. The source code is open, so feel free to read it. It's called VVFAT. It works very well for DOS for sharing any folders from host to guest with limit of 504MiB. Why only 504MiB? I guess it was because the legacy CHS disk geometry limitation without any translation. QEMU VVFAT is also bi-directional, you can share an empty folder and have the guest write to it to extract files from guest.

It works very well for DOS, somewhat OK on Win98 (write can sometimes crashed), but not very well with Linux or WinXP. But for anything other than DOS, the network share drives are always more reliable with QEMU's solid enterprise-grade networking frontend and backend implementations.

TheMechanist wrote on 2021-03-12, 15:20:

the main concern - a neat GUI debugger interface would be nice 😀

From pragmatism perspective, the best debugger support is a simple internal gdbstub. If one needs a fancy GUI, then just use whatever is available for the gdb frontend. Both Bochs & QEMU started out with gdb stub, a very logical design decision. Bochs later added a visual debugger but QEMU remained committed to gdbstub-only debugger support. IMHO, QEMU did the right decision at saving development resources. Bochs visual debugger, while looked good, doesn't really offer anything better than gdbstub with gdb frontend. In fact, it is worse because the visual debugger lacks the scripting capabilities from gdb frontend.

Yeah, I guess for bang-for-buck (features gained versus developer hours wasted) you can't really beat that. Combine gdb with rr and you've got a winning team. In terms of userfriendliness however...well 😁
Choice is good, but choosing a frontend is about the same level of confusion as choosing a Linux distro. So takes time first time around.
Using gdb without any frontend is possible, but requires some learning and is probably not very appealing for most ppl.

@UselessSoftware, which debugger are you using to develop (if any)? Anything homebrewn cooking there? 😁
(referring to a debugging solution of course)