kjliew wrote on 2021-03-12, 22:07:

If you haven't looked at QEMU, then perhaps its scary story of "very user unfriendliness" had fogged your thoughts ... 🤣
The concept of simple block layer for virtual FAT that reflects the content of host into guest had long been available from QEMU since version 1.0, more than 10 years ago. The source code is open, so feel free to read it. It's called VVFAT. It works very well for DOS for sharing any folders from host to guest with limit of 504MiB. Why only 504MiB? I guess it was because the legacy CHS disk geometry limitation without any translation. QEMU VVFAT is also bi-directional, you can share an empty folder and have the guest write to it to extract files from guest.

Nice to know! ... 10 years later with a "new" idea (including the "handle with care warning") 😀 - but it's obviously possible 🤣

kjliew wrote on 2021-03-12, 22:07:

From pragmatism perspective, the best debugger support is a simple internal gdbstub. If one needs a fancy GUI, then just use whatever is available for the gdb frontend. Both Bochs & QEMU started out with gdb stub, a very logical design decision. Bochs later added a visual debugger but QEMU remained committed to gdbstub-only debugger support. IMHO, QEMU did the right decision at saving development resources. Bochs visual debugger, while looked good, doesn't really offer anything better than gdbstub with gdb frontend. In fact, it is worse because the visual debugger lacks the scripting capabilities from gdb frontend.

okay .. I'll give it a try 😀 .. everything seems to be better than fiddling with TD in limited textmode ...

the debugger GUI was a suggestion to the XTulator Dev , just a wish, nobody expects something like gdb - a disam & memory window, basic features like breakpoints and that's it .. I've coded some time ago for the GBA and it was just a joy using the internal VisualBoyAdvance Debugger - for hobby projects something like this is quite sufficient ...

The main benefit of an integrated debugger is probably just that, simplicity & no hassle setting it up, just go hack away...
Another angle to it is that if you want to find out how debuggers work, one way to find out is to write one 😁

TheMechanist wrote on 2021-03-13, 05:51:

Nice to know! ... 10 years later with a "new" idea (including the "handle with care warning") :) - but it's obviously possible l […]
Show full quote

kjliew wrote on 2021-03-12, 22:07:

If you haven't looked at QEMU, then perhaps its scary story of "very user unfriendliness" had fogged your thoughts ... 🤣
The concept of simple block layer for virtual FAT that reflects the content of host into guest had long been available from QEMU since version 1.0, more than 10 years ago. The source code is open, so feel free to read it. It's called VVFAT. It works very well for DOS for sharing any folders from host to guest with limit of 504MiB. Why only 504MiB? I guess it was because the legacy CHS disk geometry limitation without any translation. QEMU VVFAT is also bi-directional, you can share an empty folder and have the guest write to it to extract files from guest.

Nice to know! ... 10 years later with a "new" idea (including the "handle with care warning") 😀 - but it's obviously possible 🤣

kjliew wrote on 2021-03-12, 22:07:

From pragmatism perspective, the best debugger support is a simple internal gdbstub. If one needs a fancy GUI, then just use whatever is available for the gdb frontend. Both Bochs & QEMU started out with gdb stub, a very logical design decision. Bochs later added a visual debugger but QEMU remained committed to gdbstub-only debugger support. IMHO, QEMU did the right decision at saving development resources. Bochs visual debugger, while looked good, doesn't really offer anything better than gdbstub with gdb frontend. In fact, it is worse because the visual debugger lacks the scripting capabilities from gdb frontend.

okay .. I'll give it a try 😀 .. everything seems to be better than fiddling with TD in limited textmode ...

the debugger GUI was a suggestion to the XTulator Dev , just a wish, nobody expects something like gdb - a disam & memory window, basic features like breakpoints and that's it .. I've coded some time ago for the GBA and it was just a joy using the internal VisualBoyAdvance Debugger - for hobby projects something like this is quite sufficient ...

Hmmm... Good idea though. UniPCemu already has a disassembler and 5 breakpoints supported... It also currently supports dumping memory to a file(which you then can use with your favourite hex viewer to view what's in memory).
Should I also add a memory viewer inside the app itself? Or would the current memory dump be enough? (esp. mobile devices would benefit from the memory viewer in-app?)
I´m thinking just entering a start address and then filling the display with a hex dump formatted in pure text as a simple table? Perhaps add a little coloured pointer to it like the PSP File manager did on the PSP?

superfury wrote on 2021-03-13, 11:17:

Hmmm... Good idea though. UniPCemu already has a disassembler and 5 breakpoints supported... It also currently supports dumping memory to a file(which you then can use with your favourite hex viewer to view what's in memory).
Should I also add a memory viewer inside the app itself? Or would the current memory dump be enough? (esp. mobile devices would benefit from the memory viewer in-app?)
I´m thinking just entering a start address and then filling the display with a hex dump formatted in pure text as a simple table? Perhaps add a little coloured pointer to it like the PSP File manager did on the PSP?

from my personal point of view it would be nice to have a window for disassembler & registers (where you can set breakpoints and step through code etc.) and a window for memory (classic hex mode table view like nc/mc has), where you set an address and get the content (advanced features like changing the data would be nice too, never needed dumping to file, but others could find that useful) ... since I do only real mode stuff, I need something like seg:ofs to point to a needed memory location .. have no clue how it's done and what is preferred for protected mode 'n stuff and all the fancy thinks PCs can do since 1992 🤣 ...

but I understand the arguments that including a gdbstub is the easiest & most efficient way both for developers of the emulators and most users ...

Just finished implementing a simple 16x16 grid of hex displaying RAM (only the RAM that's installed in the machine, obviously no memory mapped devices (since they can affect emulator state, like VGA VRAM) or BIOS ROMs (since they, for example when using flash memory emulated, can also affect the state of the machine)).
Although the memory ROM shadowing can still be viewed, as they reside in physical RAM instead of ROM (used by more advanced BIOSes).

It's basically having a simple pointer to allow to use the cursor keys(or dpad on PSP) to select a row and column), where the text would be white, while the row header and column header(displaying the low 4 bits and upper 4 bits of said address respectively). Also, the starting address is inputted when starting it up from the debugger. It only supports the cursor keys to move around the window and pressing the cross/NUM2 key closes it and returns to the normal debugger interface that's still in single-stepping mode.

Some examples:

Said examples use a base address of 00000005 (the top-left row/column).

The headers just display the address of the first item in said row/column. The columns are the modulo 16 addresses and the rows are the multiple of 16 addresses.

superfury wrote on 2021-03-12, 07:36:

Like I said, it's the MS-DOS system that caches the entire FAT that the emulator would provide into it's own data structures, just like normal hard disks.
... Even if you would update the RAM disk's FAT to reflect those changes, MS-DOS would fully ignore them since it will only use it's own copy of the FAT only loaded from the disk during system boot or driver load(when mounting the medium). So it will simply corrupt the entire FAT that the emulator provides, including any affected files.
Edit: Ever wondered why the virtual directory mounts of Dosbox won't work when using the boot command for full system emulation? That might be the very reason: MS-DOS HDD caching. The same can be found with the command that's related. Part of it is what Dosbox also provides: the rescan command. So automatic client flush detection probably also won't work, otherwise said command wouldn't exist (only usable while overriding int 21h through Dosbox's own MS-DOS handler).

where did you get the information, that MS DOS caches the FAT and holds his own copy of it? I thought tools like smartdrv became popular to speed up disc operations because MS DOS doesn't cache anything ? ... the only thing that seems to be loaded at boot up is the partition table?

I'm playing around with a Guest-Host-Mapper and manged to instantly overwrite the FAT by my host program without problems (testing with PCem/MS DOS 5.0) ...

TheMechanist wrote on 2021-03-15, 10:16:

superfury wrote on 2021-03-12, 07:36:

Like I said, it's the MS-DOS system that caches the entire FAT that the emulator would provide into it's own data structures, just like normal hard disks.
... Even if you would update the RAM disk's FAT to reflect those changes, MS-DOS would fully ignore them since it will only use it's own copy of the FAT only loaded from the disk during system boot or driver load(when mounting the medium). So it will simply corrupt the entire FAT that the emulator provides, including any affected files.
Edit: Ever wondered why the virtual directory mounts of Dosbox won't work when using the boot command for full system emulation? That might be the very reason: MS-DOS HDD caching. The same can be found with the command that's related. Part of it is what Dosbox also provides: the rescan command. So automatic client flush detection probably also won't work, otherwise said command wouldn't exist (only usable while overriding int 21h through Dosbox's own MS-DOS handler).

where did you get the information, that MS DOS caches the FAT and holds his own copy of it? I thought tools like smartdrv became popular to speed up disc operations because MS DOS doesn't cache anything ? ... the only thing that seems to be loaded at boot up is the partition table?

I'm playing around with a Guest-Host-Mapper and manged to instantly overwrite the FAT by my host program without problems (testing with PCem/MS DOS 5.0) ...

Well, it's showing itself when mounting an image(like in Dosbox/UniPCemu), then using an image editor to change directory contents etc.). When you perform dir after that, MS-DOS still shows what it was before the disk image edit(or it even being corrupted from the MS-DOS point of view at least). From my memory of it happening at least.
That's also happening with CD-ROM disc images when not notifying the emulator manually to remove and reinsert the CD-ROM to make MS-DOS detect that the disc has changed. It will keep listing the old disc contents.

Edit: At least floppy disk caches are confirmed: https://devblogs.microsoft.com/oldnewthing/20 … 24-00/?p=102915
Wouldn't be surprised if the same counted for hard disks(otherwise they would be hot swappable, which afaik they're not).
Edit: Hmmm... https://web.csulb.edu/~murdock/buffers.html seems to confirm that it's indeed always caching some data from the HDD.
Also, compared to floppy disks or SCSI/ATAPI devices, it can't detect if a disk has been replaced by another, making it even more likely that such a cache always exists for the HDD.
Edit: https://forums.virtualbox.org/viewtopic.php?f=3&t=82254 seems to imply that, indeed, hot-swapping IDE hard drives cannot be done. It can only be done with SATA drives(which actually supports hot-swapping specfically).

superfury wrote on 2021-03-15, 15:17:

Well, it's showing itself when mounting an image(like in Dosbox/UniPCemu), then using an image editor to change directory contents etc.). When you perform dir after that, MS-DOS still shows what it was before the disk image edit(or it even being corrupted from the MS-DOS point of view at least). From my memory of it happening at least.

Well, this could also have something to do with how the host OS handles file buffering & write backs regarding the image and how emulators mounts file images ... do they write back instantly to file or do the hold the image in memory ...

superfury wrote on 2021-03-15, 15:17:

Edit: At least floppy disk caches are confirmed: https://devblogs.microsoft.com/oldnewthing/20 … 24-00/?p=102915 Wouldn't be sur […]
Show full quote

Edit: At least floppy disk caches are confirmed: https://devblogs.microsoft.com/oldnewthing/20 … 24-00/?p=102915
Wouldn't be surprised if the same counted for hard disks(otherwise they would be hot swappable, which afaik they're not).
Edit: Hmmm... https://web.csulb.edu/~murdock/buffers.html seems to confirm that it's indeed always caching some data from the HDD.

Also, compared to floppy disks or SCSI/ATAPI devices, it can't detect if a disk has been replaced by another, making it even more likely that such a cache always exists for the HDD.
Edit: https://forums.virtualbox.org/viewtopic.php?f=3&t=82254 seems to imply that, indeed, hot-swapping IDE hard drives cannot be done. It can only be done with SATA drives(which actually supports hot-swapping specfically).

as mentioned before, partition table is perhaps only read at boot time (that's why FDISK seems to ask for reboot) ... hot-swapping/plugging is a later feature that has nothing to do with a msdos cache ...

A cache isn't very likely, since just the FAT16 can take up to 128kb of space, not counting directory-table and data area ... a design decision that Bill "640 kB ought to be enough for anybody" G. wouldn't have made 😀 .. maybe MS DOS caches something (will check the BUFFERS directive), but it can't be a lot - and if that causes problems, I'll set it to zero (or find a way to delete the reserved section in RAM - does anyone have a link to a "MSDOS inside" site/book?)

If "real" caching was a internal feature of msdos, why smartdrv?

I'm just a layman, maybe some more experienced dev could give some hint 😉 .. my tool works as expected (Linux) and I'll release it as soon as version 0.01 is ready 😀

Edit: function 0Dh is interesting Microsoft MS-DOS Programmer's Reference

okay ... I set BUFFERS to max (99,8, MSDOS 5.0), that took about 80kb of RAM - and like you said - the disc content wasn't updated after resetting the image (this didn't happen here before without BUFFERS directive in config.sys) ... BUT - function 0xD does the job! After calling DOS shows the right disc content ... will include it in my small ms dos rescan tool ...

Neville wrote on 2021-02-19, 14:09:

DOS Harddisk images don't work yet, right? […]
Show full quote

DOS Harddisk images don't work yet, right?

I could run a few PC Booters, everything looks fine.

Glad to have another PC emulator on the scene.

I haven't been here for a while!

No, DOS hard disk images should work perfectly fine, I use them all the time to play larger games.

They need to be raw images. You can't use vdi, vmdk, qcow, etc. And it should have a head/sector geometry of 16/63.

Great stuff... maybe just shooting crap here, but while it's still simple, do you think there's any hope of getting it to full XT speed on something like an ARM cortex M3 at 200Mhz or so??? Am thinking of XT on an arduino sized PCB kind of deally, probably bitbang composite out, maybe USB keyboard, or abuse searle.wales console designs.

BitWrangler wrote on 2021-07-27, 03:40:

Great stuff... maybe just shooting crap here, but while it's still simple, do you think there's any hope of getting it to full XT speed on something like an ARM cortex M3 at 200Mhz or so??? Am thinking of XT on an arduino sized PCB kind of deally, probably bitbang composite out, maybe USB keyboard, or abuse searle.wales console designs.

Thanks! And maybe! Probably? I ported it to ESP32 with CGA only support and it was roughly full XT speed. The M3 should be more capable?

Is that dual core? You'll probably need it. I'd have never gotten the ESP anywhere near full speed without a second core to render video.

Great. Though last night I was thinking "Doh! What about the Pi pico with programable IO..." several 8 bit emulators done on that already. However, just overnight, this appears... Emulation of an IBM PC on an ESP32 board well I guess it took him longer than overnight, but the info is just spreading to me today. Looks good, not sure about speeds yet though.