DaveDDS wrote on Yesterday, 13:06:

I'm not certain that's entirely correct - I fired up one of my Win10 systems, run martypc.exe - and get: […]
Show full quote

I'm not certain that's entirely correct - I fired up one of my Win10 systems,
run martypc.exe - and get:

Copy code to clipboard

1(X) The code execution cannot proceed because
2    VCRUNTIME140.dll was not found. Reinstalling the program
3    may fix this problem.

Well, it was certainly supposed to be. I'll check my flags.

DaveDDS wrote on Yesterday, 13:06:

btw - my little 15k DVM.EXE (also statically linked and really needs no
extras) runs fine, even on a fresh Windows install.

I'd love it if MartyPC was smaller. I watch its increasing bloat every time I add a new dependency with trepidation.
The gamepad mapping file that is loaded to support USB controllers is larger than your entire DVM.EXE.
The MartyPC logo PNG used in the About window is twice as large as your DVM.EXE.

It's a good thing my goal is to write to best emulator I can, not the smallest one. If you are going to judge my worth as a programmer based on the size of my binary then I suppose I measure up quite poorly.

DaveDDS wrote on Yesterday, 13:06:

oh - and also btw/fyi - trying to look at README.md brings up a prompt to look in Microsoft store to find an application. […]
Show full quote

oh - and also btw/fyi - trying to look at README.md brings up a prompt to look in Microsoft store to find an application.

I can mostly read it if I select wordpad, however formatting information for whatever program you used to create it soils the look
a bit - it looks like simple text (perhaps some of the visible garbage is to set font sizes, fonts etc.) - why not just give it
out as a .TXT - every system back to DOS can read that!

It's markdown format. https://en.wikipedia.org/wiki/Markdown
It is the same file that gets rendered on my Github project's main page:
https://github.com/dbalsom/martypc

So you can see what it is supposed to look like. There are a number of ways to view markdown on Windows. I agree it is a bit inconvenient that the average Windows user is going to try to double-click it and get some useless suggestion from Microsoft on what to open it with.

DaveDDS wrote on Yesterday, 13:06:

If you really want to "fancy it up", why not use WordPad --every recent release of Winblows has that!

They actually removed Wordpad from Windows 11.

DaveDDS wrote on Yesterday, 13:36:

I think full .IMD support is probably overkill for a PC emulator ... I designed the format to be able to represent any disk format the Nec765
(original PC FDC) could read - specifically pre-standard and oddball formats used on some classic systems (of which the more off
ones weren't PC compatible at all)

ImageDisk was an excellent tool and IMD was a good format for its time. IMD as a disk image format today is obsolete, or more charitably, "historical", and I only support it so that people can load images that they might have in IMD format from old collections that were never re-imaged.

There is a lengthy list of copy protection schemes that IMD has no hope of representing. Contemporary efforts to archive disks have mostly agreed to concentrate on the capture of flux images, and emulator support of at least bitstream level images.

DaveDDS wrote on Yesterday, 13:36:

For example, IMD supports single density, oddball sector sizes or other things I've never seen done on a PC. It also does
things like preserving the sector ordering within differing tracks, so it can recreate as close to "exactly" the original disk.

Oddball sector sizes are actually quite common - something that requires the extension you mention in your format documentation.

IMD can't:

• reasonably represent sectors nested in other sectors, such as Superlok,
• provide the bitcell position of sectors and time-to-read, so that time-sensitive copy protections such as Rob Norten's Copylock can work
• indicate that a sector IDAM has a bad address crc, as in some versions of Formaster CopyLock
• handle weak/fuzzy bit regions, seen in several different copy protection schemes
• represent disk surface damage / holes, such as in Vault Prolok
• encode data between sectors that may be read via a Read Track command with sector-size override

DaveDDS wrote on Yesterday, 13:36:

Implementing .IMD seems like a lot of unnecessary work, although at least for reading, you don't have to regenerate the image - […]
Show full quote

Implementing .IMD seems like a lot of unnecessary work, although at least for reading, you don't have to regenerate
the image - writing however... IMD "compresses" all-same sectors (very common in unwritten portions of formatted
disks) by setting a flag to indicate this is the case, and then supplying only the single byte value. So you would have to
regenerate the image to enlarge formerly compressed sectors that get written...

IMD is actually one of the simpler disk image formats I support.
Last fall I spent four months writing FluxFox, a disk image library I intended to use for MartyPC.
https://github.com/dbalsom/fluxfox/

It supports everything from IMD to Kryoflux raw flux stream captures.

DaveDDS wrote on Yesterday, 13:36:

I don't think it is at all unreasonable to tell people that if they want to use (especially write) .IMDs in an
emulation, they should use IMDU to convert it to a simple raw binary image containing sequential sectors
first - almost all emulators support that.

My goal was to make users not have to care about converting things. They should be able to drop a disk image of any format and have it "just work". They may not be able to save a disk image in the format they opened it in, but that's fine. In most cases MartyPC will "upgrade" the imported image to a modern format such as 86F or PRI.

GloriousCow wrote on Yesterday, 13:54:

I'd love it if MartyPC was smaller. I watch its increasing bloat every time I add a new dependency with trepidation ... If you […]
Show full quote

I'd love it if MartyPC was smaller. I watch its increasing bloat every time I
add a new dependency with trepidation ... If you are going to judge my worth
as a programmer based on the size of my binary then I suppose I measure up
quite poorly.

Oh don't get me wrong, I don't equate "big" with "bad"... I admit I care about
code size more than most ... as a creator of development tools, some of which
were targeted at processors with <4k (or even <1k) of code space, you do care
about the size of the code you are generating ... now that 4G is a "tiny"
system, it has little relevance (I still worry sometimes about all the extra
libs etc. that get linked in by some tools) - but mainly reference to tiny
code nowadays is kinda my own form of "bad joke" (something that I seem to be
exceptionally good at 😀

They actually removed Wordpad from Windows 11.

Why am I not surprised? I tend to make my "readme"s simple .TXT files
(does Win11 still know how to display those?)

It may not be "fancy", but very readable and almost universally supported...
FWIW few things annoy me about someone elses software more than having to
"install" something else to read "how to get started"!

re: .IMD format
I agree completely, and it's nice that you support .IMD ... but given IMDs
targeted users, and I document the format enough that some who wanted to
could recover data from a .IMD -- I always kinda figured users would know
what it is and how to convert to more common/useful formats (like binary
sectors) - of course I never expected it to become as well known as it has, and
you don't know how .IMDs are going to "get around"!

I also agree that recording the flux transitions along with the index
signal timing is by far the best way to archive any disk... (although it is
harder to "get the data" from such an archive)

IMD was created so that people with "classic" computers but didn't have a
flux based hardware copier - had a much better chance of being able to
backup/archive those systems diskettes than typical disk copy/imaging programs
... it is therefor limited by what the PC FDC (Nec 765) can do, and it was
NEVER intended as a way to circumvent copy protection! (The best of which
use techniques the PC FDC can detect/identify, but not write)

I wouldn't call the .IMD format historical or obsolete, and I'm not saying that just because Dave is in this thread. 😉 Obviously flux images can do everything non-flux formats can do and more, but for one, they can only be created using specialized hardware. Most people who have access to an old machine with old diskettes likely don't have a Kryoflux or similar, and realistically we can't expect every one of them to get one. As long as the need exists, we're likely going to continue seeing .IMD dumps cropping up as the only available format for certain software, e.g. on bitsavers and similar, where in most cases they do just fine for their intended purpose.

Copy protected, professionally duplicated software is not everything. If that's the type of artifact you need to preserve, then sure, a flux image is really the only acceptable option. But if you're just dealing with system or utilities disks for some oddball CP/M-86 machine or whatever, or someone's homebrew programs and text files, or really any other scenario where only the data itself is important, there's no advantage to the flux format - if you get a kfx just to do that, you're only wasting your money and the bandwidth/storage for the images.

Thankfully it's a moot point, since MartyPC supports both .imd and flux streams, but both have their place.

IMD is the closest thing my emulator supports to actual flux, so I'm sure glad it exists.
The only alternative is DSK, but my implementation just supports normal reads/writes, and only formatting it if the format is exactly the same.
On IMD, all commands are supported (except for flux-specific stuff, like weak bits, nested sectors etc., which aren't supported on IMD images anyway). It's the only format my emulator can properly format with any different sectors on. I don't cache it in memory for small memory device purposes, instead perform formatting and compression using two temporary file caches instead (one for previous tracks buffer and one for the next tracks). The same applies to it's automatic single-byte sector compression: it will save the sectors before and after into two files (cache them on disk), write the sectors before (reading back from the first FIFO cache), then the compressed data, then reading the second FIFO cache for the next sectors' data. So if you write a different single-byte sectors (fill operation) it will simply replace the fill byte without buffers, but if the single byte fill status changes (from non-fill to fill or vise versa), it will compress or uncompress (to full sector) the data using the two on-disk cache FIFO files.
Basically keeping the file optimized as possible for written sectors.
So for example, disk creation has just the tracks created. Formatting adds sectors with just fill bytes specified. Then, the file system being written will keep those 2-byte (compressed ID+fill byte) where possible, while non-fill sectors are expanded where needed (or compressed when possible).
Thus UniPCemu optimizes the disk image as sectors are written, keeping size minimal. It does need to check each sector written for fillability, which is relatively slow (memory scanning an entire sector), but not a problem due to being a slow emulated device anyways.
Most of the disk image is parsed using fseek and fread/fwrite anyways, skipping past sectors and tracks as needed.

Basically when formatting a track or compressing/expanding a sector the same thing is done: cache the data before into the first file, cache the data after into the second file, create a new file, empty the first cache into it, write the new (un)compressed sector, empty the second cache into it.

The two caches can run in two modes (determined at compile-time): either cache on disk (using a file) or cache into memory (on the heap). A simple compile-time define specifies which of the two is used. Right now it defaults to memory-based, except on Sony PSP, where it uses a disk-based strategy due to it not having much memory (only some 16MB after the executable is loaded. Much of that is lost to even a basic SoundFont (audio buffers and precalcs) and emulator structures and some basic RAM (even with 1MB emulated) and emulated VRAM.
I'm glad that UniPCemu can even still load on it (even the base executable is 4MB, due to the architecture (being a MIPS RS4000 CPU)!).

VileR wrote on Yesterday, 17:32:

I wouldn't call the .IMD format historical or obsolete ...

Thanks! (I guess in "software terms" it coule be so considered because I
started IMD in 2005 ... how much softare still gets regularily used after
20 years?:-)

It has done what I set to accomplish, there are a lot more vintage system disk
images available now... and.. anyone can recreate them - you don't need special
hardware!

Like I said before, it was never my goal to "get around" copy-protection, yes,
some of the simpler types IMD can image... but that was never the goal (and
better types couldn't be rewritten on "stardard PC hardware" even if I had
wanted to) - fortunately the Nec765 based PC FDC can read/write a lot more
formats than IBM decided the PC should support.

Most ancient vendors didn't try to protect their system disks (they usually
wanted ou to make backkups so you wouldn't need their help if a diskette died).
ie: If you needed a new boot disk for you Lobo MAX-80, you could just make a
copy from another system - problem was, computers were not nearly as common or
consistant then ... you might not know another Lobo MAX-80 owner, and if you
did they might be 100s of miles from you. And lets not even think about trying
to restore an obsolete/ancient system that "nobody has".

One of the biggest advantages of IMD is that it lets you make and re-create
disk for "systemX" without having to actually have "systemX" ... this makes
it much easier to maintain and make use of repositories of obsolete disks.

... MartyPC supports both .imd and flux streams ...

Very cool .. I hadn't gotten far enough with it yet to get into such details,
it must have all the logic to completely replicate the bit-stream functions of
an FDC.

This whole thing has gotten me "back into it" enough that I actually got 86box
to boot up my OS images today! (yay)

--

Playing with it a bit bring up a question about MartyPC... 80box emulates a
network card, but as far as I can tell, it doesn't do so at the raw packet
level - it supports "Null driver" or "SliRP" (which I'm guessing is some sort
of network protocol - like DosBox supports IPX communication)

I like to use DDLINK to put stuff on/off DOS Instances over network. I use
DosBox which makes use of WINPCAP to allow raw ethernet packets to be sent &
received via a virtual NE2000.. I can literaly run DosBox on a host, and use
DDLINK to transfer files over the network to/from a real DOS system (with only
a packet driver needed) directly from that host - or "real DOS" booted under
another instance of DosBox (same machine or not).

Does MartyPC support raw packet access to host network over a virtual network
card?

superfury wrote on Yesterday, 21:08:

... UniPCemu ...

Another one I should check out ... where's the best place to get a "ready to go" one?

I see (and can download) the repository at the link in your sig... but I didn't see any executables and I don't
want delve into "building it" - just want to "check out" a recent win version.

DaveDDS wrote on Yesterday, 21:31:

Another one I should check out ... where's the best place to get a "ready to go" one? […]
Show full quote

superfury wrote on Yesterday, 21:08:

... UniPCemu ...

Another one I should check out ... where's the best place to get a "ready to go" one?

I see (and can download) the repository at the link in your sig... but I didn't see any executables and I don't
want delve into "building it" - just want to "check out" a recent win version.

The last release I made was a bit over a year ago. It's found on it's itch.io page.
It doesn't have the latest bells and whistles (as I'm right now figuring out some remaining CPU bugs before releasing a new build), but it runs pretty well without many errors in that older build. Getting roughly 20% speed at 3MIPS configured speed in realtime back with that build (i7-4790K@4GHz). Right now dropped a bit with the latest additions (mainly CPU changes and slightly heavier and more accurate hardware emulation), despite some slightly improved RAM caching of translation tables. Don't get me started on the PSP, where it's so slow it might as well stand still (over 5 minutes to get a basic XT POST screen 😖 ). Don't know why exactly the PSP is that slow though, maybe the CPU is a bit too weak to handle the more accurate emulation somehow? Mobile (Android) seems comparable to the i7, albeit slightly slower, assuming you keep the video graphical settings low enough (below 4K resolution). Jazz Jackrabbit for example runs fine, but a bit choppy and like in slow-motion. Audio pretty choppy though, but that's mainly due to the slow emulation speed. The main bottleneck seems to be the RAM/ROM mapping at the byte level. The translation caches help a bit, but it's still the main bottleneck, due to the x86 CPU producing too random access patterns for the simple 1-entry TLB (kept per read/write/code fetch). DMA and hardware don't seem to affect it too much: 20% on video card emulation, 30-40% at least on the CPU emulation (which is mainly emulated RAM reads) and the remainder is other hardware and connecting logic to keep it in sync.

The downloads are at the bottom of the itch.io page.

DaveDDS wrote on Yesterday, 21:18:

Does MartyPC support raw packet access to host network over a virtual network
card?

No. one of those "Someday would be nice" features. I don't even have a finished VGA, or sound blaster, or any number of things that I'd probably want to tackle before a network card.

GloriousCow wrote on Yesterday, 22:12:

DaveDDS wrote on Yesterday, 21:18:

Does MartyPC support raw packet access to host network over a virtual network
card?

No. one of those "Someday would be nice" features. I don't even have a finished VGA, or sound blaster, or any number of things that I'd probably want to tackle before a network card.

Fair enough - as long as it's in the back of you mine for "someday"... I've got enough to work for me for now - thanks!

Did a quick build of the current version (although I'm still working on 8080 emulation support and the 0F prefixed opcodes on the NEC V20/V30 aren't implemented yet).

Stuff like ROMs etc. you'll need to supply yourself of course. There is an internal BIOS available for 80(1)86 CPUs (based on Dosbox's interrupts for the video interrupt partly), which might still be able to boot floppies. Otherwise, use a normal BIOS ROM and video BIOS ROM (required for EGA and higher).

I'm also still searching for some weird CPU bugs (if they are bugs) occurring in some specific software (like test-sbp.exe crashing to a reboot during sampled audio playback), although Windows versions I've tested (up to NT 4.0 workstation and Windows 98) seem to boot just fine. Windows 2000 and higher fail to boot due to not finding the hard disk correctly for some reason, maybe some obscure CPU bug in configuration of the drivers? Debugging the machine over (COM0COM with UniPCemu server's virtual connection) serial seems fine though (assuming the server and COM0COM are properly configured). Used Windows XP home's WinDBG for that (itself running on Windows 10).

This emulation talk reminds me of something...

Playing with these emulators, I'm kinda surprised how slow they are...
Although it's entirely understandable -- I'm sure these are portable, and written in a "high level language",
which means the CPU flags aren't directly accessible, hence you would have to do a lot of work after
any instruction setting flag (which is most) to work out what those flags should be.

Back in the day, I wrote several emulators under 16-bit DOS - These include (at least) my Altair (8080),
Heathkit H8 (8080), Vector 1+ (8080), North Star Horizon (Z80), D6809 (6809) and MIL MOD-8 (8008)

These were all written for DOS using my own C compiler... but I implemented the CPU cores
in assembly language, for the most part I could just save the PSW after instructions setting flags,
and refer to that saved value later to see what the flags should be (with a little bit of adjustment in
some cases depending on the processor being emulated)

And the given how much faster the "modern" CPU was, the resulting emulated processors were FAST,
so fast I had the artificially slow them down!

At one point I did a test running a BASIC interpreter I had written in the 70s for the Altair through
four processors (ie: 3 levels of emulation) - X86-64 -> X86-16 -> 6809 -> 8080

and it STILL runs faster than it does on the actual Altair with native 8080!

I described it in more detail here:

Re: How do todays CPUs compare over time, even back to the 1970s and 80s?

Really shows how much faster systems are now than they were at the beginning!

DaveDDS wrote on Today, 03:58:

This emulation talk reminds me of something... […]
Show full quote

This emulation talk reminds me of something...

Playing with these emulators, I'm kinda surprised how slow they are...
Although it's entirely understandable -- I'm sure these are portable, and written in a "high level language",
which means the CPU flags aren't directly accessible, hence you would have to do a lot of work after
any instruction setting flag (which is most) to work out what those flags should be.

Back in the day, I wrote several emulators under 16-bit DOS - These include (at least) my Altair (8080),
Heathkit H8 (8080), Vector 1+ (8080), North Star Horizon (Z80), D6809 (6809) and MIL MOD-8 (8008)

MartyPC is primarily slow because it does more work per CPU cycle than many emulators do for a single instruction. The 8088 bus interface unit runs independently of the execution unit and they must be updated in lockstep to replicate the 8088's behavior exactly. If I want to execute the MUL instruction, I can't just use Rust's multiplication operator or an assembler intrinsic then move the product into the destination registers, fix up the flags, then do 'cycles_spent + 68' - I have to execute the actual cordic multiplication algorithm, advancing the bus on each cycle so that while I am multiplying numbers the bus can go out and prefetch instructions on exactly the same cycles a real 8088 would. This happens everywhere. Even for all three cycles of a NOP, I have to tick the BIU.

The CGA card, which I can tick up to 14Mhz, is 7x faster than the CPU in the Altair and replicates the display field by emulating a Motorola 6845 CRTC at 1.7Mhz tick by tick. This allows you to watch the raster beam cross the screen as you step instructions. If you issue an OUT to change the background color of the display, it will take effect that pixel, as the OUT instruction fires off a state update of the CGA that advances it to match the exact clock in which the color was updated.

The 8253 timer is also cycle-precise at another 1.17Mhz. The sound you hear out of the PC speaker is not a calculated square wave from the values set in the Channel 2 reload register, you're hearing an actual downsampled 1.17Mhz signal of the timer's output.

The CPU can also be interrupted at any moment, mid-instruction, by DMA, which is also ticked at 4.77Mhz, so even per-cycle, we have to tick the DMA controller and emulate the motherboard's HOLDA/DMA logic circuitry.

Every single cycle can also be logged and annotated with comments during execution. It's also calculating pin states for the 8088 and the 8288 bus controller that can be used to export cycle-by-cycle execution in a format that a logic analyzer can read, so you can directly compare emulator execution against hardware bus traces. Many devices also keep track of updated states, keeping dirty flags when registers have been updated so that the GUI can show the user when values update. This is surprisingly expensive as well, but you can watch certain values glow blue in the UI when they change. Every IO operation involves polling the destination device to see if it holds the READY line down in order to calculate an appropriate number of wait states. There's more that I'm missing, I'm sure. It all adds up.

I realize I've left a lot of performance on the table, as it has to date been more important to be correct than to be fast. A cached decoder would help, a system scheduler that could tick devices only at the precision each requires would help.
Believe it or not, this is my first emulator and my first program in Rust. I'd say I did a pretty decent job for what is essentially a very large Hello World.

Yeah, the same with UniPCemu. In my case, the BIU holds a basic counter that it ticks down, loaded by the EU, updating single cycles until it runs out, at which point the EU usually gets control back. The CPU has 2 kinds of options for cycle modes. The first one is simple: whenever the CPU executes a clock cycle at the set speed, the hardware does too (based on the time elapsed). DMA can interrupt the BIU when it's inactive only (the CPU has full control).
The other (IPS clocking mode) ticks hardware roughly the same, except the CPU keeps running for entire instructions (From fetch until EU phase finishes), after which the hardware is ticked by the time of a single CPU clock at the specified clocking speed.
The hardware updating is relatively simple: it's keeping nanoseconds elapsed by the CPU clock only. Said nanoseconds are either directly fed to the timers of hardware (like audio cards and modern video cards with non-14MHz clock ratew) or given the elapsed time of the CPU cycle in nanoseconds.
The end effect is usually the same: all hardware either take the 14MHz clocks passed or nanoseconds passed, add it to their timing state, perform division and modulo to divide it (remainder kept for next cycles) and then tick their own clocks by the amount of cycles passed depending on their required hardware timing.
For example, Game Blaster ticks it's 8MHz clock together with a 44.1kHz clock, rendering sames with lowpass at a 8MHz rate, feeding into a free-running stereo output (two floating poin5 numbers). Then the other clock of 44.1kHz picks up the current stereo sample at that time and moves it to the audio output double buffer. Said double buffer is directly used by the audio channel callback (running realtime) which is then mixed by my homewritten mixer to the hardware audio stream, which is given to the SDL library in blocks of samples to render to the host operating system.

The Sound Blaster works much the same. It just uses a 1MHz sample output instead of 8MHz. But in direct mode, said output is updated when it receives the data. The 1MHz clock simply performs the same, except using DMA to time itself actually instead of direct writes (DMA writes and reads writing to the current sample or reading the current recording sample output), with the 1MHz clock deciding when to push the sample gotten from DMA to the free-running renderer.

The Covox Speech Thing / Disney Sound Source use such a free-running approach as well. Both output samples immediately (Covox) or timed (Disney), with the live output being recorded by the rendering at a normal 44kHz sample rate and sent through a double buffer to the mixer.