VOGONS


First post, by superfury

User metadata
Rank l33t++
Rank
l33t++

Is there any software that tests all or most functionality of a 82077aa FDC?

I'm having some issues with some software(Windows NT 4.0 and 95 RTM boot disks and winnt /ox for disk creation) and notice that it seems to get errors while verifying(when creating files) and reading(when booting it) the disks(perhaps due to turning off the disk motor while access is in progress(command 6h)?

I see that happening when verifying a file on the very first disk being created, as well as when loading NTOSKRNL from the floppy booting NT4's installation disk?

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 2 of 55, by superfury

User metadata
Rank l33t++
Rank
l33t++

OK. Found out something more: it keeps hanging and/or turning the drive motor off on 'Verifying: abiosdsk.sys' in the lower right corner of the screen?

Is there something special with that one compared to the earlier files?

Edit: OK. It seems to try to read 19/0/18 while being on track 18? And since implied seek is disabled, it errors out?

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 3 of 55, by superfury

User metadata
Rank l33t++
Rank
l33t++

Just been thinking about something...
What happens when EOT is reached with MT used? I know that on head 1 it being reached it switches to head 0 and reports the next cylinder, but what happens to the currently seeked cylinder? Will that remain the same(it just reporting the next cylinder, but remaining on the current cylinder until a seek command is issued?)? Or will an automatic seek to the next cylinder execute?

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 4 of 55, by superfury

User metadata
Rank l33t++
Rank
l33t++

I've just improved the way the automatic increase of sectors and EOT-related things are handled.

It now doesn't affect the track number that's used when reaching the different (un)documented forms of EOT. So the controller's idea of the current track and the physical track number are unchanged.

The only thing that it still does is, the normal head processing(report the reversed head when switching from head 1 to head 0, leave the original setting unmodified otherwise(unless more is read after the switch to head 0)) is applied, but when switching from head 1 to head 0 during multitrack mode and stopping at said location(e.g. finishing the last read sector at head 1 sector EOT(the disk's final sector on the track)) it reports the next cylinder number instead of the actual current cylinder number.

So the table in http://www.isdaman.com/alsos/hardware/fdc/floppy.htm ("Sector ID in the Result Phase") is still given when finishing the sectors, it's just that a normal seek operation is required for the next head to be actually accessed.

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 5 of 55, by superfury

User metadata
Rank l33t++
Rank
l33t++

Well all that seems to have fixed the floppy disk issue with booting NT4 workstation's first 2 disks.

I now get the old 0x0000006B PROCESS1_INITIALIZATION_FAILED error again!

So there's still a CPU issue with NTDLL.DLL?

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 6 of 55, by superfury

User metadata
Rank l33t++
Rank
l33t++

OK. Having fixed the FDC to no longer fault during recalibration when executing a recalibrate command without a disk being inserted, Windows NT 3.1 setup once again boots normally without issues now.

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 7 of 55, by superfury

User metadata
Rank l33t++
Rank
l33t++

OK. I've just found a little 'bug' with the FDC. It seems that the seek command sets the active head, which it didn't do until now.

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 8 of 55, by superfury

User metadata
Rank l33t++
Rank
l33t++

Hmmm... For some weird reason, it executes a Read ID command on drive 0 for the detection of a floppy disk being inserted? And said Read ID command has the head in it's second parameter of 1, thus the result has that in ST0 as well. Perhaps it doesn't like that for some reason?

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 9 of 55, by superfury

User metadata
Rank l33t++
Rank
l33t++

Can TestFDC be used with standard formatted(and not reformattable/writable) hard sectored disk images like plain .img files? Does it require IMD(In R/O mode after I implement it in the FDC&settings?)/DSK(R/W implemented) disk and/or write/format support? Just managed to add IMD disk read support(not write/format, if I'll add it due to the complexity with the different sector formats(compressed vs uncompressed))), although haven't added said read support to the FDC itself yet(just a support module for reading said format atm), which is largely in the same way as the DSK image format's handling as far as I've implemented it(just 3 functions atm: image type detection, sector/track information reading and the actual sector read function). Although I've implemented it using the same mechanics as the DSK image format uses, except for a different track information/sector information block(which is merged into one function in this case). Although DSK images are semi-soft-sectored(you can't change the soft-sectored, dynamic format on the tracks(the track layout and result codes for reads), but you can reformat them with new data in their containers).

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 10 of 55, by superfury

User metadata
Rank l33t++
Rank
l33t++

Just finished creating write support as well. Now I'll only still need to add the FDC support for the IMD files.

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 11 of 55, by superfury

User metadata
Rank l33t++
Rank
l33t++

Just finished implementing the IMD image support in the disk manager module, Settings menu and FDC. The FDC was the most work out of them, but even then it was relatively easier due to it using the same kind of disk accesses as the DSK image format(the main difference being IMD requires manual head searching and filtering/skipping within a track(which the read/read info/write routines only supporting cylinder and sector on a cylinder lookups). The DSK format handles said head lookup automatically when parsing the cylinder/head/requested sector number(physical sector number on the side of the track(linearly increasing from the index hole), not the CHS value that's in the Sector ID). So besides some added head filtering on writes/reads and skipping the other head's sectors during formatting it's exactly the same mechanic as used for the DSK image parsing in the FDC.

Originally thought that write support was going to be more difficult(due to requiring decompression of compressed sectors when they weren't compressed), but eventually it was relatively easier to perform than I thought. It simply buffers the head (all data up to the sector, when performing undo only) and tail(from after the compressed sector until EOF) into two large buffers, then simply writing the new ID(0x01) and sector data to the file, finishing by writing the buffered tail after that.
The most difficult part there was eventually the undo mechanism(which was triggered if the write of the sector or tail fails). That one had to read the head data, then try to write the sector ID, old compressed data, followed by the tail buffer. Then, if EOF wasn't reached at said point(meaning that the write of the original sector actually added corrupt data to the tail of the disk image, close the file and reopen it to make it empty, write the head buffer, then the original data until the EOF point check was once again reached, which should return EOF at that point(with a simple variable added to check if the EOF check failed before). After that, it would return the error result(used by the FDC), with the disk image being restored to it's old data if possible(if it didn't, the disk image would contain corrupted data anyways, containing anything from nothing up to the original file size(the remainder of data couldn't be written after all)). So the disk repair functionality would work properly if the filesystem the disk image was on would allow rewriting the file in it's entirety with the original file size. Otherwise, the disk image would have been left corrupted(which still happens in that case, as it can't be solved by the emulator itself).

One nice thing is that it can now read and write the IMD disk images just like DSK images(even formatting(with the assumption of keeping the same formatting layout, otherwise erroring out partway during the format due to C/H/S/sectorsize mismatch)). One specific it doesn't allow currently, though, is the changing from a unreabable sector ID(which has value 0x00) into a normal sector(which has value 0x01). Those will simply error out before actually writing said sector to the disk.

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 12 of 55, by superfury

User metadata
Rank l33t++
Rank
l33t++

OK. Just found out that IMD disk images do actually have a filter on their head values in their track information(along with a track number). Both said head number and track number in the track information block were so far ignored.

Now it will properly filter out the track to read/write with proper track numbering and head filtering applied.

I've also implemented a function to create a new formatted track for the IMD disk images. Formatting will use all optional fields as well, filling them with the values from the packets in the FDC. All sectors will then start out in compressed format(since it can actually do that with formatting). When first being written, they're probably decompressed when writing it.

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 13 of 55, by superfury

User metadata
Rank l33t++
Rank
l33t++

Just been thinking: what is the exact difference between the head field of the read/write commands and the HD field in the first command byte? Does the HD field specify the physical head to use and the Head field(the full byte at the CHS value) the logical sector head that's on the track?

Edit: If so, that would mean that just the HD field is used for selecting the head, while the Head field is the one that's compared to the sector ID on the disk? So the HD field is used to select the head, while the head byte is used in the comparision against the sector ID that's read from the disk to match?

So, when a disk has C/H/S 1/0/1 on cylinder 1; head 1, it would return said sector when trying to read C/H/S 1/0/1 when on track 1, the HD set to 1(to select said head) and the C/H/S fields on bytes 2/3/4 set to 1/0/1?

That should be easy to verify: just format the track using a format track command with HD=1 while having seeked to cylinder 1 and using one sector ID on the format parameters set to 1/0/1?
Can anyone confirm this (perhaps on real hardware)?

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 14 of 55, by superfury

User metadata
Rank l33t++
Rank
l33t++

Just added the new formatting function to the FDC emulation. It now should be (theoretically) able to format the IMD tracks in a real way(changing the soft-sector format for said track).

There also was a little issue with the head selection on IMD tracks. It was assuming each track only existed once, with the head values being recorded on the head fields when using two heads. It's now corrected to actually search for a combination of the cylinder and head on the disk image, then start parsing the data within said track block(which is just for one head) when reading/writing the sectors on said head(instead of on said track in total(which included both heads)) and formatting being applied to the single head instead of the first track that appears at such an index. Also, said index was being parsed incorrectly: it would take the nth track in the disk image(which when using two heads would place the formatting/read/write operation on half of the track/head combination on a double sided disk (in effect taking the track number directly instead of the more correct (track*sides)+side approach, or even more wrongly addressed when the side number within a track wasn't in ascending order(when it uses track 0 side 0, track 0 side 1, track 1 side 0, track 1 side 1 etc.), which would mess up even more wrongly)).

Luckily I've found that out before trying to test it.

And because it can now (untested as of yet) format the tracks specified with the format command(current physical track as specified by the command and parameters and HD value), the TESTFDC utility should theoretically work.

Although the HD selection vs CHS distinction isn't applied yet currently.

Still, anyone can confirm the HD vs Head byte distinction on real hardware?
Edit: Just implemented the physical(HD) vs logical(Head) seperation in UniPCemu. So the trick said above with formatting a physical HD with a different sector ID(which has the HD reversed) should work now, both on reads, writes, Read ID(just reading on the HD-th head, ignoring Head of the sector ID) and format(which uses HD for selecting the proper track(top or bottom), then the Head in the sector ID to format a sector on the HD-th head with said Head value).

So that should make (theoretically) the TESTFDC utility an usable testsuite now?

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 15 of 55, by superfury

User metadata
Rank l33t++
Rank
l33t++

Just found a few more bugs when testing the disk mounting itself(mainly on the selected sector detection, which assumed it was finished correctly, counting the finished value as invalid, thus erroring out).

Edit: Another more general bug solved: The I/O disk manager was checking the filename for it's type(DSKand IMD) using the relative path instead of the full path. So it would check for e.g. "test.IMD" instead of the proper "disks/test.IMD", which of course doesn't work properly(it searches in the current working directory(which isn't accessable on Android) or even in the program's folder(which also is incorrect)).

Edit: Just also found more locations where it was checking for DSK images to not be unmounted when not readable through the direct access function(which only supports static and dynamic disk images, not supporting DSK/IMD image formats).

Edit: And an infinite loop in the FDC with DSK image geometry detection(detecting SPT).

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 16 of 55, by superfury

User metadata
Rank l33t++
Rank
l33t++

OK. Some things are actually working now. It says for the first available mode: "Format Write Verify!No FDC interrupt", then for the second "Format!No FDC interrupt"?

Now I'm getting "Format!No FDC interrupt" again? Weird?
Edit: OK. After trying it again with a correct disk image, it once again gets to the Verify stage, where I see the read sector command erroring out into a deadlock state(same as no media in the drive)?

Perhaps it's still OK when formatting, but after writing everything goes haywire somehow?

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 17 of 55, by superfury

User metadata
Rank l33t++
Rank
l33t++

OK. So somehow, either during formatting or writing the IMD disk image, the amount of detected tracks decreases, which causes the disk geometry detection to limit the seeked head down to lower numbers(due to it otherwise becoming out of range). It, when it crashes, ends up at 76 tracks(it started out with 80). I know it uses the final 5 tracks of the disk for it's testing, so either during the formatting process or during the writing process, something goes quite wrong with the disk image(some kind of invalid buffer being created?)!

Edit: OK. After formatting, it's still 80 tracks. But when writing a sector, it suddenly becomes 76 tracks?

I see the file size changing correctly(adding 127 bytes to the total size for the 128-byte sector)?

Edit: OK. It incorrectly handled the sector size map at the various track skip locations. It was deallocating the map from memory and making it unused(thus defaulting back to the track information block's sector size(which usually is 128<<0xFF(oof!)).

Edit: Then, fixing some tail buffer allocation properly being optional(which isn't present when it's the last sector of the last track(no more data following the last sector) or when it's the last track during formatting, since no tracks follow it) should make it fully compatible with the disk format.

So far TESTFDC has tested quite a lot of modes:

250k Single-Density: Format Write Verify Ok
250k Double-Density: Format Write Verify Ok
250k Double/128byte: Format Write Verify Ok
500k Single-Density: Format Write Verify Ok
500k Double-Density: Format Write Verify Ok
500k Double/128byte: Format Write Verify Ok

So all checks pass with the testing software!

Although some features and checks aren't implemented yet, like data rates(except filtering on them, which is unfiltered with this file format), deleted sector read/write(not implemented at all in the FDC), Density filtering when reading/writing(it's only used to select between FM/MFM mode when formatting).

All those other features the application uses(sector sizes, normal allocated(0x01 /0x03 ID)/invalid sector(0x00 ID) are implemented in the FDC.

One thing about the FDC is that it seems to be extremely slow in accessing the file somehow? It uses lots of fseek64(the SDL version of it) for it's skipping of any unused data(like skipping tracks on the sector/block level as determined by the format), only reading bytes/words from the medium that it needs to know(all others are skipped by seeking over the relevant data, depending on the Sector ID byte's value).

It seems to somehow take quite a toll on the emulation speed(seeing as almost all processing time is going to the sector accessing on the medium)? It goes all the way down from the usual ~25% to 0%, so it must be quite a heavy processing it's doing?

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 18 of 55, by superfury

User metadata
Rank l33t++
Rank
l33t++

OK. After that testsuite fully succeeding, I now tried running format.com from MS-DOS 6.22. It originally tried to do something in protected mode(perhaps a remainder of the TESTFDC program?) that triple faults the CPU. Then after fixing format.com itself(and other system files) by copying them from a fresh MS-DOS 6.22 boot disk, it now seems to try to execute something at 0006:0006, with EFLAGS=000C?
That's kind of weird behaviour? Perhaps the MS-DOS kernel is corrupted somehow?
Edit: So, now format.com isn't working anymore? Isn't it a 8088 program in MS-DOS 6.22? Why would it be crashing?

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io

Reply 19 of 55, by superfury

User metadata
Rank l33t++
Rank
l33t++

OK. So now, apparently the fread64 functions take about 40% CPU time, while fseek64 seems to take about 22%. Totalling 40% in CPU context. So that's about 62% of CPU time spent only on reading and seeking through the IMD disks?

Would buffering the entire IMD image in one go(for each read sector/write sector/format track operation) and mostly parsing it from the all0cated memory instead of using fseek64/fread64 make it lightweight again(at the cost of a single fread for the entire file when starting the sector r/w and track format functions help in this case?
Edit: Managed to get the CPU usage for said functions down by about a factor of 4(down to 8% instead of the old ~40%) by loading the entire IMD file into memory (when requesting the IMD sector information), using said memory buffer instead of the real file until it reaches the track it wants to know about.

Author of the UniPCemu emulator.
UniPCemu Git repository
UniPCemu for Android, Windows, PSP, Vita and Switch on itch.io