Dear Frankenstein.
How did you create the EXFAT partition?
1) Please use no newer system than Windows XP / Server 2003 to create partitions on HDDs / SSDs you want to use on Win9x. Vista, 7 and newer do not care about CHS alignment.
2) For perfect alignment your partition must start at a sector number can be divided by 8. Otherwise on every write the SSD have to read the previous block before writing the data.
3) Since Microsoft did lot like big FAT32 partitions, they cannot be created with the tools supplied by newer operating systems. You either have to create them with Win9x FDISK or 3rd party tools.
CHS 0 0 1 - 0 0 63 is occupied by MBR - 63 sectors
CHS 0 1 0 - 0 7 63 - the default starting sector - should not be used to achieve 8 sector alignment - I use to create this dummy partition manually on SSDs I use with Win9x
Then I create a big NTFS partition with 2000/XP/2003.
Then I look that the data partition is aligned to divisor 8 alignment.
Then I create the data partition.
Then I hide the data partition (change partition type temporarily).
I delete the NTFS partiton.
And now I try to create the primary FAT32 partition with Win9x.
I also delete the dummy partition on the start of the disk.
And I unhide the data partition.

It may be easier when you have the data partition in front of the data partition.

That was helpful. If anyone comes across this after a google search, here's how I resolved this.

1. Downloaded a FreeDOS boot disk
2. Partitioned and formatted with the utilities provided by the FreeDOS boot disk
3. Attached the SSD to my Windows 10 PC and copied the installation media to the SSD again on the fresh partition
4. Reattached the SSD to my Retro PC, booted back into FreeDOS, ran the Windows 98 setup utility from the SSD

Prior to this, I have always installed from the Windows 98 CD. This was a successful experiment installing from installation media on a hard drive. It went pretty fast too.