I've had a lot of issues with CF in my DOS Pentium 2 machine that I tried them on. I eventually switched to an IDE->SATA converter (a few bucks on ebay) and use a 120GB Intel SSD and it's been working perfectly ever since. With the CF, sometimes it just wouldn't boot the system or have some random issues, and even playing games like Doom, it would freeze for a noticeable amount of time mid-level when it went to access the drive. My CF cards weren't too heavily used and I bought them new, so I doubt wear was the issue. I do use IDE->CF adapters in an NEC PC-9821 and Amiga A4000 but I don't do a lot of writes (at least not yet) and it seems to be working fine for now but I can't say how long that will last for.
I'd recommend you try the IDE->SATA converters and see if that solves your issues.

If you're getting weird corruption like that with everything the chances are your adaptors are garbage, CF is sensitive to mechanical connection and if get specific bits go bad this is most likely the problem. I haven't had any sort of issues with CF myself, I mostly use them in laptops. CF adaptor should also have a capacitor on the power pins too near the socket and the cable should be fairly short. They're not meant to drive long cables like IDE HDDs are made to drive.

*wonders if you're the same Cooljerk I know* 🤣

No issues with CF cards so far.. I'm using quality CF-IDE adapters with voltage regulators and cheap ones (with diodes added manually).

Tiido wrote:

CF adaptor should also have a capacitor on the power pins too near the socket and the cable should be fairly short.
They're not meant to drive long cables like IDE HDDs are made to drive.

Tiido, you're remembering me of an older Compact Flash FAQ site! 😀
"How to kill CF cards ? Easy... Start a write, and immediately (less than 1 second or so) do a system reset."
Source: http://www.pcengines.ch/cfwear.htm

I'm using CF cards in many of my retro systems as well. The most important lesson to learn: never *ever* buy used CF cards - unless you need them for testing only. And don't buy cheap ones either. Unlike hard drives, the number of writes to a Flash cell is limited by physics and each write damages the cell so that the data lasts for a shorter period of time. Used CF cards may have already reached the allowed number of writes and there is no way to verify that. Just stay away from used cards.

Buy good brands (SanDisk, Kingston, Transcent e.g.) and - if possible - industrial grade cards. Those use SLC instead of MLC, which can takes 10x as many writes and reliably is much higher. Never fill a CF card with more than 50% of data, otherwise the wear leveling becomes ineffective. And when using Win9x, install enough RAM so that you can disable the swap file - or create a separate partition for it. Obeying to that, I have systems that run from CF cards for more than 4 years now. And should one fail, I made images of all cards after the system was fully set up and can restore it on a new card within minutes.

I use CF cards all the time. The oldest installed is for my A1200 that I setup sometime around 2010. So it's in use a few times a year since then and still haven't found any problems with it. It's 4GB Sandisk Ultra.

For PC DOS/Win95 setups I use smaller CF cards (256, 512 and 1024MB). But I change files more regularly (adding files, deleting...) and I still didn't notice any problems with them.

I did read somewhere that cells in SSDs will degrade slowly if not in use regularly.

kixs wrote:

I did read somewhere that cells in SSDs will degrade slowly if not in use regularly.

All Flash cells do. Imagine it that way: a Flash cell is like a bucket. Storing a bit in it is like filling the bucket with water. But whenever doing so, a pinhole is stabbed into the bucket. Now one pinhole is not an issue and you can still see that there's water inside much later, so the information is still intact. But the more pinholes you stab into the bucket, the faster the water runs out. Eventually you reach a point at which the bucket becomes unable to hold any water.

When using a Flash-based storage device regularly, the controller inside will detect degraded cells and copy them over to spare cells.

This thread reminds me of an older thread in which we discussed wear-leveling of SD cards.
It was Using a SD card instead of hard disk

Shamino raised some very interesting points in that respect, IMHO.

Also notable is the nasty issue of filesystem/partition alignment. While NTFS can be placed nicely in matching pieces,
FAT and FAT32 are another matter. From what I read, FAT can be off even though it was aligned to 4K boundaries
by using GParted or other tools. Doing a correct alignment here is quite difficult and has to be measured manually
by calculating starting addresses, length of FAT(s), cluster size, etc.

Edit: What I'm referring to is the socalled read-modify-write.
It also happens with modern HDD drives, because they now use 512e (512byte emulation on 4K tech) or 4Kn (native 4K),
but nolonger operate with 512byes/per sector (last drives of that kind were made in late 2000s).
Such a read-modify write causes performance to driop by 50% on HDDs and is even more stressing for SSDs.
In essence, write amount gets doubled which causes huge wear in comparison to a normal write.

Jo22 wrote:

This thread reminds me of an older thread in which we discussed wear-leveling of SD cards.
It was Using a SD card instead of hard disk

Shamino raised some very interesting points in that respect, IMHO.

Which ones? Alignment will only affect write speed. You would need to ensure not only the correct start address but also that the cluster size matches the block size of the Flash memory. Since you can not freely manipulate both and may not even be able to tell the block size of the Flash memory, it's something no one really needs to care. That's more of important for real SSDs, not for storage cards.

And leaving free space is *not* overprovisioning. Overprovisioning is pre-defined by the factory. The space reserved for that is invisible to the user. Only the overprovisioning area is used to replace bad blocks. Wear leveling on the other hand works with free blocks in the user-visible area. When a block for writing is requested, the next free block with least writes to it is used. Though it depends on the controller. Some keep track of the number of writes, others only distinguish between "free" and "used" and select one free block randomly. Still, leaving free space is important, since distribution of writes is more evenly the more free space is left (obviously).

You shouldn't have to leave 50% free on a CF card to have it work properly.

CF cards do NOT have the TRIM feature, so there is NO way to release blocks that have been written by the OS back to over-provisioning once the OS stops using them. So it doesn't really matter how much free space the OS sees, because the CF card has no way to know which blocks are in use and which are not.

canthearu wrote:

CF cards do NOT have the TRIM feature, so there is NO way to release blocks that have been written by the OS

This is true for older cards only. TRIM was added in 2010 with CF 5.0. But anyway, that's why you should better not use more than 50% of the card (and put the swap file onto a separate partition). This way, it takes quite a bit of time before you run out of unassigned blocks. If you fill the card up to 100% once, there's no way back except by flushing the card.

Old CF's are old tech and it won't reallocate data.

CF cards are still current-tech. Just like with SSDs, the controller used by the card makes the features. TRIM was added to the standard in 2010, for example.

I'm talking about old CFs that most of us use.

derSammler wrote:

CF cards are still current-tech. Just like with SSDs, the controller used by the card makes the features. TRIM was added to the standard in 2010, for example.

Ah, didn't know about that.

Still, DOS does not use TRIM.

I'm out. 🙁 The discussion gets a bit too hot for me (I don't want to start a fight or something by saying something wrong).
Just let me quote a last line from that page a linked to before.
How well does the wear leveling work ? CF controllers perform wear leveling to spread the erase cycles across multiple blocks,
so frequently written blocks such as directories or file allocation tables don't wear out prematurely.
Wear leveling algorithms are proprietary and undocumented - "secret sauce".
Edit: edited.

Tiido wrote:

If you're getting weird corruption like that with everything the chances are your adaptors are garbage, CF is sensitive to mechanical connection and if get specific bits go bad this is most likely the problem. I haven't had any sort of issues with CF myself, I mostly use them in laptops. CF adaptor should also have a capacitor on the power pins too near the socket and the cable should be fairly short. They're not meant to drive long cables like IDE HDDs are made to drive.

*wonders if you're the same Cooljerk I know* 🤣

SMALLLLL world, had no idea you posted here as well 😜

Thing is with my CF cards is it's not a single adapter, across all these PCs I've used a variety of CF kits and adapters and they exhibit the same sorts of problems. Specifically with the Amigas, in fact, I used a kit from AmigaKit that came specifically with a "plug and play, no hassle" CF card already formatted for the machine. I see them recommended in lots of videos.

And ya, I'm using short cables, usually maybe about an inch or two long.

EDIT: For benefit beyond "yup, it's you" 🤣, would you be able to recommend a CF adapter and card brand that isn't crap? I'm interested in giving all this another go.

This is strange indeed, then perhaps a number of those cards are used to death or something ?

Hahaha, small world indeed 😜

I have only used brand new UDMA capable Kingston cards with lifetime warranty, never got any problems. These are the adaptors and cards I have had no problems with :
http://www.tmeeco.eu/BitShit/PCschit/CFstuff.jpg

I've used these for years without any sort of problems.

You certainly need good quality CF adapters to make cards work properly for an extended amount of time, and also make things like UDMA and IDE 80 conductor detection work as intended. UDMA usually works, but some commercial-grade CF cards don't implement it properly that would throw errors until the system puts the card to PIO modes when connected to a CF-IDE adapter. The IDE 80 conductor detection often doesn't work well with adapters that plug directly to the IDE slots rather than through conventional IDE cables (the issue also affects some IDE DOMs), limiting speed to UDMA/33.

Most CF adapters have jumpers for voltage and slave/master settings. As for voltage, whether 5V or 3.3V works best depends on the card itself, but most cards should work under either voltage as per CF card specification (provided the card followed the specification well).

You probably need to look for some industrial-grade CF cards that are properly designed for IDE use. Commercial-grade cards are often configured as "Removable" (which causes issues with modern Windows) and don't implement all the IDE functions properly (which might lead to issues faster). This depends on the manufacturers, some follow the CF card specs well, others don't. AVOID SanDisk CF cards at all cost. Those cards have all kinds of issues and SanDisk were trying their best to discourage users from using them as IDE drives.

Tiido wrote:

If you're getting weird corruption like that with everything the chances are your adaptors are garbage, CF is sensitive to mechanical connection and if get specific bits go bad this is most likely the problem. I haven't had any sort of issues with CF myself, I mostly use them in laptops. CF adaptor should also have a capacitor on the power pins too near the socket and the cable should be fairly short. They're not meant to drive long cables like IDE HDDs are made to drive.

At one time I did have a CF card misbehaving on a card reader, that the text files copied to and from the CF card became messed up (the text became something different from the source file), and turned out some pins on the card reader were bent and the card wasn't connected properly. CF slot pins seem to be really fragile that they get loose or bent very easily.

I've only once attempted to use a CF card on a PC. It was a Kingston 8GB card that I bought new at a major retailer so I'm pretty confident it was legit. I made a 2GB FAT16 partition, and for whatever reasons, I installed Daggerfall three times and on the 3rd try I started getting disk errors. I was pretty disappointed.
I have a couple Transcend 4GB cards that I meant to use for a low power system that was dedicated to a Windows application, but I got so bogged down in the minutia of how to set things up that I just used a laptop hard drive instead. The need has passed and I never used those CF cards.

I'm a bit prejudiced against cheap Flash devices like CF, SD, thumb drives, game system memory cards, etc. I've had enough problems with them not to trust them in general, but to be fair my attempt with CF was very short and with only one card. If I wanted to judge CF fairly then I should try it again.
But I don't see much need anymore. 2nd hand SSDs are cheaply available nowadays so I'd be more inclined to use those. That does add the need for SATA->IDE conversion which I've never gotten into, but from what I've read it doesn't seem like this is much of a problem. Maybe it is for an Amiga, I have no idea.
In your case though, I understand you had a large supply of CF cards already and so it makes sense to want to use them.

I wonder if the lot of cards you purchased might be hacked or fakes. What I mean by "hacked" is that some sleazebags will reprogram CF/SD/etc cards to report more capacity than they actually have, and then resell them as larger drives. Eventually they corrupt themselves because they're trying to put 10lbs of storage addresses in a 5lb bag.

====
In case it isn't already clear, I just want to point out that in order to leave free space that will actually be treated as "free" from a Flash device's point of view, that space needs to be left *unpartitioned* so that the OS never targets it for a write operation. When the OS "erases" something, it's really just another write operation and the Flash device doesn't know it's significance. It will still think that any logical address that has been the target of a write operation is still associated with important data. Thus whatever physical Flash cell(s) that were previously associated to that logical address will continue to be maintained when they don't actually need to be.
TRIM solves that problem, but obviously old OSes like MSDOS don't send TRIM commands.
A workaround is to TRIM everything in some modern OS (or start with a new drive), and then do something to ensure that some portion of the logical addresses will never be used. Leaving some space unpartitioned accomplishes that.

derSammler wrote:

And leaving free space is *not* overprovisioning. Overprovisioning is pre-defined by the factory. The space reserved for that is invisible to the user. Only the overprovisioning area is used to replace bad blocks. Wear leveling on the other hand works with free blocks in the user-visible area. When a block for writing is requested, the next free block with least writes to it is used. Though it depends on the controller. Some keep track of the number of writes, others only distinguish between "free" and "used" and select one free block randomly. Still, leaving free space is important, since distribution of writes is more evenly the more free space is left (obviously).

I'm surprised if there's any distinction between user-visible and overprovisioned space. Surely it must be a common pool. The drive has say something like 34GB of Flash storage, and then the controller is programmed to show the user that they have 32GB worth of logical storage addresses that can be targeted. Each write to some logical address (LBA) leads to an association between the LBA and some physical Flash location which could lie anywhere within the 34GB of actual space. Unless CF cards have a more fixed mapping scheme than I'm presuming.

In that example, the factory overprovisioning would ensure that there is always at least 2GB of unallocated Flash. If physical Flash storage gets close to full, it leads to a thrashing effect when the drive needs to rearrange data (ie write amplification). This kills the drive's performance and longevity, so the 2GB overprovisioning keeps that somewhat in check. But usually the factory overprovisioning is skimpy because it's "wasted" space that can't be advertised to the consumer.
So-called "enterprise" SSDs (and I assume CF cards) have more factory overprovisioning than consumer drives do, but you can get a similar effect by doing it yourself with partitioning.

If the user only partitions 20GB of space, then the other 12GB worth of logical addresses will never be targeted for a write operation, so in effect, you've increased the over-provisioning by that amount. As long as you never touch those unpartitioned addresses, then the effect is as good as if it came from the factory as a "20GB" drive with 34GB of physical Flash capacity. It will never get full enough to appreciably degrade, even without TRIM.
However, I'm really thinking of SSDs here. I don't know how dumb CF cards are. If they have a more fixed mapping between LBA -> physical Flash, and can't map the locations with as much flexibility as I'm assuming, then maybe they don't respond as well to this.

TRIM was added in 2010 with CF 5.0.

Interesting. I had no idea there was any form of non-SSD Flash devices that had this feature. I'm glad somebody did it.