I chose CF because I know that they will work with the BIOS and booting is also supported.
I found that the SD adapters are pricier, but the cards cheaper and with CF it's the other way round.
The CF solution works really well for me (tried heaps of boards with no issues whatsoever) so that's why I recommend it.
Another option are notebook HDDs. A 2.5 > 3.25" IDE adapter is cheap as chips and notebook HDDs are also quite easy to obtain or you might got one lying around already.
From what I read on the Internet, platter drivers seem to be recommended for Windows and CF cards are fine with DOS because DOS hardly ever writes to the drive (apart from the programs you use).
Another cheap option for moving data into retro PCs is Iomega ZIP. Especially the parallel port drives. They are very cheap (nobody uses parallel port anymore), much cheaper than the SCSI, USB or IDE versions.
They also "just work" with any machine that has a parallel port. But it needs to be a 386 I believe? Doesn't work on a 286 I read somewhere...
You just need to run guest.exe and BAM you have a drive letter. In contrast do CF cards you can remove the drive whenever you want. CF cards you need to shut down the machine and remove the card.
IDE and SCSI ZIP drives should be fast enough to install DOS and games. I believe that the higher capacity drives are faster (e.g. 750MB) but they also cost a lot more. Wheras the 100MB parallel ports don't sell well at all. So great for the buyer!