That's a nice system you're getting bit-by-bit.
It's looking less like a DOS system and more like a Win95 system, which doesn't have to be bad - and every one of the choices you've made pushing it in that direction still supports DOS fully (Riva128 has great VESA compatibility, Yamaha 7xx is great in DOS, P233MMX etc).
I saw a suggestion to go for Win98SE instead of Win95. Even though you can't seriously run Win98 games on this, there'e a lot of merit in that suggestion. Win98SE can run everything Win95 can and it's a significantly more modern OS. I'm not even talking about support for things like USB, but installing drivers is a lot less painful. IMHO the only downside is that Win98SE has a heavier footprint and you'll really want that 64MB of RAM, where dropping to 32MB is a viable option for Win95. Note that in DOS you can limit RAM in HIMEM.SYS anyway, it's just more effort than removing a few SIMMs.
As for storage, my preferred option for this era is DiskOnModule devices, basically an SSD in a plastic lump you stick onto your IDE connector. I find them more reliable than CF cards for OSs that write a lot to disk, and easier to use than real SSDs (plus much smaller, which is an advantage when talking of this kind of machine). The 40p PATA IDE versions were dirt cheap up to about a year ago but have shot up in price recently - but the 44p PATA versions (for mobile) are still very affordable; I picked up two 16GB ones recently for less than EUR 10 each. You just need an adapter to connect them to 40p IDE and a mini-MOLEX power connector. They can be found cheaply on eBay too.
In terms of floppy, unless you have a pile of physical floppies, I'd suggest either dropping this requirement, or adding a Gotek drive instead.
CD can be useful for audio and are cheap enough. You have a PATA system so go for a PATA drive. Plextor is the Rolls Royce of CD drives, but anything that works should be fine. Bear in mind most drives have a rubber drive belt that has probably started desintegrating. If so, it's a fairly easy (if fiddly) fix.