Actually back then I did manage to install XP on a Removable CF card.
It definitely brought up some problems:
1. During initial install phase, the text mode Windows Setup might not install the NTLDR to the correct place, when installing system to the removable CF card, resulting in system complaining about "Error Loading Operating System". This can be solved with a forced FIXBOOT and FIXMBR on the correct partition through recovery console. (Not to mention the text mode setup of W2K/XP can be a pain when it comes to multiple disks due to the way it assigns drive letters, and in most cases, removable CF cards get assigned with an undesired drive letter which can cause this problem and could possibly overwrite another drive's bootloader such as GRUB)
2. Partitions other than the first one are not visible, and the swap file cannot be created on a removable partition. While this doesn't stop you from adding a swap file on a rotational drive, you lose the ability to get more information in case of a BSoD as it requires some swap spaces on system partition to do so. I'm not sure about Windows Embedded POSReady 2009, which appears to support installing to and booting from a removable partition unlike others, but the 1 partition limit retains. (Good side of POSReady 2009 is that you can choose the drive letter of the system drive, bad side is that the Setup cannot see logical partitions, only primary partitions, whereas original XP's text mode Setup can)
3. Some system methods used by installers to get available drives do not return those that were marked Removable, so those installers might complain about something like being unable to locate the system or temp directory (with some resorting to tap the floppy drive if you have one installed as a fallback) and fail with an error, if the system or temp directory is on a removable partition. .NET Framework's installers were among those affected. This can be solved with a post-installation install of the cfadisk driver which turns your Removable disk into a Fixed one.
I was never aware of this Removable bit (and thought CF cards are just like IDE drives as I read about its description from Wikipedia) until I finally got the Windows XP installed to the CF card and found out my system partition turned out to be a "Removable Disk". 🙁
CF cards nowadays are shipped with it set Removable with no options to change, despite the CF specification stated the possibility to change this. Only Windows 2000 and higher cares about this bit, though there was a registry trick under Windows 2000 (in Japanese) to forcefully set specific cards as Fixed (For XP, the cfadisk driver can be used post-install to get a CF card recognized as fixed disk). DOS, Linux and Windows up to 9x and NT4 do not care and will happily install and boot from a Removable CF card.
Recent Sandisk CF cards are rather notorious for being IDE incompatible, and they officially stated their current stance regarding the Removable bit. Note that they appears to no longer offer FIxed mode CF cards through OEM channels anymore. However, if you can manage to get by an older Sandisk CF card with FW earlier than 5.00 you might be able to change it to Fixed mode through ATCFWCHG.
I've a few of other brands as well. They're all Removable, but compared to Sandisk they're better behaved. I do recommend Transcend CF cards as they are among the card that are best-behaved (400x ones, for example) when connected using IDE. Toshiba can also be a good option though they tended to be more expensive. On the other hand, a Lexar 32GB (800x) CF card I had does behave well, but somehow the card's device name string is empty.
Aside from that, it's better off avoid Sandisk CF cards altogether, as those cards often don't correctly report their sizes.
They show up correctly in AWARD BIOSes up to 32GB, and 0Bytes in AMI BIOSes. Also, DOS and some disk utilities treat those cards using CHS which limits them to 8GB. For 64GB and higher cards things are getting nastier: With AWARD BIOSes you can only get to access the first 32GB using IDE, and with AMI BIOSes, while you can access to all the spaces using IDE with Linux, format it and use it there, cards formatted that way will always show up as an unformatted 32GB partition under Windows (notable Vista/7 or later), and there's virtually nothing protecting you, apps, or the system from formatting the "unformatted" card by accident under Windows.
Unfortunately, it's no longer possible to install Windows 7 to a removable CF card anymore (neither does POSReady 7 nor Thin PC). The install destination list will not show the removable CF card. Also, if your PC is configured to boot from the removable CF card (that is, your removable CF card is the first one in the hard disk boot order), Windows 7 will refuse to install as it cannot find the place it's supposed to install BOOTMGR to, most likely due to the same logic as those installer issues. The installer attempts to locate the drive that is the first of the hard disk boot order, but since the boot drive is Removable, the method used to get the available disks returned a list without the drive the installer wanted, so the attempt failed and the installer displays the error message.
PS: CF cards of 8GB in size and with decent speed are best for building fast legacy rigs without crossing the 8.4GB (CHS) barrier that some good legacy operating systems (MS-DOS 6.2x, PC-DOS 7.0/2000, Windows NT3.51/4.0) incurred.
EDIT: Should tell you that while there is a cfadisk driver compiled for 64-bit, due to the driver signing enforcement in 64-bit Windows, if you use that driver you'll need to F8 and choose disable driver signing enforcement every time or the system will refuse to boot. In Windows 8 and higher, this will make Startup Repair appear every boot and you'll need to disable driver signing enforcement in advanced startup every time as well.
