Here's what I can think of for now :
Being able to boot from CD drives
Breaking capacity limits
Being able to use more drives in a computer

etc ...

You can always use IDE to SATA adapters which are about $3. That way you are not stuck with having to use old IDE drives.

For newer IDE HDDs, ATA100 or ATA133 should be fine.

If the IDE controller is below ATA133 and you want to use an SSD and really want the max performance, then get a PCI SATA controller. If using Windows ME or older, you will want to stick with a SATA 150 controller that has driver support for Windows 98 or you will not get the full speed.

As you get into chipsets from around the P4 era onward, sometimes the onboard controller is preferable because it bypasses the PCI bus.
On P2/P3 systems, onboard ATA33 controllers could be a significant limitation compared to ATA66 or ATA100. But people building these systems today might not care about the speed limitation anymore. 32GB limits can usually be fixed with a BIOS update.

When you get back into socket-7 and older, performance is probably pretty poor, and as you get into older chipsets I think they more often had serious bugs.

It's actually a misnomer to refer to an IDE interface card as a "controller".
I think this stems from the days when we still used MFM (or RLL) controllers. In such a case, the controller was actually on the interface card itself but, with IDE interfaces, the IDE "controller" cards are just host adapters, since the controller is embedded (built into) the drive itself.

The ability to boot from a CD or any capacity constraints is more a limitation of the system BIOS and has very little to do with the IDE interface (host adapter) card.
Naturally, there are limitations in terms of transfer speeds that these IDE interface cards can handle, which speaks to the PC bus speeds and whether it supports the later 80-wire version IDE cables.

This is some particularly useful info!

Edit: The question I followed up with was already answered.

It's actually a misnomer to refer to an IDE interface card as a "controller".

Yup. Esp. simple IDE cards are just bus interfaces based around a few off the shelf chips (74ls08p, 74ls245n, pal16l8acn).
Because of the relationship to ISA, back in time IDE drives were sometimes named AT-Bus drives.
Don't know if that's still valid, though. Modern PCI host controllers support Bus-mastering, etc.
And then there are also caching IDE controllers or RAID controllers.

Only reason I would not use onboard is if I want a different interface, such as SATA, SCSI. and in SATA's case I'd use a converter rather then a whole new card
The performance gains of say using a ATA133 in a board that has ATA-66 at best isn't worth the extra hassle IMHO