For my builds I try to keep the ISA/PCI/VLB/AGP/etc cards within +/- 2 years of the release of the CPU. But when it comes to HDD/CD/floppy drives I care a lot less about period-correctness. As long as it is beige and the machine is beige, I'm fine. Although I wouldn't go as far as putting a DVD drive in a 486 or something like that, I actually prefer having newer drives in my machines as they are generally more reliable, quieter and usually faster.
My 486 machine is a little weird as it's a later (mid 1997) industrial board with a PCI slot, which 90% of Socket 3 machines did not have. I'm still keeping it within the 1992-1996 range for the rest of the parts. It has an early 1996 S3 Vision968 card, probably the oldest PCI graphics card I own, and just barely at the cutoff point.
The Pentium machine is sort of aiming at a early-mid 1997 range, a maxed out PMMX just before the first P2s released. A machine like this should probably have a Voodoo1 instead of a Voodoo2, but the former is getting prohibitively expensive now, and the PMMX233 can still get a little more performance out of a V2 compared to the V1.
Over time my Dell Dimension 4100 has become a "mid-late 2000" machine. In terms of equipment it is probably the most period-correct of all my machines. Right now with a Geforce2 Ultra and Turtle Beach Santa Cruz it is very close to the top-end configuration Dell offered for the 4100, whereas it was a low-mid range config with a GF2 MX and SB Live Value when I first got it.
Pluto, the maxed out Dell Dimension 4100: Pentium III 1400S | 256MB | GeForce4 Ti4200 + Voodoo4 4500 | SB Live! 5.1
Charon, the DOS and early Windows time machine: K6-III+ 600 | 256MB | TNT2 Ultra + Voodoo3 2000 | Audician 32 Plus