And building period specific machines also tie down with the same limitations existed during that period.
For convenience and efficiency, it's much more preferable to build old systems with current peripherals.
For e.g. - as everyone has mentioned,
1. Using an LCD saves desk space, produces less heat and free from flickers, and there is no need to tinker with refresh rates other than enabling v-sync in windows games for smooth graphics scrolling.
2. A PS/2 optical mouse is better - easily available, no need to regularly clean the ball and bearing roll-pins, and less mechanical moving parts, may translate to easy maintenance.
3. With RAMs becoming so affordable, it's nice to flesh out the once dream machine we had (but was never able to upgrade more than how it was last time due to costs,) with more than enough RAMs into it.
4. Graphics and Sound Cards are preferred to be maintained period-specific, especially for eg., games that support only Voodoo1 or games supporting only Glide, DOS systems with ISA sound cards, games that make use of the AWE ports, etc.
Other than that, it's all personal preference.
It is nice to have a period specific components, like a serial mouse, CRT, AT-keyboard, etc, however. Just for that "authentic" feel. (Provided all the components are taken care of well - these are harder to replace.)
It pays to build from the ground-up keeping in mind the current techs - like looking for a 486-mobo with a PS/2 mouse connector pins built-in and so on.