Well you should first check if the case even fits standard ATX motherboards. The PSU will probably be underpowered for an Athlon XP, and there's also a chance it uses a proprietary pinout, although searching around it looks like you might be fine. A modern PSU might fit, you're in luck that a bottom-mounted fan is standard nowadays. I also really doubt cooling will be adequate.

Larger OEM machines of the time tend to be more standards-conforming, whereas small form factor units destined for offices used proprietary design to reduce size. A quick image search indicates the Deskpro EXD houses a microATX board (4 full height slots) with a standard I/O shield. Not so sure about the PSU dimensions, though (it resembles SFX with the bolted on fan). The specs list 120W on the original PSU.

So I looked for more information about the power requirements of the Athlon XP CPUs, and it looks like it would be a big hassle just to find a power supply that fits the requirements.
More Socket/Slot A 5v PSU discussion
PSU for Athlon XP 3200+, Radeon 9800 XT?

After looking at Wikipedia's list of CPU power dissipation figures, it looks like the easiest thing to do is a change of plans. I'd get a microATX board with Socket 775, to mount a Celeron 450, which only requires 35 W, as opposed to 74.3 W for an Athlon XP, and a GeForce 6600 PCI-Express.