Hardest builds for me personally - I mean these are legitimately personal reasons;
AMD Athlon XP
Not a specific system. All of the ones I have built have been temperamental and finding reliable motherboards that actually have the features I want was never easy. I've finally gotten there, but I have to fix an unusually large PSU to make it run at all.
Socket 4 Pentium
Awkward systems. My Abit PS5 one went easier, but it still throws a fit if you do certain things to the hardware and it hates my IDE controller. It's quirky though, so I like it, whereas the Batman system was just a pain in the ass.
Pentium II
For some reason, the hard drives have failed quickly (like within two weeks) on every Pentium II I have ever built. Last time it was my fault, because I used Nikimi branded drives and I knew they were bad. I would move to SCSI but it costs too much, I might just get more modern drives and make lots of small partitions as the Pentium II is new enough to justify this. Nothing over 120GB though, that would just be wrong.
Pentium OverDrive
A bad tempered machine and literally everything has gone wrong, leaving my over a year into the build with no end in sight. The problems with pushing every aspect of a technology to its upper limits are always bound to rear their head and this system is a fine example. I nearly had it working, then AOL broke my Windows install. I've had to learn new skills and how to use tools I've never owned before just to get it off the ground.
VLB 486 System
Not hard as such, but deserves a mention for the sheer difficulty and cost of obtaining VLB cards in the UK.
Yeah, what a pain... And I love every minute of it. If it just worked, I'd probably be bored, it's fun figuring out what works and what doesn't and because I don't just buy the same stuff as everyone else (which a lot of people do) or cheat and use modern hardware to build some ugly abomination, I'm effectively fumbling in the dark with little or no documentation a lot of the time.
Ha, on that note, I feel like posting a thread about my Alphameric card just to see if anyone replies to it.