Tricky question — I like both 😀
486 is nice if you like tinkering with hardware, want a very authentic build, or you have strong nostalgic feelings about the platform. Otherwise, it's a way to spend loads of money to get a computer that barely compares to the cheapest Pentium out there. Like you've said, there is nothing special about 486 in terms of performance, features or compatibility.
Socket 7 ATX is great because it's flexible and covers a huge time period. If you get a Super Socket 7 motherboard, you'll be able to use CPUs from 75MHz down to 600MHz (early to mid Pentium 2 performance). You also get a huge selection of video cards, sound cards and so on. Like you've said, you don't need ancient cases, power supplies, mice or keyboard.
ATX is particularly nice because it's very easy to build systems. A typical AT system will be littered with cables — because even late 486 PCI boards only have the AT keyboard port and everything else needs a breakout panel. Furthermore, the big thing about ATX was the fact that the CPU and memory don't obstruct any of the expansion slots. If you want, you can install full-length cards in every slot — and I mean stuff like AWE32 or some of the more exotic video cards. With AT you'll have to look for a better motherboard layout.
The downside is aesthetics, of course. Doing a period correct ATX build might be harder compared to AT. You might end up with a system that has impressive specs, but will look like a plain "old box" with no visual features like turbo button or CPU frequency indicator.
Will this be your only build? If so, I would actually recommend going with a Slot 1 system. You still get an ISA slot for true DOS sound, but the CPU will be enough for running even the latest DOS games at high resolutions. You do lose the setmul functionality and compatibility with some games — but you'll get a much better experience in most games. Also, Slot 1 motherboards are much easier to find compared to Socket 7 ATX motherboards.
My Telegram blog about retro hardware (in Russian)
Pentium 133, 32 MB RAM, S3 Trio64V+, Crystal 4232, Dreamblaster X2 and Roland MT-32
Pentium III 1000, 512 MB RAM, Voodoo 5 5500 AGP, SB Live 5.1, SB32 CT3930, Gravis Ultrasound Max rev2.1