Towns units are not that expensive. Not cheap, sure, but a modern PC (CPU+GPU, plus everything else to make those two work) are way more expensive. Also consider direct JP import unless you want a no-hassle deal where you get a known good, working unit (perhaps repaired), then ebay is a good source. But some of those prices do qualify as expensive, yes.

Anyway, a Fresh desktop model is the safest bet simply because you get VGA connector and output "upscaling", that is re-timed to meet 31k VGA monitor standards. With tower models you'd need some external scan converter like OSSC. On the other hand if you want/need HDMI and perhaps already have OSSC, then you don't much care. All games with very few exceptions will work on all Towns, even Street Fighter 2 that tends to lag a bit (unless a different than default video mode is selected) does not really use Fast mode of the newer machines. It does require 4MiB of RAM but pretty much all Towns except Marty console can accept a SIMM stick or two (or three).

I have both white desktop Fresh-ET and a 3rd generation grey tower (two of those, 20F and 40H) and I mostly use the 40H tower I got first.
- Fresh-ET has VGA output and stero speakers, but if you want quality sound you'll be using external speakers anyway. As all desktops of that era the monitor fits best on top of the unit. Only one floppy drive by default, HDD was missing.
- 40H/20F have a mono built-in speaker, again you'd want external amp and pair of stereo speakers for it anyway. Two floppy drives, HDD in 40H actually still lives but required many scans and remapping bad and weak sectors to be reliable enough to use. Right now I have it replaced with SCSI2SD.

Things you want to consider, other than monitor requirements I mentioned:
- All-in-one units (with monitor) are sort of a mixed bag. You get a monitor that can accept external input too (useful for some other devices) but it's so-so regarding dot pitch. These Towns are hard to service, expensive, and pretty much require repair to be even usable (the monitor caps leak and if not addressed it'll cause massive damage to the PCBs). The CPU is either 386SX or 486, doesn't matter that much in terms of performance but it is not upgradeable at all. You do get SIMM slot at least.
- Many games require a floppy drive to save. In fact some will not even run (or boot properly) if a floppy is missing, and some need a boot floopy too. So you want a working floppy drive or you'll be replacing it with an emulator, which don't look all that great if you ask me. I prefer the real deal. Therefore two floppy drives are better since you can swap them around if one dies and you can't repair it.
- Some newer games require HDD install. If you have a working SCSI HDD (can be external), great, but it's better to use SCSI2SD or something of the sort. Quieter too but every now and then I use a mechanical HDD for the nostalgia factor.
- All of these have PSU issues, towers especially. Some people swap the PSU for a combo of 12V wall brick and ATX pico PSU. That can cause problems in some cases, personally I prefer to repair the original PSU and swap the fan for something less noisy. My Fresh did not require any PSU repairs except the fan was badly worn out and just had to be replaced. Obviously you need a step-down transformer as these are not universal PSUs and only accept 100V input.
- Some Towns games have bugs. Having too much RAM is a dealbreaker in some cases, 6-8 MiB is pretty much how high you can go without major issues.
- You want a mouse for some games. So pad is a must and mouse is often required. Keyboard - less so but consider it. Not cheap and bulky. There are some adapters/emulators for PS2/USB PC keyboards, work well enough for games.

As for looks, keep in mind the early desktops are diffrent from Fresh ones. It's grey tower -> grey desktop -> white desktop -> Fresh desktop -> FMV series.