Well my minimal setup for llama.cpp was Core2Duo (as a CPU-only setup) so I know a little bit about this 😁
(Yes, it did work, slowly.)

My FX-8320E has AVX1 which is a huge benefit compared to C2D but as you said, packages usually go for AVX2 minimum.
So Haswell is a good choice in that sense.
I usually try to keep somewhat up to date with llama.cpp so I'd compile things myself anyway.

GT730 and 1x riser though...well, let's just say I'll be interested to hear what you can get out of them 😁
(I did dabble a bit with opencl+low VRAM combo myself a few years ago, it didn't go well.)

ElectroSoldier wrote on Today, 10:05:

Im struggling to come up with a constructive answer to this buddy because Im not really sure what it is youre wanting to do...

It sounds like you want to find the slowest PC possible to run an LLM in LMStudio... Right?

Hah, not exactly it's more like how can I use what I already have sitting here to best advantage. Which happens to be about the slowest possible.

I am not sure what I am trying to do, beyond learning about it. I'm about in the state like it's 1995 I've got a 386 with 16MB in it and I want to install Win95 and learn web development, pretty much you can bang out html in MSDOS Edit on a 5150, buuut, I'm trying to be running a browser so I can see what I am doing. The "pros" are using a Pentium 133 right now, but if I dump 2000 into one of those, I'm not using it effectively for a year or so, so could have got better hardware by the time I am able to use it.

Not planning to make a career on this per se, just use it to enable my further interests, but if it were a generic business plan of the "collect underpants, ???, profit" type, I am collecting underpants on the way to ??? stage. After all, how will you know what the underpants can do for you, until you have the underpants.

mr.cat wrote on Today, 12:08:

GT730 and 1x riser though...well, let's just say I'll be interested to hear what you can get out of them 😁
(I did dabble a bit with opencl+low VRAM combo myself a few years ago, it didn't go well.)

Yeah I am not expecting a real lot, but from what I am gathering things have changed in just the last year or two as to how the models can divide experts according to resource type such that less capable hardware can also contribute. I am viewing it like the situation with windows swap files. You've got your fast hard drive, but if both system and swap are using it, they are basically using it at half speed or less each because of dividing the resource and the seeking back and forth. But even if you put in an extra HDD that is half as fast to dedicate to swap, because it gets the full speed of that, and seeks less the whole system speeds up, because now system can use full speed of original hdd and the swap gets the effective speed it has before without so much seeking penalty. That's the minimum example, the best setup would be that your swap is on the faster of course, which might be inconvenient due to that one being huge and your extra being tiny, but I assume the resource scheduling would be more efficient. I'm just saying in worst dumb case of adding slower hardware it can make a difference.

edit: