interesting and forgive any ignorance, is the LLM that runs separately offline (eg ollama seems to have that option)? that in itself is interesting, takes a lot of GB and fast CPU to cope with all that. Or maybe not for this purpose.

natural language interface may be useful for those not versed in XP commands, and as it can run things that's interesting - look like diagnostic output an input for the LLM.

And i just thought od having an offline LLM on a single thread Pentium 3 😀 there's a challenge!

Hope you get some interest 😀

gerry wrote on Today, 17:18:

interesting and forgive any ignorance, is the LLM that runs separately offline (eg ollama seems to have that option)? that in […]
Show full quote

interesting and forgive any ignorance, is the LLM that runs separately offline (eg ollama seems to have that option)? that in itself is interesting, takes a lot of GB and fast CPU to cope with all that. Or maybe not for this purpose.

natural language interface may be useful for those not versed in XP commands, and as it can run things that's interesting - look like diagnostic output an input for the LLM.

And i just thought od having an offline LLM on a single thread Pentium 3 😀 there's a challenge!

Hope you get some interest 😀

Thanks garry, appreciate the kind words!

Good question — yes, Ollama can run fully offline once you've downloaded
a model. No internet needed after that. The XP machine and the Ollama
host just need to see each other on the local network (plain HTTP, no
TLS — XP's TLS stack is too old anyway).

For hardware on the Ollama side, it depends on the model. Something like
llama3 8B quantized (Q4) / Qwen2.5:7b needs about 4-6 GB RAM and runs fine on any
modern CPU — even without a GPU, just slower.

BUT — if you don't want to dedicate hardware to running the LLM, Ollama
also has cloud-hosted models since v0.12. You just run:

ollama run gpt-oss:20b-cloud

and it connects to their cloud backend. Models like gpt-oss (20B and
120B), deepseek-v3.1, and qwen3-coder are available. They work through
the same OpenAI-compatible API at /v1/chat/completions, so retro-agent
works with them out of the box — just point --url at the machine running
Ollama.

There's also a free tier, so you can test the whole setup without spending
anything. Useful if you just want to try it without setting up a big model
locally.

And since retro-agent works with any OpenAI-compatible endpoint, you
could also point it at other providers (OpenRouter, Together.ai, etc.)
if you prefer.

The XP machine itself does almost nothing compute-wise — it sends JSON
over HTTP, parses the tool calls, runs the local commands, and sends back
the output. Heavy lifting is 100% on the Ollama side (local or cloud).

> And I just thought of having an offline LLM on a single thread Pentium 3
> there's a challenge!

Ha! That would be something. I've seen tinyllama running on a Raspberry
Pi at ~2 tokens/second — a Pentium III would probably manage... eventually.
Maybe one day someone will get llama.cpp cross-compiled for XP x86. Now
THAT would be the ultimate retro AI setup.

If you or anyone here wants to try it, the prebuilt binary is in the
GitHub releases — just copy the .exe over and point it at an Ollama
instance (local or cloud). No install needed.