RetroPCCupboard wrote on 2026-11-01, 22:28:

I have tried using an LLM on my phone (Gemma 4) and it was useless. Context was tiny. The only positive was that it was faster t […]
Show full quote

kingcake wrote on 2026-11-01, 19:28:

Man, you are out of touch. Even phones run and accelerate local models now.

I have tried using an LLM on my phone (Gemma 4) and it was useless. Context was tiny. The only positive was that it was faster than I thought it would be.

I have also tried larger models on my RTX 3080 Ti 12Gb and was also unimpressed. Local LLMs, for me, are a waste of time.

Even the frontier models aren't perfect. I have to use copilot at work for software development. The code it generates by default is poorly structured, verbose, overly defensive, fragile and not very readable. Don't get me wrong, it is helpful when guided, but I wouldn't trust it architect a full software solution itself. I also don't trust it to do only what I asked it to do, as often it spontaneously changes things you didn't ask it to.

Ok so you don't understand how AI and prompts work at all. That explains your opinions.

liqmat wrote on 2026-11-01, 22:30:

I know nothing about the inner workings of AI LLMs etc, but I have been browsing various dev forums and it is fascinating to see the gearhead talk about hardware and the various versions of their models. Completely new language to me. People buying 10K Apple MacBook Pros with 8TB NVMe drives and 128GB of RAM to run 70B models. The newly refreshed Apple Mac Studio workstations that can climb to 30K geared with 16TB NVMe drives and 512GB RAM so they can run massive LLM models. Of course I got educated in other forums of how smaller models run lightning fast on 32GB 5090 cards. Last, but not least, I browsed through all the NVidia DGX Spark stations that you can attach as a breakout box to your current system. Gotta say, I'm lost in the wilderness in all this and the gear I see people using is certainly a rich man's game if you want powerful localized models (especially at the current prices of hardware).

Apple silicon is downright magic for AI acceleration, Apple is one of the few corporations that realised from the start that local AI is the future and has sunk billions into making it happen at a scale the general public can afford. (Not reasonably afford, I mean it is Apple)
IIRC their devices since the launch of their M series silicon has AI acceleration and NPUs built into it, their phones from the last ~4 years also have an NPU and AI acceleration.

Most android phones from the last 4 years also have this hardware for AI acceleration.

So people that assume the general public don't have the hardware for local AI ...assume wrongly, the hardware is already out there, its not Datacenter levels of capability but it doesn't need to be for local LLMs and the types of usage your average Joe is likely to want from it and for anything beyond that the cloud is there. The AI bubble popping wont change any of this at scale, Gen AI in the cloud will always be a thing now, the only thing that will change is the corporation running it.

When the pop does happen I'm expecting Apple to come out the other side mostly unscathed from it all, Microslop, Oracle and Amazon will get hit hard but I doubt they will suffer badly ..its the hardware fabs they will get hit hardest due to how deep they are into fabrication of DC parts.

My only hope is that nVidia gets hit hardest and is forced to break up and spin off parts of itself to avoid imploding, I wouldn't mind seeing Micron get the same treatment and be forced to split up.

All of this is just my own speculation and should be seen as such.

kingcake wrote on 2026-11-01, 23:20:

Ok so you don't understand how AI and prompts work at all. That explains your opinions.

I wouldn't say that. But clearly I am doing something different to you, or I have a different opinion of how good AI is at following instructions than you do.

For example in chatGPT I did a test project on my phone. Edited the project instructions to give it the exact rules I wanted it to follow for naming conventions, code structure etc. It gave me the code in zip files and I would look at it and then give a next prompt. It seemed to work fine at first, but over time it started to ignore the instructions and do its own thing. When I asked it why it ignored the instructions it's response was this:

Yes. Project instructions are supposed to apply automatically to every conversation within that project. You shouldn't have to r […]
Show full quote

Yes. Project instructions are supposed to apply automatically to every conversation within that project. You shouldn't have to repeat them in every request.

I can confirm that your project instructions are available to me in this conversation.

The problem isn't that your instructions were missing. It's that I failed to apply them consistently.

There is an important distinction between an AI having instructions available and reliably enforcing them. I can do the former, but my past work demonstrates that I haven't consistently done the latter.

On another occasion I asked it why it didnt follow instructions and it came back with something like

Oh, I am sorry, I was so distracted with trying to solve the problem that you gave me that I wasn't careful to adhere to the coding standards in the project instructions

For work we use co-pilot with an agents.md file. I try to keep chats short and focused on one problem or feature per chat.

Claude AI is not only restricted to not clone existing games, but also apparently restricting reverse engineering: Anthropic confirms new Claude Code limits to prevent game cloning — users report Opus 5.5 is also actively blocking reverse engineering tasks

Allegedly this is having unintentional impacts on legitimate projects:

In our previous report, we had already linked to recent issues raised at Claude Code's own GitHub repository, comprised of users complaining that the AI assistant was refusing to work on their games or on reverse-engineering tasks. As far as we can ascertain, it appears the game-related blocking is limited to the combination of using Claude Code and the Opus 5.5 model. Should that prove true, then users should be able to sidestep the limitation by calling upon Opus using another harness like Ollama or forwarding specific requests through a wrapper of some sort.

Regarding the blocking of reverse engineering, we have received direct confirmations. The first comes from our own alumnus Jarred Walton, who was trying to run checks on the software he's developing. He notes that such work had been performed with no issue until yesterday, October 31, and that switching to Opus 4.8 let him continue his work.

I don't like Microsoft but do like Word. Can I use AI to reverse-engineer my own custom version of Word?

https://www.youtube.com/watch?v=95bp9ehWCiU

Errius wrote on 2026-11-02, 21:25:

I don't like Microsoft but do like Word. Can I use AI to reverse-engineer my own custom version of Word?

Someone already did along with pretty much every adobe application and solidworks too.

There's a decided hissing sound....

https://abcnews.com/Business/dow-plunges-1200 … ry?id=137880888

I like the new AI made KFC yodelling song. Quite catchy

wierd_w wrote on Yesterday, 22:02:

There's a decided hissing sound....

https://abcnews.com/Business/dow-plunges-1200 … ry?id=137880888

Altman spouting off against local AI instances made me smell fear in Big AI, therefore it seemed to me that local AI is important enough to make these guys sweat. So local AI has a future. When billionaires say run, I run, but in the direction 180 degrees from what they tell me to.

Anyway, I have no illusions as to it's real thinking and reasoning capacity, but I sure could use a bunch of trained apes.

RetroPCCupboard wrote on 2026-11-01, 22:28:

I have tried using an LLM on my phone (Gemma 4) and it was useless. Context was tiny. The only positive was that it was faster t […]
Show full quote

kingcake wrote on 2026-11-01, 19:28:

Man, you are out of touch. Even phones run and accelerate local models now.

I have tried using an LLM on my phone (Gemma 4) and it was useless. Context was tiny. The only positive was that it was faster than I thought it would be.

I have also tried larger models on my RTX 3080 Ti 12Gb and was also unimpressed. Local LLMs, for me, are a waste of time.

Even the frontier models aren't perfect. I have to use copilot at work for software development. The code it generates by default is poorly structured, verbose, overly defensive, fragile and not very readable. Don't get me wrong, it is helpful when guided, but I wouldn't trust it architect a full software solution itself. I also don't trust it to do only what I asked it to do, as often it spontaneously changes things you didn't ask it to.

They do what they are directed to do though.

BitWrangler wrote on Yesterday, 22:45:

wierd_w wrote on Yesterday, 22:02:

There's a decided hissing sound....

https://abcnews.com/Business/dow-plunges-1200 … ry?id=137880888

Altman spouting off against local AI instances made me smell fear in Big AI, therefore it seemed to me that local AI is important enough to make these guys sweat. So local AI has a future. When billionaires say run, I run, but in the direction 180 degrees from what they tell me to.

Anyway, I have no illusions as to it's real thinking and reasoning capacity, but I sure could use a bunch of trained apes.

Just look at all the lobbying the AI Bros are doing against local AI models, they are trying their hardest to get local AI restricted to only the models they believe are "safe" for the general public to have. They really do not want people running local AI, its a very real threat to them because if you can run it at home why would you buy or rent it from them?

Ive been saying for a while that even if you don't believe it the current state the consumer PC hardware space is in is not an accident, its 100% deliberate to make local compute too expensive for your average Joe to own. I will admit some of it is because of a certain US president fouling up the oil markets which has a knock on effect for transport and supply lines but the vast majority of it is from the AI bros buying out any spare fabrication capacity till sometime in 2030 along with hoarding current stocks of Ram, Storage and GPUs.

Its no mistake, its not an accident its very deliberate to force us to buy our compute from them .. to sell fucking tokens because right now AI is simply not profitable.

What is highly amusing to me is watching them scurry about as the markets recoil from them ..they are scared.

Trashbytes wrote on Yesterday, 23:29:

Ive been saying for a while that even if you don't believe it the current state the consumer PC hardware space is in is not an accident, its 100% deliberate to make local compute too expensive for your average Joe to own.

Interesting take but that's too conspirational for me. My view is that it's all just supply and demand doing their thing. I guess consumer market just isn't profitable enough?
The problem is glut: Not enough capacity and very expensive/slow to get more.

The fear-mongering part is to keep VCs interested. They've been at it since 2019 at least, if not before.

EDIT: That's such a depressing scenario I really hope you're wrong...unfortunately I don't think that it can be totally ruled out.
Not having local compute would be like returning to the bad old times before microcomputers.

mr.cat wrote on Today, 12:23:

Interesting take but that's too conspirational for me. My view is that it's all just supply and demand doing their thing. I gues […]
Show full quote

Trashbytes wrote on Yesterday, 23:29:

Ive been saying for a while that even if you don't believe it the current state the consumer PC hardware space is in is not an accident, its 100% deliberate to make local compute too expensive for your average Joe to own.

Interesting take but that's too conspirational for me. My view is that it's all just supply and demand doing their thing. I guess consumer market just isn't profitable enough?
The problem is glut: Not enough capacity and very expensive/slow to get more.

The fear-mongering part is to keep VCs interested. They've been at it since 2019 at least, if not before.

People have been feeding each other the supply and demand cope for a while now even if it makes exactly no sense when looking at the intentional behaviour of the Ram Cartel, Nvidia and the AI bros, throw in an unplanned war over oil and you get what we have now.

There is no conspiracy, just corporations being the perennial assholes they have always been and a government happy to war monger over fossil fuel.

ElectroSoldier wrote on Yesterday, 23:27:

They do what they are directed to do though.

My point was that even if you give them directions they do not always follow it. Sometimes they "forget" things as the context size grows. Also they will tend to do unasked for things unless you explicitly add instructions to not do it. For example it may choose to refactor some code when it wasn't required to do the change you asked for.

There was a piece of info not often reported in connection with the "disturbance in the farce" that is the RAM industry at present which bears mentioning. The type of RAM required for application specific AI acceleration at scale is a 3D stacked high bandwidth RAM, and apparently every piece of that made displaces the production of three pieces of DDR5.

Previous disruptions like when demand spiked due to new version of windows needing more RAM, were the new product increasing interest, plus existing users upgrading to meet performance demands. Here we have that plus the effective production capacity to supply that demand being slashed by 66%. So like a windows multiple version jump, like they went from 3.1 right to XP.

No doubt energy and transport cost problems have further worsened this, like covid worsened the semiconductor supply chain problems in 2020, but that had already started also, with vehicles for example suddenly being designed with 200 microcontrollers instead of one engine computer, one transmission computer, one body module, now each thing from window actuators to the rear wiper have a uC on the system bus. So that was the demand spike before then we went into covid with that.

A lot of the helium supply used in chip making also competes with energy supply because in the main it comes out of natural gas.

Keep compute local.

ElectroSoldier wrote on Yesterday, 23:27:

They do what they are directed to do though.

This video here explains exactly what I, and others, have experienced with these frontier models. They definitely do, for some people at least, ignore instructions:

https://youtu.be/SZgyuBbMy7c?si=QGSLuqjyPr4BuSlo

From the sounds of it they are tweaking model parameters (to save money), potentially to different groups of people, to see if anyone complains.

Trashbytes wrote on Yesterday, 23:29:

Just look at all the lobbying the AI Bros are doing against local AI models, they are trying their hardest to get local AI restr […]
Show full quote

BitWrangler wrote on Yesterday, 22:45:

wierd_w wrote on Yesterday, 22:02:

There's a decided hissing sound....

https://abcnews.com/Business/dow-plunges-1200 … ry?id=137880888

Altman spouting off against local AI instances made me smell fear in Big AI, therefore it seemed to me that local AI is important enough to make these guys sweat. So local AI has a future. When billionaires say run, I run, but in the direction 180 degrees from what they tell me to.

Anyway, I have no illusions as to it's real thinking and reasoning capacity, but I sure could use a bunch of trained apes.

Just look at all the lobbying the AI Bros are doing against local AI models, they are trying their hardest to get local AI restricted to only the models they believe are "safe" for the general public to have. They really do not want people running local AI, its a very real threat to them because if you can run it at home why would you buy or rent it from them?

Ive been saying for a while that even if you don't believe it the current state the consumer PC hardware space is in is not an accident, its 100% deliberate to make local compute too expensive for your average Joe to own. I will admit some of it is because of a certain US president fouling up the oil markets which has a knock on effect for transport and supply lines but the vast majority of it is from the AI bros buying out any spare fabrication capacity till sometime in 2030 along with hoarding current stocks of Ram, Storage and GPUs.

Its no mistake, its not an accident its very deliberate to force us to buy our compute from them .. to sell fucking tokens because right now AI is simply not profitable.

What is highly amusing to me is watching them scurry about as the markets recoil from them ..they are scared.

Not that youre wrong, in fact youre probably right but once you understand what an agent can do, and is actually capable of once free then I would imagine you would also want similar safeguards once you know who is asking them to do things.