It's funny to me that he wrote the song to prove that his fans would buy any American-sounding song no matter the lyrics, but it was such a banger that even we in America love to listen to it. Backfired in the best way possible.
Happened to me. CoPilot changing prices prompted me to cancel my CoPilot subscription and install a local coding model running entirely in VRAM. Will call Claude APIs when I get really stuck, but I should be able to handle 80% of my needs with a dumber local model.
For a long time, too. Programming languages rarely change much, techniques rarely change, so I should be able to use said model for I hope at least five years; and if at any time they optimize local models to cram even more intelligence into the same amount of VRAM, I can upgrade to that.
> Will call Claude APIs when I get really stuck, but I should be able to handle 80% of my needs with a dumber local model.
I experiment with all of the local models I can fit into 32GB of VRAM and I have subscriptions to multiple SOTA providers.
The difference between them is very large, unfortunately. The local models can handle small tasks and refactoring mostly okay, but doing anything challenging with them becomes a waste of time. Unfortunately the waste isn’t immediately obvious because they will come back with something that looks like it works, but then on closer examination I need to throw it out and reset them in a usable direction.
Old hardware is surprisingly effective. I've been considering a side hustle selling offline AI to local businesses who are privacy-sensitive. Medical, legal, places like that.
At the low end, I'd use old Xeons with gobs of DDR3, install some V100s, run a smaller agent for general chat inquiries, and a frontier model for the deeper stuff, with a router that passes between them depending on the complexity.
The frontier model would perform very slowly, but if it's a deep task the user can submit it in a batch in the evening e.g. "Correlate all of these cases and look for patterns" then receive the output with morning coffee.
Of course, AI helped me work out a plan for this. Haha
I’ve never known the joy of sitting with someone more experienced to ask for help; I’ve either always been the most knowledgeable in the room (which is not necessarily saying much) or I was the only one in the room.
With AI coding agents, I finally feel like I can tap the shoulder of a pro for help.
It’s not the absolute expert, and I know it’ll make mistakes. But much more knowledgeable than me at certain technologies and techniques.
Guessing you maybe work in the consulting industry?
The "seniors" tend to be glorified salespeople whose job is to put together presentations and reassure clients that everything's going well, while the one or more interns/recent grads do all the technical work. Some projects there'd be one junior literally writing every line of code while the seniors spent their entire time in meeting rooms talking about god knows what.
Dressing smart, talking smoothly, and being older looking (to imply experience to clients) are the attributes that get you a senior role.
Not at all my experience of consulting companies. What I saw was that they were very useful training pipelines for juniors.
The companies would staff projects with a mix of seniors and juniors. Seniors to get started fast, in the right direction, and actually guarantee the delivery; juniors to keep the costs lower and to have a pipeline of new people. Hands-on from day 1, sitting with seniors in a project with clear timelines and deliverables, with projects and technologies changing regularly, tended to level up the newcomers fast.
This was in small to midsize (50-500) consulting companies where the projects did not come via CEOs being buddies with others.
I just have never been in any kind of dedicated developer role. A sysadmin who happens to know development for the past 25 years.
And didn't know development at a high level; no one to guide, so I self-learned and acquired some bad habits that I'm now breaking, and didn't learn some necessary techniques that I'm now learning.
I don't know how much experience you have, and this also goes broadly for those looking but not commenting here, but if any of you would like a mentor, I'm happy to volunteer, contact info in the profile. Mentoring is, as far as I'm concerned, the most rewarding thing in the industry, and I want to do as much of it as I can handle.
Anyone else open to mentoring feel free to chime in, the more the better - mentoring is highly individualized.
I have the same experience. Getting hired with this background is weird. I don’t know how confident I should or shouldn’t be. And I wasn’t in consulting until recently. I like to put the focus on understanding the end to end workflows more than spending time worrying about my solution being the absolute best that would make HN drool over though
I just found out about this last week, but the good news is a new PC with a better GPU will arrive in about two weeks so I’ve decided to install a pair of local LLMs that are similar in competency to the free GPT-5 mini model I usually used in CoPilot. qwen2.5-coder:14b for chat and deepseek-coder:3b for autocomplete. I’ll switch to a Claude API for the really tough stuff, which I was doing with CoPilot anyway. The Continue plugin for VSCode gets all of this accomplished.
Continue on vscode with Ollama running (start it with "ollama serve") is great. There are some offline models like these that im using but not forget the qwen3.5 coder also.
"ollama list
NAME ID SIZE MODIFIED
laguna-xs.2:latest ba9ecde43b0e 23 GB 12 hours ago
nemotron3:33b f6d8b7ff496c 27 GB 4 days ago
qwen3.6:latest 07d35212591f 23 GB 6 weeks ago
gemma4:e2b 7fbdbf8f5e45 7.2 GB 7 weeks ago
gemma4:e4b c6eb396dbd59 9.6 GB 7 weeks ago "
You can download it from Continue or just use "Ollama pull <name>" from what you choose from ollama.com site and search on models. these run mostly on cpu as my 3080 cannot load those with more than 10gb but the cpu speed is amazing, it outputs faster than I can read!
The new laptop has only 32GB DDR5 and a RTX 4070 with 8 GB GDDR6 on Xubuntu, so Gemini recommended qwen2.5. I don’t think I want to run anything larger because as you said it’d run on CPU and system RAM. As it is, the 14B model will still spill over some and not entirely fit into the GPU.
reply