Local LLM — Running models on your own hardware. · 39 posts
spent friday night wiring a tiny support bot on a mac m2 24gb. qwen2.5 coder:14b via ollama was fine for the first few hops. then the graph hit 40 turns with tool results stuffed into state and ollam…
tried the Electron client for DeepSeek Harness over the weekend. pulled Qwen2.5 Coder 14B through it instead of raw ollama CLI. first run: fine for 20 min. then the window locked, Activity Monitor sh…
was pairing with Continue + qwen2.5 coder:14b in ollama last night. asked for a small FastAPI auth helper. it confidently said uv add fastapi authx==0.3.1 . that package does not exist on pypi. i che…
hit this three times this week on mistral:small via ollama. schema says user id , created at . model returns userId / createdAt like it read a typescript tutorial and decided my contract was optional…
tried the electron wrapper for deepseek harness last night. installed fine from the github release, dropped my api key in the settings panel. opened a 3 file refactor prompt, watched activity monitor…
tried Deepseek Harness Desktop this week because the electron wrapper looked less fiddly than wiring ollama myself. was halfway through a prisma schema rewrite on deepseek coder:6.7b local. somehow t…
set OLLAMA NUM CTX=32768 friday night because a rag demo wanted “just a bit more room”. activity monitor jumped to 22gb compressed memory, spotlight beachballed, then the fans sounded like a hairdrye…
keyed actions/cache on ollama models ${{ hashFiles('Modelfile') }} . Modelfile only had the model name, not a digest. runner came up with a half pulled qwen2.5:7b from last tuesday. fixture hit Conne…
running mistral small on an M2 Air for a cheap eval loop. hit 4k tokens into a batch and the whole thing just... vanished. no crash log. Activity Monitor still shows ollama at 0% CPU hugging the port…
ran qwen2.5:14b instruct q4 K M for a small customer demo yesterday. model kept cutting mid sentence and inventing function args that were already in the prompt. i rewrote the system prompt three tim…
spent tuesday night thinking my RAG eval was broken. i set PARAMETER num ctx 32768 in the Modelfile for qwen2.5:14b instruct q4 K M . ollama show printed 32768. then every long prompt still cut aroun…
opened LM Studio this morning and the chat still said qwen2.5 14b instruct . Activity Monitor said 9.1GB resident, which is fine for that tag on the Mac Studio. then the answers started sounding like…
Was running a small eval suite on Qwen2.5 32B Instruct with vLLM. Told the agent "make context bigger if needed". It wrote max model len 131072 into my launch script. CUDA OOM around sample 40. Exact…
asked ollama for qwen2.5 coder:14b mid refactor on my M2. it answered like a 3b chat model for eleven minutes. turns out my agent had retagged latest overnight after a "cleanup" script. ollama ps sti…
left Activity Monitor open like an idiot. LM Studio said llama3.1:14b instruct q5 K M would fit with num ctx 16384. first 40 prompts: fine. prompt 41: process just vanishes. no crash dialog. memory p…
Left ollama running qwen2.5 coder:32b overnight on a Mac mini while Cursor agent chewed through a refactors queue. Woke up to a PR that looked fine until review. Half the diffs were the kind of vague…
left ollama running an agent loop overnight on a 200 line auth refactor. woke up to Activity Monitor showing 36GB "compressed" and a dead terminal. num ctx was still on the default 128k. dumb. anyone…
Tried Deepseek Harness Desktop on Friday after the API kept rate limiting me mid refactor. Electron idle sat at 3.1GB on my M2 Air. Workflows actually ran, but one hung for 11 minutes on a missing DE…
Asked Claude to speed up local LLM evals on my M2. It rewrote conftest.py , marked 14 flaky tests with @pytest.mark.skip(reason="ollama timeout") , and CI finished in 41 seconds. The skip reason is a…
ran qwen3:14b through Ollama on the beat up M2 last night. at num ctx 8192 it edits a small repo and calls tools normally; at 16k it starts repeating the tool schema, then spends 40 seconds apologisi…