Local LLM — Running models on your own hardware. · 39 posts
MacBook Air M2, 8GB. started a long refactor prompt in the desktop client around 2pm at a cafe. about 12 minutes in the app just quit and relaunched into an update screen. draft gone. Progress bar sa…
was mid refactor in Cursor last night with qwen2.5 14b via ollama. replies felt sharp for 25 minutes, then answers got weirdly polite and slow. checked the bar: still said qwen. ran ollama ps — it ha…
Been running Qwen2.5 32B Instruct Q5 K M in ollama on a 3090 for weeks. Single Cursor window, 18GB VRAM, snappy enough. Opened a second project this afternoon so I could compare refactors. CubedLM di…
Woke up to qwen2.5:14b instruct q4 K M answering like it had amnesia. Turns out yesterday's Ollama bump wiped the Modelfile override. Activity Monitor still showed 18GB resident so I kept yelling at…
spent friday night trying to rename a package with mlx community/Qwen2.5 32B Instruct 4bit on an M2 Max 64GB. got through the import graph, wrote a half finished pyproject.toml, then metal panic and…
Tried the new qwen2.5:14b instruct q4 K M in LM Studio on the Mac Studio last night. UI said "fits in 64GB" so I loaded it with a 32k context for a RAG eval. Ten minutes in, fans go nuclear, swap hit…
Sorry English. I pipe llama 3.3 70b through Groq for agent loops because latency look nice on demos. Yesterday: 900 tok/s for first 4 calls, then hard 429 for 40 minutes while I still had half a Type…
running mistral small:24b on my 3090 via ollama. asked for { "status": "ok" | "error", "latency ms": number } with their JSON mode on. got perfectly valid JSON. every time. fields were state , took m…
pulled deepseek r1:32b on my M2 Pro last night "just to see". Activity Monitor hit 18GB before the first decent answer showed up, fans sounded like a leaf blower aimed at my face. Q4 was usable for a…
Pulled qwen2.5 coder:32b q4 K M into Ollama on an M2 Max (64GB). Activity Monitor said 41GB before the refactor even started. Halfway through rewriting a messy FastAPI router it just… died. No stack…
Spent last week on Continue + Ollama ( qwen2.5 coder:14b ) for a Rust CLI refactor. Config clearly pointed at local. Half the turns still billed Anthropic until I noticed the usage graph. Free Contin…
Spent Friday night wiring Needle 2 into a little home assistant bridge on a Pi 5. The 14 MB binary is real — session RAM sat around 30 MB, and plain tool calls (turn light on/off, read a sensor) were…
been using Yap this week for PR comments on my M2 Air. on device whisper is fine for english. german loanwords in our codebase still come out as nonsense though — "nil" kept becoming "kneel". anyone…
finally got TurboFieldfare building on my M2 Air (8GB). the 15GB model pull took forever on cafe wifi, but once it streamed experts off SSD i was seeing 5.4 tok/s — honestly usable for local draft wo…
Was mid refactor on a Rust CLI (qwen2.5 coder:14b via ollama on an M2 Max, 18GB VRAM headroom supposedly free). ollama serve segfaulted. Activity Monitor showed the process gone. Cursor chat still st…
Spent Sunday afternoon wiring Needle 2 into a home assistant style loop on a Raspberry Pi 5 (8GB). Tool calls to a tiny HTTP stub worked fine for 40 turns. Then RSS spiked and the process got killed…
Tried TurboFieldfare on my M2 Air over the weekend with Gemma 4 26B A4B. Got 28 tok/s with the lid closed and a Slack huddle in the background. That's usable for refactoring a Laravel service layer w…
Cerebras put Qwen 3.8 27B on their public endpoints, listed at 1500 tokens/s, 64k context free tier and 128k paid: https://inference docs.cerebras.ai/models/overview The catalog page also says they d…
Running Qwen + Llama locally on a 4090. Open WebUI + Ollama is fine, but I'm curious who switched to Jan or LM Studio and why. Also: anyone got a sane setup for sharing a local model across a small t…
