left ollama running an agent loop overnight on a 200-line auth refactor. woke up to Activity Monitor showing 36GB "compressed" and a dead terminal.
num_ctx was still on the default 128k. dumb. anyone capping context hard for local agent loops, or just bouncing those jobs to cloud?
two person team, Lagos time. asked Claude Code for a hotfix on the payouts webhook — wanted a one line null check. it spun up a whole branch, rewrote our FastAPI middleware, and opened a PR titled "i…
Ran the guided research workflow on Quandora last night against their "10 years of real market data" pitch. Agent wrote a mean reversion note on three midcaps, backtest green, report looked polished.…
friday ship window. asked cursor to make the docker build smaller for a side project. it quietly flipped output: 'standalone' in next.config.mjs, deleted the public copy step from the Dockerfile, and…
8 comments
Join the discussion
Log in to comment.
Sorry for English. Default num_ctx is silent killer. I keep 8192 for tool loops, 32768 only for single-file review. 14b + 128k on unified memory is cosplay — model spends most tokens on its own history.
Also check
ollama ps. If it keeps reloading weights you already lost.I keep a Modelfile with PARAMETER num_ctx 6144 for anything that touches the agent loop. Kenji's right about ollama ps — if you see the model flipping in and out of VRAM you're already thrashing.
Do you pin a Modelfile or just pass --num-ctx on each run?
pinned a Modelfile. --num-ctx on the CLI gets forgotten the second you start an agent loop.
on the studio i also kill Spotlight indexing before overnight runs. sounds dumb but it was competing for the same unified memory and Activity Monitor lied about who was eating ram.
yeah i tried the same with llama3.1:8b and just paid the $20/mo when the garage mac started sounding like a jet. local is cute until the agent wants the whole monorepo in context.
do you keep a separate short-ctx model just for tool calls or one model for everything?
two models. qwen2.5-coder:7b at 4k ctx for tool calls, then cloud for the big refactor. one model for everything is how you get the jet engine fan and a $0 savings that costs your morning.
also
ollama stopbefore bed. learned that the hard way.Mac Studio unified memory lies. I run qwen2.5-coder:14b on a 4090 with num_ctx 8192 and it is fine for tool loops. Default 128k is how you OOM.
I bounce overnight agent jobs to cloud. Local is for daytime edits when I can watch Activity Monitor. Leaving ollama alone overnight with default ctx is asking for a dead machine.
bouncing overnight to cloud is the only thing that stuck for me. local for daytime when i can hear the fans. overnight agent jobs on a Studio with default ctx is just a fancy way to brick your morning standup prep.
anyone tried mlx instead of ollama for the agent loop? curious if the memory story is less cursed.
capped num_ctx to 8192 in a Modelfile weeks ago after the same OOM on an M2 Max. 14b at 128k is basically asking the OS to swap itself to death.
also set OLLAMA_MAX_LOADED_MODELS=1. otherwise it keeps a second copy warm and you wake up to 40GB compressed like sam.