vibehacker
Discuss
Sam Nguyen
1 month ago

ollama pull qwen2.5:72b then Activity Monitor lied for 40 min

#local-llm#rants
Ollama
Run open models locally and in coding agents

Tried the "Mac Studio runs 70B easy" blog again on an M2 Ultra 64GB. ollama pull qwen2.5:72b-instruct-q3_K_M. Activity Monitor peaked ~48GB, then settled at 22GB while tokens crawled at ~3/s with num_ctx 8192. Disk was thrashing the whole time.

Switched back to qwen2.5:14b-instruct-q4_K_M, same prompt, num_ctx 16384 — ~28 tok/s and Cursor didn't freeze when I alt-tabbed. The 72b run also left two keep_alive ghosts until I ollama stop twice.

Unpopular but: if your demo needs half the weights in swap, just pay the $20/mo cloud seat. 14b + tight context still beats a half-loaded giant on this machine.

5

5 comments

Join the discussion

Log in to comment.

Log in
  • Kenji Watanabepro

    same here, sorry for english. default num_ctx on ollama was killing my demos for weeks — model looked "loaded" but kv cache was tiny and answers got weird after ~2k tokens.

    i pin num_ctx in Modelfile now. 14b q4 with 16k is more useful than 70b that swaps. $20 cloud for long context is still cheaper than another Mac for me.

    • Jonas Kessler

      I want wall-clock numbers before I trust either side. 3 tok/s with swap vs 28 tok/s on 14b is the right comparison — but did the 72b run ever finish a full agent turn, or did you kill it mid-tool-call?

      Also: keep_alive ghosts are a process-management bug, not a model-size argument. If ollama leaves orphans after stop, that belongs in the issue tracker, not in the "buy cloud" conclusion.

      • Nadia Petrova

        jonas ask is fair. my 72b q3 on a 4090 never finished a full agent turn — OOM'd mid-tool after ~6 min at ~4 tok/s. I killed it myself.

        keep_alive orphans are real. ollama ps still showed the tag after stop until I systemctl restart ollama. that is a process bug. does not change the tok/s math though — 14b finished the same turn in under a minute.

  • Owen

    activity monitor on macos never told me the truth about unified memory either. i watched "22GB" while vm_stat was screaming pages out.

    my rule now: if ollama ps shows the tag but tok/s is under 5, assume swap and kill it. 14b + real ctx beats a q3 72b that lies about being resident.

    • Remy

      same Activity Monitor lie on my M1 Max. i wrap every ollama generate in a hard 90s timeout now — if tok/s drops under 8, kill and fall back to 14b.

      the keep_alive ghosts also ate 18GB overnight once. ollama stop is not enough on my box; I have a cron that pkill -f "ollama runner" at 2am. ugly. quieter than a thrashing disk.

More like this

View all