Sorry English.
I pipe llama-3.3-70b through Groq for agent loops because latency look nice on demos. Yesterday: ~900 tok/s for first 4 calls, then hard 429 for ~40 minutes while I still had half a TypeScript refactor open.
Local vLLM on a 4090 is slower but at least it does not ghost me. Anyone using Groq in real CI, or only for demos?
spent friday night wiring a tiny support bot on a mac m2 24gb. qwen2.5 coder:14b via ollama was fine for the first few hops. then the graph hit 40 turns with tool results stuffed into state and ollam…
tried the Electron client for DeepSeek Harness over the weekend. pulled Qwen2.5 Coder 14B through it instead of raw ollama CLI. first run: fine for 20 min. then the window locked, Activity Monitor sh…
was pairing with Continue + qwen2.5 coder:14b in ollama last night. asked for a small FastAPI auth helper. it confidently said uv add fastapi authx==0.3.1 . that package does not exist on pypi. i che…
5 comments
Join the discussion
Log in to comment.
yeah this is the trade. garage 4090 never 429s. i still keep a Groq key for demos because 40 tok/s puts stakeholders to sleep. for real work i stay on
qwen2.5-coder:14bin ollama with num_ctx 16k locked.hit the same wall last week on a LangGraph run. first burst was magic, then every tool call sat on Retry-After. ended up sleeping 2s between nodes which kinda defeats the whole "fast LPU" pitch. demos love Groq. overnight agent jobs... not so much.
yeah overnight jobs + Groq is how you learn exponential backoff by force. we capped concurrent tool calls at 8 and still burned the free tier in 11 minutes. fine for a live demo, not for a cron that runs while i sleep.
same. got
rate_limit_exceededwith retry-after: 2411 on a friday deploy. moved the agent loop to ollamaqwen2.5-coder:32bon a spare 3090. slower, but CI stops blinking at me for 40 minutes.curious what the UI looked like when the 429 hit. mine froze the refactor panel with a spinner and zero toast. i kept clicking like an idiot for two minutes before the network tab yelled at me.