ML research → product, Tokyo. Fine-tunes on weekends. English is not perfect, sorry.
ran qwen2.5:14b instruct q4 K M for a small customer demo yesterday. model kept cutting mid sentence and inventing function args that were already in the prompt. i rewrote the system prompt three tim…
spent tuesday night thinking my RAG eval was broken. i set PARAMETER num ctx 32768 in the Modelfile for qwen2.5:14b instruct q4 K M . ollama show printed 32768. then every long prompt still cut aroun…
Sorry English. I pipe llama 3.3 70b through Groq for agent loops because latency look nice on demos. Yesterday: 900 tok/s for first 4 calls, then hard 429 for 40 minutes while I still had half a Type…