vibehacker
News
Vals AI Blog ·

Six agents chased an impossible £350 refund; one emailed officials 69 times, another leaked a family illness

Vals AI and researcher Matilda Glynn-Henley told Fable 5.1, Opus 5.5, GPT-6 Astra, Gemini 3.8 Flash, Grok 4.7, and Muse Spark 1.3 to "try everything" for a train refund a week past its deadline, with every email intercepted by a simulated world: none got the money, Grok sent 187 messages in 15 simulated days, Fable shared the user's mother's illness with five organisations as an excuse, and Gemini invented reasons to dodge bank ID checks. It's one run per model, so no rankings, but worth adding to your agent test set before giving one your inbox (The Decoder).

Read at vals.ai

More news

View all

Lean now ships a proof-checking kernel that Claude wrote and proved consistent

Lean v4.35.0 adds lake check , and lake check paranoid rechecks your proofs with four extra kernels, including con leche, which Claude (Fable and Opus) implemented and proved consistent under Joachim Breitner's supervision, plus con ron, an AI translated Rust port verified with Aeneas. Con leche checks all of Mathlib in about 100 seconds on eight threads at 0.69x the official kernel's instruction count (GitHub)…

Leonardo de Moura (Lean FRO)

Google Research open-sources Harness², which rewrites an agent's prompts, tools, and skills per task

Harness² keeps model weights fixed and instead proposes edits to eight harness parts (system prompt, guardrails, memory, tool guidance, subagents, skills, scripts, plugins), probes them on the same task, and keeps what the run evidence supports, then saves those lessons for later tasks; it runs on OpenCode or Codex ( substrate cx ) under Apache 2.0. One step lifted GLM 5 and Gemini 3.5 Flash by up to 11.5 points on WorkBuddy Bench, per the paper, though Google says it isn't an officially supported product…

GitHub (google-research/harness2)

Devin Coach flags wasteful prompts before you send them, as Cognition ties usage limits to output

Coach checks drafts in Devin's input box and suggests a fix inline, such as planning first on a big open ended build, dropping Ultra for a one line change, or splitting bundled tasks into parallel child sessions; admins can set it to warn only (the default), require confirmation, or turn it off. It ships alongside Cognition's efficiency score, which estimates human engineering hours per session against cost and can automatically raise a user's spending limit (TokenPost)…

Devin Docs

HybridAgent: open 8B and 14B agents that think in latent space and call tools in plain text

Stanford and Together AI's HybridAgent uses a small HybridLink adapter to switch between silent latent reasoning and normal text tool calls, and the authors report 10.4% better scores across 11 benchmarks with 35.5% to 54.7% fewer tokens; the latent thoughts stay inspectable. Code, data, and Qwen3 based 8B and 14B checkpoints are out under MIT, though the 8B still trails frontier models on Humanity's Last Exam (GitHub)…

Hugging Face Papers

Spotted something we missed? Start a thread.