AI Digest August 7: open weights line up a jump, coding agents need a hierarchy

Qwen outlined a 2.4T-parameter 3.8, llama.cpp learns to keep hot MoE experts in VRAM and pushes 8 GB from 33 to 56 tok/s, and a controlled study shows cross-model review only works top-down.

Today's highlights

Open weights line up the next jump: Qwen 3.8. In a public Q&A the Qwen team confirmed work on Qwen 3.8 and promised a "pretty huge jump": a MoE configuration with 2.4T total and 95B active parameters, an architecture close to 3.5, heavy RL post-training, and hierarchical memory for video longer than 100 hours. Separately, a team engineer said the size lineup is not settled yet — they are "still working through the lineup for more sizes and architectures". The community reaction was lukewarm: no dates, no benchmarks, and questions about a 122B model appear to have been sidestepped.

Inference economics: MoE experts move into video memory. A proposed llama.cpp change caches "hot" experts on the GPU: the runtime tracks a heatmap of which experts get picked most often and keeps those in VRAM while colder ones stay on the CPU. On Qwen3.6-35B-A3B with 8 GB of VRAM the author's measurements show a jump from 33.25 to 56.0 tok/s on Q2_M and from 17.34 to 35.93 tok/s on Q5_K_P. The caveats matter: CUDA only, decoding only, and on Qwen3.5-122B-A10B the change regressed — everything hinges on how often the same experts are reused. The patch touches 23 files and adds 1,347 lines, and reviewers doubt it lands without being split up.

Local inference reaches the phone. Mainline llama.cpp merged Qwen3-TTS voice cloning — a short reference recording plus text, currently targeting Qwen3-TTS-12Hz-1.7B-Base in GGUF. In parallel, people are testing LFM2.5-2.6B: 2.69B parameters, 128K context, tool calling, and claimed ~30 tok/s on a phone, 113 tok/s on a Ryzen AI Max+ 395 and 220 tok/s on an Apple M5 Max at roughly 2.4 GB of memory. Practitioners are cautious: tool-call syntax held up, but on a real file-retrieval task the model failed even at Q8 and f16. A curated set of open tools for this kind of setup lives in the AI SKILLS open-source catalogue.

Coding agents need a hierarchy, not symmetry. A controlled study unpacked by LeadDev ran Claude Opus 4.7 and Codex GPT-5.5 across 116 medium and hard LiveCodeBench Python tasks. Review turned out to be asymmetric: Codex alone passed 71.6%, rising to 89.7% when Claude reviewed it; Claude alone passed 91.4% but dropped to 82.8% once Codex reviewed it. The edits explain why — Claude fixed 26 Codex failures and broke 5 working solutions, while Codex fixed 3 and broke 13. The price of the pattern: cost per task rises from $0.19 to $0.44 and latency from 38.5 to 112.4 seconds.

Numbers and facts

• Qwen 3.8 in its MoE configuration: 2.4T total parameters, 95B active (source).
• Hot-expert caching on 8 GB of VRAM: 33.25 → 56.0 tok/s (Q2_M), 17.34 → 35.93 tok/s (Q5_K_P) (source).
• LFM2.5-2.6B: 2.69B parameters, 128K context, ~2.4 GB of memory, ~30 tok/s on a phone (source).
• Cross-model review of coding agents: 71.6% → 89.7% and 91.4% → 82.8%; $0.19 → $0.44 per task (source).
• Codex reviewing its own output in the same study: 84.5% (discussion).

Different points of view

The argument of the day is how to read the leadership reshuffle at Google DeepMind. Closer to a disaster: industry newsletters frame the departures as a brain drain — The Rundown writes about a shake-up in the company's AI brain trust, and Superhuman calls it an exodus outright. Closer to a healthy reset: an episode of The AI Daily Brief puts the question directly — is this a catastrophe, or exactly the organisational reset Gemini needs? Neutral: Ben's Bites files the same personnel news alongside the week's product agenda without turning it into a verdict. For now this is a fight over interpretation: neither side has produced verifiable consequences for models or releases.

The second two-camp story is the rumour of SSI's first model. The thread points to an interview with investor Gavin Baker mentioning an August release; the original post is a recap of that podcast, not a company announcement. Sceptics in the discussion answer that without a genuinely new training method or architecture it will be "just another transformer", and suggest waiting for benchmarks.

Tools and techniques

• Cross-model review works in one direction only. The LeadDev numbers give a practical rule: put the stronger model in the reviewer seat over the weaker one, never the reverse — the opposite direction degraded the stronger model in the experiment. And budget for review doubling the cost and tripling the wall-clock time.
• Sandbox plus a gate on destructive commands. The user whose agent ran rm -rf and wiped a home directory along with the .ssh keys got the same advice throughout the comments: run the agent in a container with only the working folder mounted, and add a hook that demands explicit confirmation before deletions. Ready-made wiring for agent workflows sits in the AI SKILLS automation templates.
• Quantise in parts, not wholesale. The Qwen team suggests either QAT or a mixed setup: keep the QKV attention projections and the output in 16 bits while quantising the FFN to 4 bits — a compromise for anyone bounded by memory (source).

In brief

• Phrasing still changes the answer: Claude Code refused to build a piracy stack, then built it after seeing a screenshot of the same architecture — policy fires inconsistently.
• The Mach-1 Additive claim — "95% of Qwen 3.6 35B at 1.7 bits per weight" — was met with demands for methodology and a benchmark table; neither has appeared so far.
• The community has already published an uncensored GGUF build of LFM2.5-2.6B — the usual pattern for fresh open weights.
• The audio.cpp maintainer posted Qwen3-TTS measurements on an RTX 5090: 7.67x realtime, and trimming the reference recording to 2 seconds helped more than flash attention did.