16 November 2026 · 4 min read
Cmpnd Releases DSPy-WSP: Weight-Space Programming Optimizer for DSPy
Cmpnd has open-sourced DSPy-WSP, an experimental optimizer that lets a frontier LLM edit smaller model weights without gradients, improving decision model accuracy by up to 16 points.
On 14 November 2026, AI tooling organisation Cmpnd released dspy-wsp, an open-source experimental optimizer that brings weight-space programming to the DSPy framework [1][2]. The package, published under the MIT licence, allows a frontier language model to inspect and directly modify the neural network weights of smaller target models without conventional gradient descent or backpropagation [1].
What DSPy-WSP does
Weight-space programming treats a model’s internal tensors as mutable program memory. A frontier ‘writer’ model examines the target ‘student’ model’s layers, formulates targeted tensor patches, and evaluates performance iteratively across task examples [1]. No training dataset, training loop, or GPU is required for the optimisation step; the writer uses only forward passes and a sandboxed code execution environment [2].
How it works
The system has three components [1][2]:
- Student model: the compact, open-weights neural network being optimised (tested on models from 600M to 9B parameters).
- Writer model: a frontier reasoning engine such as GPT-6-Astra or GPT-6.1-Sol, which inspects error traces and proposes weight edits.
- Sandbox: an isolated execution environment. On Linux, Bubblewrap enforces isolation; on macOS, the built-in
sandbox-execis used. The sandbox provides safe utilities for loading tensor shapes, proposing modifications, and computing statistics.
The writer operates in turns. Each turn includes a short explanation and a code block that either runs Python to read or edit weights, or executes a forward pass of the student model on practice examples. Forward passes are drawn from a fixed budget (typically 25 passes per run). When the budget is exhausted, the best-performing tensor modifications are saved as a compact patch consisting of a manifest file and compressed NumPy arrays, each indexed with SHA-256 hashes of the baseline and patched weights [1][2].
Benchmark results
Cmpnd evaluated DSPy-WSP on the Typed Decisions benchmark using five open-source decision models ranging from 600 million to 9 billion parameters, with 400 held-out test examples never seen by the writer [1][2].
| Model | Parameters | Unoptimised Accuracy | WeightSpace Gain | Stacked with ReAnchor |
|---|---|---|---|---|
| Liquid AI d1-omni-600M | 600M | 50.9% | +16.0 points (66.9%) | +18.2 points (69.1%) |
| Liquid AI d1-2B | 2.0B | 60.6% | +13.4 points (74.0%) | +17.3 points (77.9%) |
| NanoJev (Qwen3 backbone) | ~1.0B | 52.3% | +13.3 points (65.6%) | +16.8 points (69.1%) |
| Strands Decider 2B | 2.0B | 58.4% | +10.4 points (68.8%) | +20.0 points (78.4%) |
| Clef-Flash (Cloudflare) | 9.0B | 68.8% | +1.9 points (70.7%) | +6.5 points (75.3%) |
Weight-space edits compounded with ReAnchor, a complementary DSPy optimizer that adjusts decision boundary thresholds without altering weights [1]. A complete 25-pass run took between 68 and 167 minutes and cost approximately $5.42 to $10.78 in reasoning API calls [1]. In one example, optimising d1-omni-600M took 78 minutes and cost $7.63; a run on NanoJev took 167 minutes and cost $6.92 [2].
Research background
The technique builds on work from MIT CSAIL. In early November 2026, researchers Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab published Sparks of Weight-Space Programming, showing that a frontier model could edit a Qwen3.5-0.8B model’s weights to fix a degenerative reasoning loop. By modifying only 0.016% of the weights, they raised the student’s MATH-500 score from 7.9% to 48.1% [1][2]. Subsequent independent projects by Hector He (llm-honing) and Michael Isaac (GEPA) extended the idea to decision models and genetic search [1]. DSPy-WSP packages these concepts into a standard DSPy optimizer.
What it means for developers
For developers building with DSPy, WeightSpace offers a new optimisation lever alongside prompt engineering and fine-tuning. A typical workflow involves installing dspy-wsp (requires DSPy 3.5), defining a DSPy signature, and calling optimizer.compile() with a training set and a validation set [2]. The resulting compiled program runs on the locally edited weights, and the patch can be shared as a small, verifiable artefact rather than a full model checkpoint [1].
Because the writer only needs API access to a reasoning model, teams can experiment with weight-space optimisation without provisioning GPUs or curating fine-tuning datasets. The current prototype is experimental: gains are smallest on larger models (Clef-Flash 9B gained only 1.9 points from WeightSpace alone), and run times are measured in hours [1][2]. Still, the ability to ship a targeted, gradient-free edit as a verifiable patch marks a practical step toward automated parameter engineering.
Sources
- Cmpnd Launches DSPy-WSP Weight-Space Optimizer — CASRAI
- Cmpnd Releases dspy-wsp: Weight-Space Programming Comes to DSPy
- dspy
- weight-space-programming
- llm-optimization
- cmpnd
- open-source
Researched and drafted with AI from the sources above.
Learn to build with AI
Hands-on tutorials on Cursor, Claude Code, AI agents and more.