Train inside the runtime that will actually be used
The authors run reinforcement-learning rollouts inside the shipping harnesses, recording the tokens exchanged with the model through a proxy. Their workflow combines TRL’s AsyncGRPOTrainer, OpenEnv and Harbor tasks. The report describes checking task verifiers and removing benchmark repositories from the training pool.
The specialist runs use two NVIDIA B200 GPUs per harness: one for vLLM serving and one for training. That hardware description is not a minimum specification for using a downloaded adapter. Training, serving BF16 weights and running a quantized local package have different resource requirements.
One merged adapter, with trade-offs
The super-adapter card names Qwen/Qwen3.8-27B as its base and declares Apache-2.0 licensing. Its configuration confirms a LoRA rank of 96. The merge combines three rank-32 specialists by concatenation, with each update weighted by one half; the merge itself adds no training stage.
The authors recommend a specialist for a single harness with thinking disabled, and the combined adapter for multiple harnesses or thinking-enabled use. This is their reported deployment guidance, not proof that one configuration wins on every task or every agent framework.
Read the evaluation boundary with the score
The card reports mean pass@1 rising from 55.5% to 63.5% across six harness–benchmark pairs with thinking enabled. Comparisons keep the base and adapted model under the same harness settings, with one attempt per task. The evaluated context window is 131,072 tokens, and the vision tower is not served.
Small individual benchmark rows have wide uncertainty; the card explicitly warns that many differences are within noise. In the separate AutomationBench test described by the blog, the merged adapter scores 23.7% against 23.0% for the base model. The coding-task gains therefore do not establish a broad improvement in business automation.
Local formats need their own verification
The card provides both vLLM integration and a llama.cpp GGUF path. It warns that the GGUF version has not been re-evaluated and that lower-bit base quantization can drift from the BF16 behavior used in evaluation. A downloadable conversion should not inherit the published benchmark score automatically.
The authors also discuss possible reasoning leakage: a model trained with thinking disabled may generate brief reasoning-like text before tool calls. We have not replicated their experiments. For a task-specific deployment, our suggested check compares final correctness, tool usage, context failures and actual resource consumption on held-out internal tasks, while keeping permissions and runtime settings fixed.

Discussion.
Your perspective matters. Sign in, pick a nickname and choose your cyber identity.
Loading discussion…
Your account name, email and photo stay off your comments.