Most “Frontier” Coding Tasks Are Useless for RL [Part 1]

As part of pre.dev Labs, we build frontier-difficulty, long-horizon reinforcement learning environments to push our coding agent to its limits. But do these complex RL tasks actually move the needle? To prove it, we trained Qwen3.8-27B on ~100 long-horizon coding tasks, moving its Terminal-Bench 3.0 score from 1.4% to 5.4%. Here’s how we did it.

TLDR; A frontier coding task is only useful for RL if the current policy can produce reward variance on it. Otherwise it isn’t a hard training example. It’s an expensive batch of zeros.

We trained Qwen3.8-27B on roughly 100 long-horizon coding tasks and moved it from 1/74 to 4/74 on Terminal-Bench 3.0. The interesting part was not the optimizer. It was figuring out which tasks could produce gradient before burning money on them.

Most “frontier” coding tasks are useless for RL.

Not because they are fake. Not because they are easy. Because the current policy cannot learn from them.

A task is only useful for RL if the model can produce reward variance on it. If every rollout gets zero, you do not have a hard training example. You have an expensive batch of zeros.

This sounds obvious, but a lot of coding RL still gets described as: generate environments, run GRPO, scale rollouts, win. GRPO came out of DeepSeekMath, and the optimizer matters. It just does not solve the data problem.

The hard part is everything before the optimizer. You need to procure real work, turn it into a clean environment, build a verifier that measures actual progress, and prove that the task sits at the model’s frontier before you spend serious GPU time on it.

We built our pipeline around that idea.

We post-trained Qwen3.8-27B on roughly 100 long-horizon coding environments from pre.dev. The broader model family is described in the Qwen3 technical report.

The base model solved 1 of 74 tasks on Terminal-Bench 3.0. After training, the same 27B model, served the same way through the same barebones Terminus-2 harness, solved 4 of 74.

That is 1.4% to 5.4%, a +4 percentage-point controlled gain.

Four tasks is still a low absolute score. Of course it is. This is Terminal-Bench 3.0 (aka Frontier Bench). Low is the point. There is no bank of easy tasks handing out free passes.

What matters is that three previously unsolved long-horizon tasks flipped after we changed the weights. We did not add a better harness. We did not give the model more tools. We did not change the benchmark parameters.

For context, GLM 5.2 + Claude Code reports 4.6% ± 1.0% on the official leaderboard, averaged over five trials per task. That is not directly rank-comparable to our single paired sweep. It is just useful grounding. A 27B model running inside Terminus-2 ended up in the same ballpark as a 744B-total, 40B-active MoE running inside Claude Code.

The claim is not that Qwen beat GLM. The claim is that the data moved Qwen.

The task is not the unit of training data

People talk about task difficulty as if it is a fixed property. It is not.

A task can be impossible for one policy, saturated for another, and useful for a third. The actual training artifact is a task-policy pair with reachable reward variance.

If every rollout gets zero, the optimizer cannot distinguish between a model that never opened the repo and one that implemented 95% of the feature but missed a concurrency invariant. If every rollout gets one, there is nothing left to teach.

Gradient lives in the middle.

This is related to the degenerate-group problem that DAPO addresses with dynamic sampling. If a group has no reward variance, keep sampling until it does. That makes sense when the task can eventually produce a useful group.

Our approach starts one level earlier. We increase the resolution of the verifier, then screen the task against the actual policy before admitting it to training.Every environment gets one of three jobs:

• If reward varies across attempts, train it with RL.
• If reward is flat but a strong demonstration scores higher, use targeted SFT to pull the policy off the floor, then probe it again.
• If reward is flat and there is no demonstrated margin, drop it.

Deleting tasks is part of the product.

GPUs are a stupid place to discover that an environment is broken, saturated, or unreachable.

From a real commit to an RL task

Every environment in this run came from real commit history inside a production codebase.

We pin the repo before and after a feature or fix that actually shipped. We rewrite the change as a behavioral specification, package the full stack, and build an independent verifier around the intended behavior.

This follows the same broad direction as SWE-RL, which learns from software-evolution history, and R2E-Gym, which treats executable environments and verifiers as the substrate for training software agents.

Our bet is that provenance, horizon, and QA matter more than people think.

These are not isolated functions with four unit tests. The training set spans multi-module backends, TypeScript monorepos, speech and media ML, blockchain systems with Rust FFI, databases, third-party integrations, payments, auth, and runtime infrastructure.

The median rollout took 50 agent steps and roughly 88K completion tokens. The longest took 802 steps and approached 288K completion tokens.

The horizon is the product.

A model can look great on a short patch and still fall apart when it has to understand an unfamiliar system, form a theory, touch several components, run experiments, interpret failures, and remember the original spec for an hour. SWE-EVO documents the same broad gap. Performance drops hard when software work expands from an isolated issue into sustained multi-file evolution.

But a real repo and a plausible ticket still do not make a good RL task.

Before an environment can train the model, we QA it for solvability, isolation, verifier alignment, leakage, reward integrity, and policy position. We prove the learning ladder before the large training batch starts.

I am leaving out the parts that create most of the defensibility: how we procure changes, construct and repair verifiers, calibrate check weights, test for reward hacks, and screen for contamination.

The general setup is not the secret. Producing clean environments consistently is.

“Failed” rollouts need to say more than failed

Binary reward is brutal at long horizons.

Imagine an 88K-token episode that understands the repo, preserves the build, implements eleven of fifteen behavioral requirements, and exercises three of five runtime paths. It misses one final integration condition.

Binary reward says zero.

That is insane information loss.

Our verifiers return a continuous reward from 0 to 1 using ordered, weighted check vectors. Early layers cover setup and build. Later layers cover behavior, integration, and runtime properties.

This is conceptually aligned with recent work on dense hierarchical rewards for code, which separates syntax, execution, functional correctness, and structure instead of compressing everything into one bit.

Our reward is fully verifier-computed. We are not using a learned reward model to judge prose or hidden reasoning. We also screen the ordered check scales for monotone coherence using Loevinger’s H in the Mokken-scale tradition.

One real rollout scored 0.6486. It passed setup 1/1, build 0/1, behavior 11/15, and runtime 3/5, for a weighted score of 24/37.

The rollout did not solve the full task. It still contained far more of the behavior we wanted than a rollout that never compiled. The optimizer should be able to see that.

This changed the economics of the entire run:

• 44 of 70 RL environments produced usable variance at G=2.
• Only seven needed G=8.
• None needed G=16.
• Under binary pass/fail grading, only 9 of 70 would have shown any reward variance in the groups we observed.

Binary reward would have thrown away 87% of the environments that made up the RL set.We did not make long rollouts cheap. We made failed rollouts useful.Shaped rewards exposed variance on 44 of 70 tasks at G=2. Binary grading would have made 87% of the corpus zero-gradient.

That wraps up Part 1. In Part 2, we’ll take a look under the hood to cover:

• Why SFT is a rescue operation, not just a phase
• The full training setup and what actually changed behind the scenes
• How catching bad signal early created our primary cost advantage
• The real verdict: What these results actually prove—and what they don't

Catch you in Part 2!

Let Us Know Your Thoughts

We want to hear from you! Join the conversation and share your feedback over on our X and LinkedIn pages.

About predev

pre.dev is built to provide professional engineering teams with hyper-intelligent and cost-efficient coding agents. These agents deliver scalable, self-verifying, and model-agnostic software development.

Professional software development requires agentic coding that deeply understands system architecture, mitigates technical debt, and respects your compute budget.