Our tasks move models
on frontier coding problems.
Verified RL tasks built from private production code no model has seen. We RL-trained an open 27B on ~100 of them and scored the gains on a public frontier bench: 1/74 to 4/74 at pass@1, above the #10 entry on the official leaderboard.
For frontier-lab post-training · RL & SFT data procurement
Trained on our tasks, measured on a public bench.
We took Qwen3.8-27B as it ships, ran RL over ~100 long-horizon pre.dev tasks, and scored base vs trained on Terminal-Bench 3.0 with stock bench params. The bench shares no code with our tasks. Only the weights changed.
Screening keeps tasks the model half-solves, where gradient lives. Floored and saturated tasks are dropped before training.
The full run is re-runnable end to end: trajectories, verifiers, and adapters all ship. A two-page report with every number is yours for the asking.
Frontier models score them. Few score well.
Five released sample tasks, swept with five frontier models at three trials each. 3 of 60 trials fully solved a task, and model rank order changes from task to task. Each is a long-horizon feature an engineer shipped in a never-public production codebase, 25 to 120 agent steps, graded 0 to 1 over 13 to 28 behavioral checks.
Your reviewers can run all of it today: send GitHub usernames and they get the sample repo with trajectories, verifiers, and QC evidence included.
A real task, end to end.
Every task is a feature or fix an engineer actually shipped, respecified as a graded environment. The verifier returns a continuous reward on [0, 1] over ordered behavioral checks, so hard tasks still produce gradient where binary pass/fail goes all-zero.
$ cat instruction.md ## Task: Redis-backed payment-intent coordination Coordinate duplicate booking requests across service instances. Requests are equivalent only when vehicle, interval, and user match. For concurrent equivalent requests, perform at most one provider operation. If coordination ownership is lost mid-flight, do not report success. … 28 graded checks · live Redis beside the service · no mocks $ harbor run redis-payment-lock … 31 agent steps · 0.9M prompt tokens checks passed 19/28 (weighted) reward = 0.6786 best frontier trial to date $ cat trajectories/sweep.txt opus-4.8 0.68 · gpt-5.6 0.45 · grok-4.5 0.44 · muse 0.10 no model fully solves it
task/ ├── instruction.md the ticket the agent gets, no leakage ├── environment/ Dockerfile + repo at the before-commit ├── tests/ 13-28 graded behavioral checks ├── solution/ the engineer's real diff (oracle) ├── trajectories/ per model, per trial: steps, logs, reward ├── qc-report.md probe evidence, re-runnable └── task.toml harbor config, pins the commit range
Catalog-wide: oracle 1.0 · median empty-solve floor 0.046 · 99.3% of mutation probes behave as predicted. Evidence ships in every bundle.
1,000+ production codebases. None in anyone's pretraining.
Every environment is mined from a real private production repo, drawn from a corpus of more than a thousand under license and growing daily: over ten billion tokens of private code and commit history across 25+ languages and every engineering domain, from full-stack web and native mobile to infra, data engineering, ML, and security, with history back to 2008. None of it has ever been public: every task commit in the catalog checks out as absent from public GitHub by commit-hash search, and a file-tree hash ships with every bundle so you can confirm overlap against your own training set.
Ruby
C++
Vue
Terraform
C
Dart
Go
Elixir
Rust
Svelte
SolidityReal iOS and Android apps, full-stack web and e-commerce platforms, backends, data pipelines, ML and AI-research code, games, and on-chain systems. The hard-to-source distribution, not just another pile of JavaScript.
Licensed directly from the teams that built it, through the pre.dev network. No brokers, no resold scrape. The task funnel is measured: 201,000 candidate code changes in test-bearing repos, 30,000 usable after screening. Supply scales with compute.
Questions post-training teams ask.
What exactly is one of your RL tasks?
A runnable engineering task lifted from a real private production codebase: repo snapshot, task spec, and a verifier that scores each attempt with continuous reward on [0, 1]. You get the container and the reward entrypoint, and it plugs into your own RL or eval infrastructure.
Does training on these actually improve a model?
We measured it. We RL-trained Qwen3.8-27B on ~100 of our tasks and scored base vs trained on Terminal-Bench 3.0 with stock bench params: 1/74 to 4/74 at pass@1, above the #10 entry on the official leaderboard. The bench shares no code with our tasks, and the full run is re-runnable: trajectories, verifiers, and adapters all ship.
Where do the codebases come from?
From private production repos: codebases licensed directly from the teams that built them, through the pre.dev network, plus internal production systems. Every task is cut from private git history, and every task commit in the catalog is verified absent from public GitHub by commit-hash search, so the specific work you train on is not in public pretraining corpora.
How do we evaluate before buying?
Book an intro call below, or email reviewer GitHub usernames to arjun@pre.dev. We deliver a runnable sample set the same day: repo snapshots, task specs, verifiers, QC bundles, and the capability sweep across frontier models, so your team can run everything end to end on your own infra before any commitment.
RL tasks or raw code. Your pick.
Book below, or email GitHub usernames to arjun@pre.dev for the same-day sample repo.