DeepGym documentation
RL environments with verifiable rewards for coding and computer-use agents. Run model output in a sandbox, score it with a verifier, and train on the result.
DeepGym runs model output in a sandbox, checks it with a verifier and returns a score with a per-test breakdown. The score is the reward: it plugs into TRL, verl, OpenRLHF, Axolotl or your own GRPO, DAPO or PPO loop.
For agents, one sandbox stays alive for the whole episode, either a shell or a full desktop. The agent acts step by step, and the verifier scores the state it leaves behind.
pip install deepgymHow a run works
The model answers
A function, a patch or a shell script. An agent instead acts one step at a time in a live sandbox.
DeepGym runs the answer in a sandbox
Locally while you develop, or in a Docker, Apple container, Daytona, Modal or E2B sandbox.
The verifier scores it
A score from 0 to 1 and a pass flag, plus a result for each test case when the verifier reports them.
Your trainer gets the reward
Through the TRL, verl, OpenRLHF or Axolotl adapters, or your own loop. A run that failed for reasons outside the model’s control gets no verdict, so it can be kept out of training instead of scored 0. See the reward contract.
Start here
Quickstart
Score a solution, score a batch for GRPO and run the QA gate in a few minutes.
Core concepts
Environments, verifiers, sandboxes, scores and verdicts.
Explore
Environments
31 built-ins, benchmark importers, Harbor tasks and computer use.
Authoring
Scaffold, check, QA-gate and approve your own environments and verifiers.
Rewards and training
The reward contract and the TRL, verl, OpenRLHF, Axolotl and lm-eval adapters.
Traces
Record graded episodes with provenance and export them as training data.
Sandboxes
Docker, Apple container, Daytona, Modal and E2B behind one API.
CLI reference
Every deepgym command and its main flags.
License
PolyForm Noncommercial License 1.0.0. Commercial use needs a license: email contact@deepgym.io.