# DeepGym Docs > RL environments with verifiable rewards for coding and computer-use agents: sandboxes, verifiers, a QA gate and reward functions for TRL, verl and OpenRLHF. ## Getting started - [DeepGym documentation](https://deepgym.io/docs): RL environments with verifiable rewards for coding and computer-use agents. Run model output in a sandbox, score it with a verifier, and train on the result. - [Installation](https://deepgym.io/docs/installation): Install DeepGym from PyPI with pip or uv, add the extras for the sandboxes and integrations you use, and check that the CLI runs. - [Quickstart](https://deepgym.io/docs/quickstart): Score a solution against a built-in environment, score a batch the way GRPO does, and run the QA gate, all on your own machine. - [Core concepts](https://deepgym.io/docs/concepts): This page covers the five ideas DeepGym is built on (environments, verifiers, sandbox providers, the reward contract and traces) and how one run moves through them. Each section links to the page with the details. - [Sandboxes](https://deepgym.io/docs/sandboxes): This page covers where DeepGym runs answers: the providers, how to set each one up, the isolation the sandbox modes give, and per-environment images. ## Environments - [Built-in environments](https://deepgym.io/docs/environments/built-ins): This page lists the 31 environments that ship with the package and shows how to load them, run them as suites, and read their source. - [Benchmark importers](https://deepgym.io/docs/environments/importers): This page covers the public benchmarks DeepGym can load or import: the code benchmarks (HumanEval, MBPP, EvalPlus, BigCodeBench), repo-level patch tasks (SWE-bench Pro, MiMo), shell tasks (Terminal-Bench 2.0 and any Harbor dataset), CyberGym, OSWorld desktop tasks, mixing benchmarks behind one reward function, and sharing environments on the Hugging Face Hub. - [Multi-turn sessions](https://deepgym.io/docs/environments/sessions): This page covers live multi-turn sessions: one sandbox kept alive for a whole episode, actions applied step by step, and a final verifier that scores the state the agent left. It covers the Python API, writing a multi-turn environment, step rewards, and running sessions over HTTP. - [Computer use](https://deepgym.io/docs/environments/computer-use): This page covers desktop environments: how an episode runs, the actions an agent can take, seeded variants, how verifiers score the result, and how the desktop is locked down. Computer-use environments need Daytona (pip install "deepgym[daytona]" and DAYTONA_API_KEY). - [CyberBench-style DeepGym environments](https://deepgym.io/docs/cyberbench): DeepGym cyber tasks should be sandbox-first, deterministic, and reward-shaped. Use live models such as GLM-5.1 only as candidate policies; rewards come from local verifiers. ## Authoring - [Write an environment: new, check, qa](https://deepgym.io/docs/authoring/new-check-qa): This page covers going from an idea to a QA-approved environment folder: scaffolding it with deepgym new, the files it contains, the four kinds, validating it with deepgym check, and gating it with deepgym qa. - [Write a verifier](https://deepgym.io/docs/authoring/verifiers): This page covers writing a verifier: the shapes it can take, the deepgym_solution helpers it must use to run the solution, how the solution is isolated from it, and the JSON protocol it prints. - [The QA gate](https://deepgym.io/docs/authoring/qa-gate): This page covers deepgym qa, which checks an environment before you train on it: the checks for single-turn and multi-turn environments, mutation testing, the difficulty probe, statuses and exit codes, and the report it writes. - [Approvals](https://deepgym.io/docs/authoring/approvals): This page covers approval.json: how deepgym qa --approve records that an environment folder passed, what the file holds, and how deepgym check tells when the folder has changed since. ## Rewards and training - [The reward contract](https://deepgym.io/docs/training/reward-contract): This page covers how DeepGym decides whether a run was judged, what each trainer integration does with a run that wasn't, and how the totals DeepGym reports stay conservative. - [TRL](https://deepgym.io/docs/training/trl): This page covers DeepGym's reward functions for TRL's GRPOTrainer: setup, what runs, how failed runs are kept out of training, and the DAPO and mixed-benchmark variants. - [verl](https://deepgym.io/docs/training/verl): This page covers training with verl on DeepGym environments: the reward function verl loads from the package, the agent loops that keep failed runs out of training, multi-turn agent RL on live sandboxes, and DAPO recipes. - [OpenRLHF](https://deepgym.io/docs/training/openrlhf): This page covers DeepGym's reward endpoint for OpenRLHF's remote reward model option: setup, how requests are graded, and what a sample gets when DeepGym can't grade it. - [Axolotl](https://deepgym.io/docs/training/axolotl): This page covers GRPO training with Axolotl on DeepGym environments, from Python or from a generated config with no code, plus the process reward model config. - [lm-evaluation-harness](https://deepgym.io/docs/training/lm-eval): This page covers evaluating a model on DeepGym environments with lm-evaluation-harness (0.4.9 or later): building the tasks, what runs, and the metrics. - [Traces](https://deepgym.io/docs/traces): This page covers recording graded agent episodes as traces with dg.trace(), what a trace holds, and exporting traces as training data in JSONL, Parquet (verl and TRL rows) or ATIF. ## Reference - [Agent harnesses](https://deepgym.io/docs/agents) - [CLI reference](https://deepgym.io/docs/cli): This page lists every deepgym command with its main flags. Run deepgym --help for the full list. - [API server](https://deepgym.io/docs/api-server): This page covers deepgym serve, the HTTP API for scoring from any language: starting it, authentication, the endpoints, and the limits it enforces. - [Console](https://deepgym.io/docs/console): This page covers deepgym console, the local web console for the environments, runs, episodes, sandboxes and QA reports on your machine: what each page shows, where data is stored, and how the console protects itself. - [FAQ](https://deepgym.io/docs/faq): Short answers on what DeepGym does, where it runs, what a score means, where it keeps its files and how it is licensed. - [Changelog](https://deepgym.io/docs/changelog): What changed in each DeepGym release, newest first. ## Other - [v0.1.0](https://deepgym.io/docs/changelog/0.1.0): Released March 17, 2026. - [v0.2.0](https://deepgym.io/docs/changelog/0.2.0): Released April 2, 2026. - [v0.3.0](https://deepgym.io/docs/changelog/0.3.0): Released April 2, 2026. - [v0.4.0](https://deepgym.io/docs/changelog/0.4.0): Released October 13, 2026. - [v0.4.1](https://deepgym.io/docs/changelog/0.4.1): Released October 13, 2026. - [v0.4.2](https://deepgym.io/docs/changelog/0.4.2): Released October 13, 2026. - [v0.5.0](https://deepgym.io/docs/changelog/0.5.0): Released October 14, 2026. ## RSS Feeds - [DeepGym Docs — Changelog](https://deepgym.io/docs/changelog/rss.xml) ## Agent skills - [deepgym-docs](https://deepgym.io/docs/.well-known/agent-skills/deepgym-docs/SKILL.md): Read the DeepGym Docs instead of answering from memory. RL environments with verifiable rewards for coding and computer-use agents: sandboxes, verifiers, a QA gate and reward functions for TRL, verl and OpenRLHF. Use when a task involves DeepGym Docs: how it works, how to set it up or configure it, or its API. ## Agent resources - [llms-full.txt](https://deepgym.io/docs/llms-full.txt): The full Markdown of every page in one file. - [Page Markdown](https://deepgym.io/docs/index.md): Append `.md` to any page URL to fetch that page as raw Markdown. - [JSON API](https://deepgym.io/docs/api/docs/pages.json): Page index of the JSON docs API; each entry links the page's JSON and Markdown forms. Described by the OpenAPI document at https://deepgym.io/docs/openapi.json. - [Agent skills](https://deepgym.io/docs/.well-known/agent-skills/index.json): Agent Skills discovery index of the skills this site publishes. - [API catalog](https://deepgym.io/docs/.well-known/api-catalog): RFC 9727 linkset of the APIs documented here. - [AI catalog](https://deepgym.io/docs/.well-known/ai-catalog.json): ARD manifest of the agent-facing resources on this site (MCP server, skills, APIs). - [agent-readability.json](https://deepgym.io/docs/agent-readability.json): Manifest of every agent-facing artifact on this site. - [Sitemap](https://deepgym.io/docs/sitemap.xml): Every indexable page URL with its last-modified date.