FAQ
Short answers on what DeepGym does, where it runs, what a score means, where it keeps its files and how it is licensed.
Does DeepGym train models?
No. DeepGym gives a trainer tasks and rewards: it runs each answer in a sandbox, scores it with a verifier and returns the result. Training runs in TRL, verl, OpenRLHF, Axolotl or your own loop, through the reward functions DeepGym provides. DeepGym itself needs no GPU.
Which sandbox should I use?
localwhile you develop with code you trust. Repo, terminal and cyber tasks refuse it.dockerorapplefor isolated runs on your own machine, with no account.daytona,modalore2bto run in the cloud. Desktop (computer-use) environments need Daytona.
Every sandbox mode gives the same isolation. See sandboxes.
What is the difference between a score of 0 and no verdict?
A score of 0 is a judgment: the answer failed. A run without a verdict (result.has_verdict is False) means the sandbox, the provider or the verifier failed before it could judge, and its 0.0 is only a placeholder. The trainer integrations keep those runs out of training. See the reward contract.
Can a model game the reward?
The verifier reaches the answer only through deepgym_solution, which runs it in a separate process: only plain data crosses back, and nothing the answer prints reaches the verifier’s output. In a sandbox mode, the answer also runs as its own user with no network and no access to the verifier’s files. Before you train, deepgym qa checks that empty and stub answers earn nothing and that simple reward-hack attacks stay below the exploit threshold.
A reward is still only as strong as its tests: an answer that satisfies weak tests scores. Run the QA gate with --mutants to find behavior the verifier never checks, and watch how many runs get no verdict while you train. The verl agent loops and the OpenRLHF router log a warning when more than 5% of a batch has none.
Where does DeepGym keep its files?
Under ~/.deepgym/:
| Path | What it holds |
|---|---|
environments/ |
Benchmarks imported with the import scripts |
benchmarks/harbor/ |
Harbor datasets cloned by deepgym harbor import |
qa/ |
deepgym qa reports (DEEPGYM_QA_DIR moves them) |
traces/ |
Traces recorded with dg.trace() (DEEPGYM_TRACES_DIR moves them) |
console.db |
The console’s runs and episodes |
Can I use DeepGym from another language?
Yes. deepgym serve exposes scoring and live sessions over HTTP, so a trainer in any language can call it. See the API server.
Which Python versions does it support?
Python 3.10 or newer.
How is DeepGym licensed?
DeepGym is licensed under the PolyForm Noncommercial License 1.0.0. Commercial use needs a license: email contact@deepgym.io.
How do I contribute?
New environments, verifiers, benchmark importers and integrations are welcome. Start with a good first issue and read CONTRIBUTING.md.