Security
Last updated October 14, 2026
Our approach
Security is central to what DeepGym does. We execute untrusted code generated by AI models, which means isolation and containment are not afterthoughts. They are the product.
Sandbox isolation
Sandboxed runs use Docker, Apple container, Modal, E2B or Daytona. Each run or session gets a throwaway sandbox. Inside it, a single-file solution that the verifier loads through deepgym_solution runs as a separate Linux user with no external network (a verifier can open one localhost port to it) that cannot read the verifier's files or write its output, and in sessions the agent's commands and the verifier run as separate users, the verifier from a file the agent cannot read. Repo and cyber benchmark tasks apply the model's patch, then score the result, as root without this judge isolation, so the sandbox itself is their boundary. Terminal (Harbor) tasks run the model's commands as an unprivileged user with no network and grade a copy of its work directory in a fresh sandbox, though tests that run the model's code as root can still let it write the reward. Imported BigCodeBench tests run alongside the solution, so a determined model can still forge that result. Docker and Apple container sandboxes are capped at 2 CPUs and 2 GB of memory.
Local execution mode
The library supports a local execution mode for development and testing. It runs on your machine without a container: solutions loaded through deepgym_solution run in a separate process under sandbox-exec on macOS or bwrap on Linux, with the verifier and the deepgym package hidden, or with process separation only and a warning when neither is available. Local multi-turn sessions have no isolation: the agent's commands run with bash in a temporary directory, and the verifier shares the agent's user. Repo, terminal and cyber benchmark tasks refuse local mode and need a sandbox. For anything shared or untrusted, use a sandbox.
Adversarial testing
DeepGym includes built in adversarial testing capabilities that probe environments for reward hacking vulnerabilities. This includes detection of empty solutions, hardcoded outputs, pattern exploits, and LLM-guided iterative exploit search. The optional deepgym qa quality gate runs the heuristic attacks, along with reference, empty and (where applicable) function-stub checks, on supported single-turn environments before you train on them. We use these tools on our own built in environments.
API authentication
The DeepGym API server requires API key authentication in production. Keys are passed via the DEEPGYM_API_KEY environment variable. Development mode can optionally disable auth for local testing.
Dependencies
We keep our dependency surface small. The core library requires only pydantic and httpx. Optional dependencies for specific integrations are clearly separated. We monitor for known vulnerabilities in our dependency tree.
Reporting vulnerabilities
If you discover a security vulnerability in DeepGym, please report it responsibly by emailing contact@deepgym.io with the subject line "Security". We will acknowledge your report within 48 hours and work with you to understand and address the issue. Please do not open a public GitHub issue for security vulnerabilities.
Scope
This security page covers the DeepGym library and the deepgym.io website. Third party services we integrate with such as HuggingFace Hub, Daytona, and lm-eval have their own security practices and policies.