Quickstart

Score a solution against a built-in environment, score a batch the way GRPO does, and run the QA gate, all on your own machine.

This walkthrough runs everything in local mode, which needs no sandbox account. Local runs execute the solution on your machine, under sandbox-exec (macOS) or bwrap (Linux) when available. For untrusted model output, use a sandbox.

Install

pip install deepgym

uv add deepgym

Score a solution

Load a built-in environment and run a candidate solution against its verifier. model_output is whatever your model wrote.

from deepgym import DeepGym, load_environment

dg = DeepGym(mode='local')
env = load_environment('coin_change')

solution = '''
def coin_change(coins, amount):
    dp = [float('inf')] * (amount + 1)
    dp[0] = 0
    for coin in coins:
        for x in range(coin, amount + 1):
            dp[x] = min(dp[x], dp[x - coin] + 1)
    return dp[amount] if dp[amount] != float('inf') else -1
'''

result = dg.run(env, model_output=solution)
print(f'Score: {result.score}, Passed: {result.passed}')

$ python score.py
Score: 1.0, Passed: True

result.cases holds one entry per test case, so a wrong answer shows which inputs it failed. A solution that returns 0 for every input scores 0.08 here: 2 of 25 cases pass.

Score a batch

GRPO samples several completions per prompt and compares their rewards. run_batch scores them in parallel:

completions = [model.generate(env.task) for _ in range(8)]
batch = dg.run_batch(env, completions, max_parallel=8)

scores = [r.score for r in batch.results]
mean = sum(scores) / len(scores)
std = (sum((s - mean) ** 2 for s in scores) / len(scores)) ** 0.5
advantages = [(s - mean) / (std + 1e-8) for s in scores]

In a batch, a run whose sandbox or verifier failed comes back without a verdict: result.has_verdict is False, and its score of 0.0 means nothing. (A single dg.run raises on sandbox failures and timeouts instead.) Keep those runs out of the group rather than training on them as zeros. The trainer integrations do this for you.

Check the environment

Before you train on an environment, the QA gate checks three things. Its reference solution must score 1.0 on every rerun, empty and stub solutions must earn nothing, and no heuristic reward-hack attack may reach the exploit threshold:

$ deepgym qa coin_change
environment  ref    det    empty  stub   busy   rand   hack   mut    diff   result
coin_change  ok     ok     ok     ok     -      -      ok     -      -      PASS

1 passed, 0 failed, 0 inconclusive, 0 skipped

Next steps