Bot soak testing

A short test shows a game starts and a round ends. It does not show a bot that wedges itself in a corner after four minutes, a mover that falls out of the world, or an object count that climbs every round. A soak runs the game’s bots headless for a number of simulated minutes and answers one question: did anything go wrong that a short test would not show.

Run one

In a reference game:

./game.sh soak                  # the default map, the default length
./game.sh soak some_map 10      # a map and a number of minutes

It runs the game’s tools/soak.tscn, prints the SOAK lines and any errors, writes the report to .godot/test-logs/soak-<map>.json, and exits non-zero when the run failed. A game with no tools/soak.tscn says so. Like any headless run it is capped by --timeout.

What it samples, and what fails a run

DotBotSoak samples every sample_sec and judges the run at the end against thresholds, returning a report with ok and a list of reasons.

Thresholds are a DotBotSoakConfig, which is a DotConfig: defaults, a JSON file, DOT_SOAK_* environment variables, then --soak-* on the command line. Growth is measured from the first sample after the warm-up, so loading the map is not counted as a leak.

Wiring a game

A game owns tools/soak.tscn, which builds its normal headless world with bots and runs the harness. dot-bot needs only dot-core; stuck, fell-out and unreachable detection come from dot-spawn’s DotSpawnWatch, handed in by duck typing, and are skipped (the report says "watch": false) without one.

var soak := DotBotSoak.new(cfg)   # cfg = DotBotSoakConfig.new(), cfg.apply_cli()
soak.movers_fn = func() -> Array: return world.bots_as_movers()   # {id, position, kind, has_goal}
soak.stats_fn = world.combat_totals                               # {kills, deaths, damage}
var report := await soak.run(get_tree())
print(DotBotSoak.summary(report))
print(DotBotSoak.report_line(report))                             # one SOAK {...} line for CI to grep
get_tree().quit(0 if report["ok"] else 1)

A host that steps its own world sets step_fn and calls advance(tick) in a loop. Optional hooks are players_fn, object_count_fn, tick_cost_fn and describers, whose describe_lines() output is copied into the report.