How scanning works

The scanner — internally, Spy — is what turns an address into a live listing. It runs continuously against roughly two million servers, and how often it gets to yours is not arbitrary.

The three processes

A scanner deployment runs three independent things:

Scanners

Pull servers off the due-queue and query them. This is the bulk of the work and the source of everything on a server’s page.

Seeds

Discovery. They find servers that are not in the database yet. Four kinds — see below.

The party queue

A short, separate queue for servers hosting a live third-party party. It is separate from the scanners because it must not compete with them for the rotation, it creates no servers so it is unrelated to seeds, and it is bounded by the number of live parties rather than the size of the fleet — which is what makes a much shorter interval affordable.

Discovery: the four seed types

SWEEP is the interesting one. Every other type finds servers somebody published. This one finds the ones nobody did: game servers come in blocks, so a box running one is usually running more on the ports next to it. It only ever visits addresses already in the database — there is no netblock field and deliberately no way to add one.

Enrichment

Whatever a seed produces is a claim. A follow-up query replaces it with what the server itself reports, dropping candidates that answer nothing.

A queried server can also correct which app it is filed under, from the game directory its own query answers with — the one identifier the person running it cannot casually change. That correction is off by default, because it is only right when the query is the more trustworthy source: a scraped listing that says “tf2” is a claim worth checking; Valve’s master server already answered the question.

Query protocols

PALWORLD_REST is the odd one: it is the only protocol that needs a credential (the server’s admin password), so the scanner holds those in its own local configuration and skips any server it has none for. Pair it with A2S rather than instead of it — A2S answers for every Palworld server but returns an empty player list, and the REST API is the only source of the roster, the real maximum player count and the settings.

Which protocols are tried, on which ports, with what timeouts and retries, is configured per app. See Apps.

Finding the query port

A server’s query port is the one thing discovery cannot infer. The address is published, the game port usually is, and the query port is whatever the host’s control panel handed out — which is also why a recorded query port is sometimes simply wrong, pointing at a neighbouring server or at nothing at all.

There is plenty to go on: each protocol has a handful of ports it is nearly always on, because games ship a default and panels allocate in blocks from it. That knowledge is a table an operator can edit — so somebody who notices a new hosting company allocating Rust servers at 28080 can act on it without a scanner release.

Two repair passes run over listings whose port looks wrong:

The quick pass

Tries the likely ports for the protocol. Repairs most rows.

The deep pass

Walks the whole port space at one probe every thirty seconds — which is the only thing separating a repair from a port scan of somebody else’s box, and which makes a full sweep of one address about 23 days.

No run lives that long, so the progress belongs to the row: each pass covers a chunk, writes down where it stopped, and the next one carries on from there.

Each pass keeps its own cursor. They are independent, and sharing one would mean an hour-long deep chunk registering as a quick check — hiding the row from the pass that repairs most rows — and every quick pass resetting the deep pass’s rest period, so no deep sweep would ever advance.

Both record their result even when they found nothing, which is what stops the same hopeless address being re-probed nightly.

The scan queue

Every server carries its own due time and a tier. The queue reads “what is due, most urgent tier first”.

Why not one flat queue

The scanner used to pull from one list ordered by “least recently scanned”. At two million rows that reduces “keep everything fresh” to “scan the entire table every cycle”, which is not possible — so a set of counters periodically swapped the whole filter set to give a different slice of the table a turn. A server with 200 players and a claimed owner was scanned at exactly the same rate as a dead IP that had never once responded.

Per-server due times replace that. The load stops being a race and becomes a knob.

The tiers

Ordered by urgency. These cadences are defaults and are operator-configurable — what is fixed is the lane each class of server lands in.

Ordering by due-time alone would treat “six hours overdue on a 24-hour cadence” as more urgent than “one minute overdue on a two-minute cadence” — so whenever the scanner fell behind, the slow junk crowded out precisely the servers the cadences exist to keep fresh. The tier column is what prevents that.

Tier 0 holds two populations on purpose. A forced rescan is written there directly with a sentinel due-time. A never-scanned row lands there too, because “we have no idea what this address is” is the one question a scan answers instantly and permanently — one scan re-tiers it onto its true cadence, so tier 0 drains and does not refill. Within the lane, a forced rescan still sorts ahead of the cold-start sweep.

The honest cost: a bulk import of N addresses puts N rows ahead of the scheduled fleet until each has been looked at once.

Flagged servers are slowed, not excluded

A flagged server drops to the slowest tier rather than being filtered out. That is what makes flags self-healing: a server that fixes whatever got it flagged is re-examined eventually and unflagged automatically, with nobody having to notice.

Automated detectors

Three checks run independently, and each owns exactly one flag sub-type:

Scan filters

Raised during a normal scan. A server’s name matching a bad-word pattern, its address inside a bad-IP range, or its network inside a bad ASN.

A good-IP list overrides these, and an official-server table recognises company-run servers by name pattern, address range or ASN.

Duplicate detection

Redirect and proxy farms: too many visible servers sharing one IPv4.

Fake-player detection

A population that barely moves across thousands of scans. Bounds are configured per app, since what counts as suspiciously flat differs between a 64-slot shooter and a persistent survival server.

Each detector may raise and lift its own flag and must leave the others alone. A flag with no sub-type was raised by a human, and no automated job ever touches it. The reasoning is on Penalties.

Auto-detection

Alongside the live state, the scanner works out:

Each can be switched off per server once you have set it correctly by hand (autoDetectApp, autoDetectCat, autoDetectCountry).

Category matching uses the pattern rules on the category itself — see Categories.

What the scanner will not overwrite

An owner can take a field over with a query override, and the scanner then stops writing it. See Reporting your own statistics.