Scanning at scale

Repositories at once is how many clones run side by side. Scans at once decides how many of those runs there are. Processes for semgrep and threads for TruffleHog are what each of those may use inside its own repository. All four are on the Limits page, they multiply, and that is the thing worth understanding before raising any of them. The ceiling on repositories at once went from sixteen to sixty four.

gitleaks, TruffleHog, syft and grype are all Go, and a Go program with nothing said helps itself to every core. That is right for one repository and wrong for sixteen, where it becomes a thousand threads fighting over the same cores and the scan gets slower the more of the machine it is given. Each worker is given the core count divided by what is actually running, so a scan on its own still gets the whole machine.

The per repository deadline used to signal the wrapper and nothing the wrapper had started, so on one account seven workers outlived their own limit by twenty minutes: nineteen cores, doing work nobody could read. The deadline takes down the whole tree now, children first, and anything that ignores the first signal is killed twenty seconds later. The log says when a repository ran past its limit, which it never used to.

A repository used to be all or nothing: one stopped at its deadline wrote nothing at all. Credentials, code, SBOM and dependencies each write what the repository has as they finish, so a worker stopped later still reports the work that was done. Such a repository is reported as partial, and the Scans page counts it beside how many failed and how many were skipped. It is deliberately not counted as scanned, because a repository stopped half way did not look for the findings it is missing and calling them fixed would be the scanner inventing good news.

Two scans of the same account watch one rate limit file rather than each tracking half of it and both walking into a limit neither was counting. The reserve setting is measured against the credential's real remaining budget, across runs and across the workers inside one.

A connection set up before the team fills it scans cleanly, finds nothing, and used to be filed as a failure every night, which teaches everybody reading that page to ignore a red row. It is recorded as empty now, with its own status and its own audit event. Empty rather than finished, because zero repositories is also what a mistyped organization looks like, and a scan that quietly reports success while covering nothing is the one outcome this must never produce.

Every feature page carries one of those. The comparison page scores the same ground against five products people pay for, including the rows that go against us.