NanoScraper · verification
What we check, and what that does not prove.
The gates that run, the dated results of the last full pass, four defects that failed completely silently, and an explicit account of what none of it establishes.
The command run → the line it printed
- Type checking, strict
mypy .
Success: no issues found in 430 source files
- Secret scan, full history
gitleaks detect --source . --redact
no leaks found
- Engine network topology
scripts/check_topology.py
Topology check passed: postgres, redis unreachable from any engine
3 of 12 measures, run 2026-08-22.The full ledger
- Where
- On the server that serves production, invoked by a person.
- Against
- Real PostgreSQL 17 and a real Chromium, in an isolated stack.
- How current
- Exactly as current as the date on it. Nothing runs on a push.
Verification runs on the server, by policy.
The workflow that would run on every push still exists, unchanged — no job removed, no check relaxed — but it is invoked by hand. A permanent red cross against every commit is indistinguishable from a real failure.
So read this as a server-side gate a person runs, not as continuous integration.
The gates
What runs, and what each one can catch. No figures here — those are dated, and they are next.
- 01Backend suite
- Every backend test against a real PostgreSQL 17 in an isolated stack. Never SQLite, whose constraint and JSON behaviour differs from what production rejects.
- 02Cross-tenant negatives
- For each tenant-scoped resource, proof that a user in one workspace is denied the equivalent in another: read, list, write, delete, export.
- 03Browser checks
- Playwright against a running stack, at the same origin the backend's CSRF and CORS settings name — so a pass cannot be an accident of hostname.
- 04Accessibility sweep
- axe on every page at 390, 768 and 1280 pixels, under WCAG 2.0 and 2.1 A/AA, plus what axe cannot judge: focus order, scrollable regions, duplicate ids, heading levels.
- 05Frontend gates
- Lint, strict TypeScript with no unchecked index access, unit tests, and a production build — the build specifically, because it catches boundary violations the dev server tolerates.
- 06API type drift
- The TypeScript client is regenerated from the OpenAPI schema and compared byte for byte with the committed copy. A hand-edited type is a lie the compiler agrees with.
- 07Static and security
- ruff, ruff format, mypy in strict mode, a secret scan over the repository, Python and JavaScript dependency audits, and Django's deployment check against production settings.
- 08Engine isolation
- The engines' network is asserted to have no route to the database or the broker — from the declared configuration and from inside a live container, which a static check cannot see.
- 09Production smoke
- Read-only over the live site: no sign-in, no form submission, no unsafe request. It probes the internal frontend-to-backend hop, the failure that once passed every anonymous check while sign-in was impossible.
2026-08-22 · last full pass
Every figure here is a line a tool printed.
Never a hand count, never a grep, never arithmetic across runs. Every one came from a run made for this release, and this is the only block on the site carrying a measured figure.
Published
Four figures lifted out of the ledger below, each read from its own row so the two cannot disagree.
Backend tests, real PostgreSQL
2483
2483 passed in 343.96s
Of them cross-tenant
231
231 passed, 2252 deselected
Browser checks, real Chromium
119
119 passed
Frontend unit tests
575
Tests 575 passed (575), across 49 files
Withheld
Four measurements that print no summary line. A number for any of them would be an inference, so none is published.
Accessibility sweep
Prints no count of pages or viewports. It asserts zero serious or critical violations at all three widths — a result, not a figure.
Schema validation
Exits cleanly while printing nothing at all. A “zero errors” number would be an inference from an exit code.
Extension checks
No command in this list produces a figure for them, so none is published.
Authenticated production journey
Needs credentials that were not supplied, and it writes to production — creating and deleting a synthetic tenant, which contradicts the read-only rule.
The ledger, in full
| Measure | What it printed |
|---|---|
| Backend suite, real PostgreSQL 17 | 2483 passed in 343.96s |
| — of which cross-tenant and isolation | 231 passed, 2252 deselected |
| Browser checks, real Chromium | 119 passed |
| Frontend unit tests | Tests 575 passed (575), across 49 files |
| Type checking, strict | Success: no issues found in 430 source files |
| Lint and formatting | All checks passed! / 434 files already formatted |
| Django deployment check | System check identified no issues (0 silenced). |
| Engine network topology | Topology check passed: postgres, redis unreachable from any engine |
| Secret scan, full history | no leaks found |
| Dependency audits | No known vulnerabilities found / found 0 vulnerabilities in runtime scope |
| Production services, at the time of the run | 7 of 7 reporting healthy |
| Records held in production | 706 observations, 472 companies, 434 contacts, 0 datasets, 0 workflows |
The same table, with the full command invocations, is recorded in docs/execution/verification/.
The service and record counts are a snapshot from that run, not a monitored value. Nothing watches production continuously, and this page makes no claim about availability. Companies rose while the observation count did not, because a matching fix re-derived records from evidence already stored — that is the raw/canonical split working.
Four defects that failed silently.
None of these errored. None raised. Each produced a plausible, confident, wrong result, and each was found by looking rather than by being told.
During the product-expansion review of 2026-08, every fix shipped a test verified to fail with the fix removed — neutered, watched fail, restored, watched pass.
“A regression test nobody has seen fail is a statement of intent, not a guard.”
One review, on its own fixes, on one date — not a claim about every test in this repository's history, which would be an unverifiable universal.
01
Schedules never fired
The dispatcher lived in a module Celery's autodiscovery never imports, so the scheduler published a task name to a worker that had never heard of it — and no scheduler process ran in production at all. Its own unit tests passed throughout: they imported the function directly.
Now — every scheduled entry must name a task a worker would actually have, on a queue it consumes. The fix that mattered was to the test.
02
A filter was accepted and ignored
A page asking for one job's runs was served every run in the workspace, so any total computed from it was the workspace's total. No tenant boundary was crossed — the numbers were simply wrong, which is why nothing caught it.
03
Provenance claimed what the source never said
The evidence table stored only the canonical value, beneath a column reading Value as observed.
Now — both halves are stored, on every field. Told in full on /docs/data.
04
A diagnostic connector sat beside real sources
One connector exists to exercise failure paths and fabricates its records. Once stored, the companies it produced were indistinguishable from collected ones.
Now — marked diagnostic, off unless an operator opts in, labelled everywhere it appears, and refused at creation.
What none of this proves.
- Product layer
- Nobody has used it on data they care about. Datasets, workflows, quality, monitoring and sharing have only ever been exercised by synthetic tenants, and this installation holds no datasets and no workflows. Deployed and reachable; never depended on.
- Volume
- Nothing has been exercised at volume. No load test, no concurrency test, no quota-pressure test. Collection runs here have been small. No throughput or latency target is defined or measured, so no page on this site states one.
- Accessibility
- Checked at three widths, by a machine. A defect appearing only between 390, 768 and 1280 pixels would be missed, axe cannot judge whether wording is comprehensible, and no screen reader was driven by a human. The sweep also signs in first, so the anonymous render is covered only by the production pass.
- Verify stack
- It runs development settings. Production-only security behaviour — secure cookies, HSTS, the proxy SSL header — is proven by Django's deployment check and by tests importing the production settings module directly, not by the browser suite.
- Diagrams
- Sources are not diff-checked against the images. They sit beside the SVGs they produce and the renderer is pinned by digest, but nothing re-renders and compares. Three diagrams here were found to disagree with the code and nothing had noticed. A flaky gate would be worse than a stated gap.
- Backups
- There is no point-in-time recovery. Backups are encrypted before they leave, asymmetrically, so the server cannot read back what it wrote; an off-host copy is refused unless encrypted, and one has been fetched back and restored into a database as its own explicit step. Recovery is to the last nightly dump, and Redis is not backed up, by design.
- Certification
- None of any kind. No SOC-style attestation, no ISO certification, no external audit, no third-party security test. The only adversarial review this code has had was internal.
Every claim on this page is drawn from the code and from runs executed against production.