What the product does
A dataset that stays current, and proves every value in it.
Collecting the businesses is where it starts. This is what happens next: keeping records current, showing what changed, checking what can be checked, and sharing part of it without handing over all of it.
Status of this installation
counted
- Datasets held here
- None
- Workflows defined
- None
- Exercised by
- Test tenants
Everything below is built, deployed and reachable. None of it has been used on data anyone depends on. What that does and does not establish.
The dataset is the product.
A dataset holds records of one kind and keeps them current — what is true now, what was true before, what changed between. Each record carries one of three states.
One absence is far more often a truncated update than a closed business, which is why the third state is the hardest one to reach.
- Present
- Seen by the most recent update.
- Missing
- Not seen last time, not gone long enough to write off.
- Gone
- Missing from enough complete updates in a row to count as gone.
Gone needs two independent things to be true
The update covered the whole source.
and
The record was absent from that many complete updates in a row — two by default, adjustable per dataset.
Either one unproven and the record stays where it is. A partial update moves nothing.
A field the update never looked at keeps its own timestamps, so re-confirming a record does not re-date the values inside it that nobody checked.
Every value shows its working.
Open any record and each field carries its own evidence, not the record's.
- 01The value
- 02How it was arrived at
- 03Which source
- 04Which reference
- 05First seen
- 06Last confirmed
- 07Next re-check due
- 08Shortened, cleaned extract
How a value was arrived at
Observed
A source stated it.
Cleaned
Derived from what a source stated, by our rules.
Filled in
Supplied later, because the first source lacked it.
Set by a person
Chosen here, with no source claimed.
Each origin carries a mark of its own shape, never colour alone. If the evidence cannot be loaded the values still appear and the evidence is marked unavailable — an empty panel and a failed lookup are different facts.
A rating counts checks. It never estimates whether a value is true.
Five things are counted, and none of them asks whether a value is correct. That is not knowable from inside NanoScraper.
- 01Required details filled in
- 02Values that look usable
- 03Values that trace back to a source
- 04Sources that agree with each other
- 05Values still within their expected life
Two results, and they are not the same result
no rating is not a zero
A check that cannot be made is skipped rather than scored zero, and the total comes only from the checks that applied.
Not a confidence, a certainty or a probability: a test asserts the rating never describes itself in any of those words.
One change stream, four things reading it.
Every difference an update finds becomes one change record, identified by what changed. Run the same update again and the identities repeat: no second copy, no second alert.
The catch-up view, your alert rules, the opportunity list and the rules that send data onward all read that one stream, so they cannot disagree. A correction made by hand is written into the same stream.
Four features, one difference. Anything re-deriving its own diff could disagree, with no way to tell which was right.
How alerts behave
- Raised by an update
- Two things ask for a check: an update that actually changed something, as it finishes, and you. Nothing sweeps your data on a clock.
- Delivered in the app
- The alert is already stored and in your list, so that channel cannot be misconfigured. Sending one onward to another system is offered but not available here — attempts are recorded as blocked, not failed. Dataset delivery is a separate path, and it does send.
- Email needs mail set up
- Out of the box no sending address is configured, so the channel reports itself unavailable rather than quietly dropping messages.
A workflow is built from these steps and no others.
- 01Collect
- 02Filter
- 03Fill in missing details
- 04Remove duplicates
- 05Check quality
- 06Save to a dataset
- 07Notify
- 08Export
Anything else is refused when the definition is checked.
Closed on purpose: a definition arrives over the API as data, and an open vocabulary cannot be checked. It is validated against this list, bounded in size, checked for cycles and for whether one step can feed the next — and what runs is the checked version.
Four counts, never collapsed into one
Each narrows the one above it, which is exactly why they are never summed into a headline number.
- 01Things seen
- 02Distinct things that was
- 03Those with every required detail
- 04Those also matching your rule
A stop condition names which of the four it means, so “stop at 100” cannot quietly mean 100 web pages.
A step that cannot work is recorded as blocked and the run finishes with gaps — its own state, never read as success. Six situations produce it, including a notification with nowhere to go, a source that is not ready, and a missing destination.
Four ways data leaves.
Datasets themselves have no download. They are read through the app, the change feed, or one of these four. None of the four is a named third-party service.
Export
Companies and contacts download as CSV or Excel, with cells that would otherwise be read as formulas neutralised first. An export over your limit is refused rather than trimmed, and no partial file comes back.
Delivery to an address you control
Rows are sent to a web address you choose, or produced as a spreadsheet instead. The attempt is recorded before anything goes out, so one that fails mid-flight still exists to look at. Set a signing secret and each delivery is signed over its contents and its send time, which your own system can check.
Client portal — an API, not a page
A named recipient gets a key and reads JSON: the columns you shared, their changes, and the proof behind each value. NanoScraper hosts no page for them. The key travels in an authorization header, never in a web address — an address ends up in browser history and every log between.
Read connection
For a subscriber's own system, scoped to the columns you chose and revocable at any time. Same key handling, same per-request re-check that the data is still shared.
What the product is opinionated about.
Each is a decision recorded in code, with the reason attached — not a gap waiting to be filled.
- 01LinkedIn
- Permanently refused — not deferred, not behind a switch. Its readiness check never consults the enablement flags, because an operator cannot switch off a licence contradiction, and the module imports no LinkedIn library.
- 02Invented records
- One source exists to exercise failure paths, and it makes its records up. Off unless an operator opts in, labelled diagnostic everywhere, refused at creation: once stored, an invented record is indistinguishable from a collected one.
- 03Payments
- Checkout, payouts, tax, refunds and revenue share all report themselves unavailable; asking to obtain a paid item raises the same refusal.
- 04No model, anywhere
- Finding similar businesses is weighted overlap of facts already held, and it returns the facets it scored on. Nothing is trained or inferred, and the backend's dependency list holds no library that could be.
- 05Filling in details
- Two sources only: a web address worked out offline from a site already held, and an email or phone read off the record's own site. The result is recorded as looks well formed, never as verified — nothing was sent and nothing was dialled.
- 06Map imagery
- Free to use on condition its source is credited, so a tile address with no credit line counts as not configured. The feature falls back to a list and says why.
- 07Branding
- Data only: a name, a few colours, a support address. No stylesheet field, because a stylesheet can reposition anything, including a confirmation button. Angle brackets are refused outright rather than stripped, and a colour pair too faint to read is refused, not saved.
The engineering detail is in the documentation, and what has actually been run is on /docs/verification.