feat(analytics): identify the companies browsing the site - #167
Merged
Conversation
Adds /admin/companies: which organisations are on the site, what they looked at, and whether they are in the pipeline. Company-level only — the Clearbit-Reveal shape, explicitly not the RB2B person-level one. The privacy contract in 006 is amended, not broken. A visitor-day may now carry a company DOMAIN, from the network it came from or from the domain of a work email consented into /sales. No email, no name, and no identity-graph product. The address is never written. Resolution waterfall (server/utils/company.ts): self_reported work email on /sales — certain ip IPLocate netblock -> domain, enriched by Brandfetch asn operator known, no company nothing consumer ISP, mobile, hosting, VPN — most traffic Cost control is the design, and it is what makes this fit a free tier: a gate (high intent, or a second page view, counted in-process so it costs no query), a free reverse-DNS pre-filter that rejects consumer and hosting networks before any paid call, network_verdict so each telco is paid for at most once ever, then per-day and per-domain caches. An office of forty people costs one lookup a day; Comcast costs one lookup in total. IPLOCATE_DAILY_BUDGET is a hard ceiling that logs when hit. Firmographics reuse the existing Brandfetch integration, which was already fetching and discarding employee range, industry and HQ. Conversion is answered twice because the answers differ: submittedInSession is same-day and closest to causal but is a floor (the hash rotates at UTC midnight); conversion joins companies.domain to sales_leads.email_domain — domain to domain, never touching visitor_hash — which is complete across all time but is not attribution. Both are labelled in the UI, and "no lead yet" sorted by intent is the outreach list. Education and government resolve to real domains (mit.edu, usgs.gov) and count as organisations; only isp and hosting are excluded. IPLOCATE_API_KEY is optional. Without it, countries (free, from Cloudflare) and self-reported companies still work, and the page says identification is off rather than rendering an empty table. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Verified the firmographics parser against Brandfetch's own documented
response schema rather than an assumed one, which surfaced two defects:
* `kind` is an enum token ('PRIVATELY_HELD'), not a display string, so
the table would have printed raw enum text. Humanised.
* `employees` is an INTEGER (1001), not the range string the comments
claimed. Formatted with separators for display; the parsed lower
bound the size filter depends on was already correct.
Checked against the documented stripe.com payload, the parser now
returns employees "1,001" / min 1001, kind "Privately held", industry
taken from the highest-scored entry, and countryCode "US".
Also adds docs/privacy-company-identification.md: paste-ready policy
copy for the disclosure this feature requires, plus a pre-publish
checklist. The policy itself is CMS content (singletons, key 'privacy')
edited at /admin/pages, so it cannot ship in code — but the copy can.
Note the safe intermediate state, now documented: with IPLOCATE_API_KEY
unset, no IP address reaches any third party, so the feature can ship
before the policy is updated and be switched on afterwards.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… lookup
Tested the Brandfetch parser against the real API rather than the
documented schema. Siemens, MIT, IBM, Ford and USGS all parse correctly
— sizes, industries, "Public company"/"Government agency" kinds — and
two defects fell out:
* foundedYear is 0, not null, when Brandfetch does not know it (IBM).
num() accepts 0 as finite, so the column would have filled with a
year rendering as "Founded 0". Rejected explicitly.
* Brandfetch has no location for plenty of real organisations — MIT
and Ford both return null country — which left the flag column blank
on rows we demonstrably observed a country for. Falls back to the
country the visit actually came from.
Also reads process.env alongside runtimeConfig for the IPLocate key.
This is not redundant: runtimeConfig bakes at BUILD time, and this app
already scopes ANALYTICS_SALT_SECRET as RUN_TIME on DigitalOcean. An
IPLOCATE_API_KEY added with that same scope would never reach
runtimeConfig, and the feature would stay dark with no error anywhere.
Reading both means the key works under either scope, and under the
NUXT_-prefixed name.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
/admin/companies— which organisations are on the site, what they looked at, and whether they are in the pipeline.Company-level only: the Clearbit-Reveal shape, explicitly not the RB2B person-level one, which would destroy the legitimate-interest basis the analytics tables rest on.
This is safe to merge as-is
With
IPLOCATE_API_KEYunset (the default), no IP address reaches any third party. The page shows country data — already covered by existing analytics disclosure — plus companies that self-identified by typing a work email into a form they chose to fill in. Nothing new is disclosed to anyone.So the recommended path is: merge now, switch on after the privacy policy is updated.
To switch network identification on
Order matters.
runtimeConfigbakes at build time (documented in.env.example) and the deploy builds from the environment, so the vars must exist before the deploy that enables them — otherwise the feature ships dark and needs a second deploy.NUXT_IPLOCATE_API_KEYoverrides at runtime without a rebuild.IPLOCATE_API_KEYBRANDFETCH_API_KEYIPLOCATE_DAILY_BUDGETBefore setting
IPLOCATE_API_KEY, publish the privacy disclosure. Paste-ready copy and a pre-publish checklist are indocs/privacy-company-identification.md. The policy is CMS content (singletons, keyprivacy) edited at/admin/pages, so it can't ship in this PR — but the copy is written and reviewable.008_firmographics.sqlruns in the PRE_DEPLOY job.IF NOT EXISTSthroughout; the oneALTER TABLEadds a nullable column (instant, no rewrite).How it works
Waterfall in
server/utils/company.ts:self_reported(work email on /sales — certain) →ip(IPLocate → domain, enriched by Brandfetch) →asn→ nothing.Cost control is the design: a gate (high intent, or a second page view, counted in-process so it adds no query), a free reverse-DNS pre-filter rejecting consumer/hosting networks before any paid call,
network_verdictso each telco is paid for at most once ever, then per-day and per-domain caches. An office of forty costs one lookup a day; Comcast costs one lookup in total.Firmographics reuse the existing Brandfetch call, which was already fetching and discarding employee range, industry and HQ. No new vendor.
Conversion is answered twice, deliberately
form_submitted. Closest to causal, but a floor: the hash rotates at UTC midnight, so browse-Tuesday-convert-Friday is invisible.companies.domain⋈sales_leads.email_domain. Domain-to-domain, never touchingvisitor_hash, so the 006 contract holds. Complete across all time but not attribution; leads predating the visit are marked "earlier".No lead yetsorted by intent is the outreach list.Verified
complete/partialderive correctly. The lead subquery is pre-grouped so it structurally cannot fan out and inflate event counts.kindis an enum (PRIVATELY_HELD) that would have printed raw, andemployeesis an integer, not the range string the comments claimed.ip→self_reportedand refuses the downgrade.Not verified
network_verdictconverges; the budget cap logs when hit.🤖 Generated with Claude Code