Skip to content

feat(analytics): identify the companies browsing the site - #167

Merged
ashrafsamhouri merged 3 commits into
mainfrom
feat/company-identification
Aug 7, 2026
Merged

feat(analytics): identify the companies browsing the site#167
ashrafsamhouri merged 3 commits into
mainfrom
feat/company-identification

Conversation

@ashrafsamhouri

@ashrafsamhouri ashrafsamhouri commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Adds /admin/companies — which organisations are on the site, what they looked at, and whether they are in the pipeline.

Company-level only: the Clearbit-Reveal shape, explicitly not the RB2B person-level one, which would destroy the legitimate-interest basis the analytics tables rest on.

This is safe to merge as-is

With IPLOCATE_API_KEY unset (the default), no IP address reaches any third party. The page shows country data — already covered by existing analytics disclosure — plus companies that self-identified by typing a work email into a form they chose to fill in. Nothing new is disclosed to anyone.

So the recommended path is: merge now, switch on after the privacy policy is updated.

To switch network identification on

Order matters. runtimeConfig bakes at build time (documented in .env.example) and the deploy builds from the environment, so the vars must exist before the deploy that enables them — otherwise the feature ships dark and needs a second deploy. NUXT_IPLOCATE_API_KEY overrides at runtime without a rebuild.

Var Purpose Notes
IPLOCATE_API_KEY IP → company iplocate.io, free 1,000 lookups/day
BRANDFETCH_API_KEY size / industry / HQ Already used for logos — please confirm it is set in prod. Without it companies are identified but have no firmographics to filter on
IPLOCATE_DAILY_BUDGET optional Defaults to 900

Before setting IPLOCATE_API_KEY, publish the privacy disclosure. Paste-ready copy and a pre-publish checklist are in docs/privacy-company-identification.md. The policy is CMS content (singletons, key privacy) edited at /admin/pages, so it can't ship in this PR — but the copy is written and reviewable.

008_firmographics.sql runs in the PRE_DEPLOY job. IF NOT EXISTS throughout; the one ALTER TABLE adds a nullable column (instant, no rewrite).

How it works

Waterfall in server/utils/company.ts: self_reported (work email on /sales — certain) → ip (IPLocate → domain, enriched by Brandfetch) → asn → nothing.

Cost control is the design: a gate (high intent, or a second page view, counted in-process so it adds no query), a free reverse-DNS pre-filter rejecting consumer/hosting networks before any paid call, network_verdict so each telco is paid for at most once ever, then per-day and per-domain caches. An office of forty costs one lookup a day; Comcast costs one lookup in total.

Firmographics reuse the existing Brandfetch call, which was already fetching and discarding employee range, industry and HQ. No new vendor.

Conversion is answered twice, deliberately

  • Converted on site — same-day form_submitted. Closest to causal, but a floor: the hash rotates at UTC midnight, so browse-Tuesday-convert-Friday is invisible.
  • In the pipelinecompanies.domainsales_leads.email_domain. Domain-to-domain, never touching visitor_hash, so the 006 contract holds. Complete across all time but not attribution; leads predating the visit are marked "earlier".

No lead yet sorted by intent is the outreach list.

Verified

  • Migration applies; every query runs against the real schema.
  • Lead join tested against the 11 real lead domains — 11/11 match, complete/partial derive correctly. The lead subquery is pre-grouped so it structurally cannot fan out and inflate event counts.
  • IPLocate tested live: IBM, Siemens, Ford, MIT, Stanford, USGS identified; Comcast, AWS and GitHub-edge correctly skipped.
  • Brandfetch parser checked against Brandfetch's own documented schema — caught two defects, both fixed in d7b8084: kind is an enum (PRIVATELY_HELD) that would have printed raw, and employees is an integer, not the range string the comments claimed.
  • Reverse-DNS classifier tested on real IPs — caught a false positive (GitHub's LB embeds its IP like a dynamic pool) that would have silently skipped corporate networks.
  • Confidence ladder upgrades ipself_reported and refuses the downgrade.
  • Build, types, format, knip clean.

Not verified

  • The authenticated page render. Verifying it needs an admin session, and both routes to one were correctly blocked by security controls (forging a session row; reading a live OTP from logs). The template compiles and the endpoint builds, but no one has seen it render with rows. Worth a look on first login.
  • Expect a thin match rate (single- to low-double-digit % of looked-up visitors), weighted to corporate networks. The page states its denominator out loud.
  • The widened gate will spend more lookups until network_verdict converges; the budget cap logs when hit.

🤖 Generated with Claude Code

Ashraf Samhouri and others added 3 commits August 5, 2026 21:53
Adds /admin/companies: which organisations are on the site, what they
looked at, and whether they are in the pipeline. Company-level only —
the Clearbit-Reveal shape, explicitly not the RB2B person-level one.

The privacy contract in 006 is amended, not broken. A visitor-day may
now carry a company DOMAIN, from the network it came from or from the
domain of a work email consented into /sales. No email, no name, and no
identity-graph product. The address is never written.

Resolution waterfall (server/utils/company.ts):
  self_reported  work email on /sales — certain
  ip             IPLocate netblock -> domain, enriched by Brandfetch
  asn            operator known, no company
  nothing        consumer ISP, mobile, hosting, VPN — most traffic

Cost control is the design, and it is what makes this fit a free tier:
a gate (high intent, or a second page view, counted in-process so it
costs no query), a free reverse-DNS pre-filter that rejects consumer and
hosting networks before any paid call, network_verdict so each telco is
paid for at most once ever, then per-day and per-domain caches. An
office of forty people costs one lookup a day; Comcast costs one lookup
in total. IPLOCATE_DAILY_BUDGET is a hard ceiling that logs when hit.

Firmographics reuse the existing Brandfetch integration, which was
already fetching and discarding employee range, industry and HQ.

Conversion is answered twice because the answers differ:
submittedInSession is same-day and closest to causal but is a floor (the
hash rotates at UTC midnight); conversion joins companies.domain to
sales_leads.email_domain — domain to domain, never touching
visitor_hash — which is complete across all time but is not attribution.
Both are labelled in the UI, and "no lead yet" sorted by intent is the
outreach list.

Education and government resolve to real domains (mit.edu, usgs.gov) and
count as organisations; only isp and hosting are excluded.

IPLOCATE_API_KEY is optional. Without it, countries (free, from
Cloudflare) and self-reported companies still work, and the page says
identification is off rather than rendering an empty table.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Verified the firmographics parser against Brandfetch's own documented
response schema rather than an assumed one, which surfaced two defects:

  * `kind` is an enum token ('PRIVATELY_HELD'), not a display string, so
    the table would have printed raw enum text. Humanised.
  * `employees` is an INTEGER (1001), not the range string the comments
    claimed. Formatted with separators for display; the parsed lower
    bound the size filter depends on was already correct.

Checked against the documented stripe.com payload, the parser now
returns employees "1,001" / min 1001, kind "Privately held", industry
taken from the highest-scored entry, and countryCode "US".

Also adds docs/privacy-company-identification.md: paste-ready policy
copy for the disclosure this feature requires, plus a pre-publish
checklist. The policy itself is CMS content (singletons, key 'privacy')
edited at /admin/pages, so it cannot ship in code — but the copy can.

Note the safe intermediate state, now documented: with IPLOCATE_API_KEY
unset, no IP address reaches any third party, so the feature can ship
before the policy is updated and be switched on afterwards.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… lookup

Tested the Brandfetch parser against the real API rather than the
documented schema. Siemens, MIT, IBM, Ford and USGS all parse correctly
— sizes, industries, "Public company"/"Government agency" kinds — and
two defects fell out:

  * foundedYear is 0, not null, when Brandfetch does not know it (IBM).
    num() accepts 0 as finite, so the column would have filled with a
    year rendering as "Founded 0". Rejected explicitly.
  * Brandfetch has no location for plenty of real organisations — MIT
    and Ford both return null country — which left the flag column blank
    on rows we demonstrably observed a country for. Falls back to the
    country the visit actually came from.

Also reads process.env alongside runtimeConfig for the IPLocate key.
This is not redundant: runtimeConfig bakes at BUILD time, and this app
already scopes ANALYTICS_SALT_SECRET as RUN_TIME on DigitalOcean. An
IPLOCATE_API_KEY added with that same scope would never reach
runtimeConfig, and the feature would stay dark with no error anywhere.
Reading both means the key works under either scope, and under the
NUXT_-prefixed name.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@ashrafsamhouri
ashrafsamhouri merged commit ec77ee4 into main Aug 7, 2026
@ashrafsamhouri
ashrafsamhouri deleted the feat/company-identification branch August 7, 2026 18:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant