diff --git a/docs/changelog/2026-W47.md b/docs/changelog/2026-W47.md index 4522b5d..3098c30 100644 --- a/docs/changelog/2026-W47.md +++ b/docs/changelog/2026-W47.md @@ -33,3 +33,19 @@ **Files touched**: `engine/entities/samples.ts`, `engine/entities/aquifers.ts`, `engine/entities/streams.ts` **Prevention**: `engine/tests/blockWording.test.ts` checks the aquifer and streams wording; `engine/tests/stepLabels.test.ts` checks the sample step labels, and its raw-type-name check now allows labels that contain their type name; `engine/tests/__snapshots__/question-text.txt` re-recorded. + +3. **Added: Counts Next to Every Filter Dropdown Option** (2026-11-17) + + **Component**: `buildOptionCountQuery` in `engine/templates/fusedQueries.ts`, `options` on the entity modules in `engine/entities/`, `useOptionCounts` in `ui/hooks/useDiscoveryQueries.ts` + + **Change**: Issue #165, a follow-up from #70. Industry, Feature Type, Aquifer Type and Well classification now show how many entities each option would find: in the selected state or counties, or nationwide with none. Industry used to show counts only once a state was picked, Well classification only for the whole of Illinois or Maine, and Feature Type and Aquifer Type none. Feature Type parents show the sum of their children; Aquifer Type sums the raw values each option covers. + + **How**: each count is the run's own count step with the filter left open and grouped by option, so it checks the region exactly as the run does. Hancock County wells: 7,296, where a count by subdivision link alone gave 11,644. Live queries on `federation`, cached for the session, nothing stored. A failed count leaves the dropdown working, with no numbers. + + **Cost**: measured live on 2026-11-17, 1.8s or less for every type, nationwide and for Maine, Hancock and Illinois, except Illinois aquifers cold (30.1s, 0.46s on a rerun; the run's own count step for that block took 25.8s), where the first open may show no numbers. + + **Unchanged**: the SPARQL every question sends (`query-shapes.txt` did not move). Substance and Material counts. + + **Files touched**: `engine/templates/fusedQueries.ts`, `engine/templates/regions.ts` (`buildIndustryCountsQuery` removed), `engine/templates/wells.ts`, `engine/entities/*.ts`, `engine/optionCounts.ts` (new), `ui/hooks/useDiscoveryQueries.ts`, `ui/components/QueryEditor/` (Facility, SurfaceWater, Aquifer and Well filters, `EntityBlock.tsx`, `FlatSelect/withCount.ts` new), `docs/wiki/Dropdowns Industry Counts.md` (now "Option counts"), `docs/wiki/Dropdowns.md` + + **Prevention**: `engine/tests/optionCounts.test.ts` checks that every counted type's count query carries the same region lines as its count step, for no region, a state, a county and two counties, and how rows become counts; `ui/tests/optionCounts.test.tsx` checks each dropdown shows its counts for the right region, a failed count, and that a region already seen is not fetched again. diff --git a/docs/wiki/Dropdowns Industry Counts.md b/docs/wiki/Dropdowns Industry Counts.md index de3dc2b..e606f3f 100644 --- a/docs/wiki/Dropdowns Industry Counts.md +++ b/docs/wiki/Dropdowns Industry Counts.md @@ -1,132 +1,125 @@ -# Industry counts +# Option counts -Not a dropdown of its own. These are the facility counts shown next to each entry in the -[Industry dropdown](Dropdowns%20Industry), and they come from a separate query against a -different endpoint, so they get their own page. +Not a dropdown of its own. These are the numbers next to each option in the +[Industry](Dropdowns%20Industry), Feature Type, Aquifer Type and Well classification +dropdowns, and they all come from one query shape, so they share this page. (Substance and +Material count observations with their own queries, on their own pages.) ![Industry counts](images/dropdown_industry_counts.png) ## What it does -Once a state or some counties are selected, each node in the industry tree picks up a number: -how many facilities of that kind actually exist in that region. It turns an abstract list of -5596 NAICS codes into something you can steer by, because you can see at a glance which -industries are worth filtering on and which will return nothing. +Each option shows how many entities picking it would find. With a state or counties +selected, the count is for that region. With none, it is nationwide. An option the region +has none of shows no number and is dimmed in the tree dropdowns. -With no region selected there are no counts at all. The query is gated behind having a state, -because counting every facility in the country is both slow and not useful for picking a -filter. +A count previews the answer, so it has to find exactly what the run finds. It is the run's +own count step (`buildEntityCountQuery`, the step that opens every run) with the filter left +open and grouped by option. That matters most for types whose region check looks at the +entity as well as its S2 cell. Wells in Hancock County, Maine (2026-11-17): -Counts roll up the tree. A leaf shows what the endpoint returned for that exact code. A -parent shows the sum of itself and everything beneath it, computed on the client by -`rollupCounts()`. So a sector node showing 400 means four hundred facilities spread across -its descendants, not four hundred facilities tagged with the bare sector code. +| Region check | Wells | +|---|---| +| Subdivision link only | 11,644 | +| S2 cell only | 7,428 | +| Both, as the run checks wells | 7,296 | -There is **no fallback**. If the query fails you get an empty object and the tree simply -shows no numbers. Nothing breaks, you just lose the guidance. +A count built any other way would promise one number and the map would show another. + +Tree dropdowns roll counts up: a parent shows the sum of its children, computed on the +client by `rollupCounts()`. That is sound because the children partition the parent (an +FTYPE value or a NAICS code belongs to one parent only). + +There is **no fallback**. A failed count leaves the dropdown working, with no numbers. ## The SPARQL query -Endpoint is `federation`, not `fiokg`. This is the important detail: the industry list comes -from `fiokg` but the counts cannot, because counting facilities per region needs the industry -graph and the spatial graph joined together, and only `federation` has both. +Endpoint `federation` for every type. Built by `buildOptionCountQuery(type, regionCodes)`. +Industry in Hancock County: ```sparql -SELECT ?industryCode (COUNT(DISTINCT ?facility) AS ?num) WHERE { - VALUES ?_regionRoot { kwgr:administrativeRegion.USA.23 } - ?s2 rdf:type kwg-ont:S2Cell_Level13 ; - spatial:connectedTo ?_regionRoot ; - kwg-ont:sfContains ?facility . - ?facility fio:ofIndustry ?industryCode . - ?industryCode a naics:NAICS-IndustryCode . -} GROUP BY ?industryCode +SELECT ?option (COUNT(DISTINCT ?facilityP) AS ?count) WHERE { + ?s2probe rdf:type kwg-ont:S2Cell_Level13 . + ?s2probe spatial:connectedTo kwgr:administrativeRegion.USA.23009 . + ?facilityP spatial:connectedTo ?_regionLinkP . + ?_regionLinkP kwg-ont:administrativePartOf* kwgr:administrativeRegion.USA.23009 . + ?s2probe kwg-ont:sfContains ?facilityP . + ?facilityP fio:ofIndustry ?option . + ?option a naics:NAICS-IndustryCode . +} GROUP BY ?option ``` -Line by line: +* The three lines after the S2 cell line are the run's region check for this type, taken from + the same code (`regionClause` and the type's `regionCheck`). With no region they are + absent and the count is nationwide. +* The last lines are what the type binds when one option is picked, with the option left + open in `?option`. Each type's module in `engine/entities/` supplies it as + `options.bind`: -* `VALUES ?_regionRoot { ... }` is the selected region. One entry for a whole state, or one - per county when counties are picked. `23` is Maine. -* `?s2 rdf:type kwg-ont:S2Cell_Level13` brings in S2 cells, the grid the whole app uses for - spatial joins. Each cell covers roughly 1.2 square kilometres. -* `spatial:connectedTo ?_regionRoot` keeps only the cells inside the chosen region. -* `kwg-ont:sfContains ?facility` gets the facilities inside those cells. -* `?facility fio:ofIndustry ?industryCode` is what kind of facility it is, and the - `naics:NAICS-IndustryCode` line keeps only real NAICS codes rather than any other - classification hanging off the same predicate. -* `COUNT(DISTINCT ?facility)` per code. `DISTINCT` matters, because a facility sitting on a - cell boundary can be reached more than once. +| Dropdown | Type | `?option` is | +|---|---|---| +| Industry | facilities | `fio:ofIndustry`, a NAICS code IRI. No geometry check, as when an industry is picked | +| Feature Type | waterBodies, streams | `nhdplusv2:hasFTYPE`, e.g. `"LakePond"` | +| Aquifer Type | aquifers | `saw_water:aquiferType`, a raw value such as `"sand_gravel"` | +| Well classification | wells | Illinois `wellPurpose`, Maine `ofWellType` or `hasUse`, an IRI | -Note the route through S2 cells rather than asking directly which facilities are in Maine. -That is the standard pattern across this app: everything spatial goes through S2 Level 13 -cells. See [`docs/ARCHITECTURE.md`](https://github.com/SAWGraph/explorer-app/blob/main/docs/ARCHITECTURE.md) -for why. +### What comes back, and how it is keyed -Verified against the live endpoint. Maine returns 664 distinct industry codes with counts, -from single facility codes like `111110` Soybean Farming through to the busier ones. +One row per option, `?option` and `?count`. `optionCountsFrom()` keys them the way each +dropdown keys its options, using the type's `options.key`: -### What comes back +* Industry: the IRI is cut to the bare code (`NAICS-111110` to `111110`). +* Aquifer Type: raw values are summed into the two options through `AQUIFER_TYPE_VALUES`. + Sand & gravel covers `sand and gravel`, `coarse-grain_materials` and `sand_gravel`. +* Well classification: keyed by IRI. An Illinois entry stands for an active and a plugged + purpose, so `wellTokenCount()` sums the IRIs in the entry's token. +* Feature Type: the FTYPE value as is. -Full URIs, not bare codes: +### Measured -``` -http://w3id.org/fio/v1/naics#NAICS-111110 1 -http://w3id.org/fio/v1/naics#NAICS-11121 4 -``` +Live, 2026-11-17, first run unless noted: -Unlike the industry list query, this one does not strip the prefix in SPARQL. The hook does -it, splitting on `#` or `/` and then removing the leading `NAICS-`. The result has to match -the bare codes the tree is keyed on, so if either side changes shape the numbers silently -stop appearing. +| Region | Feature Type (water bodies / flowlines) | Aquifer Type | Wells | Industry | +|---|---|---|---|---| +| Nationwide | 0.33s / 0.14s | 0.22s | 0.22s | 0.15s | +| Maine | 0.31s / 0.17s | 0.40s | 0.43s | 1.42s | +| Hancock | 0.18s / 0.24s | 0.36s | 0.43s | 0.35s | +| Illinois | 0.17s / 0.54s | 30.1s cold, 0.46s rerun | 0.66s | 1.74s | + +Illinois aquifers are slow cold because the subdivision check touches every Illinois +subdivision an aquifer overlaps. The run's own count step for the same block took 25.8s +cold. + +Sample counts from the same run: nationwide Lake / Pond 60,083, Stream / River 353,003; +Hancock Lake / Pond 348, sand & gravel aquifers 181, Maine bedrock wells 7,236. ## How it is wired up -* There is no control of its own. The numbers are passed into `HierarchicalSelect` as a - `counts` prop at - [`FacilityFilters.tsx:24`](https://github.com/SAWGraph/explorer-app/blob/main/src/components/QueryEditor/FacilityFilters.tsx#L24). -* The data comes from - [`useIndustryCounts(region)`](https://github.com/SAWGraph/explorer-app/blob/main/src/hooks/useDiscoveryQueries.ts#L76), - called at - [`FacilityFilters.tsx:13`](https://github.com/SAWGraph/explorer-app/blob/main/src/components/QueryEditor/FacilityFilters.tsx#L13). -* The query text is built by - [`buildIndustryCountsQuery()`](https://github.com/SAWGraph/explorer-app/blob/main/src/engine/templates/regions.ts#L14). -* It runs through - [`executeSparql()`](https://github.com/SAWGraph/explorer-app/blob/main/engine/endpoint.ts) - against the `federation` URL in - [`endpoints.ts`](https://github.com/SAWGraph/explorer-app/blob/main/src/constants/endpoints.ts). -* The rollup is - [`rollupCounts()`](https://github.com/SAWGraph/explorer-app/blob/main/src/components/QueryEditor/HierarchicalSelect/useNaicsTree.ts#L148), - now shared with the [Material](Dropdowns%20Material) dropdown. Summing children into a - parent is only sound when the children partition the parent. That holds for NAICS by - construction, and for material types because `MIN(?bucketPrio)` forces each one into - exactly one group. It would not hold for a set of overlapping categories. - -Cached under `['industryCounts', ]` with `staleTime: Infinity` and gated by -`enabled: !!region?.stateCode`. The region key sorts the county codes before joining them, so -picking the same counties in a different order reuses the cache instead of refetching. - -The hook returns a plain `Record` keyed by bare code, which is what -`rollupCounts()` expects as its leaf counts. +* `useOptionCounts(type, region)` in `ui/hooks/useDiscoveryQueries.ts` runs the query and + returns `Record`. Cached under `['optionCounts', type, ]` with + `staleTime: Infinity`, so each type and region is fetched once a session. The region key + sorts the county codes, so the same counties in another order reuse the cache. +* It runs when the dropdown mounts: Feature Type as soon as the Type is Surface Water (both + water bodies and flowlines are counted, since the tree holds both), the others once + "+ Add Filters" is open. +* `FacilityFilters` and `SurfaceWaterFilters` pass the counts to `HierarchicalSelect` as its + `counts` prop. `AquiferFilters` and `WellFilters` put them in the label with `withCount()`. +* Well classification lists its options from `hydrologykg` with counts for the whole of + Illinois or Maine. Those are not shown; the region counts replace them. ## If it looks wrong -**No numbers anywhere.** Either no state is selected, which is expected, or the `federation` -query failed. There is no fallback, so failure looks exactly like "not selected yet". Check -the network tab to tell them apart. - -**Numbers appear on some nodes but not others.** Those codes have no facilities in the -selected region. Zero is not returned, so nothing shows. +**No numbers anywhere.** The count query failed, or has not come back yet. Check the +network tab for a `federation` request with `GROUP BY ?option`. -**A parent shows a bigger number than the sum of its visible children.** Expected when a -child code exists in the counts result but not in the industry list, or when the tree is -flatter than NAICS because of a missing intermediate code. The rollup only sums nodes that -are in the tree. +**Numbers on some options but not others.** The region has none of those. Zero is not +returned, so nothing shows. -**Numbers vanish after switching from a state to counties.** New cache key, new fetch. If -they do not come back, one of the county codes is not resolving to a region root. +**A count does not match the map.** Compare the region lines of the count with the run's +count step for the same block. `engine/tests/optionCounts.test.ts` asserts they are the same +for every type and region form. -**Counts look too high for a small county.** Facilities on S2 cell boundaries can appear in -cells belonging to more than one region. `COUNT(DISTINCT ?facility)` dedupes within a single -result, but the cell to region join is inclusive by design. +**Numbers vanish after switching from a state to counties.** New cache key, new fetch. ## See also diff --git a/docs/wiki/Dropdowns Industry.md b/docs/wiki/Dropdowns Industry.md index 0b94c42..c994ca2 100644 --- a/docs/wiki/Dropdowns Industry.md +++ b/docs/wiki/Dropdowns Industry.md @@ -19,9 +19,9 @@ selected. Tick some of a branch and the parent shows an indeterminate state. The full list is large, 5596 rows from the endpoint, which is why it is a searchable tree rather than a list you scroll. Typing filters the tree and auto expands whatever matched. -Each node also shows how many facilities of that kind exist in the currently selected region. -Those numbers come from a completely separate query, documented in -[Dropdowns Industry Counts](Dropdowns%20Industry%20Counts). +Each node also shows how many facilities of that kind exist in the selected region, or +nationwide with none. Those numbers come from a separate query, documented in +[Option counts](Dropdowns%20Industry%20Counts). ### Nothing ticked is not the same as everything ticked @@ -152,8 +152,8 @@ what the query actually returned before assuming the tree builder is broken. **Labels with odd trailing spaces.** Source data. They come through as is. -**No counts next to anything.** No region is selected, or the counts query failed. See -[Dropdowns Industry Counts](Dropdowns%20Industry%20Counts). +**No counts next to anything.** The counts query failed or has not come back. See +[Option counts](Dropdowns%20Industry%20Counts). **Selecting a parent does not match the facilities you expected.** Selections are expanded to descendant codes, so a parent only covers children that exist in this list. If the facility @@ -162,4 +162,4 @@ you want carries a code absent from `fiokg`, no amount of parent ticking will re ## See also * [Dropdowns](Dropdowns), the index of all filter controls -* [Dropdowns Industry Counts](Dropdowns%20Industry%20Counts), the numbers shown on each node +* [Option counts](Dropdowns%20Industry%20Counts), the numbers shown on each node diff --git a/docs/wiki/Dropdowns.md b/docs/wiki/Dropdowns.md index 3f3e603..d8d08d0 100644 --- a/docs/wiki/Dropdowns.md +++ b/docs/wiki/Dropdowns.md @@ -20,9 +20,10 @@ themselves tickable and collapsible. Falls back to six material types. NAICS codes, as a searchable tree. Shown for Facilities. Endpoint `fiokg`. Falls back to eight codes covering the prebuilt questions. -**[Industry counts](Dropdowns%20Industry%20Counts)** -The facility counts shown on each industry tree node. Endpoint `federation`. No fallback, -and nothing at all until a state is selected. +**[Option counts](Dropdowns%20Industry%20Counts)** +The numbers next to each option in Industry, Feature Type, Aquifer Type and Well +classification: for the selected region, or nationwide with none. Endpoint `federation`, one +query per type and region. No fallback. **[County](Dropdowns%20County)** Narrows to counties inside the selected state. Endpoint `spatialkg`. No fallback, on purpose. @@ -66,15 +67,25 @@ with two parents, Surface Water Bodies (Lake or Pond, Swamp or Marsh, Reservoir) and Flowlines (Stream or River, Artificial Path, Canal or Ditch, Connector, Pipeline). Only one parent can hold ticks at a time, and that parent sets the block's type. It starts with Surface Water Bodies ticked; unticking everything shows an empty tree, and applying it empty runs -Surface Water Bodies. A whole parent is stored as no filter, so it sends the same query as an unfiltered block. NHDPlus +Surface Water Bodies. A whole parent is stored as no filter, so it sends the same query as an unfiltered block. Each +option shows its count, and each parent the sum of its children +([Option counts](Dropdowns%20Industry%20Counts)). NHDPlus FTYPEs that are rare or absent in the source files (Inundation Area, Estuary, Ice Mass, Playa, Drainageway, Underground Conduit) are left out (#70). [`SurfaceWaterFilters.tsx`](https://github.com/SAWGraph/explorer-app/blob/main/src/components/QueryEditor/SurfaceWaterFilters.tsx), [`constants/featureTypes.ts`](https://github.com/SAWGraph/explorer-app/blob/main/src/constants/featureTypes.ts) -**Well Source**. Illinois Wells and Maine Wells. Each is tied to one state, and picking a -source that contradicts the selected state raises a warning. -[`WellFilters.tsx`](https://github.com/SAWGraph/explorer-app/blob/main/src/components/QueryEditor/WellFilters.tsx) +**Aquifer Type**, shown for Ground Water (Aquifers). Two options, Sand & gravel (surficial) +and Bedrock, each standing for one or more raw `saw_water:aquiferType` values +(`AQUIFER_TYPE_VALUES`). Each shows its count +([Option counts](Dropdowns%20Industry%20Counts)). +[`AquiferFilters.tsx`](https://github.com/SAWGraph/explorer-app/blob/main/ui/components/QueryEditor/AquiferFilters.tsx) + +**Well classification**, shown for Wells. Not a fixed list: the options come from +`hydrologykg` (Illinois well purpose, Maine well type and use), grouped by state and field. A +selected state hides the other state's groups. Each option shows its count for the region +([Option counts](Dropdowns%20Industry%20Counts)). +[`WellFilters.tsx`](https://github.com/SAWGraph/explorer-app/blob/main/ui/components/QueryEditor/WellFilters.tsx) **Basemap**, on the map itself. Five tile layers. [`basemaps.ts`](https://github.com/SAWGraph/explorer-app/blob/main/src/components/Map/basemaps.ts) diff --git a/docs/wiki/Home.md b/docs/wiki/Home.md index 1e8deba..380c9d6 100644 --- a/docs/wiki/Home.md +++ b/docs/wiki/Home.md @@ -16,7 +16,7 @@ Where the options in each filter control come from. * [Substance](Dropdowns%20Substance) * [Material](Dropdowns%20Material) * [Industry](Dropdowns%20Industry) -* [Industry counts](Dropdowns%20Industry%20Counts) +* [Option counts](Dropdowns%20Industry%20Counts) * [County](Dropdowns%20County) * [Include non-detects](Samples%20Non%20Detects), the Samples checkbox that moves the numbers most diff --git a/docs/wiki/_Sidebar.md b/docs/wiki/_Sidebar.md index 12c9a07..2cc97ff 100644 --- a/docs/wiki/_Sidebar.md +++ b/docs/wiki/_Sidebar.md @@ -8,7 +8,7 @@ * [Substance](Dropdowns%20Substance) * [Material](Dropdowns%20Material) * [Industry](Dropdowns%20Industry) -* [Industry counts](Dropdowns%20Industry%20Counts) +* [Option counts](Dropdowns%20Industry%20Counts) * [County](Dropdowns%20County) **Samples** diff --git a/engine/entities/aquifers.ts b/engine/entities/aquifers.ts index 6b18291..6ef1b0c 100644 --- a/engine/entities/aquifers.ts +++ b/engine/entities/aquifers.ts @@ -4,7 +4,14 @@ import { ownFilterSet, subdivisionRegionCheck, type EntityModule } from './types const LABEL = 'ground water aquifers'; +// Each raw aquiferType value, to the dropdown option that covers it. +const AQUIFER_OPTION_OF: Record = Object.fromEntries( + Object.entries(AQUIFER_TYPE_VALUES).flatMap(([option, raws]) => raws.map((raw) => [raw, option])), +); + const iriVar = (suffix: string): string => `?aquifer${suffix}`; +const inCell = (s2Var: string, suffix: string): string => `${s2Var} spatial:connectedTo ${iriVar(suffix)} . + ${iriVar(suffix)} rdf:type gwml2:GW_Aquifer .`; // One kind selected reads as an adjective ("bedrock ground water aquifers"). const KIND_WORDS: Record = { surficial: 'sand & gravel', @@ -34,10 +41,15 @@ export const aquifers: EntityModule = { iriVar, bindInCell(block, s2Var, suffix) { const typeFilter = aquiferTypeFilter(block.aquiferFilters, suffix); - return `${s2Var} spatial:connectedTo ?aquifer${suffix} . - ?aquifer${suffix} rdf:type gwml2:GW_Aquifer . + return `${inCell(s2Var, suffix)} ${typeFilter}`; }, + // The bind finds raw aquiferType values; each dropdown option covers several. + options: { + bind: (s2Var, suffix, optionVar) => `${inCell(s2Var, suffix)} + ${iriVar(suffix)} saw_water:aquiferType ${optionVar} .`, + key: (raw) => AQUIFER_OPTION_OF[raw], + }, // The bind alone is one row per entity and cell (measured in Maine, // 2026-11-06). The region check is not: it joins the aquifer to each of its // subdivisions in the region. Collapsed first, Hancock "samples downstream diff --git a/engine/entities/facilities.ts b/engine/entities/facilities.ts index c3c80ef..2501f7a 100644 --- a/engine/entities/facilities.ts +++ b/engine/entities/facilities.ts @@ -8,6 +8,11 @@ import { namedOrCounted } from './wording'; const LABEL = 'facilities'; const iriVar = (suffix: string): string => `?facility${suffix}`; +const inCell = (s2Var: string, suffix: string): string => `${s2Var} kwg-ont:sfContains ${iriVar(suffix)} .`; +// The facility's industry, in `codeVar`: what an industry filter and the +// dropdown's counts both bind. +const ofIndustry = (suffix: string, codeVar: string): string => `${iriVar(suffix)} fio:ofIndustry ${codeVar} . + ${codeVar} a naics:NAICS-IndustryCode .`; export const facilities: EntityModule = { type: 'facilities', @@ -24,18 +29,22 @@ export const facilities: EntityModule = { // 2026-11-16, #145). The geometry is required because the details step // cannot draw a facility without one (Cumberland: 2,595 without the check). bindInCell(block, s2Var, suffix) { - const facility = iriVar(suffix); - const inCell = `${s2Var} kwg-ont:sfContains ${facility} .`; const codes = (block.facilityFilters as FacilityFilters | undefined)?.industryCodes; if (!codes?.length) { - return `${inCell} - ${facility} a fio:Facility ; geo:hasGeometry ?_facilityGeom${suffix} .`; + return `${inCell(s2Var, suffix)} + ${iriVar(suffix)} a fio:Facility ; geo:hasGeometry ?_facilityGeom${suffix} .`; } - return `${inCell} - ${facility} fio:ofIndustry ?industryCode${suffix} . - ?industryCode${suffix} a naics:NAICS-IndustryCode . + return `${inCell(s2Var, suffix)} + ${ofIndustry(suffix, `?industryCode${suffix}`)} ${buildIndustryValues(codes, suffix)}`; }, + // As the filtered bind above: a picked industry reads no geometry. The + // dropdown keys industries by bare code. + options: { + bind: (s2Var, suffix, optionVar) => `${inCell(s2Var, suffix)} + ${ofIndustry(suffix, optionVar)}`, + key: (iri) => iri.split(/[#/]/).pop()?.replace(/^NAICS-/, '') || undefined, + }, // A facility has one row per industry code that passes the filter. A full // 6-digit code matches itself only, so those sides are one row per facility // already (landfills and DoD sites nationwide: 5,922 rows for 5,908); a diff --git a/engine/entities/streams.ts b/engine/entities/streams.ts index 4dcf3b3..cd9a76f 100644 --- a/engine/entities/streams.ts +++ b/engine/entities/streams.ts @@ -6,6 +6,10 @@ import { ftypeLabels, namedOrCounted } from './wording'; const LABEL = 'streams and flowlines'; +const iriVar = (suffix: string): string => `?stream${suffix}`; +const inCell = (s2Var: string, suffix: string): string => `${s2Var} spatial:connectedTo ${iriVar(suffix)} . + ${iriVar(suffix)} rdf:type hyf:HY_FlowPath .`; + // FTYPE restriction for flowlines, shared by the cell bind below and by the // trace query, where a streams target is the traced flowline itself. export function streamFtypeFilter(filters: StreamFilters | undefined, suffix: string): string { @@ -24,12 +28,15 @@ export const streams: EntityModule = { endpoint: 'hydrologykg', buildQuery: (iris, block) => buildStreamsByIri(iris, block.streamFilters), }, - iriVar: (suffix) => `?stream${suffix}`, + iriVar, bindInCell(block, s2Var, suffix) { - return `${s2Var} spatial:connectedTo ?stream${suffix} . - ?stream${suffix} rdf:type hyf:HY_FlowPath . + return `${inCell(s2Var, suffix)} ${streamFtypeFilter(block.streamFilters, suffix)}`; }, + options: { + bind: (s2Var, suffix, optionVar) => `${inCell(s2Var, suffix)} + ${iriVar(suffix)} nhdplusv2:hasFTYPE ${optionVar} .`, + }, // One row per entity and cell already (measured in Maine, 2026-11-06). fansOut: () => false, filterSlot: 'streamFilters', diff --git a/engine/entities/types.ts b/engine/entities/types.ts index 9e116bf..f7ac71a 100644 --- a/engine/entities/types.ts +++ b/engine/entities/types.ts @@ -21,6 +21,10 @@ export interface EntityModule { // Binds the entity inside the S2 cell `s2Var`, with the block's filters, and // only entities the map can draw. bindInCell(block: EntityBlock, s2Var: string, suffix: string): string; + // The type's filter dropdown, for counting each option + // (buildOptionCountQuery, #165). Absent for a type whose dropdowns are + // counted elsewhere (samples). + readonly options?: OptionCounting; // Whether the bind can yield more than one row per entity and cell (a join // on a field with several values: a sample's measurements, a facility's // industry codes). Such a side is collapsed to one row per entity and cell @@ -41,6 +45,15 @@ export interface EntityModule { regionCheck?(suffix: string, anyOf: (variable: string) => RegionMatch): string; } +export interface OptionCounting { + // Binds the entity inside the cell as bindInCell does when one option is + // picked, with the option left open in `optionVar`. + bind(s2Var: string, suffix: string, optionVar: string): string; + // The key the dropdown stores for an option the bind found, or undefined + // when it lists no such option. Absent: the value itself. + key?(option: string): string | undefined; +} + export type DetailsLoad = // Load the IRIs discovery found, in request-size chunks. | { kind: 'by-iri'; endpoint: EndpointKey; buildQuery(iris: string[], block: EntityBlock): string } diff --git a/engine/entities/waterBodies.ts b/engine/entities/waterBodies.ts index 30ca186..5198f99 100644 --- a/engine/entities/waterBodies.ts +++ b/engine/entities/waterBodies.ts @@ -5,6 +5,10 @@ import { ftypeLabels, namedOrCounted } from './wording'; const LABEL = 'surface water bodies'; +const iriVar = (suffix: string): string => `?waterBody${suffix}`; +const inCell = (s2Var: string, suffix: string): string => `${s2Var} spatial:connectedTo ${iriVar(suffix)} . + ${iriVar(suffix)} rdf:type hyf:HY_WaterBody .`; + export const waterBodies: EntityModule = { type: 'waterBodies', label: LABEL, @@ -14,7 +18,7 @@ export const waterBodies: EntityModule = { endpoint: 'hydrologykg', buildQuery: (iris, block) => buildWaterBodiesByIri(iris, block.waterBodyFilters), }, - iriVar: (suffix) => `?waterBody${suffix}`, + iriVar, bindInCell(block, s2Var, suffix) { let filterClauses = ''; if (block.waterBodyFilters?.ftypes?.length) { @@ -22,10 +26,13 @@ export const waterBodies: EntityModule = { filterClauses = `?waterBody${suffix} nhdplusv2:hasFTYPE ?ftype${suffix} . VALUES ?ftype${suffix} { ${ftypeValues} }`; } - return `${s2Var} spatial:connectedTo ?waterBody${suffix} . - ?waterBody${suffix} rdf:type hyf:HY_WaterBody . + return `${inCell(s2Var, suffix)} ${filterClauses}`; }, + options: { + bind: (s2Var, suffix, optionVar) => `${inCell(s2Var, suffix)} + ${iriVar(suffix)} nhdplusv2:hasFTYPE ${optionVar} .`, + }, // One row per entity and cell already (measured in Maine, 2026-11-06). fansOut: () => false, filterSlot: 'waterBodyFilters', diff --git a/engine/entities/wells.ts b/engine/entities/wells.ts index 4360f42..5b661d0 100644 --- a/engine/entities/wells.ts +++ b/engine/entities/wells.ts @@ -1,4 +1,4 @@ -import { buildWellCategoryFilter } from '../templates/wells'; +import { buildWellCategoryFilter, buildWellOptionPattern } from '../templates/wells'; import { ownFilterSet, subdivisionRegionCheck, type EntityModule } from './types'; import { extractLabel, namedOrCounted } from './wording'; @@ -19,6 +19,11 @@ export const wells: EntityModule = { return `${s2Var} spatial:connectedTo ?well${suffix} . ${typeFilter}`; }, + // Keyed by classification IRI; an entry can stand for several (wellTokenCount). + options: { + bind: (s2Var, suffix, optionVar) => `${s2Var} spatial:connectedTo ${iriVar(suffix)} . + ${buildWellOptionPattern(iriVar(suffix), optionVar)}`, + }, // One row per entity and cell already (measured in Maine, 2026-11-06). fansOut: () => false, filterSlot: 'wellFilters', diff --git a/engine/optionCounts.ts b/engine/optionCounts.ts new file mode 100644 index 0000000..06b56ec --- /dev/null +++ b/engine/optionCounts.ts @@ -0,0 +1,29 @@ +import type { EntityType } from './types/query'; +import type { SparqlRow } from './types/sparql'; +import { entityFor } from './entities'; +import { parseWellToken } from './constants/wellClassifications'; + +// The rows of buildOptionCountQuery, keyed the way the type's dropdown keys +// its options: an FTYPE value, an aquifer option ('surficial', 'bedrock'), a +// bare NAICS code, or a well classification IRI (see wellTokenCount). Rows +// that share a key are summed: an aquifer has one type, so the raw values an +// option covers count different aquifers. +export function optionCountsFrom(type: EntityType, rows: SparqlRow[]): Record { + const keyOf = entityFor(type).options?.key ?? ((option: string) => option); + const counts: Record = {}; + for (const row of rows) { + if (!row.option || !row.count) continue; + const key = keyOf(row.option); + if (!key) continue; + counts[key] = (counts[key] ?? 0) + Number(row.count); + } + return counts; +} + +// A well classification entry stands for one or more IRIs (Illinois active and +// plugged twins share an entry), each well carrying one of them. +export function wellTokenCount(counts: Record, token: string): number | undefined { + const found = parseWellToken(token).uris.filter((uri) => uri in counts); + if (!found.length) return undefined; + return found.reduce((sum, uri) => sum + counts[uri], 0); +} diff --git a/engine/templates/fusedQueries.ts b/engine/templates/fusedQueries.ts index 6ea7453..aeaeb81 100644 --- a/engine/templates/fusedQueries.ts +++ b/engine/templates/fusedQueries.ts @@ -1,5 +1,5 @@ import { PREFIXES } from '../constants/prefixes'; -import type { EntityBlock } from '../types/query'; +import type { EntityBlock, EntityType } from '../types/query'; import { entityFor, streamFtypeFilter } from '../entities'; import { leadSide } from '../sides'; import type { QuestionKind } from '../question'; @@ -46,6 +46,25 @@ export function buildEntityCountQuery(block: EntityBlock, regionCodes: string[] `; } +// How many entities each option of a type's filter dropdown would find in the +// region: one row per option, `?option` and `?count` (#165). The count step's +// body with the filter left open, so a count previews what picking that option +// returns. Measured live on 2026-11-17: every type, nationwide and for Maine, +// Hancock and Illinois, 1.8s or less. +export function buildOptionCountQuery(type: EntityType, regionCodes: string[] | undefined): string { + const entity = entityFor(type); + if (!entity.options) throw new Error(`${type} has no filter options to count`); + const region = regionClause({ type }, regionCodes, '?s2probe', 'P'); + return ` + ${PREFIXES} + SELECT ?option (COUNT(DISTINCT ${entity.iriVar('P')}) AS ?count) WHERE { + ?s2probe rdf:type kwg-ont:S2Cell_Level13 . + ${region} + ${entity.options.bind('?s2probe', 'P', '?option')} + } GROUP BY ?option + `; +} + // One side on its own: its cells in its region, its entities in them, its // filters. The probe lists entity IRIs, and the list it returns becomes the // chunk membership (engine/slicing.ts). A narrowing join here removes entities diff --git a/engine/templates/regions.ts b/engine/templates/regions.ts index 20d2f8a..d4ea452 100644 --- a/engine/templates/regions.ts +++ b/engine/templates/regions.ts @@ -11,24 +11,6 @@ export function buildDiscoverCountiesQuery(stateCode: string): string { `; } -export function buildIndustryCountsQuery(region: { stateCode: string; countyCodes?: string[] }): string { - const codes = region.countyCodes?.length ? region.countyCodes : [region.stateCode]; - const values = codes - .map((c) => `kwgr:administrativeRegion.USA.${c}`) - .join(' '); - return ` - ${PREFIXES} - SELECT ?industryCode (COUNT(DISTINCT ?facility) AS ?num) WHERE { - VALUES ?_regionRoot { ${values} } - ?s2 rdf:type kwg-ont:S2Cell_Level13 ; - spatial:connectedTo ?_regionRoot ; - kwg-ont:sfContains ?facility . - ?facility fio:ofIndustry ?industryCode . - ?industryCode a naics:NAICS-IndustryCode . - } GROUP BY ?industryCode - `; -} - export function buildDiscoverIndustriesQuery(): string { return ` ${PREFIXES} diff --git a/engine/templates/wells.ts b/engine/templates/wells.ts index 3f6bf9a..baea360 100644 --- a/engine/templates/wells.ts +++ b/engine/templates/wells.ts @@ -51,3 +51,12 @@ export function buildWellCategoryFilter( } return blocks.join(' UNION '); } + +// One well classification left open in `optionVar`, for counting each option of +// the dropdown (#165). The same three fields buildWellCategoryFilter reads; a +// single pick is one field, so the AND across Maine type and use never applies. +export function buildWellOptionPattern(wellVar: string, optionVar: string): string { + return `{ ${wellVar} rdf:type il_isgs:ISGS-Well ; il_isgs:wellPurpose ${optionVar} } + UNION { ${wellVar} rdf:type me_mgs:MGS-Well ; me_mgs:ofWellType ${optionVar} } + UNION { ${wellVar} rdf:type me_mgs:MGS-Well ; me_mgs:hasUse ${optionVar} }`; +} diff --git a/engine/tests/optionCounts.test.ts b/engine/tests/optionCounts.test.ts new file mode 100644 index 0000000..ccef679 --- /dev/null +++ b/engine/tests/optionCounts.test.ts @@ -0,0 +1,120 @@ +// The counts next to each filter option (#165). A count previews what picking +// that option returns, so it has to check the region the way the pipeline's +// own count step does. Counting Hancock wells by their subdivision link alone +// gave 11,644; the pipeline, which checks the cell and the subdivision, finds +// 7,296 (2026-11-17). +// +// npm test # query text only, no network +import { describe, expect, it } from 'vitest'; +import { buildEntityCountQuery, buildOptionCountQuery } from '../templates/fusedQueries'; +import { ENTITY_TYPES, entityFor } from '../entities'; +import type { EntityType } from '../types/query'; +import { optionCountsFrom, wellTokenCount } from '../optionCounts'; +import { encodeWellToken } from '../constants/wellClassifications'; + +const REGIONS: (string[] | undefined)[] = [undefined, ['23'], ['23009'], ['23009', '23005']]; + +// The lines of a query that restrict it to a region. +const regionLines = (q: string): string[] => + q + .split('\n') + .map((line) => line.trim()) + .filter((line) => /administrativeRegion|_region|administrativePartOf|_subdivision|_regionLink/.test(line)); + +const countedTypes = ENTITY_TYPES.filter((t) => entityFor(t).options); + +describe('option counts check the region like the count step', () => { + it('covers every type with a filter dropdown', () => { + expect([...countedTypes].sort()).toEqual(['aquifers', 'facilities', 'streams', 'waterBodies', 'wells']); + }); + + let checked = 0; + for (const type of countedTypes) { + for (const region of REGIONS) { + it(`${type} in ${region?.join(',') ?? 'no region'}`, () => { + const step = buildEntityCountQuery({ type }, region); + const option = buildOptionCountQuery(type, region); + const expected = regionLines(step); + if (region) expect(expected.length).toBeGreaterThan(0); + else expect(expected).toEqual([]); + expect(regionLines(option)).toEqual(expected); + checked++; + }); + } + } + + it('ran for every type and region', () => { + expect(checked).toBe(5 * REGIONS.length); + }); +}); + +describe('option count query', () => { + const flat = (q: string) => q.replace(/\s+/g, ' '); + const cases: [EntityType, string][] = [ + ['waterBodies', '?waterBodyP rdf:type hyf:HY_WaterBody . ?waterBodyP nhdplusv2:hasFTYPE ?option .'], + ['streams', '?streamP rdf:type hyf:HY_FlowPath . ?streamP nhdplusv2:hasFTYPE ?option .'], + ['aquifers', '?aquiferP rdf:type gwml2:GW_Aquifer . ?aquiferP saw_water:aquiferType ?option .'], + ['facilities', '?facilityP fio:ofIndustry ?option . ?option a naics:NAICS-IndustryCode .'], + ['wells', '{ ?wellP rdf:type il_isgs:ISGS-Well ; il_isgs:wellPurpose ?option } UNION { ?wellP rdf:type me_mgs:MGS-Well ; me_mgs:ofWellType ?option } UNION { ?wellP rdf:type me_mgs:MGS-Well ; me_mgs:hasUse ?option }'], + ]; + for (const [type, pattern] of cases) { + it(`${type} counts distinct entities per option`, () => { + const q = flat(buildOptionCountQuery(type, ['23'])); + expect(q).toContain(pattern); + expect(q).toContain(`SELECT ?option (COUNT(DISTINCT ${entityFor(type).iriVar('P')}) AS ?count)`); + expect(q).toContain('GROUP BY ?option'); + }); + } + + it('reads no facility geometry: a picked industry does not ask for one', () => { + expect(buildOptionCountQuery('facilities', ['23'])).not.toContain('geo:hasGeometry'); + }); +}); + +describe('option counts from rows', () => { + const row = (option: string, count: number) => ({ option, count: String(count) }); + + it('keys feature types by their FTYPE value', () => { + expect(optionCountsFrom('waterBodies', [row('LakePond', 348), row('SwampMarsh', 170)])).toEqual({ + LakePond: 348, + SwampMarsh: 170, + }); + }); + + it('sums raw aquifer types into the two dropdown options', () => { + const rows = [row('coarse-grain_materials', 96), row('sand_gravel', 2), row('bedrock', 1), row('sand and gravel', 5)]; + expect(optionCountsFrom('aquifers', rows)).toEqual({ surficial: 103, bedrock: 1 }); + }); + + it('drops an aquifer type no option covers', () => { + expect(optionCountsFrom('aquifers', [row('clay', 4)])).toEqual({}); + }); + + it('keys industries by bare NAICS code', () => { + const rows = [row('http://w3id.org/fio/v1/naics#NAICS-111110', 1), row('http://w3id.org/fio/v1/naics#NAICS-11121', 4)]; + expect(optionCountsFrom('facilities', rows)).toEqual({ '111110': 1, '11121': 4 }); + }); + + it('keys well classifications by IRI', () => { + const iri = 'http://sawgraph.spatialai.org/v1/me-mgs-data#d.wellType.Bedrock'; + expect(optionCountsFrom('wells', [row(iri, 7236)])).toEqual({ [iri]: 7236 }); + }); + + it('skips rows with no count', () => { + expect(optionCountsFrom('streams', [{ option: 'Pipeline' }])).toEqual({}); + }); +}); + +describe('well classification count', () => { + const active = 'http://sawgraph.spatialai.org/v1/il-isgs-data#d.ISGS-WellPurpose.WATER'; + const plugged = 'http://sawgraph.spatialai.org/v1/il-isgs-data#d.ISGS-WellPurpose.WATERP'; + + it('sums the active and plugged twins one entry stands for', () => { + const token = encodeWellToken('ilPurpose', [active, plugged]); + expect(wellTokenCount({ [active]: 300, [plugged]: 20 }, token)).toBe(320); + }); + + it('is undefined when no IRI of the entry was counted', () => { + expect(wellTokenCount({}, encodeWellToken('ilPurpose', [active]))).toBeUndefined(); + }); +}); diff --git a/ui/components/QueryEditor/AquiferFilters.tsx b/ui/components/QueryEditor/AquiferFilters.tsx index 98f7f16..3a5e8df 100644 --- a/ui/components/QueryEditor/AquiferFilters.tsx +++ b/ui/components/QueryEditor/AquiferFilters.tsx @@ -1,9 +1,12 @@ -import type { AquiferFilters as AquiferFiltersType } from '../../../engine/types/query'; +import type { AquiferFilters as AquiferFiltersType, RegionFilter } from '../../../engine/types/query'; import { FlatSelect } from './FlatSelect/FlatSelect'; +import { withCount } from './FlatSelect/withCount'; +import { useOptionCounts } from '../../hooks/useDiscoveryQueries'; interface AquiferFiltersProps { value?: AquiferFiltersType; onChange: (filters: AquiferFiltersType) => void; + region?: RegionFilter; } // Two meaningful choices; the query builder expands each to the raw (messy) @@ -14,15 +17,17 @@ const AQUIFER_TYPE_OPTIONS = [ { value: 'bedrock', label: 'Bedrock' }, ]; -export function AquiferFilters({ value, onChange }: AquiferFiltersProps) { +export function AquiferFilters({ value, onChange, region }: AquiferFiltersProps) { const selected = value?.aquiferTypes ?? []; + const { data: counts } = useOptionCounts('aquifers', region); + const options = AQUIFER_TYPE_OPTIONS.map((o) => ({ ...o, label: withCount(o.label, counts?.[o.value]) })); return (
onChange({ ...value, aquiferTypes: vals })} placeholder="All aquifers..." diff --git a/ui/components/QueryEditor/EntityBlock.tsx b/ui/components/QueryEditor/EntityBlock.tsx index 8b949cc..3c2792f 100644 --- a/ui/components/QueryEditor/EntityBlock.tsx +++ b/ui/components/QueryEditor/EntityBlock.tsx @@ -52,6 +52,7 @@ export function EntityBlock({ label, value, onChange }: EntityBlockProps) { {surfaceWaterType && ( onChange({ @@ -86,13 +87,14 @@ export function EntityBlock({ label, value, onChange }: EntityBlockProps) { onChange({ ...value, wellFilters: f })} - stateCode={value.region?.stateCode} + region={value.region} /> )} {value.type === 'aquifers' && ( onChange({ ...value, aquiferFilters: f })} + region={value.region} /> )} diff --git a/ui/components/QueryEditor/FacilityFilters.tsx b/ui/components/QueryEditor/FacilityFilters.tsx index 0f4c57a..79e3e64 100644 --- a/ui/components/QueryEditor/FacilityFilters.tsx +++ b/ui/components/QueryEditor/FacilityFilters.tsx @@ -1,6 +1,6 @@ import { HierarchicalSelect } from './HierarchicalSelect/HierarchicalSelect'; import type { FacilityFilters as FacilityFiltersType, RegionFilter } from '../../../engine/types/query'; -import { useIndustries, useIndustryCounts } from '../../hooks/useDiscoveryQueries'; +import { useIndustries, useOptionCounts } from '../../hooks/useDiscoveryQueries'; interface FacilityFiltersProps { value?: FacilityFiltersType; @@ -10,7 +10,7 @@ interface FacilityFiltersProps { export function FacilityFilters({ value, onChange, region }: FacilityFiltersProps) { const { data: industries = [] } = useIndustries(); - const { data: counts } = useIndustryCounts(region); + const { data: counts } = useOptionCounts('facilities', region); return (
diff --git a/ui/components/QueryEditor/FlatSelect/withCount.ts b/ui/components/QueryEditor/FlatSelect/withCount.ts new file mode 100644 index 0000000..b690373 --- /dev/null +++ b/ui/components/QueryEditor/FlatSelect/withCount.ts @@ -0,0 +1,4 @@ +// An option label with its count, when there is one to show. +export function withCount(label: string, count?: number): string { + return count && count > 0 ? `${label} (${count.toLocaleString()})` : label; +} diff --git a/ui/components/QueryEditor/SampleFilters.tsx b/ui/components/QueryEditor/SampleFilters.tsx index ff2dc3b..c9b2fca 100644 --- a/ui/components/QueryEditor/SampleFilters.tsx +++ b/ui/components/QueryEditor/SampleFilters.tsx @@ -5,6 +5,7 @@ import { useMaterialTypes, } from '../../hooks/useDiscoveryQueries'; import { FlatSelect } from './FlatSelect/FlatSelect'; +import { withCount } from './FlatSelect/withCount'; import { HierarchicalSelect } from './HierarchicalSelect/HierarchicalSelect'; // Material types have no hierarchy of their own — discovery buckets each one into @@ -19,10 +20,6 @@ interface SampleFiltersProps { region?: RegionFilter; } -function withCount(label: string, count?: number): string { - return count && count > 0 ? `${label} (${count.toLocaleString()})` : label; -} - export function SampleFilters({ value, onChange, region }: SampleFiltersProps) { const { data: substances = [] } = useSubstances(region); const { data: materialTypes = [] } = useMaterialTypes(region); diff --git a/ui/components/QueryEditor/SurfaceWaterFilters.tsx b/ui/components/QueryEditor/SurfaceWaterFilters.tsx index eb5018d..9a1802c 100644 --- a/ui/components/QueryEditor/SurfaceWaterFilters.tsx +++ b/ui/components/QueryEditor/SurfaceWaterFilters.tsx @@ -1,4 +1,4 @@ -import { useState } from 'react'; +import { useMemo, useState } from 'react'; import { HierarchicalSelect } from './HierarchicalSelect/HierarchicalSelect'; import type { TreeItem } from './HierarchicalSelect/useNaicsTree'; import { @@ -7,11 +7,14 @@ import { toSurfaceWaterCodes, type SurfaceWaterType, } from '../../../engine/constants/featureTypes'; +import type { RegionFilter } from '../../../engine/types/query'; +import { useOptionCounts } from '../../hooks/useDiscoveryQueries'; interface SurfaceWaterFiltersProps { type: SurfaceWaterType; ftypes?: string[]; onChange: (type: SurfaceWaterType, ftypes: string[]) => void; + region?: RegionFilter; } // Parents first, so buildTree sees each one before its children. @@ -22,7 +25,16 @@ const ITEMS: TreeItem[] = SURFACE_WATER_GROUPS.flatMap((g) => [ const PARENTS = SURFACE_WATER_GROUPS.map((g) => g.code); -export function SurfaceWaterFilters({ type, ftypes = [], onChange }: SurfaceWaterFiltersProps) { +export function SurfaceWaterFilters({ type, ftypes = [], onChange, region }: SurfaceWaterFiltersProps) { + // FTYPE values differ between the two parents, so one map holds both. The + // tree sums each parent from its children. + const { data: waterBodyCounts } = useOptionCounts('waterBodies', region); + const { data: streamCounts } = useOptionCounts('streams', region); + const counts = useMemo( + () => (waterBodyCounts || streamCounts ? { ...waterBodyCounts, ...streamCounts } : undefined), + [waterBodyCounts, streamCounts], + ); + // Unticking everything shows an empty tree so a child can be picked next, // while the block already holds the default (Surface Water Bodies, any type). // Display only: the editor remounts after a run, showing the default it used. @@ -46,6 +58,7 @@ export function SurfaceWaterFilters({ type, ftypes = [], onChange }: SurfaceWate labelOnly selectAll={false} initiallyExpanded={PARENTS} + counts={counts} />
diff --git a/ui/components/QueryEditor/WellFilters.tsx b/ui/components/QueryEditor/WellFilters.tsx index 6b5d1cb..d7230be 100644 --- a/ui/components/QueryEditor/WellFilters.tsx +++ b/ui/components/QueryEditor/WellFilters.tsx @@ -1,29 +1,29 @@ -import type { WellFilters as WellFiltersType } from '../../../engine/types/query'; +import type { RegionFilter, WellFilters as WellFiltersType } from '../../../engine/types/query'; import { FlatSelect } from './FlatSelect/FlatSelect'; -import { useWellClassifications } from '../../hooks/useDiscoveryQueries'; +import { withCount } from './FlatSelect/withCount'; +import { useOptionCounts, useWellClassifications } from '../../hooks/useDiscoveryQueries'; +import { wellTokenCount } from '../../../engine/optionCounts'; import { WELL_GROUP_SHORT_LABEL, wellStateOf } from '../../../engine/constants/wellClassifications'; interface WellFiltersProps { value?: WellFiltersType; onChange: (filters: WellFiltersType) => void; - stateCode?: string; + region?: RegionFilter; } -function withCount(label: string, count?: number): string { - return count && count > 0 ? `${label} (${count.toLocaleString()})` : label; -} - -export function WellFilters({ value, onChange, stateCode }: WellFiltersProps) { +export function WellFilters({ value, onChange, region }: WellFiltersProps) { const { data: classifications = [], isLoading } = useWellClassifications(); + // The list's own counts are for the whole state; these are for the region. + const { data: counts } = useOptionCounts('wells', region); // A chosen region state limits the visible groups to that state; with no // state chosen every group shows and each pick self-scopes to its state. - const state = wellStateOf(stateCode); + const state = wellStateOf(region?.stateCode); const visible = state ? classifications.filter((c) => c.state === state) : classifications; const options = visible.map((c) => ({ value: c.key, - label: withCount(c.label, c.count), + label: withCount(c.label, counts && wellTokenCount(counts, c.key)), group: state ? WELL_GROUP_SHORT_LABEL[c.field] : c.group, })); diff --git a/ui/hooks/useDiscoveryQueries.ts b/ui/hooks/useDiscoveryQueries.ts index 05860fa..598ff58 100644 --- a/ui/hooks/useDiscoveryQueries.ts +++ b/ui/hooks/useDiscoveryQueries.ts @@ -1,12 +1,14 @@ -import { useQuery } from '@tanstack/react-query'; +import { useQuery, type UseQueryResult } from '@tanstack/react-query'; import { executeSparql } from '../../engine/endpoint'; import type { SparqlRow } from '../../engine/types/sparql'; +import type { EntityType } from '../../engine/types/query'; +import { buildOptionCountQuery } from '../../engine/templates/fusedQueries'; +import { optionCountsFrom } from '../../engine/optionCounts'; import { buildDiscoverIndustriesQuery, buildDiscoverSubstancesQuery, buildDiscoverMaterialTypesQuery, buildDiscoverCountiesQuery, - buildIndustryCountsQuery, buildDiscoverIllinoisPurposesQuery, buildDiscoverMaineTypesQuery, buildDiscoverMaineUsesQuery, @@ -73,36 +75,33 @@ interface RegionParam { countyCodes?: string[]; } -export function useIndustryCounts(region?: RegionParam) { - const key = region?.stateCode - ? (region.countyCodes?.length ? [...region.countyCodes].sort() : [region.stateCode]).join(',') - : ''; +// How many entities each option of a type's filter dropdown would find: in the +// region when one is picked, nationwide when none is (#165). Keyed the way the +// dropdown keys its options (optionCountsFrom). Runs once per type and region a +// session, when the dropdown mounts. A failed query leaves the dropdown working, +// with no numbers. +export function useOptionCounts(type: EntityType, region?: RegionParam): UseQueryResult> { + const codes = regionCodes(region); return useQuery>({ - queryKey: ['industryCounts', key], + queryKey: ['optionCounts', type, codes?.join(',') ?? ''], queryFn: async () => { - if (!region?.stateCode) return {}; - const rows = await executeSparql( - 'federation', - buildIndustryCountsQuery({ stateCode: region.stateCode, countyCodes: region.countyCodes }), - ); - const counts: Record = {}; - for (const r of rows) { - const uri = r.industryCode || ''; - const code = uri.split(/[#/]/).pop()?.replace(/^NAICS-/, ''); - if (!code) continue; - counts[code] = Number(r.num) || 0; - } - return counts; + const rows = await executeSparql('federation', buildOptionCountQuery(type, codes)); + return optionCountsFrom(type, rows); }, - enabled: !!region?.stateCode, staleTime: Infinity, + retry: 1, }); } +// The selected counties, sorted so the same counties in another order share a +// cache entry, or the state; undefined with no state. +function regionCodes(region?: RegionParam): string[] | undefined { + if (!region?.stateCode) return undefined; + return region.countyCodes?.length ? [...region.countyCodes].sort() : [region.stateCode]; +} + function regionKey(region?: RegionParam): string { - if (!region?.stateCode) return ''; - const codes = region.countyCodes?.length ? [...region.countyCodes].sort() : [region.stateCode]; - return codes.join(','); + return regionCodes(region)?.join(',') ?? ''; } export function useSubstances(region?: RegionParam) { diff --git a/ui/tests/editModalFlows.test.tsx b/ui/tests/editModalFlows.test.tsx index d111551..26989de 100644 --- a/ui/tests/editModalFlows.test.tsx +++ b/ui/tests/editModalFlows.test.tsx @@ -36,7 +36,7 @@ vi.mock('../hooks/useDiscoveryQueries', () => { const none = () => ({ data: [], isLoading: false, isError: false, error: null }); return { useIndustries: none, - useIndustryCounts: none, + useOptionCounts: () => ({ data: undefined }), useSubstances: none, useMaterialTypes: none, useCounties: none, diff --git a/ui/tests/optionCounts.test.tsx b/ui/tests/optionCounts.test.tsx new file mode 100644 index 0000000..a50a059 --- /dev/null +++ b/ui/tests/optionCounts.test.tsx @@ -0,0 +1,141 @@ +// @vitest-environment jsdom +// +// Each filter dropdown shows how many entities every option would find, in the +// block's region or nationwide with none (#165). The counts come from one live +// query per type and region (buildOptionCountQuery), faked here. +// +// npm test +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; +import { cleanup, fireEvent, render, screen } from '@testing-library/react'; +import { QueryClient, QueryClientProvider } from '@tanstack/react-query'; +import type { SparqlRow } from '../../engine/types/sparql'; + +const executeSparql = vi.fn<(endpoint: string, query: string) => Promise>(); +vi.mock('../../engine/endpoint', () => ({ + executeSparql: (endpoint: string, query: string) => executeSparql(endpoint, query), +})); + +import { EntityBlock } from '../components/QueryEditor/EntityBlock'; +import { encodeWellToken } from '../../engine/constants/wellClassifications'; +import type { EntityBlock as EntityBlockType } from '../../engine/types/query'; + +const isOptionCount = (query: string) => query.includes('GROUP BY ?option'); +const optionCountCalls = () => executeSparql.mock.calls.filter(([, q]) => isOptionCount(q)); + +// Answers every option count query with `counts`, the option list queries that +// mention `listFor` with `listRows`, and everything else empty. +function answer(counts: Record, listRows: SparqlRow[] = [], listFor = '') { + executeSparql.mockImplementation(async (_endpoint, query) => { + if (isOptionCount(query)) return Object.entries(counts).map(([option, count]) => ({ option, count: String(count) })); + return query.includes(listFor) ? listRows : []; + }); +} + +const newClient = () => new QueryClient({ defaultOptions: { queries: { retry: false } } }); + +function renderBlock(block: EntityBlockType, client = newClient()) { + const { container, unmount } = render( + + undefined} /> + , + ); + const open = (selector: string) => fireEvent.click(container.querySelector(`${selector} .hs-control`)!); + return { open, unmount }; +} + +beforeEach(() => { + executeSparql.mockReset(); +}); +afterEach(cleanup); + +describe('aquifer type counts', () => { + it('sums the raw types each option covers, in the region', async () => { + answer({ 'coarse-grain_materials': 96, sand_gravel: 2, bedrock: 1 }); + const { open } = renderBlock({ type: 'aquifers', region: { stateCode: '17', countyCodes: ['17031'] }, aquiferFilters: { aquiferTypes: ['bedrock'] } }); + open('.aquifer-filters'); + expect(await screen.findByText('Sand & gravel (surficial) (98)')).toBeTruthy(); + // The picked option's chip carries the count too. + expect(screen.getAllByText('Bedrock (1)').length).toBe(2); + const [[endpoint, query]] = optionCountCalls(); + expect(endpoint).toBe('federation'); + expect(query).toContain('administrativeRegion.USA.17031'); + }); +}); + +describe('feature type counts', () => { + it('counts each child and rolls the parents up, nationwide with no region', async () => { + answer({ LakePond: 60083, SwampMarsh: 12015, Reservoir: 886, StreamRiver: 353003 }); + const { open } = renderBlock({ type: 'waterBodies' }); + open('.surface-water-filters'); + expect(await screen.findByText('Lake / Pond (60,083)')).toBeTruthy(); + expect(screen.getByText('Surface Water Bodies (72,984)')).toBeTruthy(); + expect(screen.getByText('Stream / River (353,003)')).toBeTruthy(); + const queries = optionCountCalls().map(([, q]) => q); + expect(queries.some((q) => q.includes('hyf:HY_WaterBody'))).toBe(true); + expect(queries.some((q) => q.includes('hyf:HY_FlowPath'))).toBe(true); + expect(queries.every((q) => !q.includes('administrativeRegion'))).toBe(true); + }); +}); + +describe('industry counts', () => { + it('shows nationwide counts with no region picked', async () => { + answer({ 'http://w3id.org/fio/v1/naics#NAICS-11': 2094 }, [ + { code: '11', label: 'Agriculture', groupCode: '11', groupLabel: 'Agriculture' }, + ]); + const { open } = renderBlock({ type: 'facilities', facilityFilters: { industryCodes: ['11'] } }); + open('.facility-filters'); + expect(await screen.findByText('11 - Agriculture (2,094)')).toBeTruthy(); + expect(optionCountCalls()[0][1]).not.toContain('administrativeRegion'); + }); +}); + +describe('well classification counts', () => { + const bedrock = 'http://sawgraph.spatialai.org/v1/me-mgs-data#d.wellType.Bedrock'; + + it('counts the wells in the county, not the whole state', async () => { + answer({ [bedrock]: 7236 }, [{ value: bedrock, num: '79336' }], 'me_mgs:ofWellType'); + const { open } = renderBlock({ + type: 'wells', + region: { stateCode: '23', countyCodes: ['23009'] }, + wellFilters: { wellCategories: [encodeWellToken('meType', [bedrock])] }, + }); + open('.well-filters'); + expect((await screen.findAllByText('Bedrock (7,236)')).length).toBe(2); + expect(screen.queryByText('Bedrock (79,336)')).toBeNull(); + expect(optionCountCalls()[0][1]).toContain('administrativeRegion.USA.23009'); + }); +}); + +describe('a failed count', () => { + it('leaves the dropdown usable with no numbers', async () => { + executeSparql.mockImplementation(async (_e, query) => { + if (isOptionCount(query)) throw new Error('429 Too Many Requests'); + return []; + }); + const { open } = renderBlock({ type: 'aquifers', aquiferFilters: { aquiferTypes: ['bedrock'] } }); + open('.aquifer-filters'); + await vi.waitFor(() => expect(optionCountCalls().length).toBe(1)); + expect(screen.getAllByText('Bedrock').length).toBe(2); + }); +}); + +describe('count caching', () => { + it('fetches a type and region once a session, and again for a new region', async () => { + answer({ bedrock: 14 }); + const client = newClient(); + const block = (countyCodes: string[]): EntityBlockType => ({ + type: 'aquifers', + region: { stateCode: '17', countyCodes }, + aquiferFilters: { aquiferTypes: ['bedrock'] }, + }); + renderBlock(block(['17031', '17043']), client).unmount(); + await vi.waitFor(() => expect(optionCountCalls().length).toBe(1)); + // The same counties in another order reuse the cached counts. + renderBlock(block(['17043', '17031']), client).unmount(); + renderBlock(block(['17031']), client); + await vi.waitFor(() => expect(optionCountCalls().length).toBe(2)); + await new Promise((resolve) => setTimeout(resolve, 50)); + expect(optionCountCalls().length).toBe(2); + expect(optionCountCalls()[1][1]).not.toContain('17043'); + }); +}); diff --git a/ui/tests/wellStateGating.test.tsx b/ui/tests/wellStateGating.test.tsx index c932739..7321276 100644 --- a/ui/tests/wellStateGating.test.tsx +++ b/ui/tests/wellStateGating.test.tsx @@ -11,7 +11,7 @@ import { cleanup, fireEvent, render, screen } from '@testing-library/react'; vi.mock('../hooks/useDiscoveryQueries', () => { const none = () => ({ data: [], isLoading: false, isError: false, error: null }); - return { useCounties: none, useWellClassifications: none }; + return { useCounties: none, useWellClassifications: none, useOptionCounts: () => ({ data: undefined }) }; }); import { EntityBlock } from '../components/QueryEditor/EntityBlock';