Repository navigation
feat(wikidata): Count OSM features and names per brand - #928
Merged
Merged
Conversation
collect_wikidata_ids already visits every ATP and OSM feature with a brand:wikidata, operator:wikidata or network:wikidata tag. It now also writes two files for the spider stats: - osm-brands.tsv: OSM features per Wikidata ID, country and main category (such as shop=supermarket). A feature that names the same ID twice counts once. - brand-names.tsv: the three most common brand, operator or network names per Wikidata ID, country and key, with how often OSM and ATP use each. Names count only where the Wikidata tag has a single ID. The country of an OSM feature comes from its centroid, so features with a Wikidata tag now get their geometry decoded. Part of #924. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ttfb2LhDRExioXkdHbxLno
The names are only for telling brands apart in the spider stats, so brand-names.tsv now has one name per Wikidata ID and key, not three per country: the most common name in Latin script, or else the most common one. ATP places no longer need their country looked up. osm-brands.tsv keeps its countries, so the spider stats can compare a per-country spider with OSM in that country. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ttfb2LhDRExioXkdHbxLno
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requested by Sascha · project thread
Fourth of the PRs planned in #924, after #927.
collect_wikidata_idsalready visits every ATP and OSM feature with abrand:wikidata,operator:wikidataornetwork:wikidatatag. It now also writes two files for the spider stats of PR 5. Nothing is published yet.osm-brands.tsv: OSM features per Wikidata ID, country and main category, such asQ680727 CH shop=supermarket 1234. The category is the onecontact::main_categorygives (shop, amenity, tourism, leisure, healthcare, craft, office), or empty, as for transit stops. A feature that names the same ID twice, such as bybrand:wikidataandoperator:wikidata, counts once. This lets the spider stats compare ATP's count for a brand with OSM's, per country.brand-names.tsv: one display name per Wikidata ID and key (brand,operatorornetwork, from the tag that goes with the Wikidata tag): the most common name in Latin script, or if there is none, the most common one, with how often OSM and ATP use it. That is enough to tell brands apart in an internal tool, without a name per country. Names count only where the Wikidata tag has a single ID, since withQ1;Q2there's no telling which name goes with which. ATP names are included so that brands only ATP has get a name too. Tabs and line breaks in names become spaces.Cost. The country of an OSM feature comes from its centroid, so the features with a Wikidata tag now get their geometry decoded, as
send_deletion_candidatesalready does for unmatched ones. ATP places need no country. Both files are rebuilt when they're missing, so an existing workdir runs this step again.docs/TECHNICAL_DESIGN.mddescribes the two files.Testing. New unit tests for the totals (merging, counting an ID once per feature, names with several IDs, preferring Latin script) and for both files from
collect_wikidata_idson the ATP fixture.cargo test --lib,cargo clippy --all-targets -- -D warningsandcargo fmtpass. CI runs the integration tests, since they needtippecanoe.https://claude.ai/code/session_01Ttfb2LhDRExioXkdHbxLno
Generated by Claude Code