From dc4c93f6aaf522aed8daa72e09d900100626a24e Mon Sep 17 00:00:00 2001 From: rafapereirabr Date: Tue, 20 Oct 2026 11:29:03 -0300 Subject: [PATCH 01/31] set up claude --- .Rbuildignore | 5 + .claude/rules/orchestrator-protocol.md | 82 ++++++ .claude/rules/plan-first-workflow.md | 80 ++++++ .claude/rules/prompt-shaping.md | 18 ++ .claude/rules/quality-gates.md | 70 +++++ .claude/rules/r-package-conventions.md | 350 +++++++++++++++++++++++++ .claude/rules/session-logging.md | 20 ++ .claude/rules/summary-parity.md | 35 +++ .gitignore | 7 + CLAUDE.md | 150 +++++++++++ 10 files changed, 817 insertions(+) create mode 100644 .claude/rules/orchestrator-protocol.md create mode 100644 .claude/rules/plan-first-workflow.md create mode 100644 .claude/rules/prompt-shaping.md create mode 100644 .claude/rules/quality-gates.md create mode 100644 .claude/rules/r-package-conventions.md create mode 100644 .claude/rules/session-logging.md create mode 100644 .claude/rules/summary-parity.md create mode 100644 CLAUDE.md diff --git a/.Rbuildignore b/.Rbuildignore index 1d906a7..84a9929 100644 --- a/.Rbuildignore +++ b/.Rbuildignore @@ -12,3 +12,8 @@ ^data-raw$ ^\.pre-commit-config\.yaml$ ^testing_constrained\.Rmd$ +^\.claude$ +^CLAUDE\.md$ +^MEMORY\.md$ +^templates$ +^quality_reports$ diff --git a/.claude/rules/orchestrator-protocol.md b/.claude/rules/orchestrator-protocol.md new file mode 100644 index 0000000..5467937 --- /dev/null +++ b/.claude/rules/orchestrator-protocol.md @@ -0,0 +1,82 @@ +# Orchestrator Protocol (contractor mode) + +**After the user approves a plan, execute it autonomously:** implement, verify, +review, fix, and repeat until reviews come back dry. Return to the user only +for ambiguity, a decision that is theirs, or a blocker. Nothing in this loop +commits, pushes or opens PRs; that needs an explicit request (`/acc-commit`). + +## The loop + +``` +Approved plan + │ + 1. IMPLEMENT smallest change that satisfies the plan; follow + │ r-package-conventions.md + 2. VERIFY quality-gates.md Gate 1 (document, targeted tests, lint); + │ read the output; retry ≤ 2, then escalate + 3. REVIEW parallel forked reviewers (lenses below), each returns + │ FINDINGs + 4. REDUCE dedupe on location+finding; CRITICAL>0 → BLOCK, + │ MAJOR>0 → REVISE, else PASS + 5. FIX critical → major; minors fixed if cheap and in scope + 6. RE-VERIFY Gate 1 again (Gate 2 before a PR) + │ + └─ converged? (2 consecutive rounds with 0 new CRITICAL/MAJOR) + yes → report no → back to 3 with fresh reviewers (cap 5 rounds) +``` + +## Review lenses for package code + +Launch in **one message**, each as a separate fresh subagent (not `fork`) with only the diff, +the plan and the relevant files: + +| Lens | Agent | Looks for | +|---|---|---| +| Correctness and methodology | `general-purpose` | Formula vs the cited paper; hand-computed toy example; active vs passive; `group_by` and multiple cutoffs/decay values; `fill_missing_ids`; ids missing from `land_use_data`; NA/zero/Inf costs and opportunities; marginals and convergence (constrained/balancing measures); population weighting (inequality/poverty); output class and column order | +| Performance | `general-purpose` | Avoidable copies, row loops, repeated joins, materialising the full cutoff × OD grid, complexity regressions; asks for `bench::mark` if a speed claim is made | +| Package and CRAN | `r-package-reviewer` (local; if absent, `general-purpose` with `r-package-conventions.md`) | Conventions, roxygen, NAMESPACE, tests (including "input data sets remain unchanged" and output-class tests), CRAN policy, back-compat | + +Scale down for small changes: a docs-only change needs only the package lens. +Scale up for risky ones: add a second correctness reviewer with a different +fixture focus. + +**FINDING format:** +`{lens, severity: CRITICAL|MAJOR|MINOR, location: file:line, finding, evidence, proposed_fix}`. +A CRITICAL must be backed by evidence (a failing input, a quoted line). When +the orchestrator introduces a CRITICAL that no lens raised, it re-verifies it +in a fresh fork before acting on it; if the fork can't ground it, it is +dropped. + +## Guards + +- **Two strikes:** if the same finding survives rounds N and N+2, stop patching + and escalate to the user. The design is probably wrong. +- **Scope:** a reviewer finding outside the plan's scope goes to the backlog, + not into the diff. +- **Pre-existing failures:** don't fix them as drive-bys (see + `quality-gates.md`). +- **Cost:** keep fan-outs at 3–4 agents. For anything larger, ask first. + +## Check-in cadence + +Contractor mode means no routine check-ins. Plans may still define explicit +checkpoints (stop, summarise, wait); honour them. While a maintainer is new to +this workflow, plans include a checkpoint at each phase boundary. Drop these +checkpoints once the maintainer says so. Escalate immediately for: +- an API or behaviour decision +- a new dependency +- a gate that can't be passed without changing the plan +- disagreement between reviewers that touches user-facing behaviour + +## Report (end of execution) + +What changed (files, one line each), gate results with the actual output +summarised, review rounds and what they caught, anything deferred to the +backlog, and open questions. Then update the session log and set the plan +Status to COMPLETED. + +## Not automatic + +The loop is always started by an approved plan or an explicit skill +invocation. No background daemons, and no unattended loops against shared +branches. diff --git a/.claude/rules/plan-first-workflow.md b/.claude/rules/plan-first-workflow.md new file mode 100644 index 0000000..3214c32 --- /dev/null +++ b/.claude/rules/plan-first-workflow.md @@ -0,0 +1,80 @@ +# Plan-First Workflow + +**For any non-trivial task, enter plan mode before writing code.** Non-trivial +means more than one file, a change to exported behaviour, a performance change, +or anything taking more than a few minutes. A typo or a one-line obvious fix is +not. + +## The protocol + +1. **Enter plan mode** (`EnterPlanMode`). +2. **Recall:** read `MEMORY.md` `[LEARN]` entries and the backlog in + `quality_reports/specs/` for anything relevant. +3. **Explore:** read the affected functions, their helpers, tests and callers. + Look for existing helpers to reuse before proposing new code. +4. **Clarify:** ask only about genuine decisions (API naming, behaviour + changes, new dependencies), at most 3–5 questions via `AskUserQuestion`. For + large or vague tasks, write a spec first (below). +5. **Draft the plan:** context, approach, files to change, tests to add, + verification, and an explicit list of what is *not* being changed. +6. **Adversarial plan review:** see the next section. Mandatory. +7. **Plan file:** the harness creates the plan file in `quality_reports/plans/` + when plan mode starts (`plansDirectory` in settings), and it is the only + file writable in plan mode. It is the plan of record. Write the draft there + with Status: DRAFT and revise it in place through the review rounds. +8. **Present:** once the reviewers reach consensus, call `ExitPlanMode`. + Include a short "review outcome" note: what the reviewers cut or changed. +9. **On approval:** rename the file to + `quality_reports/plans/YYYY-MM-DD_short-description.md`, set Status to + APPROVED, start the session log, and execute via `orchestrator-protocol.md` + (contractor mode). + +## Adversarial plan review (mandatory) + +Every plan is challenged by **two independent adversarial agents** before it is +shown to the user. They are fresh `Plan` subagents (read-only; not `fork`), launched +in parallel in one message, each given the draft plan and the relevant file +paths. They never see each other's output in the first round. + +| Reviewer | Mandate | +|---|---| +| **Minimalist** | Make it smaller. Which steps, files, abstractions, arguments or dependencies are unnecessary? Can an existing helper do this? What is the smallest diff that fully solves the problem? Is anything gold-plating? | +| **Skeptic** | Make it right. What breaks? Edge cases (ids missing from `land_use_data`, NA/zero costs, empty results after a cutoff, `group_by` + multiple cutoffs/decay values, active vs passive, data.frame/tibble input), does it match the cited formula, caller mutation, performance regressions, back-compat for exported functions, CRAN issues, missing tests. | + +Each returns findings as `{severity: CRITICAL|MAJOR|MINOR, finding, proposed_change}` +and a verdict: `APPROVE` or `REVISE`. + +**Consensus loop:** +1. Revise the plan to address every CRITICAL/MAJOR finding (or write down why + it is rejected). +2. Re-run both reviewers on the revised plan, this time with the other + reviewer's previous findings attached so they can contest them. +3. **Consensus = both return APPROVE.** Stop and save. +4. **Cap: 3 rounds.** If they still disagree, present the plan with the + unresolved disagreement stated plainly as a decision for the user. + +**Simplicity bias:** when the two reviewers conflict, prefer the option with +less code change, unless the Skeptic shows a correctness or CRAN failure. + +## Requirements spec (large or ambiguous tasks) + +Use it when the task is vague ("improve performance"), has several valid +interpretations, or touches the public API in several places. Write +`quality_reports/specs/YYYY-MM-DD_description.md`. Use +`templates/requirements-spec.md` if present; it is local to the maintainer's +setup. +- Mark requirements MUST / SHOULD / MAY. +- Give each aspect a clarity status: CLEAR / ASSUMED / BLOCKED. +- Get approval, then plan. + +## Plans on disk + +Plans survive context compression. Status values: DRAFT → APPROVED → +COMPLETED. Update the status when it changes. + +## Session recovery + +After compression or in a new session: +1. Read `CLAUDE.md` and the most recent plan in `quality_reports/plans/`. +2. Run `git log --oneline -10` and `git status`. +3. State the current task and next step before continuing. diff --git a/.claude/rules/prompt-shaping.md b/.claude/rules/prompt-shaping.md new file mode 100644 index 0000000..f96ea7a --- /dev/null +++ b/.claude/rules/prompt-shaping.md @@ -0,0 +1,18 @@ +# Prompt Shaping (standing habit) + +**When a request is informal, dictated or ambiguous, shape it silently before +acting.** Resolve: + +1. **Role:** whose expertise answers this (R package maintainer, data.table + performance engineer, transport-accessibility methodologist, technical + writer)? +2. **Task:** the single concrete deliverable, as verb + object. +3. **Context:** which functions, tests, fixtures, issues and prior decisions + (`MEMORY.md`, plans) bear on it. +4. **Constraints:** conventions, gates, CRAN, back-compat, performance. +5. **Output:** a diff, a plan, a report, a benchmark? +6. **Bookend:** at the end, restate the goal and confirm it was met. + +Don't narrate this back. Ask (briefly, via `AskUserQuestion`) only when a +decision belongs to the user. Otherwise infer, and state the assumption in one +line. diff --git a/.claude/rules/quality-gates.md b/.claude/rules/quality-gates.md new file mode 100644 index 0000000..ea43576 --- /dev/null +++ b/.claude/rules/quality-gates.md @@ -0,0 +1,70 @@ +--- +paths: + - "R/**" + - "tests/**" + - "vignettes/**" + - "man/roxygen/**" + - "README.Rmd" + - "DESCRIPTION" + - "NEWS.md" + - "inst/**" + - "_pkgdown.yml" +--- + +# Quality gates (accessibility) + +Gates are **pass/fail checks**, not scores. A task is "done" only when the gate +for its checkpoint passes and the output has actually been seen; "should work" +doesn't count. + +## Gate 1: Task done / commit + +All must hold for the files touched: + +| Check | Command | Pass | +|---|---|---| +| Docs regenerated | `Rscript -e "devtools::document()"` then `git diff --stat man/ NAMESPACE` | No unexpected content drift (use `git diff`, not `git status`: with `core.autocrlf=true` EOL-only rewrites show as modified); intended changes staged with the source | +| Targeted tests | `Rscript -e "devtools::test(filter = '')"` for every touched function | 0 failures, 0 new warnings | +| Lint | `Rscript -e "lintr::lint('R/.R')"` per changed file (no `.lintr`; defaults) | No new lints on changed lines | +| Correctness | new/changed measure | Test against a hand-computed toy example / the cited formula (conventions A8) | +| README | if `README.Rmd` changed: `rmarkdown::render("README.Rmd")` | `README.md` regenerated and staged with `README.Rmd` | +| NEWS | user-facing change | Bullet under `# accessibility (development version)` | +| codemeta | if `DESCRIPTION` changed: `Rscript -e "codemetar::write_codemeta()"` | `codemeta.json` regenerated and staged (or reason noted) | +| pkgdown index | if an `@export` was added or removed: `Rscript -e "pkgdown::check_pkgdown()"` | No missing or stale topics in `_pkgdown.yml` | +| Fixture data | if `inst/extdata/*.rds` changed | Regenerated by a `data-raw/` script, not edited by hand; reason recorded | +| Staging hygiene | `git diff --cached --name-only` | No `.RData`/`.Rhistory`; `.rds` only under `inst/extdata/` (regenerated); no workflow files except `CLAUDE.md` and `.claude/rules/` | +| Conventions | `.claude/rules/r-package-conventions.md` checklist | All items hold | + +## Gate 2: Pull request + +Gate 1, plus: + +| Check | Command | Pass | +|---|---|---| +| Full suite | `Rscript -e "devtools::test()"` | 0 failures | +| CRAN check | `Rscript -e "devtools::check(args = '--as-cran')"` (background; several minutes) | 0 errors, 0 warnings; each NOTE triaged (pre-existing vs new) | +| Review loop | parallel review lenses (`orchestrator-protocol.md`) | Converged: 0 open CRITICAL/MAJOR | +| Package review | `r-package-reviewer` agent on changed files (local; if absent, apply the conventions checklist manually) | 0 CRITICAL/MAJOR | +| Performance | if the change claims or risks a speed change | `bench::mark` evidence recorded | + +## Gate 3: Release (maintainer-driven) + +Gate 2, plus `/acc-package-check` (full report), `devtools::check_win_devel()` +and/or R-hub, a reverse-dependency check, `cran-comments.md` updated, and +`_pkgdown.yml` covering all exports. Claude prepares; the maintainer bumps the +version and submits. + +## Overrides + +A gate may be skipped only on an explicit user instruction ("commit anyway"), +and the reason is recorded in the commit body and the session log. Never use +`git commit --no-verify`. The repo's `.pre-commit-config.yaml` hooks (README +rendered, codemeta synced, pkgdown reference index) run only where +`pre-commit install` has been run. That is why +Gate 1 checks README and staging explicitly instead of relying on the hooks. + +## Pre-existing failures + +If a gate fails for reasons unrelated to the change, don't fix it as a drive-by. +Report it, confirm it also fails on `main`, and add it to the backlog +(`quality_reports/specs/`). diff --git a/.claude/rules/r-package-conventions.md b/.claude/rules/r-package-conventions.md new file mode 100644 index 0000000..3f9a012 --- /dev/null +++ b/.claude/rules/r-package-conventions.md @@ -0,0 +1,350 @@ +--- +paths: + - "R/**/*.R" + - "tests/**/*.R" + - "man/**" + - "vignettes/**" + - "README.Rmd" + - "DESCRIPTION" + - "NAMESPACE" + - "NEWS.md" + - "_pkgdown.yml" + - "inst/REFERENCES.bib" +--- + +# accessibility package conventions + +**Standard:** a CRAN-ready, fast, predictable package whose numbers are right. +`R CMD check --as-cran` returns 0 errors, 0 warnings, and only explained notes. +New code reads like the code around it. + +Part A describes how accessibility is written today (verified against the +source, 2026-10-12). Items tagged **[NEW]** are decisions for new or modified +code only; never apply them to untouched code as drive-bys. Part B is the +general CRAN standard. When they conflict, Part A wins; flag the conflict in +the plan. + +--- + +# Part A: accessibility house style + +## A1. Namespacing + +- **Always call `package::function()`**, including `data.table::`, + `checkmate::`, `stats::`, `utils::`, `Hmisc::`. +- **Exceptions:** + - data.table special symbols and operators: `:=`, `.I`, `.SD`, `%chin%`, + `.GRP`, `.N`. These, plus `utils globalVariables` and `Rdpack reprompt`, + are the only `@importFrom` imports (in `R/accessibility-package.R`). + `.()` is used inside `[` and is covered by `globalVariables(".")`. + - ggplot2 functions in vignettes and README, after `library(ggplot2)`. +- Never use `library()`/`require()` in `R/`. Suggested packages (`sf`, + `ggplot2`, `tibble`) are guarded with `requireNamespace("pkg", quietly = + TRUE)`. +- Column names used with non-standard evaluation go into the + `utils::globalVariables(c(...))` block in `R/accessibility-package.R`. + Append new names at the end and check for duplicates first. + +## A2. Function skeleton + +Every accessibility measure follows the same shape (see +`R/cumulative_cutoff.R`, `R/gravity.R`): + +```r +#' @template travel_matrix +#' @template land_use_data +#' @template opportunity +#' @template travel_cost +#' @template group_by +#' @template active +#' @template fill_missing_ids_combinations +#' @template return_accessibility +#' @family +#' @examplesIf identical(tolower(Sys.getenv("NOT_CRAN")), "true") +#' ... +#' @export +some_measure <- function(travel_matrix, + land_use_data, + opportunity, + travel_cost, + group_by = character(0), + active = TRUE, + fill_missing_ids = TRUE) { + + checkmate::assert_string(opportunity) + checkmate::assert_logical(active, len = 1, any.missing = FALSE) + assert_group_by(group_by) # R/assert_and_assign.R + assert_travel_matrix(travel_matrix, travel_cost, group_by) + assert_land_use_data(land_use_data, travel_matrix, opportunity, active = active) + + # if not a dt, keep original class to assign later when returning result + + if (!inherits(travel_matrix, "data.table")) { + original_class <- class(travel_matrix) + data <- data.table::as.data.table(travel_matrix) + } else { + data <- data.table::copy(travel_matrix) + } + + # ... body ... + + if (exists("original_class")) class(access) <- original_class + + return(access[]) +} +``` + +- **Inputs:** validate with `checkmate::assert_*` first, then the shared + `assert_*()` helpers in `R/assert_and_assign.R` (`assert_travel_matrix`, + `assert_land_use_data`, `assert_accessibility_data`, + `assert_sociodemographic_data`, `assert_group_by`, + `assert_access_group_by`, `assert_decay_function`, `assert_cutoff`). Reuse + them rather than writing new checks. Enumerated string options use + `checkmate::assert_string()` + `checkmate::assert_names(x, subset.of = ...)` + (or `assert_choice()` when exactly one value is ever valid). +- **Input data contracts** (documented in `man/roxygen/templates/`): + - `travel_matrix`: `from_id`, `to_id`, the `travel_cost` columns, the + `group_by` columns. + - `land_use_data`: `id` plus the `opportunity`/`demand`/`supply` columns. + - `accessibility_data` (inequality/poverty): `id` plus `opportunity` and + `group_by`. + - `sociodemographic_data`: `id` plus `population` (and `income` where used). + - `group_by` can never contain `from_id`/`to_id` (access) or `id` + (inequality). +- **Output class mirrors input:** a `data.table` in returns a `data.table`; a + `data.frame`/`tibble` in returns that class, via the `original_class` idiom + above. Every measure has a test for it. +- **Output columns:** `id`, then `group_by` columns, then any grid columns + (cutoffs, `decay_function_arg`), then the opportunity column named after + `opportunity`. Sorted with `data.table::setorderv()` by those keys. +- **Data-quality warnings, not errors:** ids in `travel_matrix` missing from + `land_use_data`, NA in opportunity/demand columns, and extra columns in + `travel_matrix` not in `travel_cost`/`group_by` (`warn_extra_cols()`) all + `warning()` and continue. Keep that contract. +- **`fill_missing_ids`:** origins with no reachable opportunity are absent + after aggregation; when `TRUE` they are filled with 0 via + `do_fill_missing_ids()` (`R/fill_missing_ids.R`). New measures that can drop + ids expose the same argument. + +## A3. data.table discipline + +- **Never modify the caller's objects.** The input `travel_matrix` is + `data.table::copy()`ed (or converted with `as.data.table()`, which copies) + before `merge_by_reference()` adds the opportunity column. Any new `:=` / + `data.table::set*()` must act on a table the function owns. Every measure has + an "input data sets remain unchanged" test. +- `land_use_data` is only converted, never updated by reference; keep it that + way (`merge_by_reference()` reads from it with `i.` prefixes). +- data.table auto-indexing may add an `index` attribute to the caller's table + after `dt[i]`. Tests clear it with `data.table::setindex(x, NULL)` before + comparing; that is accepted, don't add `copy()` calls to prevent it. +- Use `:=` for in-place updates of owned tables; `%chin%` for character + membership; `get()` / `..var` / `eval()` with an explicit `env` for + programmatic column names (see `cumulative_cutoff()`). +- Long calls are split one argument per line; `[` blocks put the empty `i` + on its own line: + ```r + cum_opps <- cum_opps[ + , + .(access = sum(get(.opportunity_colname))), + by = eval(groups, envir = env) + ] + ``` +- End data.table returns with `[]` (`return(access[])`) so the result prints + after `:=`. +- No row-wise loops or `apply` over rows of the travel matrix. Iteration over + *parameter grids* (cutoffs, decay values) with `mapply`/`lapply` + + `data.table::rbindlist()` is the established pattern. + +## A4. Messages and errors + +- **Today:** base `stop()`/`warning()` with `call. = FALSE` and messages built + by string concatenation, quoting argument names in single quotes + (`"'land_use_data' is missing ids listed in 'travel_matrix', ..."`). Input + type errors come from checkmate. `cli` is **not** a dependency. +- The newer constrained-measure files (`total_constrained.R`, + `doubly_constrained.R`, `constrained_accessibility.R`) use `stop()` without + `call. = FALSE`. [NEW] modified code adds `call. = FALSE` to match the rest + of the package. +- Adding `cli` (or `lifecycle`) to Imports needs a plan; never migrate + messages as a drive-by. +- **Deprecations:** keep the old argument for at least one CRAN release with a + `warning()` naming the replacement, and test it with `expect_warning()`. + +## A5. Style + +- 2-space indent, `<-`, 80-column lines, snake_case, explicit `return()` at + the end of exported functions; validators end with + `return(invisible(TRUE))`. +- Function arguments aligned one per line under the opening parenthesis when + they don't fit on one line. +- Two blank lines between top-level functions in multi-function files + (`assert_and_assign.R`). Comments are lowercase prose explaining *why*, + followed by a blank line before the code they describe. +- There is no `.lintr` in this repo; run `lintr::lint()` with defaults and + treat only lints on **changed lines** as actionable. +- Internal helpers: descriptive snake_case names, `#' @keywords internal` + roxygen stub (or none, as in `fill_missing_ids.R`; match the file). + +## A6. Documentation (roxygen2, markdown, Rdpack) + +- Shared parameters use the templates in `man/roxygen/templates/` + (`@template travel_matrix`, `land_use_data`, `opportunity`, `travel_cost`, + `group_by`, `active`, `decay_function`, `demand`, `supply`, `population`, + `accessibility_data`, `return_accessibility`, …). Add a template only when a + parameter is shared by 3+ functions. Edit the template, not each `.Rd`. +- `@family` tags in use: `cumulative access`, `decay functions`, + `Floating catchment area`, `Constrained accessibility`, `inequality`. Match + the exact spelling (case matters for grouping). +- **Citations** use Rdpack: entries in `inst/REFERENCES.bib`, cited with + `\insertCite{key;textual}{accessibility}` and listed under + `@references \insertAllCited{}`. Every measure cites the paper it + implements. New bib entries need DOI/URL checked. +- **Examples** use the bundled fixtures in `inst/extdata/` and are gated so + they don't run on CRAN: + ```r + #' @examplesIf identical(tolower(Sys.getenv("NOT_CRAN")), "true") + #' data_dir <- system.file("extdata", package = "accessibility") + #' travel_matrix <- readRDS(file.path(data_dir, "travel_matrix.rds")) + #' land_use_data <- readRDS(file.path(data_dir, "land_use_data.rds")) + ``` + Tests and vignettes set `Sys.setenv(OMP_THREAD_LIMIT = 2)` for CRAN CPU + time. +- Every new export is added to a section of `_pkgdown.yml` (Accessibility, + Decay functions, Inequality, Poverty). Check with `pkgdown::check_pkgdown()`. +- Never hand-edit generated files. Edit the source and regenerate: + + | Generated file | Regenerate with | + |---|---| + | `NAMESPACE`, `man/*.Rd` | `devtools::document()` | + | `README.md` | `rmarkdown::render("README.Rmd")` | + | `codemeta.json` | `codemetar::write_codemeta()` (synced from DESCRIPTION) | + | `inst/extdata/*.rds` | scripts in `data-raw/` (never edit the `.rds` directly) | + +## A7. Tests (testthat, 2nd edition) + +- `DESCRIPTION` has no `Config/testthat/edition`, so tests run in the 2nd + edition (`expect_is()` is used widely). Don't switch editions as a + drive-by; that changes `expect_equal()` semantics across the suite. +- Shared fixtures live in `tests/testthat/setup.R`: `travel_matrix` (with a + `mode` column, two copies), `smaller_matrix` (first 10 rows), + `land_use_data`, `pareto_frontier` / `small_frontier` (multi-cost), + `small_access`. Running a test file manually needs + `source("tests/testthat/setup.R")` first. +- One file per function: `tests/testthat/test-.R`, with a + `tester()` wrapper that sets defaults, then `test_that()` blocks. +- **New or modified measures need the standard block set** (see + `test-cumulative_cutoff.R`): + - "raises errors due to incorrect input" (every argument, including + missing required columns) + - "throws warning if travel_matrix has an extra col" + - "returns a dataframe whose class is the same as travel_matrix's" + - "result has correct structure" (columns, types, row count) + - **"input data sets remain unchanged"** (clear indices with + `data.table::setindex(x, NULL)` first) + - "... is correctly calculated" against **hand-computed values** on a few + selected ids, or a tiny synthetic matrix (as in + `test-constrained-wrapper.R`). Never assert only on the current output. + - "works even if travel_matrix and land_use has specific colnames" + (columns named like internal variables) + - `fill_missing_ids` behaviour, where the argument exists +- Restore global state with `on.exit()`. `withr` is not a dependency. +- `tests/test_rafa/` is a personal scratch folder, not part of the suite. + +## A8. Correctness and performance + +- **Results must match the cited formula.** When implementing or changing a + measure, write the formula (with the paper's notation) in the plan, and test + against a hand-computed toy example. For constrained/balancing measures + check the marginals (row/column sums equal demand/supply) and convergence. +- Decay functions return a function of `travel_cost` that yields one numeric + weight per element (a named list of vectors when several decay values are + given); weights are in `[0, 1]` and non-increasing in cost. Keep that + contract; `assert_decay_function()` relies on it. +- Inequality/poverty measures weight by `population`; NA handling is tested + ("handles missing data correctly"). State `na.rm` explicitly in new code. +- Avoid materialising the full cutoff × OD grid when a join or cumulative sum + would do. **A claimed speed-up needs evidence:** `bench::mark(old = ..., + new = ...)` on the bundled `travel_matrix` (and a larger matrix when + relevant), results checked equal, timings recorded in the plan or session + log. `bench` is a dev tool, not a dependency. +- Don't add a hard dependency (Imports) without a plan that justifies it. + +## A9. NEWS.md + +Entries go under `# accessibility (development version)` (create it above the +latest release heading when missing). Section headings used in the repo: +`## New features`, `## Bug fixes`, `## Minor changes`, `## Notes`, and +`## Breaking changes` only for changes that break code or change correct +results. + +``` +- New function `fun()`, which implements ... proposed in Author et al. (Year). +- `fun()` has a new argument `arg` ... Closes [#63](https://github.com/ipeaGIT/accessibility/issues/63). +- Fixed a bug in `fun()` that ... (#issue). +``` + +Function names in backticks; link issues; credit external contributors. + +## A10. Figures (vignettes, README, pkgdown) + +- **Existing convention:** ggplot2 and sf are in Suggests; vignette chunks + gated on `requireNamespace()`. Maps use `inst/extdata/grid_bho.rds`. +- **[NEW] Publication-ready standard for new or modified figures:** one + consistent theme, a colour-blind-safe palette (viridis / Okabe-Ito), + labelled axes with units, fixed `fig.width`/`fig.height`/`dpi`. Existing + figures are restyled only through a plan. + +--- + +# Part B: general CRAN standard + +- **DESCRIPTION:** `Imports` for code in `R/`, `Suggests` for + tests/vignettes/optional paths, `Depends` only for the R version. Version + floors only when a feature requires them. `Authors@R` with roles. +- **Docs:** every export documents all `@param`s, `@return`, and runnable + `@examples`. Use `\donttest{}` / `@examplesIf` for slow-but-correct + examples; never use `\dontrun{}` to hide errors. +- **Red flags:** + +| Red flag | Do instead | +|---|---| +| `library()`/`require()` in `R/` | `pkg::fun()` | +| Writing outside `tempdir()` | Write only to `tempdir()` or user-supplied paths | +| `<<-` to global env | Return values | +| `print()`/`cat()` in functions | `message()`/`warning()`, gated on a `verbose` arg where relevant | +| `options()`/`par()`/threads changed, not restored | `on.exit(..., add = TRUE)` | +| `T`/`F` | `TRUE`/`FALSE` | +| Unvalidated enumerated arg | `checkmate::assert_names(x, subset.of = ...)` | +| `==` on doubles | `abs(x - y) < tol` or `all.equal()` | +| [NEW] Implicit `na.rm` | State `na.rm` explicitly in new or modified code | +| Growing vectors in loops | Pre-allocate or vectorise | + +- **Versioning:** dev versions use `x.y.z.9000`. The maintainer bumps release + versions; Claude never does. + +## Checklist (new or modified code) + +``` +[ ] pkg::fun() everywhere (except data.table operators / ggplot2 in docs) +[ ] checkmate + shared assert_*() helpers; group_by can't hold reserved id cols +[ ] input travel_matrix / land_use_data not modified (copy()); test proves it +[ ] output class mirrors travel_matrix (original_class idiom); test proves it +[ ] output columns/order follow A2; fill_missing_ids where ids can drop +[ ] values tested against a hand-computed toy example / the cited formula +[ ] base stop()/warning() with call. = FALSE and quoted arg names +[ ] style: 2 spaces, <-, 80 cols, return(), file's blank-line style +[ ] roxygen: shared @template, @family (exact spelling), @return, + @examplesIf NOT_CRAN, Rdpack citation for the method +[ ] new exports listed in _pkgdown.yml; new NSE names in globalVariables() +[ ] devtools::document() run; no hand-edited generated files +[ ] NEWS.md bullet for user-facing changes +[ ] perf claims backed by bench::mark with equal results +``` + +## Cross-references + +- [`quality-gates.md`](quality-gates.md): when each check must pass. +- `.claude/agents/r-package-reviewer.md` enforces this rule. It is local to + the maintainer's setup and may be absent in a fresh clone; if so, apply the + checklist manually. diff --git a/.claude/rules/session-logging.md b/.claude/rules/session-logging.md new file mode 100644 index 0000000..5b78e0d --- /dev/null +++ b/.claude/rules/session-logging.md @@ -0,0 +1,20 @@ +# Session Logging + +**Location:** `quality_reports/session_logs/YYYY-MM-DD_description.md` +(local, gitignored). **Template:** `templates/session-log.md`. + +A Stop hook also appends a mechanical change list to +`quality_reports/session_logs/auto/YYYY-MM-DD_auto.md`. The narrative log below is still Claude's job. + +## Three triggers (all proactive) + +1. **After plan approval:** goal, approach, rationale, key context, and a + link to the plan. +2. **Incrementally:** append 1–3 lines when a design decision is made, a + problem is solved, the user corrects something (also add a `[LEARN]` entry + to `MEMORY.md`), or the approach changes. Don't batch. +3. **End of session:** summary, gate results (actual numbers: tests passed, + check E/W/N), open questions, next steps. + +Record benchmark results (`bench::mark` output) in the log when a performance +claim is made. diff --git a/.claude/rules/summary-parity.md b/.claude/rules/summary-parity.md new file mode 100644 index 0000000..e2f18c6 --- /dev/null +++ b/.claude/rules/summary-parity.md @@ -0,0 +1,35 @@ +--- +paths: + - "NEWS.md" + - "README.Rmd" + - "vignettes/**" + - "DESCRIPTION" + - "cran-comments.md" + - "R/**/*.R" +--- + +# Summary–body parity + +**When editing a summary, re-check the whole summary against what it +summarises.** Don't patch just the flagged word. Summaries drift when the body +changes and the summary isn't re-verified. + +**Summaries in this repo:** +- the `Description:` field in `DESCRIPTION` +- the README introduction and section ledes +- vignette introductions +- the opening of each `NEWS.md` version entry +- `cran-comments.md` +- PR titles and `## Summary` blocks +- roxygen `@description` / title lines + +## Protocol + +1. Read the full body (the function's actual behaviour, the full NEWS section, + the vignette). +2. List every claim in the summary: counts, function names, "now supports X", + "no longer Y". +3. Check each claim against the code or body. +4. Rewrite the paragraph as a whole when any claim fails. +5. If the same summary is flagged twice in a row, rewrite it structurally and + prefer fewer enumerative claims. A summary with no counts can't drift. diff --git a/.gitignore b/.gitignore index 1fb59ac..6a4c5cb 100644 --- a/.gitignore +++ b/.gitignore @@ -41,3 +41,10 @@ vignettes/*.pdf .Renviron r5/ + +# Claude Code workflow (only CLAUDE.md + .claude/rules/ are tracked) +/.claude/* +!/.claude/rules/ +/MEMORY.md +/templates/ +/quality_reports/ diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..3b7c10d --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,150 @@ +# CLAUDE.md — accessibility + +**accessibility** is an R package of fast, convenient functions for transport +accessibility analysis. Given a pre-computed travel cost matrix and land use +data, it calculates accessibility levels (cumulative, gravity, floating +catchment area, spatial availability, balancing cost, constrained measures, +cost to closest), decay functions, and accessibility inequality and poverty +(Palma, concentration, Gini, Theil T, FGT). Built on `data.table`. Published +on CRAN; developed by Ipea (ipeaGIT). Maintainer: Daniel Herszenhut; authors +include Rafael H. M. Pereira and Anastasia Soukhov. License MIT. Docs: +. + +--- + +## Core principles + +- **Correct numbers first.** Every measure implements a published formula + (cited via Rdpack). A new or changed measure is tested against a + hand-computed toy example, not just against its own current output. +- **Performance is a feature.** Vectorised `data.table` code, no avoidable + copies. Any claimed speed-up needs a `bench::mark()` comparison with equal + results checked. +- **Never surprise the caller.** Functions never modify the input + `travel_matrix` / `land_use_data`, and the output class mirrors the input + (`data.table`, `data.frame`, `tibble`). +- **CRAN-ready at all times.** `R CMD check --as-cran`: 0 errors, 0 warnings. +- **Plan first.** Enter plan mode before non-trivial tasks. Every plan must be + reviewed by two adversarial agents that improve it and make it simpler, with + as little code intervention as possible. Present the plan only when both + agents reach consensus (max 3 rounds, then escalate the disagreement). Plans + live in `quality_reports/plans/`. +- **Verify after.** Every task ends with the gates below, not with "should work". +- **Learn from corrections.** Log `[LEARN:category] wrong → right` in + `MEMORY.md`. +- **Code style.** Always use the convention `package::function()`. Exceptions + are functions from ggplot2 and special data.table operators such as `:=`. + +Conventions are in [`.claude/rules/r-package-conventions.md`](.claude/rules/r-package-conventions.md); +gates are in [`.claude/rules/quality-gates.md`](.claude/rules/quality-gates.md). + +--- + +## Repository map + +``` +accessibility/ +├── DESCRIPTION, NAMESPACE # NAMESPACE is roxygen-generated; never hand-edit +├── R/ # one exported function per file, plus: +│ ├── accessibility-package.R # package doc, importFrom, utils::globalVariables() +│ ├── assert_and_assign.R # shared input validators (assert_travel_matrix, ...) +│ ├── merge_by_reference.R # joins land use opportunities onto the (copied) matrix +│ ├── fill_missing_ids.R # fills unreachable ids with 0 +│ ├── warn_extra_cols.R # warns about columns not in travel_cost/group_by +│ └── apply_gravity_measure.R # shared by gravity-based measures +├── tests/testthat/ # test-.R; fixtures built in setup.R +├── tests/test_rafa/ # personal scratch scripts, not part of the suite +├── inst/extdata/ # travel_matrix.rds, land_use_data.rds, pareto_frontier.rds, grid_bho.rds +├── inst/REFERENCES.bib # Rdpack citations used in the docs +├── data-raw/ # scripts that regenerate inst/extdata +├── man/roxygen/templates/ # shared @param/@return templates +├── vignettes/ # accessibility.Rmd, decay_functions.Rmd, inequality_and_poverty.Rmd +├── README.Rmd → README.md # edit the .Rmd, then re-render +├── NEWS.md # user-facing changelog +├── _pkgdown.yml # reference index; every export must be listed +├── .github/workflows/ # rcmdcheck matrix, pkgdown, test-coverage +└── .pre-commit-config.yaml # readme-rendered, codemeta, pkgdown (active only after `pre-commit install`) +``` + +Function families: accessibility (`cumulative_cutoff`, `cumulative_interval`, +`gravity`, `floating_catchment_area`, `spatial_availability`, +`balancing_cost`, `constrained_accessibility`, `cost_to_closest`) · +`decay_*` (binary, exponential, linear, logistic, power, stepped) · +inequality (`concentration_index`, `gini_index`, `palma_ratio`, `theil_t`) · +poverty (`fgt_poverty`). + +Local-only workflow files (gitignored and Rbuildignored): `MEMORY.md`, +`quality_reports/`, `templates/`, and everything in `.claude/` except `rules/`. + +--- + +## Commands + +```r +devtools::load_all() +devtools::document() # regenerate man/ + NAMESPACE +devtools::test() # full suite +devtools::test(filter = "cumulative_cutoff") +source("tests/testthat/setup.R") # fixtures, when running a test file by hand +devtools::check(args = "--as-cran") # long; run in background +lintr::lint("R/.R") # no .lintr: defaults; act on changed lines only +rmarkdown::render("README.Rmd") # after editing README.Rmd +pkgdown::build_site() +covr::package_coverage() +``` + +- CRAN checks CPU time, so threads are capped: tests and vignettes set + `Sys.setenv(OMP_THREAD_LIMIT = 2)`. Examples run only when + `NOT_CRAN=true` (`@examplesIf`). +- Tests run in testthat's 2nd edition (no `Config/testthat/edition`). + +--- + +## Quality gates (summary) + +| Checkpoint | Must hold | +|---|---| +| Commit | `document()` produces no content drift (`git diff`); tests for touched functions pass; `lintr` clean on changed lines; values checked against a hand-computed example for changed measures; README re-rendered if `README.Rmd` changed; `NEWS.md` entry for user-facing changes | +| PR | Full `test()` passes; `check(--as-cran)` gives 0 errors / 0 warnings, NOTEs triaged; review loop converged; `r-package-reviewer` 0 CRITICAL/MAJOR | +| Release | `/acc-package-check`; win-builder + rhub; reverse-dependency check; `cran-comments.md` updated | + +Never bypass `.pre-commit-config.yaml` hooks (`--no-verify`). Details: `.claude/rules/quality-gates.md`. + +--- + +## Skills and agents + +These are local to the maintainer's Claude setup and are not in git. In a fresh +clone, follow the rules in `.claude/rules/` directly. + +Project skills carry an `acc-` prefix so user-level skills with the same base +name can't shadow them. **In this repo, always use the `acc-` version:** use +`/acc-commit`, never a generic `/commit` that merges to `main`. Where +user-level rules conflict with `.claude/rules/`, the project rules win. + +- `/acc-package-check`: document, test, check `--as-cran`, triage, reviewer + report. +- `/acc-commit`: Gate 1, then branch and commit. Push and PR only when asked, + after Gate 2. **Merging to `main` only on explicit request.** +- `/acc-diagnose`: root-cause a failing test or wrong accessibility result. +- `/acc-checkpoint`, `/acc-compress-session`, `/context-status`: session state. +- `/acc-learn`: turn a session discovery into a reusable skill. +- Agent `r-package-reviewer`: read-only CRAN and convention review of + package source. + +--- + +## Working mode + +1. **Plan:** for non-trivial tasks, enter plan mode, explore, ask only about + real decisions, pass the adversarial plan review, and get approval. +2. **Execute (contractor mode):** after approval, work autonomously through + implement → verify → parallel review → fix, looping until reviews come + back dry (see `.claude/rules/orchestrator-protocol.md`). Come back only for + ambiguity or a decision that belongs to the user. +3. **Report:** summarise what changed, gate results, open questions; log the + session. + +No commits, pushes, PRs, merges, version bumps or `.github/workflows/` +changes without an explicit request. Never hand-edit generated files +(`NAMESPACE`, `man/*.Rd`, `README.md`, `codemeta.json`, `inst/extdata/*.rds`). From 999c975245c4edcb128ab6347704d4cbcffc028f Mon Sep 17 00:00:00 2001 From: rafapereirabr Date: Tue, 20 Oct 2026 11:29:25 -0300 Subject: [PATCH 02/31] bug in balancing_cost --- NEWS.md | 8 ++++++++ R/balancing_cost.R | 2 +- tests/testthat/test-balancing_cost.R | 30 ++++++++++++++++++++++++++++ 3 files changed, 39 insertions(+), 1 deletion(-) diff --git a/NEWS.md b/NEWS.md index a2aa110..ac81262 100644 --- a/NEWS.md +++ b/NEWS.md @@ -1,3 +1,11 @@ +# accessibility (development version) + +## Bug fixes + +- Fixed a bug in `balancing_cost()` in which numeric ids (e.g. `101`, `102`) + were used as positions to look up each origin's population, which resulted + in `NA` or wrong balancing costs. + # accessibility 1.5.0 ## New features diff --git a/R/balancing_cost.R b/R/balancing_cost.R index a35585a..fd9ada4 100644 --- a/R/balancing_cost.R +++ b/R/balancing_cost.R @@ -123,7 +123,7 @@ balancing_cost <- function(travel_matrix, for (i in lookup_costs) { cum_opps <- calc_cum_cutoff(data, groups, opportunity, travel_cost, i) - cum_opps[, did_balance := access > lookup_vec[from_id]] + cum_opps[, did_balance := access > lookup_vec[as.character(from_id)]] balanced <- cum_opps[did_balance == TRUE] if (nrow(balanced) > 0) { diff --git a/tests/testthat/test-balancing_cost.R b/tests/testthat/test-balancing_cost.R index 49170e4..69339cb 100644 --- a/tests/testthat/test-balancing_cost.R +++ b/tests/testthat/test-balancing_cost.R @@ -155,6 +155,36 @@ test_that("calculates balancing cost correctly", { expect_identical(result, expected_result) }) +test_that("works with numeric ids, regardless of land_use_data row order", { + toy_matrix <- data.table::data.table( + from_id = rep(101:103, each = 3), + to_id = rep(101:103, 3), + travel_time = c(0, 10, 20, 10, 0, 30, 20, 30, 0) + ) + toy_land_use <- data.table::data.table( + id = 101:103, + jobs = c(10, 20, 30), + population = c(5, 15, 50) + ) + + # 103 has 50 people: reaches 30 jobs at 0, 40 at 20 and 60 at 30 + + expected_result <- data.table::data.table( + id = 101:103, + travel_time = c(0, 0, 30) + ) + + result <- tester(toy_matrix, toy_land_use, group_by = character(0)) + expect_equal(result, expected_result) + + result <- tester( + toy_matrix, + toy_land_use[c(3, 1, 2)], + group_by = character(0) + ) + expect_equal(result, expected_result) +}) + test_that("fill_missing_ids arg works correctly", { test_matrix <- rbind( fast_smaller_matrix, From cd9f47d5784414b697502552ed8ab9b2777384b4 Mon Sep 17 00:00:00 2001 From: rafapereirabr Date: Tue, 20 Oct 2026 11:41:44 -0300 Subject: [PATCH 03/31] minor bugs --- NEWS.md | 8 ++++ R/concentration_index.R | 6 +++ R/gini_index.R | 6 +++ R/spatial_availability.R | 16 +++++-- R/theil_t.R | 6 +++ tests/testthat/test-concentration_index.R | 26 +++++++++++ tests/testthat/test-gini_index.R | 21 +++++++++ tests/testthat/test-spatial_availability.R | 50 ++++++++++++++++++++++ tests/testthat/test-theil_t.R | 32 ++++++++++++++ 9 files changed, 168 insertions(+), 3 deletions(-) diff --git a/NEWS.md b/NEWS.md index ac81262..4f8a9ee 100644 --- a/NEWS.md +++ b/NEWS.md @@ -5,6 +5,14 @@ - Fixed a bug in `balancing_cost()` in which numeric ids (e.g. `101`, `102`) were used as positions to look up each origin's population, which resulted in `NA` or wrong balancing costs. +- Fixed a bug in `spatial_availability()` in which a destination that no one + could reach (or that was only reached by origins with no demand) made the + results of every origin `NaN`. The opportunities in such destinations are + now left unallocated. +- Fixed a bug in `gini_index()`, `concentration_index()` and `theil_t()` in + which large integer opportunity and population counts overflowed when + multiplied, returning `NA` (e.g. with long travel time cutoffs or large + cities). # accessibility 1.5.0 diff --git a/R/concentration_index.R b/R/concentration_index.R index 3b948f1..bc02c05 100644 --- a/R/concentration_index.R +++ b/R/concentration_index.R @@ -104,6 +104,12 @@ concentration_index <- function(accessibility_data, warn_extra_access_cols(accessibility_data, opportunity, group_by) + # integer opportunities and population counts may overflow when multiplied, + # so we convert them to doubles first + + .numeric_cols <- c(opportunity, population) + data[, (.numeric_cols) := lapply(.SD, as.numeric), .SDcols = .numeric_cols] + .opp_colname <- opportunity .pop_colname <- population .groups <- group_by diff --git a/R/gini_index.R b/R/gini_index.R index 1e130f9..a5f1955 100644 --- a/R/gini_index.R +++ b/R/gini_index.R @@ -69,6 +69,12 @@ gini_index <- function(accessibility_data, warn_extra_access_cols(accessibility_data, opportunity, group_by) + # integer opportunities and population counts may overflow when multiplied, + # so we convert them to doubles first + + .numeric_cols <- c(opportunity, population) + data[, (.numeric_cols) := lapply(.SD, as.numeric), .SDcols = .numeric_cols] + .opp_colname <- opportunity .pop_colname <- population .groups <- group_by diff --git a/R/spatial_availability.R b/R/spatial_availability.R index 30615d3..0dbc659 100644 --- a/R/spatial_availability.R +++ b/R/spatial_availability.R @@ -129,16 +129,26 @@ spatial_availability <- function(travel_matrix, total_demand <- sum(land_use_data[[.demand_colname]] ^ alpha) data[, demand_bal_fac := (get(.demand_colname) ^ alpha) / total_demand] + # destinations that no one reaches (or that are only reached by origins + # without demand) have null denominators. their opportunities are not + # allocated to any origin, so their factors are set to 0 instead of NaN + data[ , - impedance_bal_fac := opp_weight / sum(opp_weight), + impedance_bal_fac := { + denominator <- sum(opp_weight) + if (isTRUE(denominator == 0)) 0 else opp_weight / denominator + }, by = c("to_id", group_by) ] data[ , - combined_bal_fac := demand_bal_fac * impedance_bal_fac / - sum(demand_bal_fac * impedance_bal_fac), + combined_bal_fac := { + numerator <- demand_bal_fac * impedance_bal_fac + denominator <- sum(numerator) + if (isTRUE(denominator == 0)) 0 else numerator / denominator + }, by = c("to_id", group_by) ] diff --git a/R/theil_t.R b/R/theil_t.R index 321ec35..c5eb53d 100644 --- a/R/theil_t.R +++ b/R/theil_t.R @@ -113,6 +113,12 @@ theil_t <- function(accessibility_data, warn_extra_access_cols(accessibility_data, opportunity, group_by) + # integer opportunities and population counts may overflow when multiplied, + # so we convert them to doubles first + + .numeric_cols <- c(opportunity, population) + data[, (.numeric_cols) := lapply(.SD, as.numeric), .SDcols = .numeric_cols] + if (is.null(socioeconomic_groups)) { result <- theil_without_groups(data, opportunity, population, group_by) } else { diff --git a/tests/testthat/test-concentration_index.R b/tests/testthat/test-concentration_index.R index 13d45c4..e60f586 100644 --- a/tests/testthat/test-concentration_index.R +++ b/tests/testthat/test-concentration_index.R @@ -232,3 +232,29 @@ test_that("works even if access_data and sociodem_data has specific colnames", { access_data[, group_by := NULL] }) + +test_that("doesn't overflow with large integer opportunities and population", { + # jobs * population exceeds .Machine$integer.max (~2.1e9) + + toy_access <- data.table::data.table(id = c("a", "b"), jobs = c(50000L, 100000L)) + toy_sociodem <- data.table::data.table( + id = c("a", "b"), + population = c(50000L, 30000L), + income_per_capita = c(1, 2), + income_decile = c(1, 2) + ) + + # population-weighted mean access: (5e4 * 5e4 + 3e4 * 1e5) / 8e4 = 68750 + + # fractional ranks are 0.3125 and 0.8125, population shares 0.625 and 0.375 + + expected_ci <- 2 / 68750 * ( + 0.625 * (50000 - 68750) * (0.3125 - 0.5) + + 0.375 * (100000 - 68750) * (0.8125 - 0.5) + ) + + result <- expect_silent( + tester(toy_access, toy_sociodem, group_by = character(0)) + ) + expect_equal(result$concentration_index, expected_ci) +}) diff --git a/tests/testthat/test-gini_index.R b/tests/testthat/test-gini_index.R index e83b46b..72559f0 100644 --- a/tests/testthat/test-gini_index.R +++ b/tests/testthat/test-gini_index.R @@ -171,3 +171,24 @@ test_that("works even if access_data and sociodem_data has specific colnames", { access_data[, group_by := NULL] }) + +test_that("doesn't overflow with large integer opportunities and population", { + # jobs * population exceeds .Machine$integer.max (~2.1e9) + + toy_access <- data.table::data.table(id = c("a", "b"), jobs = c(50000L, 100000L)) + toy_sociodem <- data.table::data.table( + id = c("a", "b"), + population = c(50000L, 30000L), + income_per_capita = c(1, 2), + income_decile = c(1, 2) + ) + + # population-weighted mean access: (5e4 * 5e4 + 3e4 * 1e5) / 8e4 = 68750 + + # pairwise gini: 2 * 5e4 * 3e4 * 5e4 / (2 * 68750 * 8e4^2) = 15 / 88 + + result <- expect_silent( + tester(toy_access, toy_sociodem, group_by = character(0)) + ) + expect_equal(result$gini_index, 15 / 88) +}) diff --git a/tests/testthat/test-spatial_availability.R b/tests/testthat/test-spatial_availability.R index 4b3b548..77efce6 100644 --- a/tests/testthat/test-spatial_availability.R +++ b/tests/testthat/test-spatial_availability.R @@ -281,6 +281,56 @@ test_that("calculates spatial availability correctly", { ) }) +test_that("doesn't return NaN when a destination has null denominators", { + toy_matrix <- data.table::data.table( + from_id = rep(c("a", "b", "c"), each = 3), + to_id = rep(c("a", "b", "c"), 3), + travel_time = c(0, 10, 20, 10, 0, 30, 20, 30, 0) + ) + toy_land_use <- data.table::data.table( + id = c("a", "b", "c"), + jobs = c(10, 20, 30), + population = c(10, 5, 1) + ) + + # no one reaches c, so its 30 jobs are not allocated. a and b split the jobs + # in a and b proportionally to their population (10:5) + + unreachable_matrix <- data.table::copy(toy_matrix) + unreachable_matrix[from_id == "c" & to_id == "c", travel_time := 25] + + result <- tester( + unreachable_matrix, + toy_land_use, + decay_function = decay_binary(15), + group_by = character(0) + ) + expect_equal(result$jobs, c(20, 10, 0)) + + # b is only reached by itself, and it has no population, so its 20 jobs are + # not allocated. a and c only reach themselves + + no_demand_land_use <- data.table::copy(toy_land_use) + no_demand_land_use[id == "b", population := 0] + + result <- tester( + toy_matrix, + no_demand_land_use, + decay_function = decay_binary(5), + group_by = character(0) + ) + expect_equal(result$jobs, c(10, 0, 30)) + + result <- tester( + unreachable_matrix, + toy_land_use, + decay_function = decay_binary(15), + group_by = character(0), + detailed_results = TRUE + ) + expect_false(anyNA(result$jobs)) +}) + test_that("results are grouped by decay_function_arg when needed", { small_travel_matrix <- travel_matrix[ from_id %in% c("89a88cdb57bffff", "89a88cdb597ffff") & diff --git a/tests/testthat/test-theil_t.R b/tests/testthat/test-theil_t.R index bde7667..7e00d90 100644 --- a/tests/testthat/test-theil_t.R +++ b/tests/testthat/test-theil_t.R @@ -452,3 +452,35 @@ test_that("works even if access_data and sociodem_data has specific colnames", { access_data[, group_by := NULL] }) + +test_that("doesn't overflow with large integer opportunities and population", { + # jobs * population exceeds .Machine$integer.max (~2.1e9) + + toy_access <- data.table::data.table(id = c("a", "b"), jobs = c(50000L, 100000L)) + toy_sociodem <- data.table::data.table( + id = c("a", "b"), + population = c(50000L, 30000L), + income_per_capita = c(1, 2), + income_decile = c(1, 2) + ) + + # population-weighted mean access: (5e4 * 5e4 + 3e4 * 1e5) / 8e4 = 68750 + + ratio <- c(50000, 100000) / 68750 + expected_theil <- sum(c(0.625, 0.375) * ratio * log(ratio)) + + result <- expect_silent( + tester( + toy_access, + toy_sociodem, + socioeconomic_groups = NULL, + group_by = character(0) + ) + ) + expect_equal(result$theil_t, expected_theil) + + result <- expect_silent( + tester(toy_access, toy_sociodem, group_by = character(0)) + ) + expect_equal(result$summary[component == "total"]$value, expected_theil) +}) From cfbd6062899ce4fa9e1a846d26bdcf327dd0d5b6 Mon Sep 17 00:00:00 2001 From: rafapereirabr Date: Tue, 20 Oct 2026 13:02:17 -0300 Subject: [PATCH 04/31] minor issue in doubly_constrained --- R/doubly_constrained.R | 26 +++++- tests/testthat/test-constrained-essentials.R | 95 ++++++++++++++++++++ 2 files changed, 118 insertions(+), 3 deletions(-) diff --git a/R/doubly_constrained.R b/R/doubly_constrained.R index 59c9cd1..c55d28f 100644 --- a/R/doubly_constrained.R +++ b/R/doubly_constrained.R @@ -64,8 +64,17 @@ doubly_constrained <- function(travel_matrix, # Prepare vectors and impedance matrix origins <- unique(data$from_id) destinations <- unique(data$to_id) - O <- land_use_data[id %in% origins, get(demand)] - D <- land_use_data[id %in% destinations, get(supply)] + O <- land_use_data[[demand]][match(origins, land_use_data$id)] + D <- land_use_data[[supply]][match(destinations, land_use_data$id)] + + if (anyNA(O) || anyNA(D)) { + stop( + "For doubly-constrained, 'land_use_data' must list a non-NA demand for ", + "every origin and a non-NA supply for every destination in ", + "'travel_matrix'.", + call. = FALSE + ) + } # Validate totals match if (abs(sum(O) - sum(D)) > 1e-6) { @@ -73,7 +82,18 @@ doubly_constrained <- function(travel_matrix, } # Build impedance matrix - impedance_matrix <- matrix(data$opp_weight, nrow = length(origins), ncol = length(destinations), byrow = FALSE) + # filled by id, so the result doesn't depend on the row order of + # travel_matrix. pairs missing from travel_matrix are unreachable (weight 0) + + impedance_matrix <- matrix( + 0, + nrow = length(origins), + ncol = length(destinations), + dimnames = list(origins, destinations) + ) + impedance_matrix[ + cbind(as.character(data$from_id), as.character(data$to_id)) + ] <- data$opp_weight # Iterative proportional fitting Ai <- rep(1, length(O)) diff --git a/tests/testthat/test-constrained-essentials.R b/tests/testthat/test-constrained-essentials.R index 66b5c47..59c6fe5 100644 --- a/tests/testthat/test-constrained-essentials.R +++ b/tests/testthat/test-constrained-essentials.R @@ -209,3 +209,98 @@ test_that("doubly-constrained: OD flows match marginals", { expect_equal(by_dest$sum_flow, D, tolerance = 1e-3) }) + +test_that("doubly-constrained: results don't depend on row order or sparsity", { + ids <- c("1", "2", "3") + cost <- matrix( + c(0, 10, 25, 15, 0, 5, 30, 20, 0), + nrow = 3, + byrow = TRUE, + dimnames = list(ids, ids) + ) + lu <- make_lu_match20() + + # independent iterative proportional fitting, as reference + + reference_flows <- function(weights) { + a <- b <- rep(1, 3) + for (k in 1:2000) { + a <- 1 / as.vector(weights %*% (b * lu$jobs)) + b <- 1 / as.vector(t(weights) %*% (a * lu$population)) + } + flows <- outer(a * lu$population, b * lu$jobs) * weights + dimnames(flows) <- list(ids, ids) + flows + } + + run_doubly <- function(tm, land_use = lu) { + constrained_accessibility( + constraint = "doubly", + travel_matrix = tm, + land_use_data = land_use, + travel_cost = "travel_time", + decay_function = exp_decay_01, + demand = "population", + supply = "jobs", + detailed_results = TRUE, + error_threshold = 1e-10, + improvement_threshold = 0 + ) + } + + to_major <- data.table::data.table( + expand.grid(from_id = ids, to_id = ids, stringsAsFactors = FALSE) + ) + to_major[, travel_time := cost[cbind(from_id, to_id)]] + from_major <- to_major[order(from_id, to_id)] + shuffled <- to_major[c(5, 9, 1, 7, 3, 8, 2, 6, 4)] + + expected <- reference_flows(exp(-0.1 * cost)) + for (tm in list(to_major, from_major, shuffled)) { + result <- run_doubly(tm) + expect_equal( + result$flow, + expected[cbind(result$from_id, result$to_id)], + tolerance = 1e-6 + ) + } + + result <- run_doubly(to_major, lu[c(3, 1, 2)]) + expect_equal( + result$flow, + expected[cbind(result$from_id, result$to_id)], + tolerance = 1e-6 + ) + + # pairs missing from the travel matrix are unreachable + + sparse_weights <- exp(-0.1 * cost) + sparse_weights["1", "3"] <- 0 + expected <- reference_flows(sparse_weights) + + result <- run_doubly(shuffled[!(from_id == "1" & to_id == "3")]) + expect_equal( + result$flow, + expected[cbind(result$from_id, result$to_id)], + tolerance = 1e-6 + ) + expect_identical(result[from_id == "1" & to_id == "3"]$flow, 0) +}) + +test_that("doubly-constrained: errors when land_use_data misses an id", { + # the validators also warn about the missing id + + expect_error( + suppressWarnings(constrained_accessibility( + constraint = "doubly", + travel_matrix = make_tm_3x3(), + land_use_data = make_lu_match20()[1:2], + travel_cost = "travel_time", + decay_function = exp_decay_01, + demand = "population", + supply = "jobs", + detailed_results = TRUE + )), + regexp = "must list a non-NA demand" + ) +}) From 6ab633c807cbfc95c692368ada9831593de43595 Mon Sep 17 00:00:00 2001 From: rafapereirabr Date: Tue, 20 Oct 2026 13:02:33 -0300 Subject: [PATCH 05/31] theil_t with zeros --- NEWS.md | 14 +++++++++ R/theil_t.R | 47 ++++++++++++++++++++++------ man/theil_t.Rd | 14 ++++++--- tests/testthat/test-theil_t.R | 58 +++++++++++++++++++++++++++++++++++ 4 files changed, 119 insertions(+), 14 deletions(-) diff --git a/NEWS.md b/NEWS.md index 4f8a9ee..8b83a90 100644 --- a/NEWS.md +++ b/NEWS.md @@ -13,6 +13,20 @@ which large integer opportunity and population counts overflowed when multiplied, returning `NA` (e.g. with long travel time cutoffs or large cities). +- Fixed a bug in `constrained_accessibility(constraint = "doubly")` in which + the impedance matrix and the demand and supply totals were aligned by row + position instead of by id. Results were only correct when `travel_matrix` + listed every origin-destination pair ordered by destination and + `land_use_data` was ordered like the travel matrix. Pairs missing from + `travel_matrix` are now treated as unreachable, and ids missing from + `land_use_data` raise an informative error. +- Fixed a bug in `theil_t()` in which observations with no access were + wrongly handled: without `socioeconomic_groups` they were excluded from the + index (but not from the average accessibility), and with + `socioeconomic_groups` they made the index `NaN`. They now contribute 0 to + the index, following the usual `0 * log(0) = 0` convention. `group_by` + levels in which no one has access are now kept in the output with `NA`, + instead of being dropped. # accessibility 1.5.0 diff --git a/R/theil_t.R b/R/theil_t.R index c5eb53d..4185843 100644 --- a/R/theil_t.R +++ b/R/theil_t.R @@ -8,6 +8,11 @@ #' decomposed into a between-groups inequaliy component and a within-groups #' component. #' +#' Observations with no access contribute 0 to the index, following the usual +#' convention that `0 * log(0) = 0`. If no one has access to any opportunity +#' (i.e. the average accessibility is 0), the index is undefined and `NA` is +#' returned. +#' #' @template accessibility_data #' @param sociodemographic_data A data frame. The distribution of #' sociodemographic characteristics of the population in the study area cells. @@ -146,11 +151,6 @@ theil_t <- function(accessibility_data, } theil_without_groups <- function(data, opportunity, population, group_by) { - # we have to filter by opportunity because opportunity = 0 results in - # theil_t being NaN (because log(0) is NaN) - # TODO: throw warning - - .opp_colname <- opportunity .groups <- group_by data[ @@ -162,8 +162,11 @@ theil_without_groups <- function(data, opportunity, population, group_by) { by = .groups ] + # subsetting rows (even if all of them) keeps the result empty when data is + # empty and there are no groups + theil_index <- data[ - get(.opp_colname) > 0, + seq_len(.N), .( theil_t = calc_theil_t(get(..opportunity), avg_access, get(..population)) ), @@ -245,7 +248,7 @@ theil_with_groups <- function(data, group_tot_pop ), within_group = sum( - group_theil_t * group_tot_access / sum(group_tot_access) + within_contribution(group_theil_t, group_tot_access) ) ), by = .groups @@ -266,7 +269,7 @@ theil_with_groups <- function(data, within_group <- data.table::copy(summarized_data) within_group[ , - value := group_theil_t * group_tot_access / sum(group_tot_access), + value := within_contribution(group_theil_t, group_tot_access), by = .groups ] within_group[ @@ -285,7 +288,14 @@ theil_with_groups <- function(data, # decomposed between group between_group <- data.table::copy(summarized_data) - between_group[, value := access_share * log(access_share / pop_share)] + between_group[ + , + value := data.table::fifelse( + access_share > 0, + access_share * log(access_share / pop_share), + 0 + ) + ] between_group[, eval(cols_to_drop) := NULL] data.table::setorderv(between_group, .socioecon_groups) @@ -298,7 +308,24 @@ theil_with_groups <- function(data, return(output_list) } +# observations with no access contribute 0 to the index, following the usual +# 0 * log(0) = 0 convention. if the average access is 0 the index is undefined, +# and NA is returned + calc_theil_t <- function(x, avg_x, weight) { - stats::weighted.mean(x / avg_x * log(x / avg_x), w = weight) + ratio <- x / avg_x + terms <- data.table::fifelse(ratio > 0, ratio * log(ratio), 0) + + stats::weighted.mean(terms, w = weight) +} + + +# socioeconomic groups with no access have an undefined theil index, but their +# weight in the within-group component is 0, so they contribute 0 + +within_contribution <- function(group_theil_t, group_tot_access) { + access_share <- group_tot_access / sum(group_tot_access) + + data.table::fifelse(access_share > 0, group_theil_t * access_share, 0) } diff --git a/man/theil_t.Rd b/man/theil_t.Rd index b3987a2..5bbc36a 100644 --- a/man/theil_t.Rd +++ b/man/theil_t.Rd @@ -62,6 +62,12 @@ mutually exclusive and completely exhaustive groups, the index can be decomposed into a between-groups inequaliy component and a within-groups component. } +\details{ +Observations with no access contribute 0 to the index, following the usual +convention that \code{0 * log(0) = 0}. If no one has access to any opportunity +(i.e. the average accessibility is 0), the index is undefined and \code{NA} is +returned. +} \examples{ \dontshow{if (identical(tolower(Sys.getenv("NOT_CRAN")), "true")) withAutoprint(\{ # examplesIf} data_dir <- system.file("extdata", package = "accessibility") @@ -108,9 +114,9 @@ ti \dontshow{\}) # examplesIf} } \seealso{ -Other inequality: -\code{\link{concentration_index}()}, -\code{\link{gini_index}()}, -\code{\link{palma_ratio}()} +Other inequality: +\code{\link[=concentration_index]{concentration_index()}}, +\code{\link[=gini_index]{gini_index()}}, +\code{\link[=palma_ratio]{palma_ratio()}} } \concept{inequality} diff --git a/tests/testthat/test-theil_t.R b/tests/testthat/test-theil_t.R index 7e00d90..d717428 100644 --- a/tests/testthat/test-theil_t.R +++ b/tests/testthat/test-theil_t.R @@ -484,3 +484,61 @@ test_that("doesn't overflow with large integer opportunities and population", { ) expect_equal(result$summary[component == "total"]$value, expected_theil) }) + +test_that("handles zero access with the 0 * log(0) = 0 convention", { + toy_access <- data.table::data.table( + id = rep(c("a", "b", "c", "d"), 2), + mode = rep(c("car", "walk"), each = 4), + jobs = c(0, 10, 20, 30, 0, 0, 0, 0) + ) + toy_sociodem <- data.table::data.table( + id = c("a", "b", "c", "d"), + population = c(1, 1, 1, 1), + income_decile = c(1, 1, 2, 2) + ) + + # car: mean access 15, so ratios are 0, 2/3, 4/3 and 2. walk: no one has + # access, so the index is undefined + + ratio <- c(0, 10, 20, 30) / 15 + expected_theil <- mean(c(0, ratio[-1] * log(ratio[-1]))) + + result <- tester(toy_access, toy_sociodem, socioeconomic_groups = NULL) + expect_equal(result[mode == "car"]$theil_t, expected_theil) + expect_true(is.na(result[mode == "walk"]$theil_t)) + + # decomposition: deciles hold 1/6 and 5/6 of the access and half of the + # population each + + access_share <- c(10, 50) / 60 + expected_between <- sum(access_share * log(access_share / 0.5)) + + result <- tester(toy_access, toy_sociodem) + car_summary <- result$summary[mode == "car"] + expect_equal( + car_summary[component == "between_group"]$value, + expected_between + ) + expect_equal(car_summary[component == "total"]$value, expected_theil) + expect_false(anyNA(result$within_group_component[mode == "car"]$value)) + expect_true(all(is.na(result$summary[mode == "walk"]$value))) + + # a decile with no access has weight 0 in the within-group component + + toy_access <- data.table::data.table( + id = c("a", "b", "c", "d"), + jobs = c(0, 0, 20, 30) + ) + ratio <- c(0, 0, 20, 30) / 12.5 + expected_theil <- mean(c(0, 0, ratio[3:4] * log(ratio[3:4]))) + + result <- tester(toy_access, toy_sociodem, group_by = character(0)) + expect_equal( + result$summary[component == "total"]$value, + expected_theil + ) + expect_equal( + result$within_group_component[income_decile == 1]$value, + 0 + ) +}) From 0904f09a3348aaba1f1c93afe6bfcd8f5e7fb5ec Mon Sep 17 00:00:00 2001 From: rafapereirabr Date: Tue, 20 Oct 2026 13:08:59 -0300 Subject: [PATCH 06/31] update url to ipea and minor fixes --- .claude/rules/r-package-conventions.md | 2 +- CLAUDE.md | 4 +-- DESCRIPTION | 6 ++-- NEWS.md | 2 +- R/accessibility-package.R | 6 ++-- R/assert_and_assign.R | 24 ++++++++++++++- R/balancing_cost.R | 20 +++++++++++-- R/spatial_availability.R | 6 +++- README.Rmd | 14 ++++----- README.md | 2 +- _pkgdown.yml | 2 +- codemeta.json | 30 ++++++++++++------- man/accessibility.Rd | 13 ++++---- tests/testthat/test-assert_land_use_data.R | 10 ++++++- ...st-assert_sociodemographic_data_warnings.R | 8 +++++ tests/testthat/test-balancing_cost.R | 26 ++++++++++++++++ tests/testthat/test-spatial_availability.R | 24 +++++++++++++++ vignettes/accessibility.Rmd | 4 +-- 18 files changed, 160 insertions(+), 43 deletions(-) diff --git a/.claude/rules/r-package-conventions.md b/.claude/rules/r-package-conventions.md index 3f9a012..218eb64 100644 --- a/.claude/rules/r-package-conventions.md +++ b/.claude/rules/r-package-conventions.md @@ -280,7 +280,7 @@ results. ``` - New function `fun()`, which implements ... proposed in Author et al. (Year). -- `fun()` has a new argument `arg` ... Closes [#63](https://github.com/ipeaGIT/accessibility/issues/63). +- `fun()` has a new argument `arg` ... Closes [#63](https://github.com/ipea/accessibility/issues/63). - Fixed a bug in `fun()` that ... (#issue). ``` diff --git a/CLAUDE.md b/CLAUDE.md index 3b7c10d..05c24b1 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,9 +6,9 @@ data, it calculates accessibility levels (cumulative, gravity, floating catchment area, spatial availability, balancing cost, constrained measures, cost to closest), decay functions, and accessibility inequality and poverty (Palma, concentration, Gini, Theil T, FGT). Built on `data.table`. Published -on CRAN; developed by Ipea (ipeaGIT). Maintainer: Daniel Herszenhut; authors +on CRAN; developed by Ipea (github.com/ipea). Maintainer: Daniel Herszenhut; authors include Rafael H. M. Pereira and Anastasia Soukhov. License MIT. Docs: -. +. --- diff --git a/DESCRIPTION b/DESCRIPTION index fdc5b35..0c82491 100644 --- a/DESCRIPTION +++ b/DESCRIPTION @@ -24,9 +24,9 @@ Description: A set of fast and convenient functions to help conducting metrics (such as the Palma ratio, the concentration and Theil indices and the FGT family of measures). License: MIT + file LICENSE -URL: https://github.com/ipeaGIT/accessibility, - https://ipeagit.github.io/accessibility/ -BugReports: https://github.com/ipeaGIT/accessibility/issues +URL: https://github.com/ipea/accessibility, + https://ipea.github.io/accessibility/ +BugReports: https://github.com/ipea/accessibility/issues Depends: R (>= 3.5.0) Imports: diff --git a/NEWS.md b/NEWS.md index 8b83a90..525321d 100644 --- a/NEWS.md +++ b/NEWS.md @@ -43,7 +43,7 @@ to be poor. This is the correct approach for primal accessibility measures (e.g. cumulative accessibility). If `FALSE`, then observations above the poverty line are considered to be poor. This is the correct approach for dual accessibility measures (e.g. travel time to the closest facility). When set to `FALSE`, FGT 1 -and 2 do not have an upper bound. Closes [#63](https://github.com/ipeaGIT/accessibility/issues/63) +and 2 do not have an upper bound. Closes [#63](https://github.com/ipea/accessibility/issues/63) diff --git a/R/accessibility-package.R b/R/accessibility-package.R index 8565f6d..2b0d141 100644 --- a/R/accessibility-package.R +++ b/R/accessibility-package.R @@ -4,14 +4,14 @@ #' Please check the vignettes for more on the package usage: #' - Introduction to accessibility: calculating accessibility measures. Run #' `vignette("accessibility")` or check it on the [website]( -#' https://ipeagit.github.io/accessibility/articles/accessibility.html). +#' https://ipea.github.io/accessibility/articles/accessibility.html). #' - Decay functions. Run `vignette("decay_functions", package = #' "accessibility")` or check it on the [website]( -#' https://ipeagit.github.io/accessibility/articles/decay_functions.html). +#' https://ipea.github.io/accessibility/articles/decay_functions.html). #' - Calculating accessibility inequality and poverty. Run #' `vignette("inequality_and_poverty", package = "accessibility")` or check it #' on the -#' [website](https://ipeagit.github.io/accessibility/articles/inequality_and_poverty.html). +#' [website](https://ipea.github.io/accessibility/articles/inequality_and_poverty.html). #' #' @docType package #' @name accessibility diff --git a/R/assert_and_assign.R b/R/assert_and_assign.R index 6e82099..502ece8 100644 --- a/R/assert_and_assign.R +++ b/R/assert_and_assign.R @@ -107,6 +107,17 @@ assert_land_use_data <- function(land_use_data, .var.name = "land_use_data" ) + # duplicated ids would be silently resolved by the joins with the travel + # matrix / accessibility data (only one of the rows would be used) + + if (anyDuplicated(land_use_data$id)) { + stop( + "'land_use_data' must not contain duplicated ids. Found duplicates: ", + paste0("'", unique(land_use_data$id[duplicated(land_use_data$id)]), "'", collapse = ", "), + call. = FALSE + ) + } + # if an id is present in travel_matrix but not in land_use_data, merging the # datasets will produce NAs, so we warn users about that @@ -166,6 +177,17 @@ assert_sociodemographic_data <- function(sociodemographic_data, .var.name = "sociodemographic_data" ) + # duplicated ids would be silently resolved by the joins with the travel + # matrix / accessibility data (only one of the rows would be used) + + if (anyDuplicated(sociodemographic_data$id)) { + stop( + "'sociodemographic_data' must not contain duplicated ids. Found duplicates: ", + paste0("'", unique(sociodemographic_data$id[duplicated(sociodemographic_data$id)]), "'", collapse = ", "), + call. = FALSE + ) + } + # if an id is present in accessibility_data but not in sociodemographic_data, # merging the datasets will produce NAs, so we warn users about that @@ -182,7 +204,7 @@ assert_sociodemographic_data <- function(sociodemographic_data, # we warn about that too # if NAs are found in income column, we check if the correspondent population # entries are 0. if not, we throw the warning, otherwise we ignore it. related - # to https://github.com/ipeaGIT/accessibility/issues/42 + # to https://github.com/ipea/accessibility/issues/42 non_income_cols <- c(population, extra_cols) diff --git a/R/balancing_cost.R b/R/balancing_cost.R index fd9ada4..4995f79 100644 --- a/R/balancing_cost.R +++ b/R/balancing_cost.R @@ -59,7 +59,13 @@ balancing_cost <- function(travel_matrix, checkmate::assert_string(opportunity) checkmate::assert_string(travel_cost) checkmate::assert_string(demand) - checkmate::assert_number(cost_increment, lower = 0, finite = TRUE) + checkmate::assert_number(cost_increment, finite = TRUE) + if (cost_increment <= 0) { + stop( + "Assertion on 'cost_increment' failed: Value must be greater than 0.", + call. = FALSE + ) + } checkmate::assert_logical(fill_missing_ids, len = 1, any.missing = FALSE) assert_group_by(group_by) assert_travel_matrix(travel_matrix, travel_cost, group_by) @@ -98,7 +104,15 @@ balancing_cost <- function(travel_matrix, } else { 0 } - lookup_costs <- seq.int(0, max_lookup_cost, by = cost_increment) + + # the last cost must be at least the max cost, otherwise origins that only + # balance after the last multiple of cost_increment are reported as NA + + lookup_costs <- seq.int( + 0, + ceiling(max_lookup_cost / cost_increment) * cost_increment, + by = cost_increment + ) # origins with 0 people should always have a balancing time of 0 @@ -123,7 +137,7 @@ balancing_cost <- function(travel_matrix, for (i in lookup_costs) { cum_opps <- calc_cum_cutoff(data, groups, opportunity, travel_cost, i) - cum_opps[, did_balance := access > lookup_vec[as.character(from_id)]] + cum_opps[, did_balance := access >= lookup_vec[as.character(from_id)]] balanced <- cum_opps[did_balance == TRUE] if (nrow(balanced) > 0) { diff --git a/R/spatial_availability.R b/R/spatial_availability.R index 0dbc659..4b2b6df 100644 --- a/R/spatial_availability.R +++ b/R/spatial_availability.R @@ -126,7 +126,11 @@ spatial_availability <- function(travel_matrix, ) .demand_colname <- demand - total_demand <- sum(land_use_data[[.demand_colname]] ^ alpha) + + # na.rm because ids that are not in travel_matrix may have NA demand. this + # sum cancels out in combined_bal_fac anyway + + total_demand <- sum(land_use_data[[.demand_colname]] ^ alpha, na.rm = TRUE) data[, demand_bal_fac := (get(.demand_colname) ^ alpha) / total_demand] # destinations that no one reaches (or that are only reached by origins diff --git a/README.Rmd b/README.Rmd index 8ad38f8..452b360 100644 --- a/README.Rmd +++ b/README.Rmd @@ -15,11 +15,11 @@ knitr::opts_chunk$set( [![CRAN status](https://www.r-pkg.org/badges/version/accessibility)](https://CRAN.R-project.org/package=accessibility) -[![rcmdcheck](https://github.com/ipeaGIT/accessibility/workflows/rcmdcheck/badge.svg)](https://github.com/ipeaGIT/accessibility/actions) +[![rcmdcheck](https://github.com/ipea/accessibility/workflows/rcmdcheck/badge.svg)](https://github.com/ipea/accessibility/actions) [![CRAN/METACRAN Total downloads](https://cranlogs.r-pkg.org/badges/grand-total/accessibility?color=yellow)](https://CRAN.R-project.org/package=accessibility) [![Codecov test -coverage](https://codecov.io/gh/ipeaGIT/accessibility/branch/main/graph/badge.svg)](https://app.codecov.io/gh/ipeaGIT/accessibility?branch=main) +coverage](https://codecov.io/gh/ipea/accessibility/branch/main/graph/badge.svg)](https://app.codecov.io/gh/ipea/accessibility?branch=main) [![Lifecycle: stable](https://lifecycle.r-lib.org/articles/figures/lifecycle-stable.svg)](https://lifecycle.r-lib.org/articles/stages.html) @@ -46,7 +46,7 @@ Development version: ```{r, eval = FALSE} # install.packages("remotes") -remotes::install_github("ipeaGIT/accessibility") +remotes::install_github("ipea/accessibility") ``` ## Usage @@ -55,11 +55,11 @@ This section aims to present a very brief overview of some of the packages' features. Fore more details please read the vignettes: - [Introduction to **accessibility**: calculating accessibility - measures](https://ipeagit.github.io/accessibility/articles/accessibility.html) + measures](https://ipea.github.io/accessibility/articles/accessibility.html) - [Decay - functions](https://ipeagit.github.io/accessibility/articles/decay_functions.html) + functions](https://ipea.github.io/accessibility/articles/decay_functions.html) - [Calculating accessibility inequality and - poverty](https://ipeagit.github.io/accessibility/articles/inequality_and_poverty.html) + poverty](https://ipea.github.io/accessibility/articles/inequality_and_poverty.html) To calculate accessibility levels, one simply needs a pre-calculated travel matrix and some land use data. Below we showcase some of the available @@ -128,7 +128,7 @@ poverty ## Related work: -- [r5r](https://github.com/ipeaGIT/r5r): Rapid Realistic Routing with R5 in R +- [r5r](https://github.com/ipea/r5r): Rapid Realistic Routing with R5 in R - [tracc](https://github.com/jamaps/tracc): Transport accessibility measures in Python - [access](https://access.readthedocs.io/en/latest/): Spatial Access for PySAL diff --git a/README.md b/README.md index 64938a0..b801e9b 100644 --- a/README.md +++ b/README.md @@ -148,7 +148,7 @@ poverty ## Related work: -- [r5r](https://github.com/ipeaGIT/r5r): Rapid Realistic Routing with R5 +- [r5r](https://github.com/ipea/r5r): Rapid Realistic Routing with R5 in R - [tracc](https://github.com/jamaps/tracc): Transport accessibility measures in Python diff --git a/_pkgdown.yml b/_pkgdown.yml index 0e54a65..9744649 100644 --- a/_pkgdown.yml +++ b/_pkgdown.yml @@ -1,4 +1,4 @@ -url: https://ipeagit.github.io/accessibility/ +url: https://ipea.github.io/accessibility/ template: bootstrap: 5 diff --git a/codemeta.json b/codemeta.json index b1580b2..cef4e43 100644 --- a/codemeta.json +++ b/codemeta.json @@ -2,19 +2,19 @@ "@context": "https://doi.org/10.5063/schema/codemeta-2.0", "@type": "SoftwareSourceCode", "identifier": "accessibility", - "description": "A set of fast and convenient functions to help conducting accessibility analyses. Given a pre-computed travel cost matrix and a land use dataset (containing the location of jobs, healthcare and population, for example), the package allows one to calculate accessibility levels and accessibility poverty and inequality. The package covers the majority of the most commonly used accessibility measures (such as cumulative opportunities, gravity-based and floating catchment areas methods), as well as the most frequently used inequality and poverty metrics (such as the Palma ratio, the concentration and Theil indices and the FGT family of measures).", + "description": "A set of fast and convenient functions to help conducting accessibility analyses. Given a pre-computed travel cost matrix and a land use dataset (containing for example the location of jobs, healthcare and population), the package allows one to calculate accessibility levels, and accessibility poverty and inequality. The package covers the majority of the most commonly used accessibility measures (such as cumulative opportunities, gravity-based and floating catchment areas methods), some cutting edge measures proposed in the literature (e.g. balancing cost and constrained accessibility) as well as the most frequently used inequality and poverty metrics (such as the Palma ratio, the concentration and Theil indices and the FGT family of measures).", "name": "accessibility: Transport Accessibility Measures", - "relatedLink": ["https://ipeagit.github.io/accessibility/", "https://CRAN.R-project.org/package=accessibility"], - "codeRepository": "https://github.com/ipeaGIT/accessibility", - "issueTracker": "https://github.com/ipeaGIT/accessibility/issues", + "relatedLink": ["https://ipea.github.io/accessibility/", "https://CRAN.R-project.org/package=accessibility"], + "codeRepository": "https://github.com/ipea/accessibility", + "issueTracker": "https://github.com/ipea/accessibility/issues", "license": "https://spdx.org/licenses/MIT", - "version": "1.4.0.9000", + "version": "1.5.0", "programmingLanguage": { "@type": "ComputerLanguage", "name": "R", "url": "https://r-project.org" }, - "runtimePlatform": "R version 4.3.3 (2024-02-29)", + "runtimePlatform": "R version 4.5.1 (2025-06-13 ucrt)", "provider": { "@id": "https://cran.r-project.org", "@type": "Organization", @@ -35,9 +35,19 @@ "familyName": "Herszenhut", "email": "dhersz@gmail.com", "@id": "https://orcid.org/0000-0001-8066-1105" + }, + { + "@type": "Person", + "givenName": "Anastasia", + "familyName": "Soukhov" } ], "contributor": [ + { + "@type": "Person", + "givenName": "Christopher", + "familyName": "Higgins" + }, { "@type": "Person", "givenName": "Joey", @@ -219,10 +229,10 @@ }, "SystemRequirements": null }, - "fileSize": "3382.26KB", - "releaseNotes": "https://github.com/ipeaGIT/accessibility/blob/master/NEWS.md", - "readme": "https://github.com/ipeaGIT/accessibility/blob/main/README.md", - "contIntegration": ["https://github.com/ipeaGIT/accessibility/actions", "https://app.codecov.io/gh/ipeaGIT/accessibility?branch=main"], + "fileSize": "3272.323KB", + "releaseNotes": "https://github.com/ipea/accessibility/blob/main/NEWS.md", + "readme": "https://github.com/ipea/accessibility/blob/main/README.md", + "contIntegration": ["https://github.com/ipea/accessibility/actions", "https://app.codecov.io/gh/ipea/accessibility?branch=main"], "developmentStatus": "https://lifecycle.r-lib.org/articles/stages.html", "keywords": ["accessibility", "r", "transport", "transport-accessibility"] } diff --git a/man/accessibility.Rd b/man/accessibility.Rd index c21eb18..910d851 100644 --- a/man/accessibility.Rd +++ b/man/accessibility.Rd @@ -15,21 +15,21 @@ A set of fast and convenient functions to help conducting accessibility analyses Please check the vignettes for more on the package usage: \itemize{ \item Introduction to accessibility: calculating accessibility measures. Run -\code{vignette("accessibility")} or check it on the \href{https://ipeagit.github.io/accessibility/articles/accessibility.html}{website}. -\item Decay functions. Run \code{vignette("decay_functions", package = "accessibility")} or check it on the \href{https://ipeagit.github.io/accessibility/articles/decay_functions.html}{website}. +\code{vignette("accessibility")} or check it on the \href{https://ipea.github.io/accessibility/articles/accessibility.html}{website}. +\item Decay functions. Run \code{vignette("decay_functions", package = "accessibility")} or check it on the \href{https://ipea.github.io/accessibility/articles/decay_functions.html}{website}. \item Calculating accessibility inequality and poverty. Run \code{vignette("inequality_and_poverty", package = "accessibility")} or check it on the -\href{https://ipeagit.github.io/accessibility/articles/inequality_and_poverty.html}{website}. +\href{https://ipea.github.io/accessibility/articles/inequality_and_poverty.html}{website}. } } \seealso{ Useful links: \itemize{ - \item \url{https://github.com/ipeaGIT/accessibility} - \item \url{https://ipeagit.github.io/accessibility/} - \item Report bugs at \url{https://github.com/ipeaGIT/accessibility/issues} + \item \url{https://github.com/ipea/accessibility} + \item \url{https://ipea.github.io/accessibility/} + \item Report bugs at \url{https://github.com/ipea/accessibility/issues} } } @@ -38,6 +38,7 @@ Useful links: Authors: \itemize{ + \item Daniel Herszenhut \email{dhersz@gmail.com} (\href{https://orcid.org/0000-0001-8066-1105}{ORCID}) \item Rafael H. M. Pereira \email{rafa.pereira.br@gmail.com} (\href{https://orcid.org/0000-0003-2125-7465}{ORCID}) \item Anastasia Soukhov } diff --git a/tests/testthat/test-assert_land_use_data.R b/tests/testthat/test-assert_land_use_data.R index b974225..2a57526 100644 --- a/tests/testthat/test-assert_land_use_data.R +++ b/tests/testthat/test-assert_land_use_data.R @@ -86,8 +86,16 @@ test_that("warns if either opportunity or demand cols contain NAs", { }) test_that("assert_land_use_data() doesn't error with tibbles", { - # issue 49 (https://github.com/ipeaGIT/accessibility/issues/49) + # issue 49 (https://github.com/ipea/accessibility/issues/49) # using land_use_data as tibbles would throw errors because this format has # stricter subsetting rules expect_no_error(tester(tibble::as_tibble(land_use_data))) }) + +test_that("errors if land_use_data has duplicated ids", { + duplicated_land_use <- rbind(land_use_data, land_use_data[1]) + expect_error( + tester(duplicated_land_use), + regexp = "must not contain duplicated ids" + ) +}) diff --git a/tests/testthat/test-assert_sociodemographic_data_warnings.R b/tests/testthat/test-assert_sociodemographic_data_warnings.R index de5f655..bcc9dc7 100644 --- a/tests/testthat/test-assert_sociodemographic_data_warnings.R +++ b/tests/testthat/test-assert_sociodemographic_data_warnings.R @@ -83,3 +83,11 @@ test_that("warns if cols contain NAs", { ) ) }) + +test_that("errors if sociodemographic_data has duplicated ids", { + duplicated_sociodem <- rbind(land_use_data, land_use_data[1]) + expect_error( + tester(duplicated_sociodem, population = "population"), + regexp = "must not contain duplicated ids" + ) +}) diff --git a/tests/testthat/test-balancing_cost.R b/tests/testthat/test-balancing_cost.R index 69339cb..95e8772 100644 --- a/tests/testthat/test-balancing_cost.R +++ b/tests/testthat/test-balancing_cost.R @@ -36,6 +36,7 @@ test_that("raises errors due to incorrect input", { expect_error(tester(cost_increment = "a")) expect_error(tester(cost_increment = -1)) + expect_error(tester(cost_increment = 0)) expect_error(tester(cost_increment = Inf)) expect_error(tester(group_by = 1)) @@ -185,6 +186,31 @@ test_that("works with numeric ids, regardless of land_use_data row order", { expect_equal(result, expected_result) }) +test_that("balances when access equals population, at any max cost", { + toy_matrix <- data.table::data.table( + from_id = rep(c("a", "b"), each = 2), + to_id = rep(c("a", "b"), 2), + travel_time = c(0, 25, 25, 0) + ) + toy_land_use <- data.table::data.table( + id = c("a", "b"), + jobs = c(10, 20), + population = c(10, 25) + ) + + # a reaches exactly as many jobs as people at cost 0. b only balances at + # cost 25, which is not a multiple of cost_increment, so it balances at the + # next step of the cost grid (30) + + result <- tester( + toy_matrix, + toy_land_use, + cost_increment = 10, + group_by = character(0) + ) + expect_equal(result$travel_time, c(0, 30)) +}) + test_that("fill_missing_ids arg works correctly", { test_matrix <- rbind( fast_smaller_matrix, diff --git a/tests/testthat/test-spatial_availability.R b/tests/testthat/test-spatial_availability.R index 77efce6..300fa6c 100644 --- a/tests/testthat/test-spatial_availability.R +++ b/tests/testthat/test-spatial_availability.R @@ -331,6 +331,30 @@ test_that("doesn't return NaN when a destination has null denominators", { expect_false(anyNA(result$jobs)) }) +test_that("NA demand in ids not in travel_matrix doesn't affect results", { + toy_matrix <- data.table::data.table( + from_id = rep(c("a", "b", "c"), each = 3), + to_id = rep(c("a", "b", "c"), 3), + travel_time = c(0, 10, 20, 10, 0, 30, 20, 30, 0) + ) + toy_land_use <- data.table::data.table( + id = c("a", "b", "c"), + jobs = c(10, 20, 30), + population = c(10, 5, 1) + ) + extra_land_use <- rbind( + toy_land_use, + data.table::data.table(id = "z", jobs = 5, population = NA) + ) + + expected <- tester(toy_matrix, toy_land_use, group_by = character(0)) + result <- suppressWarnings( + tester(toy_matrix, extra_land_use, group_by = character(0)) + ) + expect_false(anyNA(result$jobs)) + expect_equal(result, expected) +}) + test_that("results are grouped by decay_function_arg when needed", { small_travel_matrix <- travel_matrix[ from_id %in% c("89a88cdb57bffff", "89a88cdb597ffff") & diff --git a/vignettes/accessibility.Rmd b/vignettes/accessibility.Rmd index 07c0e19..4817b2c 100644 --- a/vignettes/accessibility.Rmd +++ b/vignettes/accessibility.Rmd @@ -40,7 +40,7 @@ install.packages("accessibility") ```{r, eval = FALSE} # install.packages("remotes") -remotes::install_github("ipeaGIT/accessibility") +remotes::install_github("ipea/accessibility") ``` # Overview of the package @@ -114,7 +114,7 @@ for demonstration purposes[^1]. [^1]: If you would like to calculate such travel cost matrices yourself, there are several computational packages to do that in R, such as -[r5r](https://github.com/ipeaGIT/r5r), +[r5r](https://github.com/ipea/r5r), [dodgr](https://github.com/UrbanAnalyst/dodgr), [gtfsrouter](https://github.com/UrbanAnalyst/gtfsrouter), [hereR](https://github.com/munterfi/hereR) and From a6831addf4e1c04366b60544a22fffc4aaeea7ed Mon Sep 17 00:00:00 2001 From: rafapereirabr Date: Tue, 20 Oct 2026 13:11:46 -0300 Subject: [PATCH 07/31] dev version --- .Rbuildignore | 4 +- DESCRIPTION | 2 +- NEWS.md | 19 +++ R/assert_and_assign.R | 15 +- tests/testthat/test-constrained-wrapper.R | 161 +++++++++++----------- 5 files changed, 116 insertions(+), 85 deletions(-) diff --git a/.Rbuildignore b/.Rbuildignore index 84a9929..65a1fb2 100644 --- a/.Rbuildignore +++ b/.Rbuildignore @@ -1,7 +1,7 @@ ^.*\.Rproj$ ^\.Rproj\.user$ ^\.github$ -^tests_rafa$ +^tests/test_rafa$ ^codecov\.yml$ ^cran-comments\.md$ ^CRAN-SUBMISSION$ @@ -11,9 +11,9 @@ ^codemeta\.json$ ^data-raw$ ^\.pre-commit-config\.yaml$ -^testing_constrained\.Rmd$ ^\.claude$ ^CLAUDE\.md$ ^MEMORY\.md$ ^templates$ ^quality_reports$ +^man/figures/decay_functions\.png$ diff --git a/DESCRIPTION b/DESCRIPTION index 0c82491..4e4ddf6 100644 --- a/DESCRIPTION +++ b/DESCRIPTION @@ -1,7 +1,7 @@ Type: Package Package: accessibility Title: Transport Accessibility Measures -Version: 1.5.0 +Version: 1.5.0.9000 Authors@R: c( person("Rafael H. M.", "Pereira", , "rafa.pereira.br@gmail.com", role = "aut", comment = c(ORCID = "0000-0003-2125-7465")), diff --git a/NEWS.md b/NEWS.md index 525321d..329c6df 100644 --- a/NEWS.md +++ b/NEWS.md @@ -27,6 +27,25 @@ the index, following the usual `0 * log(0) = 0` convention. `group_by` levels in which no one has access are now kept in the output with `NA`, instead of being dropped. +- Fixed two bugs in `balancing_cost()`: origins only balanced when they reached + more opportunities than people (instead of as many opportunities as people), + and origins that balanced at a cost greater than the last multiple of + `cost_increment` were reported as `NA`. `cost_increment` must now be greater + than 0. +- Fixed a bug in `spatial_availability()` in which an `NA` demand in + `land_use_data` made every result `NA`, even when the id with the `NA` was + not in `travel_matrix`. + +## Minor changes + +- Accessibility, inequality and poverty functions now raise an error if + `land_use_data` or `sociodemographic_data` contain duplicated ids. + Previously, only one of the duplicated rows was silently used. + +## Notes + +- Updated the package website and GitHub URLs after the repository moved from + the `ipeaGIT` to the `ipea` organization. # accessibility 1.5.0 diff --git a/R/assert_and_assign.R b/R/assert_and_assign.R index 502ece8..5b9b472 100644 --- a/R/assert_and_assign.R +++ b/R/assert_and_assign.R @@ -111,9 +111,12 @@ assert_land_use_data <- function(land_use_data, # matrix / accessibility data (only one of the rows would be used) if (anyDuplicated(land_use_data$id)) { + duplicated_ids <- unique(land_use_data$id[duplicated(land_use_data$id)]) + stop( - "'land_use_data' must not contain duplicated ids. Found duplicates: ", - paste0("'", unique(land_use_data$id[duplicated(land_use_data$id)]), "'", collapse = ", "), + "'land_use_data' must not contain duplicated ids. ", + "Found duplicates: ", + paste0("'", duplicated_ids, "'", collapse = ", "), call. = FALSE ) } @@ -181,9 +184,13 @@ assert_sociodemographic_data <- function(sociodemographic_data, # matrix / accessibility data (only one of the rows would be used) if (anyDuplicated(sociodemographic_data$id)) { + ids <- sociodemographic_data$id + duplicated_ids <- unique(ids[duplicated(ids)]) + stop( - "'sociodemographic_data' must not contain duplicated ids. Found duplicates: ", - paste0("'", unique(sociodemographic_data$id[duplicated(sociodemographic_data$id)]), "'", collapse = ", "), + "'sociodemographic_data' must not contain duplicated ids. ", + "Found duplicates: ", + paste0("'", duplicated_ids, "'", collapse = ", "), call. = FALSE ) } diff --git a/tests/testthat/test-constrained-wrapper.R b/tests/testthat/test-constrained-wrapper.R index d78d961..62ba1e2 100644 --- a/tests/testthat/test-constrained-wrapper.R +++ b/tests/testthat/test-constrained-wrapper.R @@ -1,94 +1,104 @@ -# testthat::local_edition(3) - tm <- data.table::data.table( - expand.grid(from_id = c("1","2"), to_id = c("1","2")) + expand.grid( + from_id = c("1", "2"), + to_id = c("1", "2"), + stringsAsFactors = FALSE + ) ) tm[, travel_time := c(10, 20, 15, 10)] + +# total demand equals total supply, so the doubly-constrained case also works + lu <- data.table::data.table( - id = c("1","2"), + id = c("1", "2"), population = c(5, 7), - jobs = c(100, 120) + jobs = c(4, 8) ) decay <- decay_exponential(decay_value = 0.1) - -tester <- function( - constraint = "total", - travel_matrix = tm, - land_use_data = lu, - travel_cost = "travel_time", - decay_function = decay, - demand = "population", - supply = "jobs", - active = FALSE, - error_threshold = 0.001, - improvement_threshold = 1e-6, - max_iterations = 1000, - group_by = character(0), - fill_missing_ids = TRUE, - detailed_results = FALSE -) { +tester <- function(constraint = "total", + travel_matrix = tm, + land_use_data = lu, + travel_cost = "travel_time", + demand = "population", + supply = "jobs", + decay_function = decay, + active = TRUE, + error_threshold = 0.001, + improvement_threshold = 1e-6, + max_iterations = 1000, + group_by = character(0), + fill_missing_ids = TRUE, + detailed_results = FALSE) { constrained_accessibility( - constraint, - travel_matrix, - land_use_data, - travel_cost, - decay_function, - demand, - supply, - return_demand_side, - error_threshold, - improvement_threshold, - max_iterations, - group_by, - fill_missing_ids, - detailed_results + travel_matrix = travel_matrix, + land_use_data = land_use_data, + travel_cost = travel_cost, + demand = demand, + supply = supply, + constraint = constraint, + decay_function = decay_function, + active = active, + error_threshold = error_threshold, + improvement_threshold = improvement_threshold, + max_iterations = max_iterations, + group_by = group_by, + fill_missing_ids = fill_missing_ids, + detailed_results = detailed_results ) } -test_that("constrained_accessibility: wrapper argument rules are enforced", { +test_that("runs with valid input for every constraint", { + for (constraint in c("total", "singly")) { + expect_error(tester(constraint, active = TRUE), NA) + expect_error(tester(constraint, active = FALSE), NA) + } - # invalid constraint -> any error is fine + expect_error(tester("doubly", detailed_results = TRUE), NA) +}) +test_that("wrapper argument rules are enforced", { + # total and singly require active to be TRUE or FALSE - # total/singly require TRUE or FALSE (not NULL) expect_error( - tester(constraint = "total", return_demand_side = NULL) + tester("total", active = NULL), + regexp = "active must be TRUE or FALSE" ) - expect_error( - tester(constraint = "singly", return_demand_side = NULL) + tester("singly", active = NULL), + regexp = "active must be TRUE or FALSE" ) - # doubly requires NULL (error if TRUE) - expect_error( - tester(constraint = "doubly", active = FALSE) + # active is ignored when constraint = "doubly" + + doubly_result <- tester("doubly", active = NULL, detailed_results = TRUE) + expect_identical( + tester("doubly", active = TRUE, detailed_results = TRUE), + doubly_result + ) + expect_identical( + tester("doubly", active = FALSE, detailed_results = TRUE), + doubly_result ) }) - - - - test_that("raises errors due to incorrect input", { - expect_error(tester(constraint = "banana")) expect_error(tester(decay_function = "a")) expect_error(tester(decay_function = mean)) expect_error(tester(decay_function = get)) - expect_error(tester(opportunity = 1)) - expect_error(tester(opportunity = c("schools", "jobs"))) - expect_error(tester(travel_cost = 1)) expect_error(tester(travel_cost = c("travel_time", "monetary_cost"))) - expect_error(tester(demand = 1)) - expect_error(tester(demand = c("population", "population"))) + expect_error(tester(active = FALSE, demand = 1)) + expect_error(tester(active = FALSE, demand = c("population", "population"))) + expect_error(tester(active = FALSE, demand = "banana")) expect_error(tester(supply = 1)) - # expect_error(tester(supply = c("population", "population"))) 666666666 + expect_error(tester(supply = c("jobs", "jobs"))) + expect_error(tester(supply = "banana")) expect_error(tester(group_by = 1)) expect_error(tester(group_by = NA)) @@ -99,36 +109,31 @@ test_that("raises errors due to incorrect input", { expect_error(tester(fill_missing_ids = c(TRUE, TRUE))) expect_error(tester(fill_missing_ids = NA)) - expect_error(tester(as.list(tm))) - expect_error(tester(tm[, .(oi = from_id, to_id, travel_time)])) - expect_error(tester(tm[, .(from_id, oi = to_id, travel_time)])) - expect_error( - tester( - tm[, .(from_id, to_id, oi = travel_time)], - travel_cost = "travel_time" - ) - ) + expect_error(tester(detailed_results = 1)) + expect_error(tester(detailed_results = NA)) + + expect_error(tester("doubly", max_iterations = 0)) + expect_error(tester("doubly", error_threshold = -1)) + expect_error(tester("doubly", improvement_threshold = -1)) + + expect_error(tester(travel_matrix = as.list(tm))) + expect_error(tester(travel_matrix = tm[, .(oi = from_id, to_id, travel_time)])) + expect_error(tester(travel_matrix = tm[, .(from_id, oi = to_id, travel_time)])) + expect_error(tester(travel_matrix = tm[, .(from_id, to_id, oi = travel_time)])) expect_error( tester( - tm[, .(from_id, to_id, travel_time, oi = mode)], + travel_matrix = tm[, .(from_id, to_id, travel_time, oi = 1)], group_by = "mode" ) ) - expect_error(tester(as.list(lu))) - expect_error( - tester(lu = lu[, .(oi = id, jobs, population)]) - ) - expect_error( - tester( - lu = lu[, .(id, oi = jobs, population)], - opportunity = "jobs" - ) - ) + expect_error(tester(land_use_data = as.list(lu))) + expect_error(tester(land_use_data = lu[, .(oi = id, jobs, population)])) + expect_error(tester(land_use_data = lu[, .(id, oi = jobs, population)])) expect_error( tester( - lu = lu[, .(id, jobs, oi = population)], - demand = "population" + land_use_data = lu[, .(id, jobs, oi = population)], + active = FALSE ) ) }) From edffe51ae566bfee5cfd99604252040a18e8dbaf Mon Sep 17 00:00:00 2001 From: rafapereirabr Date: Tue, 20 Oct 2026 15:33:52 -0300 Subject: [PATCH 08/31] bug fix in constrained_accessibility --- NEWS.md | 9 ++ R/doubly_constrained.R | 20 ++++- R/fill_missing_ids.R | 12 ++- R/singly_constrained.R | 18 +++- R/total_constrained.R | 18 +++- tests/testthat/test-constrained-essentials.R | 93 ++++++++++++++++++++ 6 files changed, 162 insertions(+), 8 deletions(-) diff --git a/NEWS.md b/NEWS.md index 329c6df..12e2575 100644 --- a/NEWS.md +++ b/NEWS.md @@ -35,6 +35,15 @@ - Fixed a bug in `spatial_availability()` in which an `NA` demand in `land_use_data` made every result `NA`, even when the id with the `NA` was not in `travel_matrix`. +- Fixed a bug in `constrained_accessibility(constraint = "doubly")` in which + a small constant added to the balancing factors distorted the results when + impedance weights were very small (e.g. with steep decay functions or long + travel costs). The function now also warns when the balancing procedure + stops before reaching `error_threshold`. +- Fixed a bug in `constrained_accessibility()` (`constraint = "total"` or + `"singly"`) in which `fill_missing_ids = TRUE` raised an error ("Object + 'access' not found") when `group_by` left an origin/destination and group + combination missing from the results. ## Minor changes diff --git a/R/doubly_constrained.R b/R/doubly_constrained.R index c55d28f..687f989 100644 --- a/R/doubly_constrained.R +++ b/R/doubly_constrained.R @@ -106,14 +106,19 @@ doubly_constrained <- function(travel_matrix, iteration_count <- iteration_count + 1 # Update Ai + # a null denominator means the zone reaches (or is reached by) no one, so + # its factor is 0 and it gets no flow + for (i in seq_along(O)) { - Ai[i] <- 1 / (sum(Bj * D * impedance_matrix[i, ]) + 1e-9) + denominator <- sum(Bj * D * impedance_matrix[i, ]) + Ai[i] <- if (denominator > 0) 1 / denominator else 0 } # Update Bj Bj_new <- numeric(length(D)) for (j in seq_along(D)) { - Bj_new[j] <- 1 / (sum(Ai * O * impedance_matrix[, j]) + 1e-9) + denominator <- sum(Ai * O * impedance_matrix[, j]) + Bj_new[j] <- if (denominator > 0) 1 / denominator else 0 } # Compute flows @@ -133,6 +138,17 @@ doubly_constrained <- function(travel_matrix, Bj <- Bj_new } + if (error >= error_threshold) { + warning( + "Doubly-constrained balancing did not converge (", stop_reason, " after ", + iteration_count, " iterations). Final error is ", signif(error, 3), + ", above 'error_threshold' (", error_threshold, "), so flows don't ", + "fully match the demand and supply totals. Consider increasing ", + "'max_iterations' or decreasing 'improvement_threshold'.", + call. = FALSE + ) + } + # Compute kappa_doubly # kappa_ij^D = A_i * B_j * O_i * f(c_ij) kappa_matrix <- outer(Ai * O, Bj) * impedance_matrix diff --git a/R/fill_missing_ids.R b/R/fill_missing_ids.R index e5ec3c9..8356bdc 100644 --- a/R/fill_missing_ids.R +++ b/R/fill_missing_ids.R @@ -1,4 +1,7 @@ -fill_missing_ids <- function(access_df, travel_matrix, groups) { +fill_missing_ids <- function(access_df, + travel_matrix, + groups, + access_col = "access") { unique_values <- lapply(groups, function(x) unique(travel_matrix[[x]])) names(unique_values) <- groups @@ -8,7 +11,12 @@ fill_missing_ids <- function(access_df, travel_matrix, groups) { possible_combinations <- do.call(data.table::CJ, unique_values) if (nrow(access_df) < nrow(possible_combinations)) { - access_df <- do_fill_missing_ids(access_df, possible_combinations, groups) + access_df <- do_fill_missing_ids( + access_df, + possible_combinations, + groups, + access_col = access_col + ) } return(access_df) diff --git a/R/singly_constrained.R b/R/singly_constrained.R index 61ba322..9b9d5d4 100644 --- a/R/singly_constrained.R +++ b/R/singly_constrained.R @@ -70,7 +70,14 @@ singly_constrained <- function(travel_matrix, )] } else { access <- data[, .(supply = sum(singly_access)), by = c("from_id", group_by)] - if (fill_missing_ids) access <- fill_missing_ids(access, travel_matrix, c("from_id", group_by)) + if (fill_missing_ids) { + access <- fill_missing_ids( + access, + travel_matrix, + c("from_id", group_by), + access_col = "supply" + ) + } } } else { @@ -90,7 +97,14 @@ singly_constrained <- function(travel_matrix, )] } else { access <- data[, .(demand = sum(singly_access)), by = c("to_id", group_by)] - if (fill_missing_ids) access <- fill_missing_ids(access, travel_matrix, c("to_id", group_by)) + if (fill_missing_ids) { + access <- fill_missing_ids( + access, + travel_matrix, + c("to_id", group_by), + access_col = "demand" + ) + } } } diff --git a/R/total_constrained.R b/R/total_constrained.R index 69869b3..fb0757b 100644 --- a/R/total_constrained.R +++ b/R/total_constrained.R @@ -125,11 +125,25 @@ total_constrained <- function(travel_matrix, if (active) { # supply aggregated by origin (from_id) access <- data[, .(supply = sum(constrained_opportunity)), by = c("from_id", group_by)] - if (fill_missing_ids) access <- fill_missing_ids(access, travel_matrix, c("from_id", group_by)) + if (fill_missing_ids) { + access <- fill_missing_ids( + access, + travel_matrix, + c("from_id", group_by), + access_col = "supply" + ) + } } else { # demand aggregated by destination (to_id) access <- data[, .(demand = sum(constrained_opportunity)), by = c("to_id", group_by)] - if (fill_missing_ids) access <- fill_missing_ids(access, travel_matrix, c("to_id", group_by)) + if (fill_missing_ids) { + access <- fill_missing_ids( + access, + travel_matrix, + c("to_id", group_by), + access_col = "demand" + ) + } } } diff --git a/tests/testthat/test-constrained-essentials.R b/tests/testthat/test-constrained-essentials.R index 59c6fe5..334e710 100644 --- a/tests/testthat/test-constrained-essentials.R +++ b/tests/testthat/test-constrained-essentials.R @@ -304,3 +304,96 @@ test_that("doubly-constrained: errors when land_use_data misses an id", { regexp = "must list a non-NA demand" ) }) + +test_that("doubly-constrained: converges with very small impedance weights", { + tm <- make_tm_3x3() + tm[, travel_time := travel_time + 60] + lu <- make_lu_match20() + + # exp(-0.5 * c) is ~1e-13 to 1e-19 here + + result <- expect_silent( + constrained_accessibility( + constraint = "doubly", + travel_matrix = tm, + land_use_data = lu, + travel_cost = "travel_time", + decay_function = decay_exponential(decay_value = 0.5), + demand = "population", + supply = "jobs", + detailed_results = TRUE + ) + ) + + by_origin <- result[, .(flow = sum(flow)), by = from_id] + by_destination <- result[, .(flow = sum(flow)), by = to_id] + expect_equal( + by_origin$flow, + lu$population[match(by_origin$from_id, lu$id)], + tolerance = 1e-3 + ) + expect_equal( + by_destination$flow, + lu$jobs[match(by_destination$to_id, lu$id)], + tolerance = 1e-3 + ) +}) + +test_that("doubly-constrained: warns when balancing doesn't converge", { + expect_warning( + constrained_accessibility( + constraint = "doubly", + travel_matrix = make_tm_3x3(), + land_use_data = make_lu_match20(), + travel_cost = "travel_time", + decay_function = exp_decay_01, + demand = "population", + supply = "jobs", + max_iterations = 1, + detailed_results = TRUE + ), + regexp = "did not converge" + ) +}) + +test_that("fill_missing_ids fills missing combinations when using group_by", { + tm <- make_tm_3x3() + lu <- make_lu_match20() + + # bus doesn't serve origin 3, so the (3, bus) combination is missing + + tm_modes <- data.table::rbindlist( + list(car = tm, bus = tm[from_id != "3"]), + idcol = "mode" + ) + + run <- function(constraint, travel_matrix, group_by = "mode") { + constrained_accessibility( + constraint = constraint, + travel_matrix = travel_matrix, + land_use_data = lu, + travel_cost = "travel_time", + decay_function = exp_decay_01, + demand = "population", + supply = "jobs", + active = TRUE, + group_by = group_by + ) + } + + for (constraint in c("total", "singly")) { + result <- run(constraint, tm_modes) + expect_identical(nrow(result), 6L) + expect_identical(result[from_id == "3" & mode == "bus"]$supply, 0) + } + + # singly-constrained car results don't depend on the bus rows + + result <- run("singly", tm_modes) + ungrouped <- run("singly", tm, group_by = character(0)) + car <- result[mode == "car"] + expect_equal( + car$supply, + ungrouped$supply[match(car$from_id, ungrouped$from_id)] + ) +}) From 97a883385c1ff158f8395b2601fd22970b1a178b Mon Sep 17 00:00:00 2001 From: rafapereirabr Date: Tue, 20 Oct 2026 15:40:30 -0300 Subject: [PATCH 09/31] dealing with observations with the same income in concentration_index --- NEWS.md | 5 +++ R/accessibility-package.R | 3 +- R/concentration_index.R | 12 +++++- tests/testthat/test-concentration_index.R | 46 +++++++++++++++++++++++ 4 files changed, 64 insertions(+), 2 deletions(-) diff --git a/NEWS.md b/NEWS.md index 12e2575..e54de48 100644 --- a/NEWS.md +++ b/NEWS.md @@ -44,6 +44,11 @@ `"singly"`) in which `fill_missing_ids = TRUE` raised an error ("Object 'access' not found") when `group_by` left an origin/destination and group combination missing from the results. +- Fixed a bug in `concentration_index()` in which observations with the same + income received different fractional ranks, so the result depended on the + order of the rows (especially with ordinal variables such as income + deciles). Tied observations now share the same rank, the midpoint of their + block in the cumulative population distribution. ## Minor changes diff --git a/R/accessibility-package.R b/R/accessibility-package.R index 2b0d141..a8a1f0a 100644 --- a/R/accessibility-package.R +++ b/R/accessibility-package.R @@ -91,7 +91,8 @@ utils::globalVariables( "share_of_component", "group_tot_pop", "total", - "share_of_total" + "share_of_total", + ".block_share" ) ) diff --git a/R/concentration_index.R b/R/concentration_index.R index bc02c05..e2ff022 100644 --- a/R/concentration_index.R +++ b/R/concentration_index.R @@ -119,10 +119,20 @@ concentration_index <- function(accessibility_data, # fractional rank calculation based on equation 4 of this paper: # Eddy van Doorslaer and Xander Koolman, https://doi.org/10.1002/hec.918 + # observations with the same income share the same rank: the midpoint of + # their block in the cumulative population distribution. otherwise the result + # would depend on the order of the rows + data[, pop_share := get(.pop_colname) / sum(get(.pop_colname)), by = .groups] + + .income_colname <- income + data[, .block_share := sum(pop_share), by = c(.groups, income)] data[ , - fractional_rank := c(0, cumsum(pop_share)[-.N]) + pop_share / 2, + fractional_rank := { + first_in_block <- !duplicated(get(.income_colname)) + cumsum(.block_share * first_in_block) - .block_share / 2 + }, by = .groups ] diff --git a/tests/testthat/test-concentration_index.R b/tests/testthat/test-concentration_index.R index e60f586..f87d0b6 100644 --- a/tests/testthat/test-concentration_index.R +++ b/tests/testthat/test-concentration_index.R @@ -258,3 +258,49 @@ test_that("doesn't overflow with large integer opportunities and population", { ) expect_equal(result$concentration_index, expected_ci) }) + +test_that("observations with tied income share the same rank", { + toy_access <- data.table::data.table( + id = c("a", "b", "c", "d"), + jobs = c(10, 30, 20, 40) + ) + toy_sociodem <- data.table::data.table( + id = c("a", "b", "c", "d"), + population = c(1, 1, 1, 1), + income_per_capita = c(1, 1, 2, 2) + ) + + # a/b and c/d are tied, so their fractional ranks are 0.25 and 0.75 + + access <- c(10, 30, 20, 40) + rank <- c(0.25, 0.25, 0.75, 0.75) + avg_access <- mean(access) + expected_ci <- 2 / avg_access * mean((access - avg_access) * (rank - 0.5)) + expected_corrected <- expected_ci * 4 * avg_access / (40 - 10) + + for (rows in list(1:4, 4:1, c(2, 1, 4, 3))) { + result <- tester( + toy_access[rows], + toy_sociodem[rows], + group_by = character(0) + ) + expect_equal(result$concentration_index, expected_ci) + + result <- tester( + toy_access[rows], + toy_sociodem[rows], + type = "corrected", + group_by = character(0) + ) + expect_equal(result$concentration_index, expected_corrected) + } + + # without ties, ranks are the usual cumulative midpoints + + toy_sociodem[, income_per_capita := 1:4] + rank <- c(0.125, 0.375, 0.625, 0.875) + expected_ci <- 2 / avg_access * mean((access - avg_access) * (rank - 0.5)) + + result <- tester(toy_access, toy_sociodem, group_by = character(0)) + expect_equal(result$concentration_index, expected_ci) +}) From 9d657678254a2b20ac15f1d798aea7d485812a96 Mon Sep 17 00:00:00 2001 From: rafapereirabr Date: Tue, 20 Oct 2026 16:13:01 -0300 Subject: [PATCH 10/31] fix bug corner case of cumulative_interval --- NEWS.md | 15 +++ R/accessibility-package.R | 3 +- R/cumulative_interval.R | 42 ++++++--- man/cumulative_interval.Rd | 12 ++- tests/testthat/test-cumulative_interval.R | 106 +++++++++++++++++++++- 5 files changed, 160 insertions(+), 18 deletions(-) diff --git a/NEWS.md b/NEWS.md index e54de48..4bcd50b 100644 --- a/NEWS.md +++ b/NEWS.md @@ -1,5 +1,12 @@ # accessibility (development version) +## New features + +- `cumulative_interval()` has a new argument `fill_missing_ids`. When `TRUE` + (the default), origins (or destinations) that cannot reach any opportunity + within the interval are included in the output with an accessibility of 0, + as in `cumulative_cutoff()`. Previously they were dropped from the output. + ## Bug fixes - Fixed a bug in `balancing_cost()` in which numeric ids (e.g. `101`, `102`) @@ -49,6 +56,14 @@ order of the rows (especially with ordinal variables such as income deciles). Tied observations now share the same rank, the midpoint of their block in the cumulative population distribution. +- Fixed a bug in `cumulative_interval()` in which, when several intervals + were passed to `interval`, each interval was summarised over the cutoffs of + every interval within its range, so its result depended on the other + intervals requested (e.g. `c(0, 20)` together with `c(5, 15)`). +- Fixed a bug in `cumulative_interval()` in which the upper limit of the + interval was not considered when the interval width was not a multiple of + `interval_increment` (e.g. `interval = c(0, 25)` with + `interval_increment = 10`). ## Minor changes diff --git a/R/accessibility-package.R b/R/accessibility-package.R index a8a1f0a..5d33aa8 100644 --- a/R/accessibility-package.R +++ b/R/accessibility-package.R @@ -92,7 +92,8 @@ utils::globalVariables( "group_tot_pop", "total", "share_of_total", - ".block_share" + ".block_share", + "..groups" ) ) diff --git a/R/cumulative_interval.R b/R/cumulative_interval.R index 5bfc0c4..279ef1d 100644 --- a/R/cumulative_interval.R +++ b/R/cumulative_interval.R @@ -26,6 +26,10 @@ #' to [stats::median()]. #' @template group_by #' @template active +#' @param fill_missing_ids A `logical`. Origins (or destinations, when `active` +#' is `FALSE`) that cannot reach any opportunity within the interval would +#' not be included in the output. When `TRUE` (the default), the function +#' includes them with an accessibility of 0. #' #' @template return_accessibility #' @@ -66,10 +70,12 @@ cumulative_interval <- function(travel_matrix, interval_increment = 1, summary_function = stats::median, group_by = character(0), - active = TRUE) { + active = TRUE, + fill_missing_ids = TRUE) { checkmate::assert_string(opportunity) checkmate::assert_string(travel_cost) checkmate::assert_logical(active, len = 1, any.missing = FALSE) + checkmate::assert_logical(fill_missing_ids, len = 1, any.missing = FALSE) assert_interval_increment(interval_increment) assert_summary_function(summary_function) assert_group_by(group_by) @@ -96,11 +102,17 @@ cumulative_interval <- function(travel_matrix, land_use_data <- data.table::as.data.table(land_use_data) } - cutoffs <- lapply( + # each interval is summarised over its own cutoffs only, which always include + # the interval's upper limit (even if it's not a multiple of the increment). + # accessibility is calculated once for the union of all cutoffs + + interval_cutoffs <- lapply( interval, - function(pair) seq.int(pair[1], pair[2], by = interval_increment) + function(pair) { + unique(c(seq.int(pair[1], pair[2], by = interval_increment), pair[2])) + } ) - cutoffs <- unique(unlist(cutoffs)) + cutoffs <- unique(unlist(interval_cutoffs)) cutoffs <- cutoffs[order(cutoffs)] names(cutoffs) <- as.character(cutoffs) @@ -141,12 +153,12 @@ cumulative_interval <- function(travel_matrix, # measure would be miscalculated (e.g. think of median of 1 3 5 and median of # 0 1 3 5) - nrow_more_restrictive <- nrow(access_list[[1]]) - nrow_less_restrictive <- nrow(access_list[[length(access_list)]]) + # ids that can't reach any opportunity at any cutoff are filled as well, and + # only dropped at the end if fill_missing_ids is FALSE - if (nrow_more_restrictive < nrow_less_restrictive) { - access_list <- fill_access_list(access_list, travel_matrix, groups) - } + reached_ids <- unique(data.table::rbindlist(access_list)[, ..groups]) + + access_list <- fill_access_list(access_list, travel_matrix, groups) access_list <- data.table::rbindlist(access_list, idcol = "cutoffs") access_list[, cutoffs := as.numeric(cutoffs)] @@ -157,16 +169,22 @@ cumulative_interval <- function(travel_matrix, # reach 2, but you can't reach 3) access_list <- lapply( - interval, - function(pair) { + interval_cutoffs, + function(own_cutoffs) { access_list[ - cutoffs >= pair[1] & cutoffs <= pair[2], + cutoffs %in% own_cutoffs, .(access = as.integer(summary_function(access))), by = eval(groups, envir = env) ] } ) access <- data.table::rbindlist(access_list, idcol = "interval") + + if (!fill_missing_ids) { + reached_rows <- access[reached_ids, on = groups, which = TRUE, nomatch = 0] + access <- access[sort(reached_rows)] + } + data.table::setcolorder(access, c(groups, "interval", "access")) data.table::setnames(access, c(group_id, "access"), c("id", opportunity)) diff --git a/man/cumulative_interval.Rd b/man/cumulative_interval.Rd index fa0a96c..d83df00 100644 --- a/man/cumulative_interval.Rd +++ b/man/cumulative_interval.Rd @@ -13,7 +13,8 @@ cumulative_interval( interval_increment = 1, summary_function = stats::median, group_by = character(0), - active = TRUE + active = TRUE, + fill_missing_ids = TRUE ) } \arguments{ @@ -63,6 +64,11 @@ accessibility (the quantity of opportunities that can be reached from a given origin). when \code{FALSE}, it calculates passive accessibility (by how many people each destination can be reached), which is equivalent to the notion of market potential.} + +\item{fill_missing_ids}{A \code{logical}. Origins (or destinations, when \code{active} +is \code{FALSE}) that cannot reach any opportunity within the interval would +not be included in the output. When \code{TRUE} (the default), the function +includes them with an accessibility of 0.} } \value{ A data frame containing the accessibility estimates for each @@ -108,7 +114,7 @@ head(df) \insertAllCited{} } \seealso{ -Other cumulative access: -\code{\link{cumulative_cutoff}()} +Other cumulative access: +\code{\link[=cumulative_cutoff]{cumulative_cutoff()}} } \concept{cumulative access} diff --git a/tests/testthat/test-cumulative_interval.R b/tests/testthat/test-cumulative_interval.R index 85dd667..2001735 100644 --- a/tests/testthat/test-cumulative_interval.R +++ b/tests/testthat/test-cumulative_interval.R @@ -9,7 +9,8 @@ tester <- function(travel_matrix = smaller_matrix, interval_increment = 1, summary_function = stats::median, group_by = "mode", - active = TRUE) { + active = TRUE, + fill_missing_ids = TRUE) { cumulative_interval( travel_matrix, land_use_data, @@ -19,7 +20,8 @@ tester <- function(travel_matrix = smaller_matrix, interval_increment, summary_function, group_by, - active + active, + fill_missing_ids ) } @@ -67,6 +69,10 @@ test_that("raises errors due to incorrect input", { expect_error(tester(active = c(TRUE, TRUE))) expect_error(tester(active = NA)) + expect_error(tester(fill_missing_ids = 1)) + expect_error(tester(fill_missing_ids = c(TRUE, TRUE))) + expect_error(tester(fill_missing_ids = NA)) + expect_error(tester(as.list(travel_matrix))) expect_error(tester(travel_matrix[, .(oi = from_id, to_id, travel_time)])) expect_error(tester(travel_matrix[, .(from_id, oi = to_id, travel_time)])) @@ -362,3 +368,99 @@ test_that("works even if travel_matrix and land_use has specific colnames", { land_use_data[, opportunity := NULL] }) + +test_that("each interval is summarised over its own cutoffs only", { + toy_matrix <- data.table::data.table( + from_id = rep(c("a", "b", "c"), each = 3), + to_id = rep(c("a", "b", "c"), 3), + travel_time = c(0, 10, 20, 10, 0, 30, 20, 30, 0) + ) + toy_land_use <- data.table::data.table( + id = c("a", "b", "c"), + jobs = c(10, 20, 30) + ) + + toy_tester <- function(interval) { + tester( + toy_matrix, + toy_land_use, + interval = interval, + interval_increment = 10, + summary_function = mean, + group_by = character(0) + ) + } + + # [0, 20] uses cutoffs 0, 10 and 20 (a: mean(10, 30, 60) = 33.3) and [5, 15] + # uses 5 and 15 (a: mean(10, 30) = 20), whether requested alone or together + + result <- toy_tester(list(c(0, 20), c(5, 15))) + expect_identical(result[interval == "[0,20]"]$jobs, c(33L, 26L, 33L)) + expect_identical(result[interval == "[5,15]"]$jobs, c(20L, 25L, 30L)) + + expect_identical( + result[interval == "[0,20]"]$jobs, + toy_tester(c(0, 20))$jobs + ) + expect_identical( + result[interval == "[5,15]"]$jobs, + toy_tester(c(5, 15))$jobs + ) +}) + +test_that("fill_missing_ids arg works correctly", { + toy_matrix <- data.table::data.table( + from_id = c(rep(c("a", "b", "c"), each = 3), "d"), + to_id = c(rep(c("a", "b", "c"), 3), "a"), + travel_time = c(0, 10, 20, 10, 0, 30, 20, 30, 0, 100) + ) + toy_land_use <- data.table::data.table( + id = c("a", "b", "c", "d"), + jobs = c(10, 20, 30, 40) + ) + + toy_tester <- function(...) { + tester( + toy_matrix, + toy_land_use, + interval_increment = 10, + summary_function = mean, + group_by = character(0), + ... + ) + } + + # d doesn't reach any opportunity within the interval + + result <- toy_tester(interval = c(0, 20)) + expect_identical(result$id, c("a", "b", "c", "d")) + expect_identical(result$jobs, c(33L, 26L, 33L, 0L)) + + result <- toy_tester(interval = c(0, 20), fill_missing_ids = FALSE) + expect_identical(result$id, c("a", "b", "c")) + expect_identical(result$jobs, c(33L, 26L, 33L)) +}) + +test_that("the upper limit of the interval is always considered", { + toy_matrix <- data.table::data.table( + from_id = rep(c("a", "b", "c"), each = 3), + to_id = rep(c("a", "b", "c"), 3), + travel_time = c(0, 10, 25, 10, 0, 30, 25, 30, 0) + ) + toy_land_use <- data.table::data.table( + id = c("a", "b", "c"), + jobs = c(10, 20, 30) + ) + + # cutoffs are 0, 10, 20 and 25. a reaches 10, 30, 30 and 60 jobs + + result <- tester( + toy_matrix, + toy_land_use, + interval = c(0, 25), + interval_increment = 10, + summary_function = mean, + group_by = character(0) + ) + expect_identical(result$jobs, c(32L, 27L, 32L)) +}) From 538985ada3a9649e1e347efffb90243086402e97 Mon Sep 17 00:00:00 2001 From: rafapereirabr Date: Tue, 20 Oct 2026 16:42:06 -0300 Subject: [PATCH 11/31] corner cases of cumulative_cutoff --- NEWS.md | 4 +++ R/cumulative_cutoff.R | 38 ++++++++++++------------- tests/testthat/test-cumulative_cutoff.R | 35 +++++++++++++++++++++++ 3 files changed, 57 insertions(+), 20 deletions(-) diff --git a/NEWS.md b/NEWS.md index 4bcd50b..328740c 100644 --- a/NEWS.md +++ b/NEWS.md @@ -64,6 +64,10 @@ interval was not considered when the interval width was not a multiple of `interval_increment` (e.g. `interval = c(0, 25)` with `interval_increment = 10`). +- Fixed a bug in `cumulative_cutoff()` in which cutoffs with floating point + noise (e.g. created with `seq(0, 0.5, by = 0.1)`) could have their results + replaced by 0 when filling missing ids. Travel cost columns with + non-syntactic names (e.g. `"bus fare"`) now also work. ## Minor changes diff --git a/R/cumulative_cutoff.R b/R/cumulative_cutoff.R index 303363d..cf9d8fa 100644 --- a/R/cumulative_cutoff.R +++ b/R/cumulative_cutoff.R @@ -155,16 +155,22 @@ cumulative_cutoff <- function(travel_matrix, as.list(cutoff_df), SIMPLIFY = FALSE, FUN = function(...) { - cutoff_args <- unlist(list(...)) - subset_expr <- paste( - names(cutoff_args), - cutoff_args, - sep = " <= ", - collapse = " & " + cutoff_args <- list(...) + + # the filter and the cutoff columns use the exact cutoff values, which + # must match the values used to fill missing ids later. NA costs count + # as not reachable + + within_cutoffs <- Reduce( + `&`, + Map( + function(cost, value) data[[cost]] <= value, + names(cutoff_args), + cutoff_args + ) ) - subset_expr <- parse(text = subset_expr) - cum_opps <- data[eval(subset_expr)] + cum_opps <- data[within_cutoffs] if (length(travel_cost) > 1) { .od_group <- c(setdiff(groups, group_id), "from_id", "to_id") @@ -177,19 +183,11 @@ cumulative_cutoff <- function(travel_matrix, by = eval(groups, envir = env) ] - cutoff_id_expr <- paste0( - "`:=`(", - paste( - names(cutoff_args), - cutoff_args, - sep = " = ", - collapse = ", " - ), - ")" - ) - cutoff_id_expr <- parse(text = cutoff_id_expr) + for (cost in names(cutoff_args)) { + data.table::set(cum_opps, j = cost, value = cutoff_args[[cost]]) + } - cum_opps[, eval(cutoff_id_expr)] + cum_opps } ) ) diff --git a/tests/testthat/test-cumulative_cutoff.R b/tests/testthat/test-cumulative_cutoff.R index e3891e7..1486107 100644 --- a/tests/testthat/test-cumulative_cutoff.R +++ b/tests/testthat/test-cumulative_cutoff.R @@ -379,3 +379,38 @@ test_that("works even if travel_matrix and land_use has specific colnames", { land_use_data[, opportunity := NULL] }) + +test_that("works with cutoffs with floating point noise and any colname", { + toy_matrix <- data.table::data.table( + from_id = c(rep(c("a", "b", "c"), each = 3), "d"), + to_id = c(rep(c("a", "b", "c"), 3), "a"), + fare = c(0, 0.2, 0.3, 0.2, 0, 0.4, 0.3, 0.4, 0, 1) + ) + toy_land_use <- data.table::data.table( + id = c("a", "b", "c", "d"), + jobs = c(10, 20, 30, 40) + ) + + # seq() cutoffs are not exactly 0.1, 0.2, ... (e.g. the 4th is + # 0.30000000000000004). d doesn't reach anyone, so missing ids are filled + + result <- tester( + toy_matrix, + toy_land_use, + travel_cost = "fare", + cutoff = seq(0, 0.5, by = 0.1), + group_by = character(0) + ) + expect_identical(result[id == "a"]$jobs, c(10, 10, 30, 60, 60, 60)) + expect_identical(result[id == "d"]$jobs, rep(0, 6)) + + data.table::setnames(toy_matrix, "fare", "bus fare") + result <- tester( + toy_matrix, + toy_land_use, + travel_cost = "bus fare", + cutoff = 0.3, + group_by = character(0) + ) + expect_identical(result$jobs, c(60, 30, 40, 0)) +}) From 7121263cdb6e9dec99a9b8b37105abbb633ccead Mon Sep 17 00:00:00 2001 From: rafapereirabr Date: Tue, 20 Oct 2026 16:45:04 -0300 Subject: [PATCH 12/31] update roxygen docs --- DESCRIPTION | 2 +- NAMESPACE | 14 ++++++++------ man/concentration_index.Rd | 8 ++++---- man/constrained_accessibility.Rd | 4 ++-- man/cumulative_cutoff.Rd | 4 ++-- man/decay_binary.Rd | 12 ++++++------ man/decay_exponential.Rd | 12 ++++++------ man/decay_linear.Rd | 12 ++++++------ man/decay_logistic.Rd | 12 ++++++------ man/decay_power.Rd | 12 ++++++------ man/decay_stepped.Rd | 12 ++++++------ man/gini_index.Rd | 8 ++++---- man/palma_ratio.Rd | 8 ++++---- man/spatial_availability.Rd | 4 ++-- 14 files changed, 63 insertions(+), 61 deletions(-) diff --git a/DESCRIPTION b/DESCRIPTION index 4e4ddf6..c54cb99 100644 --- a/DESCRIPTION +++ b/DESCRIPTION @@ -51,4 +51,4 @@ RdMacros: Encoding: UTF-8 NeedsCompilation: no Roxygen: list(markdown = TRUE) -RoxygenNote: 7.3.3 +Config/roxygen2/version: 8.1.0 diff --git a/NAMESPACE b/NAMESPACE index b2f77f4..982bae3 100644 --- a/NAMESPACE +++ b/NAMESPACE @@ -20,10 +20,12 @@ export(palma_ratio) export(spatial_availability) export(theil_t) importFrom(Rdpack,reprompt) -importFrom(data.table,"%chin%") -importFrom(data.table,":=") -importFrom(data.table,.GRP) -importFrom(data.table,.I) -importFrom(data.table,.N) -importFrom(data.table,.SD) +importFrom(data.table, + "%chin%", + ":=", + .GRP, + .I, + .N, + .SD +) importFrom(utils,globalVariables) diff --git a/man/concentration_index.Rd b/man/concentration_index.Rd index fc419ee..f9185ea 100644 --- a/man/concentration_index.Rd +++ b/man/concentration_index.Rd @@ -97,9 +97,9 @@ ci \insertAllCited{} } \seealso{ -Other inequality: -\code{\link{gini_index}()}, -\code{\link{palma_ratio}()}, -\code{\link{theil_t}()} +Other inequality: +\code{\link[=gini_index]{gini_index()}}, +\code{\link[=palma_ratio]{palma_ratio()}}, +\code{\link[=theil_t]{theil_t()}} } \concept{inequality} diff --git a/man/constrained_accessibility.Rd b/man/constrained_accessibility.Rd index da7b2e9..63d849d 100644 --- a/man/constrained_accessibility.Rd +++ b/man/constrained_accessibility.Rd @@ -343,7 +343,7 @@ constrained_accessibility( \insertAllCited{} } \seealso{ -Other Constrained accessibility: -\code{\link{spatial_availability}()} +Other Constrained accessibility: +\code{\link[=spatial_availability]{spatial_availability()}} } \concept{Constrained accessibility} diff --git a/man/cumulative_cutoff.Rd b/man/cumulative_cutoff.Rd index 4d6cf90..3b95bd7 100644 --- a/man/cumulative_cutoff.Rd +++ b/man/cumulative_cutoff.Rd @@ -131,7 +131,7 @@ head(df) \dontshow{\}) # examplesIf} } \seealso{ -Other cumulative access: -\code{\link{cumulative_interval}()} +Other cumulative access: +\code{\link[=cumulative_interval]{cumulative_interval()}} } \concept{cumulative access} diff --git a/man/decay_binary.Rd b/man/decay_binary.Rd index 95e91ff..66d1827 100644 --- a/man/decay_binary.Rd +++ b/man/decay_binary.Rd @@ -35,11 +35,11 @@ weighting_function(c(20, 35)) \dontshow{\}) # examplesIf} } \seealso{ -Other decay functions: -\code{\link{decay_exponential}()}, -\code{\link{decay_linear}()}, -\code{\link{decay_logistic}()}, -\code{\link{decay_power}()}, -\code{\link{decay_stepped}()} +Other decay functions: +\code{\link[=decay_exponential]{decay_exponential()}}, +\code{\link[=decay_linear]{decay_linear()}}, +\code{\link[=decay_logistic]{decay_logistic()}}, +\code{\link[=decay_power]{decay_power()}}, +\code{\link[=decay_stepped]{decay_stepped()}} } \concept{decay functions} diff --git a/man/decay_exponential.Rd b/man/decay_exponential.Rd index efd2f0a..7ac6cf1 100644 --- a/man/decay_exponential.Rd +++ b/man/decay_exponential.Rd @@ -35,11 +35,11 @@ weighting_function(c(20, 30)) \dontshow{\}) # examplesIf} } \seealso{ -Other decay functions: -\code{\link{decay_binary}()}, -\code{\link{decay_linear}()}, -\code{\link{decay_logistic}()}, -\code{\link{decay_power}()}, -\code{\link{decay_stepped}()} +Other decay functions: +\code{\link[=decay_binary]{decay_binary()}}, +\code{\link[=decay_linear]{decay_linear()}}, +\code{\link[=decay_logistic]{decay_logistic()}}, +\code{\link[=decay_power]{decay_power()}}, +\code{\link[=decay_stepped]{decay_stepped()}} } \concept{decay functions} diff --git a/man/decay_linear.Rd b/man/decay_linear.Rd index f4a083d..b016cad 100644 --- a/man/decay_linear.Rd +++ b/man/decay_linear.Rd @@ -35,11 +35,11 @@ weighting_function(c(20, 35)) \dontshow{\}) # examplesIf} } \seealso{ -Other decay functions: -\code{\link{decay_binary}()}, -\code{\link{decay_exponential}()}, -\code{\link{decay_logistic}()}, -\code{\link{decay_power}()}, -\code{\link{decay_stepped}()} +Other decay functions: +\code{\link[=decay_binary]{decay_binary()}}, +\code{\link[=decay_exponential]{decay_exponential()}}, +\code{\link[=decay_logistic]{decay_logistic()}}, +\code{\link[=decay_power]{decay_power()}}, +\code{\link[=decay_stepped]{decay_stepped()}} } \concept{decay functions} diff --git a/man/decay_logistic.Rd b/man/decay_logistic.Rd index 85e21be..b771691 100644 --- a/man/decay_logistic.Rd +++ b/man/decay_logistic.Rd @@ -55,11 +55,11 @@ weighting_function(c(0, 30, 45, 60)) \insertAllCited{} } \seealso{ -Other decay functions: -\code{\link{decay_binary}()}, -\code{\link{decay_exponential}()}, -\code{\link{decay_linear}()}, -\code{\link{decay_power}()}, -\code{\link{decay_stepped}()} +Other decay functions: +\code{\link[=decay_binary]{decay_binary()}}, +\code{\link[=decay_exponential]{decay_exponential()}}, +\code{\link[=decay_linear]{decay_linear()}}, +\code{\link[=decay_power]{decay_power()}}, +\code{\link[=decay_stepped]{decay_stepped()}} } \concept{decay functions} diff --git a/man/decay_power.Rd b/man/decay_power.Rd index ba47beb..fadf779 100644 --- a/man/decay_power.Rd +++ b/man/decay_power.Rd @@ -34,11 +34,11 @@ weighting_function(c(20, 35)) \dontshow{\}) # examplesIf} } \seealso{ -Other decay functions: -\code{\link{decay_binary}()}, -\code{\link{decay_exponential}()}, -\code{\link{decay_linear}()}, -\code{\link{decay_logistic}()}, -\code{\link{decay_stepped}()} +Other decay functions: +\code{\link[=decay_binary]{decay_binary()}}, +\code{\link[=decay_exponential]{decay_exponential()}}, +\code{\link[=decay_linear]{decay_linear()}}, +\code{\link[=decay_logistic]{decay_logistic()}}, +\code{\link[=decay_stepped]{decay_stepped()}} } \concept{decay functions} diff --git a/man/decay_stepped.Rd b/man/decay_stepped.Rd index d8d71b4..49c7cff 100644 --- a/man/decay_stepped.Rd +++ b/man/decay_stepped.Rd @@ -64,11 +64,11 @@ weighting_function(c(0, 10, 20, 30, 40)) \dontshow{\}) # examplesIf} } \seealso{ -Other decay functions: -\code{\link{decay_binary}()}, -\code{\link{decay_exponential}()}, -\code{\link{decay_linear}()}, -\code{\link{decay_logistic}()}, -\code{\link{decay_power}()} +Other decay functions: +\code{\link[=decay_binary]{decay_binary()}}, +\code{\link[=decay_exponential]{decay_exponential()}}, +\code{\link[=decay_linear]{decay_linear()}}, +\code{\link[=decay_logistic]{decay_logistic()}}, +\code{\link[=decay_power]{decay_power()}} } \concept{decay functions} diff --git a/man/gini_index.Rd b/man/gini_index.Rd index 0e9618a..e3cba3d 100644 --- a/man/gini_index.Rd +++ b/man/gini_index.Rd @@ -67,9 +67,9 @@ gini \dontshow{\}) # examplesIf} } \seealso{ -Other inequality: -\code{\link{concentration_index}()}, -\code{\link{palma_ratio}()}, -\code{\link{theil_t}()} +Other inequality: +\code{\link[=concentration_index]{concentration_index()}}, +\code{\link[=palma_ratio]{palma_ratio()}}, +\code{\link[=theil_t]{theil_t()}} } \concept{inequality} diff --git a/man/palma_ratio.Rd b/man/palma_ratio.Rd index 9108338..c4ccd81 100644 --- a/man/palma_ratio.Rd +++ b/man/palma_ratio.Rd @@ -80,9 +80,9 @@ palma \dontshow{\}) # examplesIf} } \seealso{ -Other inequality: -\code{\link{concentration_index}()}, -\code{\link{gini_index}()}, -\code{\link{theil_t}()} +Other inequality: +\code{\link[=concentration_index]{concentration_index()}}, +\code{\link[=gini_index]{gini_index()}}, +\code{\link[=theil_t]{theil_t()}} } \concept{inequality} diff --git a/man/spatial_availability.Rd b/man/spatial_availability.Rd index 006412c..9809465 100644 --- a/man/spatial_availability.Rd +++ b/man/spatial_availability.Rd @@ -134,7 +134,7 @@ detailed_df \insertAllCited{} } \seealso{ -Other Constrained accessibility: -\code{\link{constrained_accessibility}()} +Other Constrained accessibility: +\code{\link[=constrained_accessibility]{constrained_accessibility()}} } \concept{Constrained accessibility} From cbc9f44c76814a44f69376c0b0c219f0adb6a57d Mon Sep 17 00:00:00 2001 From: rafapereirabr Date: Tue, 20 Oct 2026 16:58:30 -0300 Subject: [PATCH 13/31] corner cases in equity measures --- NEWS.md | 14 ++++++++ R/accessibility-package.R | 3 +- R/concentration_index.R | 12 ++++++- R/fgt_poverty.R | 11 +++++-- R/palma_ratio.R | 14 +++++--- tests/testthat/test-concentration_index.R | 24 ++++++++++++++ tests/testthat/test-fgt_poverty.R | 9 +++++ tests/testthat/test-palma_ratio.R | 40 +++++++++++++++++++++++ 8 files changed, 119 insertions(+), 8 deletions(-) diff --git a/NEWS.md b/NEWS.md index 328740c..10d84fc 100644 --- a/NEWS.md +++ b/NEWS.md @@ -69,6 +69,20 @@ replaced by 0 when filling missing ids. Travel cost columns with non-syntactic names (e.g. `"bus fare"`) now also work. +- Fixed a bug in `palma_ratio()` in which the income cutoffs depended on the + scale of the population weights, which resulted in `NA` when population was + given as shares (e.g. summing to 1). +- Fixed a bug in `concentration_index(type = "corrected")` in which the index + was `NaN` when everyone had the same accessibility. It's now 0, as with + `type = "standard"`. +- `fgt_poverty()` now raises an error if `poverty_line` is not a single finite + number (or is 0 when `poor_below_threshold = FALSE`) and if + `poor_below_threshold` is not a single `TRUE` or `FALSE`. Previously these + inputs silently returned recycled values, `NA` or `Inf`, or a cryptic error. +- Fixed a bug in `palma_ratio()` in which, with more than one `group_by` + column, the output included rows with `NA` for group combinations that were + not present in the data. The output is also no longer keyed. + ## Minor changes - Accessibility, inequality and poverty functions now raise an error if diff --git a/R/accessibility-package.R b/R/accessibility-package.R index 5d33aa8..421f9c9 100644 --- a/R/accessibility-package.R +++ b/R/accessibility-package.R @@ -93,7 +93,8 @@ utils::globalVariables( "total", "share_of_total", ".block_share", - "..groups" + "..groups", + "..group_by" ) ) diff --git a/R/concentration_index.R b/R/concentration_index.R index e2ff022..32aa34d 100644 --- a/R/concentration_index.R +++ b/R/concentration_index.R @@ -168,7 +168,17 @@ concentration_index <- function(accessibility_data, # from Erreygers (2009), comparing equations 27 and 24, we have the # following correction factor # https://doi.org/10.1016/j.jhealeco.2008.02.003 - data[, correction_factor := 4 * avg_access / (upper - lower)] + # when access is constant (upper == lower) there is no inequality, and the + # factor would be Inf (0 * Inf = NaN). it's set to 0 so the index is 0 + + data[ + , + correction_factor := data.table::fifelse( + upper > lower, + 4 * avg_access / (upper - lower), + 0 + ) + ] ci <- data[ , diff --git a/R/fgt_poverty.R b/R/fgt_poverty.R index 90e83fd..0cdc43f 100644 --- a/R/fgt_poverty.R +++ b/R/fgt_poverty.R @@ -94,8 +94,15 @@ fgt_poverty <- function(accessibility_data, checkmate::assert_string(opportunity) checkmate::assert_string(population) - checkmate::assert_numeric(poverty_line, lower = 0) - checkmate::assert_logical(poor_below_threshold) + checkmate::assert_number(poverty_line, lower = 0, finite = TRUE) + checkmate::assert_flag(poor_below_threshold) + if (!poor_below_threshold && poverty_line == 0) { + stop( + "Assertion on 'poverty_line' failed: Value must be greater than 0 when ", + "'poor_below_threshold' is FALSE.", + call. = FALSE + ) + } assert_access_group_by(group_by) assert_accessibility_data(accessibility_data, opportunity, group_by) assert_sociodemographic_data( diff --git a/R/palma_ratio.R b/R/palma_ratio.R index 22b10a2..27993c3 100644 --- a/R/palma_ratio.R +++ b/R/palma_ratio.R @@ -83,10 +83,14 @@ palma_ratio <- function(accessibility_data, warn_extra_access_cols(accessibility_data, opportunity, group_by) + # normwt = TRUE treats population as relative weights, so the quantiles don't + # depend on its scale (e.g. counts vs. shares) + income_quantiles <- Hmisc::wtd.quantile( sociodemographic_data[[income]], weights = sociodemographic_data[[population]], - probs = c(0.4, 0.9) + probs = c(0.4, 0.9), + normwt = TRUE ) .income_colname <- income @@ -153,16 +157,18 @@ palma_ratio <- function(accessibility_data, palma[, c("avg_access", "poorest_avg_access") := NULL] } + data.table::setkeyv(palma, NULL) + if (exists("original_class")) class(palma) <- original_class return(palma[]) } fill_groups <- function(wealthiest_access, data, group_by) { - unique_values <- lapply(group_by, function(x) unique(data[[x]])) - names(unique_values) <- group_by + # only the group combinations present in the data, not all crossings of the + # values of each group_by column - possible_combinations <- do.call(data.table::CJ, unique_values) + possible_combinations <- unique(data[, ..group_by]) if (nrow(possible_combinations) > nrow(wealthiest_access)) { wealthiest_access <- possible_combinations[ diff --git a/tests/testthat/test-concentration_index.R b/tests/testthat/test-concentration_index.R index f87d0b6..a33256c 100644 --- a/tests/testthat/test-concentration_index.R +++ b/tests/testthat/test-concentration_index.R @@ -304,3 +304,27 @@ test_that("observations with tied income share the same rank", { result <- tester(toy_access, toy_sociodem, group_by = character(0)) expect_equal(result$concentration_index, expected_ci) }) + +test_that("returns 0 when access is constant, for both types", { + toy_access <- data.table::data.table( + id = rep(c("a", "b", "c"), 2), + mode = rep(c("car", "walk"), each = 3), + jobs = c(5, 5, 5, 1, 2, 6) + ) + toy_sociodem <- data.table::data.table( + id = c("a", "b", "c"), + population = c(1, 2, 3), + income_per_capita = c(1, 2, 3) + ) + + standard <- tester(toy_access, toy_sociodem) + corrected <- tester(toy_access, toy_sociodem, type = "corrected") + + expect_identical(standard[mode == "car"]$concentration_index, 0) + expect_identical(corrected[mode == "car"]$concentration_index, 0) + + # the level with some inequality is unaffected by the constant one + + expect_false(anyNA(corrected$concentration_index)) + expect_true(corrected[mode == "walk"]$concentration_index > 0) +}) diff --git a/tests/testthat/test-fgt_poverty.R b/tests/testthat/test-fgt_poverty.R index 796c5f7..4483295 100644 --- a/tests/testthat/test-fgt_poverty.R +++ b/tests/testthat/test-fgt_poverty.R @@ -28,6 +28,15 @@ test_that("raises errors due to incorrect input", { expect_error(tester(poverty_line = "a")) expect_error(tester(poverty_line = -1)) + expect_error(tester(poverty_line = c(10000, 20000))) + expect_error(tester(poverty_line = NA_real_)) + expect_error(tester(poverty_line = Inf)) + expect_error( + tester(poverty_line = 0, poor_below_threshold = FALSE), + regexp = "greater than 0" + ) + expect_error(tester(poor_below_threshold = NA)) + expect_error(tester(poor_below_threshold = c(TRUE, FALSE))) expect_error(tester(group_by = 1)) expect_error(tester(group_by = NA)) diff --git a/tests/testthat/test-palma_ratio.R b/tests/testthat/test-palma_ratio.R index daafb8e..dcc56bb 100644 --- a/tests/testthat/test-palma_ratio.R +++ b/tests/testthat/test-palma_ratio.R @@ -297,3 +297,43 @@ test_that("handles missing data correctly", { suppressWarnings(result <- tester(no_access, group_by = character())) expect_identical(result, data.table::data.table(palma_ratio = numeric())) }) + +test_that("result doesn't depend on the scale of the population weights", { + access_data <- cumulative_cutoff( + travel_matrix, + land_use_data, + opportunity = "jobs", + travel_cost = "travel_time", + cutoff = 30, + group_by = "mode" + ) + population_shares <- data.table::copy(land_use_data) + population_shares[, population := population / sum(population)] + + expected <- tester(access_data) + result <- tester(access_data, population_shares) + expect_false(anyNA(result$palma_ratio)) + expect_equal(result, expected) +}) + +test_that("only returns the group_by combinations present in the data", { + transit_matrix <- travel_matrix[mode == "transit"] + transit_matrix[, mode := NULL] + + car_am <- cumulative_cutoff( + transit_matrix, + land_use_data, + opportunity = "jobs", + travel_cost = "travel_time", + cutoff = 30 + ) + car_am[, `:=`(mode = "car", period = "am")] + + bus_pm <- data.table::copy(car_am) + bus_pm[, `:=`(mode = "bus", period = "pm")] + + result <- tester(rbind(car_am, bus_pm), group_by = c("mode", "period")) + expect_identical(nrow(result), 2L) + expect_setequal(paste(result$mode, result$period), c("car am", "bus pm")) + expect_null(data.table::key(result)) +}) From aad4d6493cafa083012c5f6a16fdf4166cdcb7dd Mon Sep 17 00:00:00 2001 From: rafapereirabr Date: Tue, 20 Oct 2026 17:10:56 -0300 Subject: [PATCH 14/31] docs and tests --- NEWS.md | 19 +++++ R/assert_and_assign.R | 30 +++++++ R/constrained_accessibility.R | 2 + R/decay_linear.R | 11 ++- R/decay_logistic.R | 11 +++ R/decay_stepped.R | 23 +++++- R/total_constrained.R | 5 +- man/decay_linear.Rd | 2 +- tests/testthat/test-assert_land_use_data.R | 2 + tests/testthat/test-constrained-essentials.R | 22 ++++++ tests/testthat/test-constrained-wrapper.R | 11 +++ tests/testthat/test-cumulative_cutoff.R | 36 +++++++++ tests/testthat/test-decay_linear.R | 5 ++ tests/testthat/test-decay_logistic.R | 8 ++ tests/testthat/test-decay_stepped.R | 22 ++++++ tests/testthat/test-fgt_poverty.R | 50 ++++++++++++ tests/testthat/test-floating_catchment_area.R | 78 +++++++++++++++++++ tests/testthat/test-gravity.R | 16 ++++ tests/testthat/test-theil_t.R | 6 +- vignettes/accessibility.Rmd | 7 +- 20 files changed, 357 insertions(+), 9 deletions(-) diff --git a/NEWS.md b/NEWS.md index 10d84fc..0718576 100644 --- a/NEWS.md +++ b/NEWS.md @@ -83,16 +83,35 @@ column, the output included rows with `NA` for group combinations that were not present in the data. The output is also no longer keyed. +- Fixed a bug in `decay_stepped()` that emitted a warning for each `NA` + travel cost (the weight is still `NA`). +- Fixed a bug in `constrained_accessibility(constraint = "total")` that added + a data.table index attribute to the user's `land_use_data`. + ## Minor changes - Accessibility, inequality and poverty functions now raise an error if `land_use_data` or `sociodemographic_data` contain duplicated ids. Previously, only one of the duplicated rows was silently used. +- Accessibility functions now raise an error if a `travel_cost` column of + `travel_matrix` contains negative values, which previously produced weights + above 1 and inflated accessibility levels. +- Accessibility functions now raise a clear error when the ids of + `travel_matrix` and `land_use_data` have incompatible types (e.g. integer + and character), instead of an obscure join error. +- `decay_linear()` now requires `cutoff` to be greater than 0 (a cutoff of 0 + produced `NaN` weights). +- `decay_logistic()` and `decay_stepped()` now raise an error when given + duplicated parameter combinations, which produced duplicated results. +- `constrained_accessibility()` now checks that `demand` and `supply` are + single strings. ## Notes - Updated the package website and GitHub URLs after the repository moved from the `ipeaGIT` to the `ipea` organization. +- Vignettes and tests now only use the suggested packages `{sf}`, + `{ggplot2}` and `{tibble}` when they are installed. # accessibility 1.5.0 diff --git a/R/assert_and_assign.R b/R/assert_and_assign.R index 5b9b472..3eeb88a 100644 --- a/R/assert_and_assign.R +++ b/R/assert_and_assign.R @@ -68,6 +68,21 @@ assert_travel_matrix <- function(travel_matrix, travel_cost, group_by) { .var.name = "travel_matrix" ) + # negative travel costs would produce weights higher than 1 in most decay + # functions, inflating accessibility levels + + for (col in travel_cost) { + cost_values <- travel_matrix[[col]] + + if (is.numeric(cost_values) && any(cost_values < 0, na.rm = TRUE)) { + stop( + "Assertion on 'travel_matrix$", col, "' failed: ", + "Travel costs must not be negative.", + call. = FALSE + ) + } + } + return(invisible(TRUE)) } @@ -132,6 +147,21 @@ assert_land_use_data <- function(land_use_data, travel_matrix$from_id } + # joining numeric ids to character/factor ids fails with an obscure + # data.table error, so we check the id types before warning about missing ids + + is_text_like <- function(x) is.character(x) || is.factor(x) + + if (is_text_like(travel_matrix_ids) != is_text_like(land_use_data$id)) { + stop( + "The id columns of 'travel_matrix' ('from_id'/'to_id') and ", + "'land_use_data' ('id') must be of compatible types. Found '", + class(travel_matrix_ids)[1], "' in 'travel_matrix' and '", + class(land_use_data$id)[1], "' in 'land_use_data'.", + call. = FALSE + ) + } + if (!all(travel_matrix_ids %in% land_use_data$id)) { warning( "'land_use_data' is missing ids listed in 'travel_matrix', which may ", diff --git a/R/constrained_accessibility.R b/R/constrained_accessibility.R index 41e862b..2f50bbf 100644 --- a/R/constrained_accessibility.R +++ b/R/constrained_accessibility.R @@ -265,6 +265,8 @@ constrained_accessibility <- function(travel_matrix, checkmate::assert_choice(constraint, c("total", "singly", "doubly")) checkmate::assert_string(travel_cost) + checkmate::assert_string(demand, null.ok = TRUE) + checkmate::assert_string(supply, null.ok = TRUE) assert_decay_function(decay_function) assert_group_by(group_by) assert_travel_matrix(travel_matrix, travel_cost, group_by) diff --git a/R/decay_linear.R b/R/decay_linear.R index eadacf8..f5018e1 100644 --- a/R/decay_linear.R +++ b/R/decay_linear.R @@ -6,7 +6,7 @@ #' #' @param cutoff A `numeric` vector. Indicates the travel cost cutoffs until #' which the weighting factor decays linearly. From this point onward the -#' weight is equal to 0. +#' weight is equal to 0. Must be greater than 0. #' #' @template return_decay_function #' @@ -32,6 +32,15 @@ decay_linear <- function(cutoff) { unique = TRUE ) + # a cutoff of 0 would produce NaN weights when travel cost is 0 (0 / 0) + + if (any(cutoff <= 0)) { + stop( + "Assertion on 'cutoff' failed: Value must be greater than 0.", + call. = FALSE + ) + } + weighting_function <- function(travel_cost) { weights_list <- lapply( cutoff, diff --git a/R/decay_logistic.R b/R/decay_logistic.R index 370c2e4..65df84c 100644 --- a/R/decay_logistic.R +++ b/R/decay_logistic.R @@ -45,6 +45,17 @@ decay_logistic <- function(cutoff, sd) { assert_logistic_cutoff(cutoff) assert_standard_deviation(sd, cutoff) + # duplicated (cutoff, sd) pairs would produce duplicated list names, which + # are merged/double counted downstream + + if (anyDuplicated(data.frame(cutoff, sd))) { + stop( + "Assertion on 'cutoff' and 'sd' failed: ", + "Must not contain duplicated (cutoff, sd) pairs.", + call. = FALSE + ) + } + weighting_function <- function(travel_cost) { g <- function(cost, median, std_dev) { 1 + exp(((cost - median) * base::pi) / (std_dev * sqrt(3))) diff --git a/R/decay_stepped.R b/R/decay_stepped.R index 8036b7e..bedc8cb 100644 --- a/R/decay_stepped.R +++ b/R/decay_stepped.R @@ -56,6 +56,24 @@ decay_stepped <- function(steps, weights) { if (!is.list(steps)) steps <- list(steps) if (!is.list(weights)) weights <- list(weights) + # duplicated (steps, weights) pairs would produce duplicated list names, + # which are merged/double counted downstream + + step_weight_pairs <- mapply( + function(stps, wghts) list(as.numeric(stps), as.numeric(wghts)), + steps, + weights, + SIMPLIFY = FALSE + ) + + if (anyDuplicated(step_weight_pairs)) { + stop( + "Assertion on 'steps' and 'weights' failed: ", + "Must not contain duplicated (steps, weights) pairs.", + call. = FALSE + ) + } + steps <- lapply(steps, function(stps) c(0, stps)) weights <- lapply(weights, function(wghts) c(1, wghts)) @@ -66,7 +84,10 @@ decay_stepped <- function(steps, weights) { FUN = function(stps, wghts) { vapply( travel_cost, - function(x) wghts[max(which(x >= stps))], + function(x) { + if (is.na(x)) return(NA_real_) + wghts[max(which(x >= stps))] + }, numeric(1) ) }, diff --git a/R/total_constrained.R b/R/total_constrained.R index fb0757b..37a560c 100644 --- a/R/total_constrained.R +++ b/R/total_constrained.R @@ -96,7 +96,10 @@ total_constrained <- function(travel_matrix, data[, (kappa_col) := get(weighted_col) / total_weighted_system] ids_for_total <- unique(data[[merge_id]]) # e.g., A,B,C when active=TRUE - total_opportunities_region <- land_use_data[id %in% ids_for_total, sum(get(opportunity_col), na.rm = TRUE)] + total_opportunities_region <- sum( + land_use_data[[opportunity_col]][match(ids_for_total, land_use_data$id)], + na.rm = TRUE + ) data[, constrained_opportunity := get(kappa_col) * total_opportunities_region] diff --git a/man/decay_linear.Rd b/man/decay_linear.Rd index b016cad..6eed5e1 100644 --- a/man/decay_linear.Rd +++ b/man/decay_linear.Rd @@ -9,7 +9,7 @@ decay_linear(cutoff) \arguments{ \item{cutoff}{A \code{numeric} vector. Indicates the travel cost cutoffs until which the weighting factor decays linearly. From this point onward the -weight is equal to 0.} +weight is equal to 0. Must be greater than 0.} } \value{ A \code{function} that takes a generic travel cost vector (\code{numeric}) as diff --git a/tests/testthat/test-assert_land_use_data.R b/tests/testthat/test-assert_land_use_data.R index 2a57526..b111f4f 100644 --- a/tests/testthat/test-assert_land_use_data.R +++ b/tests/testthat/test-assert_land_use_data.R @@ -86,6 +86,8 @@ test_that("warns if either opportunity or demand cols contain NAs", { }) test_that("assert_land_use_data() doesn't error with tibbles", { + skip_if_not_installed("tibble") + # issue 49 (https://github.com/ipea/accessibility/issues/49) # using land_use_data as tibbles would throw errors because this format has # stricter subsetting rules diff --git a/tests/testthat/test-constrained-essentials.R b/tests/testthat/test-constrained-essentials.R index 334e710..fd30247 100644 --- a/tests/testthat/test-constrained-essentials.R +++ b/tests/testthat/test-constrained-essentials.R @@ -397,3 +397,25 @@ test_that("fill_missing_ids fills missing combinations when using group_by", { ungrouped$supply[match(car$from_id, ungrouped$from_id)] ) }) + +test_that("total-constrained: doesn't add an index to land_use_data", { + lu <- make_lu_large() + + # data.table auto-indexing is only triggered with character ids + + tm <- make_tm_3x3() + tm[, `:=`(from_id = as.character(from_id), to_id = as.character(to_id))] + + constrained_accessibility( + travel_matrix = tm, + land_use_data = lu, + travel_cost = "travel_time", + supply = "jobs", + demand = "population", + constraint = "total", + decay_function = exp_decay_01, + active = TRUE + ) + + expect_true(is.null(attr(lu, "index"))) +}) diff --git a/tests/testthat/test-constrained-wrapper.R b/tests/testthat/test-constrained-wrapper.R index 62ba1e2..c52cb14 100644 --- a/tests/testthat/test-constrained-wrapper.R +++ b/tests/testthat/test-constrained-wrapper.R @@ -137,3 +137,14 @@ test_that("raises errors due to incorrect input", { ) ) }) + +test_that("raises a clear error if demand or supply has length > 1", { + expect_error( + tester(demand = c("population", "population")), + regexp = "Must have length 1" + ) + expect_error( + tester(supply = c("jobs", "jobs")), + regexp = "Must have length 1" + ) +}) diff --git a/tests/testthat/test-cumulative_cutoff.R b/tests/testthat/test-cumulative_cutoff.R index 1486107..c0c4923 100644 --- a/tests/testthat/test-cumulative_cutoff.R +++ b/tests/testthat/test-cumulative_cutoff.R @@ -414,3 +414,39 @@ test_that("works with cutoffs with floating point noise and any colname", { ) expect_identical(result$jobs, c(60, 30, 40, 0)) }) + +test_that("raises a clear error if id types are incompatible", { + int_matrix <- data.table::data.table( + from_id = 1:2, + to_id = 1:2, + travel_time = c(1, 2) + ) + chr_land_use <- data.table::data.table(id = c("1", "2"), jobs = c(1, 2)) + + expect_error( + tester(int_matrix, chr_land_use, cutoff = 5, group_by = character(0)), + regexp = "must be of compatible types" + ) + + # integer vs double and factor vs character ids join fine, so no error + + dbl_land_use <- data.table::data.table(id = c(1, 2), jobs = c(1, 2)) + expect_error( + tester(int_matrix, dbl_land_use, cutoff = 5, group_by = character(0)), + NA + ) + + chr_matrix <- data.table::data.table( + from_id = c("1", "2"), + to_id = c("1", "2"), + travel_time = c(1, 2) + ) + fct_land_use <- data.table::data.table( + id = factor(c("1", "2")), + jobs = c(1, 2) + ) + expect_error( + tester(chr_matrix, fct_land_use, cutoff = 5, group_by = character(0)), + NA + ) +}) diff --git a/tests/testthat/test-decay_linear.R b/tests/testthat/test-decay_linear.R index be66d74..ddd84ef 100644 --- a/tests/testthat/test-decay_linear.R +++ b/tests/testthat/test-decay_linear.R @@ -61,3 +61,8 @@ test_that("output fn returns empty numeric if receives empty numeric/integer", { expect_identical(output_fn(numeric())[["20"]], numeric()) expect_identical(output_fn(numeric())[["30"]], numeric()) }) + +test_that("raises an error if cutoff is 0", { + expect_error(tester(0), regexp = "Value must be greater than 0") + expect_error(tester(c(0, 30)), regexp = "Value must be greater than 0") +}) diff --git a/tests/testthat/test-decay_logistic.R b/tests/testthat/test-decay_logistic.R index e67108c..a3fed4b 100644 --- a/tests/testthat/test-decay_logistic.R +++ b/tests/testthat/test-decay_logistic.R @@ -55,3 +55,11 @@ test_that("output fn returns empty numeric if receives empty numeric/integer", { expect_identical(output_fn(integer())[["c20;sd2"]], numeric()) expect_identical(output_fn(numeric())[["c20;sd2"]], numeric()) }) + +test_that("raises an error if (cutoff, sd) pairs are duplicated", { + expect_error( + tester(c(30, 30), c(5, 5)), + regexp = "duplicated .cutoff, sd. pairs" + ) + expect_error(tester(c(30, 30), c(5, 10)), NA) +}) diff --git a/tests/testthat/test-decay_stepped.R b/tests/testthat/test-decay_stepped.R index 43f2cbb..05a91cc 100644 --- a/tests/testthat/test-decay_stepped.R +++ b/tests/testthat/test-decay_stepped.R @@ -89,3 +89,25 @@ test_that("output fn returns empty numeric if receives empty numeric/integer", { expect_identical(output_fn(integer())[[1]], numeric()) expect_identical(output_fn(numeric())[[1]], numeric()) }) + +test_that("raises an error if (steps, weights) pairs are duplicated", { + expect_error( + tester(list(c(10, 20), c(10, 20)), list(c(0.5, 0), c(0.5, 0))), + regexp = "duplicated .steps, weights. pairs" + ) + expect_error( + tester(list(c(10L, 20L), c(10, 20)), list(c(0.5, 0), c(0.5, 0))), + regexp = "duplicated .steps, weights. pairs" + ) + expect_error( + tester(list(c(10, 20), c(10, 20)), list(c(0.5, 0), c(0.6, 0))), + NA + ) +}) + +test_that("returns NA without warnings when travel cost is NA", { + output_fn <- tester(c(10, 20), c(0.5, 0)) + + expect_silent(result <- output_fn(c(NA, 5, 15, 25))) + expect_identical(result[[1]], c(NA, 1, 0.5, 0)) +}) diff --git a/tests/testthat/test-fgt_poverty.R b/tests/testthat/test-fgt_poverty.R index 4483295..d140572 100644 --- a/tests/testthat/test-fgt_poverty.R +++ b/tests/testthat/test-fgt_poverty.R @@ -220,3 +220,53 @@ test_that("works even if access_data and sociodem_data has specific colnames", { expect_identical(poor_above$FGT1, 1) expect_identical(poor_above$FGT2, 1) }) + +test_that("fgt measures match hand-computed values on a toy example", { + # access x = (0, 5, 10, 15, 20), population w = (1, 2, 3, 4, 5), total 15, + # poverty line z = 10. zone "c" sits exactly on the line and is never poor + # (strict inequalities). FGT_a = sum_i w_i * gap_i ^ a * poor_i / sum_i w_i + toy_access <- data.table::data.table( + id = c("a", "b", "c", "d", "e"), + jobs = c(0, 5, 10, 15, 20) + ) + toy_sociodem <- data.table::data.table( + id = c("a", "b", "c", "d", "e"), + population = c(1, 2, 3, 4, 5) + ) + + # poor_below_threshold = TRUE: poor if x < z, gap (z - x) / z. + # a (gap 1) and b (gap 0.5) are poor + result <- tester( + toy_access, + toy_sociodem, + poverty_line = 10, + poor_below_threshold = TRUE, + group_by = character(0) + ) + expect_equal( + result, + data.table::data.table( + FGT0 = (1 + 2) / 15, + FGT1 = (1 * 1 + 2 * 0.5) / 15, + FGT2 = (1 * 1 ^ 2 + 2 * 0.5 ^ 2) / 15 + ) + ) + + # poor_below_threshold = FALSE: poor if x > z, gap (x - z) / z. + # d (gap 0.5) and e (gap 1) are poor + result <- tester( + toy_access, + toy_sociodem, + poverty_line = 10, + poor_below_threshold = FALSE, + group_by = character(0) + ) + expect_equal( + result, + data.table::data.table( + FGT0 = (4 + 5) / 15, + FGT1 = (4 * 0.5 + 5 * 1) / 15, + FGT2 = (4 * 0.5 ^ 2 + 5 * 1 ^ 2) / 15 + ) + ) +}) diff --git a/tests/testthat/test-floating_catchment_area.R b/tests/testthat/test-floating_catchment_area.R index da3ee20..5952e9a 100644 --- a/tests/testthat/test-floating_catchment_area.R +++ b/tests/testthat/test-floating_catchment_area.R @@ -319,3 +319,81 @@ test_that("results are grouped by decay_function_arg when needed", { ) ) }) + +test_that("2sfca and bfca match hand-computed values on a toy example", { + # 3 zones, asymmetric costs (a->b = 10 but b->a = 20), linear decay with + # cutoff 40, so f(c) = 1 - c / 40: + # to a to b to c + # from a 1 0.75 0.50 row sums: a 2.25, b 1.75, c 1.75 + # from b 0.50 1 0.25 col sums: a 2.00, b 2.00, c 1.75 + # from c 0.50 0.25 1 + # opportunities S = (10, 20, 30); population P = (10, 5, 5) + toy_matrix <- data.table::data.table( + from_id = rep(c("a", "b", "c"), each = 3), + to_id = rep(c("a", "b", "c"), 3), + travel_time = c(0, 10, 20, 20, 0, 30, 20, 30, 0) + ) + toy_land_use <- data.table::data.table( + id = c("a", "b", "c"), + jobs = c(10, 20, 30), + population = c(10, 5, 5) + ) + + toy_fca <- function(method) { + floating_catchment_area( + toy_matrix, + toy_land_use, + opportunity = "jobs", + travel_cost = "travel_time", + demand = "population", + method = method, + decay_function = decay_linear(cutoff = 40) + ) + } + + # 2SFCA (Luo & Wang 2003): R_j = S_j / sum_k P_k f(c_kj); + # A_i = sum_j R_j f(c_ij) + # R_a = 10 / (10 * 1 + 5 * 0.5 + 5 * 0.5 ) = 10 / 15 = 2 / 3 + # R_b = 20 / (10 * 0.75 + 5 * 1 + 5 * 0.25) = 20 / 13.75 = 16 / 11 + # R_c = 30 / (10 * 0.5 + 5 * 0.25 + 5 * 1 ) = 30 / 11.25 = 8 / 3 + # A_a = 2/3 + 0.75 * 16/11 + 0.5 * 8/3 = 34 / 11 + # A_b = 0.5 * 2/3 + 16/11 + 0.25 * 8/3 = 27 / 11 + # A_c = 0.5 * 2/3 + 0.25 * 16/11 + 8/3 = 37 / 11 + result <- toy_fca("2sfca") + expect_equal( + result, + data.table::data.table( + id = c("a", "b", "c"), + jobs = c(34 / 11, 27 / 11, 37 / 11) + ) + ) + # 2SFCA preserves the total supply: sum_i P_i A_i = sum_j S_j = 60 + expect_equal(sum(toy_land_use$population * result$jobs), 60) + + # BFCA (Paez et al. 2019): f is balanced by origin, Wi_ij = f_ij / sum_j f_ij, + # to allocate demand, and by destination, Wj_ij = f_ij / sum_i f_ij, to + # allocate the level of service: + # R_j = S_j / sum_k P_k Wi_kj; A_i = sum_j R_j Wj_ij + # demand: D_a = 10 * 4/9 + 5 * 2/7 + 5 * 2/7 = 460 / 63 + # D_b = 10 * 1/3 + 5 * 4/7 + 5 * 1/7 = 145 / 21 + # D_c = 10 * 2/9 + 5 * 1/7 + 5 * 4/7 = 365 / 63 + # ratios: R_a = 10 / D_a = 63 / 46, R_b = 20 / D_b = 84 / 29, + # R_c = 30 / D_c = 378 / 73 + r_a <- 63 / 46 + r_b <- 84 / 29 + r_c <- 378 / 73 + result <- toy_fca("bfca") + expect_equal( + result, + data.table::data.table( + id = c("a", "b", "c"), + jobs = c( + r_a * 0.5 + r_b * 0.375 + r_c * 2 / 7, + r_a * 0.25 + r_b * 0.5 + r_c * 1 / 7, + r_a * 0.25 + r_b * 0.125 + r_c * 4 / 7 + ) + ) + ) + # BFCA does not inflate the level of service: sum_i A_i = sum_j R_j + expect_equal(sum(result$jobs), r_a + r_b + r_c) +}) diff --git a/tests/testthat/test-gravity.R b/tests/testthat/test-gravity.R index 7488349..4df5a46 100644 --- a/tests/testthat/test-gravity.R +++ b/tests/testthat/test-gravity.R @@ -321,3 +321,19 @@ test_that("decay_fn_arg is char when decay fn is stepped and num otherwise", { ) expect_is(result$decay_function_arg, "character") }) + +test_that("raises an error if travel_matrix contains negative travel costs", { + negative_matrix <- data.table::copy(smaller_matrix) + negative_matrix[1, travel_time := -10] + + expect_error( + tester(travel_matrix = negative_matrix), + regexp = "Travel costs must not be negative" + ) + + # NA costs are still allowed + + na_matrix <- data.table::copy(smaller_matrix) + na_matrix[1, travel_time := NA] + expect_error(tester(travel_matrix = na_matrix), NA) +}) diff --git a/tests/testthat/test-theil_t.R b/tests/testthat/test-theil_t.R index d717428..7446a33 100644 --- a/tests/testthat/test-theil_t.R +++ b/tests/testthat/test-theil_t.R @@ -199,7 +199,8 @@ test_that("input data sets remain unchanged", { data.table::setindexv(small_access, NULL) expect_equal(original_access_data, small_access) - expect_equal(original_sociodem_data, original_sociodem_data) + data.table::setindex(sociodem_data, NULL) + expect_equal(original_sociodem_data, sociodem_data) }) test_that("theil t is correctly calculated", { @@ -379,7 +380,8 @@ test_that("input data sets remain unchanged", { result <- null_tester() expect_equal(original_access_data, small_access) - expect_equal(original_sociodem_data, original_sociodem_data) + data.table::setindex(sociodem_data, NULL) + expect_equal(original_sociodem_data, sociodem_data) }) test_that("theil t is correctly calculated", { diff --git a/vignettes/accessibility.Rmd b/vignettes/accessibility.Rmd index 4817b2c..48915a3 100644 --- a/vignettes/accessibility.Rmd +++ b/vignettes/accessibility.Rmd @@ -95,8 +95,6 @@ First we'll need to load the libraries we'll be using: ```{r, message = FALSE, warning = FALSE} library(accessibility) library(data.table) -library(ggplot2) -library(sf) ``` ## Data requirements @@ -343,7 +341,10 @@ If you have the spatial data of your origins/destinations, you can easily merge it with the accessibility to create spatial visualizations of the results. The example below quickly shows how to create a simple map using `{ggplot2}`. -```{r, eval = requireNamespace(c("sf", "ggplot2"), quietly = TRUE), out.width = "80%", fig.width = 6, fig.height = 6} +```{r, eval = requireNamespace("sf", quietly = TRUE) && requireNamespace("ggplot2", quietly = TRUE), message = FALSE, out.width = "80%", fig.width = 6, fig.height = 6} +library(ggplot2) +library(sf) + grid <- system.file("extdata/grid_bho.rds", package = "accessibility") grid <- readRDS(grid) From 813666efc587d3b8cd3b51c864a86d1935e011e2 Mon Sep 17 00:00:00 2001 From: rafapereirabr Date: Tue, 20 Oct 2026 19:40:22 -0300 Subject: [PATCH 15/31] fast merge_by_reference --- R/merge_by_reference.R | 25 +++++++++++++++++-------- 1 file changed, 17 insertions(+), 8 deletions(-) diff --git a/R/merge_by_reference.R b/R/merge_by_reference.R index 18773ff..b99821f 100644 --- a/R/merge_by_reference.R +++ b/R/merge_by_reference.R @@ -6,14 +6,23 @@ merge_by_reference <- function(data, # when calculating active accessibility, we want the number of opportunities # in the destination. if passive, the number of opportunities in the origin - right_df_idcol <- "id" + # matching the ids and assigning with set() is much faster than an update + # join, which sorts the (large) travel matrix on every call. ids missing from + # land_use_data get NA, as in a join. land_use_data ids are unique (checked + # in the assert_*() helpers) - join_vector <- right_df_idcol - names(join_vector) <- left_df_idcol + left_ids <- data[[left_df_idcol]] + right_ids <- land_use_data$id - data[ - land_use_data, - on = join_vector, - eval(opportunity) := get(paste0("i.", ..opportunity)) - ] + match_index <- if (is.character(left_ids) && is.character(right_ids)) { + data.table::chmatch(left_ids, right_ids) + } else { + match(left_ids, right_ids) + } + + data.table::set( + data, + j = opportunity, + value = land_use_data[[opportunity]][match_index] + ) } From db0285157d29569c616403a17fa7976c2e06a86f Mon Sep 17 00:00:00 2001 From: rafapereirabr Date: Tue, 20 Oct 2026 20:26:17 -0300 Subject: [PATCH 16/31] using cli --- .claude/rules/r-package-conventions.md | 31 ++-- .github/workflows/pkgdown.yaml | 4 +- .github/workflows/rcmdcheck.yaml | 9 +- .github/workflows/test-coverage.yaml | 8 +- DESCRIPTION | 2 + NAMESPACE | 1 + NEWS.md | 22 +++ R/accessibility-package.R | 1 + R/assert_and_assign.R | 155 +++++++++---------- R/balancing_cost.R | 9 +- R/concentration_index.R | 6 +- R/constrained_accessibility.R | 5 +- R/cumulative_interval.R | 10 +- R/decay_linear.R | 6 +- R/decay_logistic.R | 8 +- R/decay_stepped.R | 15 +- R/doubly_constrained.R | 34 ++-- R/fgt_poverty.R | 8 +- R/fill_missing_ids.R | 5 + R/total_constrained.R | 12 +- R/warn_extra_cols.R | 34 ++-- codemeta.json | 38 ++++- tests/testthat/test-concentration_index.R | 4 +- tests/testthat/test-constrained-essentials.R | 4 +- tests/testthat/test-cumulative_cutoff.R | 20 +++ tests/testthat/test-cumulative_interval.R | 38 ++--- tests/testthat/test-decay_logistic.R | 2 +- tests/testthat/test-decay_stepped.R | 4 +- tests/testthat/test-gravity.R | 2 +- tests/testthat/test-merge_by_reference.R | 40 +++++ 30 files changed, 337 insertions(+), 200 deletions(-) create mode 100644 tests/testthat/test-merge_by_reference.R diff --git a/.claude/rules/r-package-conventions.md b/.claude/rules/r-package-conventions.md index 218eb64..e804767 100644 --- a/.claude/rules/r-package-conventions.md +++ b/.claude/rules/r-package-conventions.md @@ -134,7 +134,9 @@ some_measure <- function(travel_matrix, `data.table::set*()` must act on a table the function owns. Every measure has an "input data sets remain unchanged" test. - `land_use_data` is only converted, never updated by reference; keep it that - way (`merge_by_reference()` reads from it with `i.` prefixes). + way (`merge_by_reference()` only reads from it, matching on `id` with + `match()`/`chmatch()`; it relies on the `assert_*()` helpers rejecting + duplicated ids). - data.table auto-indexing may add an `index` attribute to the caller's table after `dt[i]`. Tests clear it with `data.table::setindex(x, NULL)` before comparing; that is accepted, don't add `copy()` calls to prevent it. @@ -158,18 +160,19 @@ some_measure <- function(travel_matrix, ## A4. Messages and errors -- **Today:** base `stop()`/`warning()` with `call. = FALSE` and messages built - by string concatenation, quoting argument names in single quotes - (`"'land_use_data' is missing ids listed in 'travel_matrix', ..."`). Input - type errors come from checkmate. `cli` is **not** a dependency. -- The newer constrained-measure files (`total_constrained.R`, - `doubly_constrained.R`, `constrained_accessibility.R`) use `stop()` without - `call. = FALSE`. [NEW] modified code adds `call. = FALSE` to match the rest - of the package. -- Adding `cli` (or `lifecycle`) to Imports needs a plan; never migrate - messages as a drive-by. +- The package's own messages use `cli::cli_abort(, call = NULL)` and + `cli::cli_warn()` (cli and rlang are Imports; migrated 2026-10-20). + Inline markup: `{.arg x}` for arguments and `df$col` expressions, + `{.val {x}}` for values/vectors, `{.code}` for code, `{.cls}` for classes; + bullets (`"*"`, `"i"`) instead of ` +`. Runtime values always go through + interpolation, never `paste0()` into the template. Input type errors still + come from checkmate. +- cli wraps long messages at the console width, so test `regexp`s that span + several words use `[[:space:]]+` between them. - **Deprecations:** keep the old argument for at least one CRAN release with a - `warning()` naming the replacement, and test it with `expect_warning()`. + `cli::cli_warn()` naming the replacement, and test it with + `expect_warning()`. ## A5. Style @@ -312,7 +315,7 @@ Function names in backticks; link issues; credit external contributors. | `library()`/`require()` in `R/` | `pkg::fun()` | | Writing outside `tempdir()` | Write only to `tempdir()` or user-supplied paths | | `<<-` to global env | Return values | -| `print()`/`cat()` in functions | `message()`/`warning()`, gated on a `verbose` arg where relevant | +| `print()`/`cat()` in functions | `cli::cli_inform()`/`cli::cli_warn()`, gated on a `verbose` arg where relevant | | `options()`/`par()`/threads changed, not restored | `on.exit(..., add = TRUE)` | | `T`/`F` | `TRUE`/`FALSE` | | Unvalidated enumerated arg | `checkmate::assert_names(x, subset.of = ...)` | @@ -332,7 +335,7 @@ Function names in backticks; link issues; credit external contributors. [ ] output class mirrors travel_matrix (original_class idiom); test proves it [ ] output columns/order follow A2; fill_missing_ids where ids can drop [ ] values tested against a hand-computed toy example / the cited formula -[ ] base stop()/warning() with call. = FALSE and quoted arg names +[ ] cli::cli_abort(call = NULL)/cli::cli_warn() with inline markup [ ] style: 2 spaces, <-, 80 cols, return(), file's blank-line style [ ] roxygen: shared @template, @family (exact spelling), @return, @examplesIf NOT_CRAN, Rdpack citation for the method diff --git a/.github/workflows/pkgdown.yaml b/.github/workflows/pkgdown.yaml index 750160e..7a821b8 100644 --- a/.github/workflows/pkgdown.yaml +++ b/.github/workflows/pkgdown.yaml @@ -7,10 +7,10 @@ name: pkgdown jobs: pkgdown: - runs-on: ubuntu-22.04 + runs-on: ubuntu-latest steps: - - uses: actions/checkout@v2 + - uses: actions/checkout@v4 - uses: r-lib/actions/setup-pandoc@v2 diff --git a/.github/workflows/rcmdcheck.yaml b/.github/workflows/rcmdcheck.yaml index 456da00..6bb6ec9 100644 --- a/.github/workflows/rcmdcheck.yaml +++ b/.github/workflows/rcmdcheck.yaml @@ -20,16 +20,15 @@ jobs: config: - {os: windows-latest, r: 'release'} - {os: macOS-latest, r: 'release'} - - {os: ubuntu-22.04, r: 'devel', rspm: "https://packagemanager.rstudio.com/cran/__linux__/jammy/latest"} - - {os: ubuntu-22.04, r: 'release', rspm: "https://packagemanager.rstudio.com/cran/__linux__/jammy/latest"} - - {os: ubuntu-22.04, r: 'oldrel', rspm: "https://packagemanager.rstudio.com/cran/__linux__/jammy/latest"} + - {os: ubuntu-latest, r: 'devel'} + - {os: ubuntu-latest, r: 'release'} + - {os: ubuntu-latest, r: 'oldrel'} env: - RSPM: ${{ matrix.config.rspm }} GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }} steps: - - uses: actions/checkout@v2 + - uses: actions/checkout@v4 - uses: r-lib/actions/setup-r@v2 with: diff --git a/.github/workflows/test-coverage.yaml b/.github/workflows/test-coverage.yaml index 9d9dc55..9f6c700 100644 --- a/.github/workflows/test-coverage.yaml +++ b/.github/workflows/test-coverage.yaml @@ -12,7 +12,7 @@ permissions: read-all jobs: test-coverage: - runs-on: ubuntu-22.04 + runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 @@ -34,11 +34,11 @@ jobs: covr::to_cobertura(cov) shell: Rscript {0} - - uses: codecov/codecov-action@v4 + - uses: codecov/codecov-action@v5 with: # Fail if error if not on PR, or if on PR and token is given fail_ci_if_error: ${{ github.event_name != 'pull_request' || secrets.CODECOV_TOKEN }} - file: ./cobertura.xml - plugin: noop + files: ./cobertura.xml + plugins: noop disable_search: true token: ${{ secrets.CODECOV_TOKEN }} diff --git a/DESCRIPTION b/DESCRIPTION index c54cb99..9d161ec 100644 --- a/DESCRIPTION +++ b/DESCRIPTION @@ -31,9 +31,11 @@ Depends: R (>= 3.5.0) Imports: checkmate, + cli (>= 3.0.0), data.table, Hmisc, Rdpack (>= 0.7), + rlang (>= 1.0.0), stats, utils Suggests: diff --git a/NAMESPACE b/NAMESPACE index 982bae3..1ff30c3 100644 --- a/NAMESPACE +++ b/NAMESPACE @@ -28,4 +28,5 @@ importFrom(data.table, .N, .SD ) +importFrom(rlang,abort) importFrom(utils,globalVariables) diff --git a/NEWS.md b/NEWS.md index 0718576..6e00780 100644 --- a/NEWS.md +++ b/NEWS.md @@ -7,6 +7,16 @@ within the interval are included in the output with an accessibility of 0, as in `cumulative_cutoff()`. Previously they were dropped from the output. +## Breaking changes + +- `cumulative_interval()` now returns the summary of the accessibility levels + as a `numeric` without truncation (e.g. the median of 26 and 27 + opportunities is 26.5). Previously, results were truncated to integers. +- `concentration_index(type = "corrected")` now calculates the correction + factor with the bounds of the whole accessibility distribution, instead of + the bounds of each `group_by` group, so groups are corrected on a common + scale. Results only change when `group_by` is used. + ## Bug fixes - Fixed a bug in `balancing_cost()` in which numeric ids (e.g. `101`, `102`) @@ -105,6 +115,18 @@ duplicated parameter combinations, which produced duplicated results. - `constrained_accessibility()` now checks that `demand` and `supply` are single strings. +- Results of accessibility functions are no longer returned as keyed + data.tables when missing ids are filled (previously, they were keyed only in + this case). +- Opportunities, demand, supply, population and income are now matched to + the input data by id with `match()` instead of an update join. Accessibility + functions are about 1.2 to 3.3 times faster and use about half the memory on + large travel matrices, with identical results. If `travel_matrix` already + has a column named like the opportunity column, that column is now fully + replaced by the values in `land_use_data`. +- Errors and warnings raised by the package now use `{cli}`, with consistent + formatting of argument names and values. Long lists of ids or columns in + messages are abbreviated. `{cli}` and `{rlang}` are new dependencies. ## Notes diff --git a/R/accessibility-package.R b/R/accessibility-package.R index 421f9c9..39c3752 100644 --- a/R/accessibility-package.R +++ b/R/accessibility-package.R @@ -20,6 +20,7 @@ #' @importFrom data.table := .I .SD %chin% .GRP .N #' @importFrom utils globalVariables #' @importFrom Rdpack reprompt +#' @importFrom rlang abort #' #' @keywords internal "_PACKAGE" diff --git a/R/assert_and_assign.R b/R/assert_and_assign.R index 3eeb88a..8f2cc44 100644 --- a/R/assert_and_assign.R +++ b/R/assert_and_assign.R @@ -75,10 +75,10 @@ assert_travel_matrix <- function(travel_matrix, travel_cost, group_by) { cost_values <- travel_matrix[[col]] if (is.numeric(cost_values) && any(cost_values < 0, na.rm = TRUE)) { - stop( - "Assertion on 'travel_matrix$", col, "' failed: ", - "Travel costs must not be negative.", - call. = FALSE + cli::cli_abort( + "Assertion on {.arg travel_matrix${col}} failed: Travel costs must + not be negative.", + call = NULL ) } } @@ -128,11 +128,10 @@ assert_land_use_data <- function(land_use_data, if (anyDuplicated(land_use_data$id)) { duplicated_ids <- unique(land_use_data$id[duplicated(land_use_data$id)]) - stop( - "'land_use_data' must not contain duplicated ids. ", - "Found duplicates: ", - paste0("'", duplicated_ids, "'", collapse = ", "), - call. = FALSE + cli::cli_abort( + "{.arg land_use_data} must not contain duplicated ids. Found + duplicates: {.val {duplicated_ids}}.", + call = NULL ) } @@ -153,20 +152,19 @@ assert_land_use_data <- function(land_use_data, is_text_like <- function(x) is.character(x) || is.factor(x) if (is_text_like(travel_matrix_ids) != is_text_like(land_use_data$id)) { - stop( - "The id columns of 'travel_matrix' ('from_id'/'to_id') and ", - "'land_use_data' ('id') must be of compatible types. Found '", - class(travel_matrix_ids)[1], "' in 'travel_matrix' and '", - class(land_use_data$id)[1], "' in 'land_use_data'.", - call. = FALSE + cli::cli_abort( + "The id columns of {.arg travel_matrix} ({.field from_id}/{.field to_id}) + and {.arg land_use_data} ({.field id}) must be of compatible types. + Found {.cls {class(travel_matrix_ids)[1]}} in {.arg travel_matrix} and + {.cls {class(land_use_data$id)[1]}} in {.arg land_use_data}.", + call = NULL ) } if (!all(travel_matrix_ids %in% land_use_data$id)) { - warning( - "'land_use_data' is missing ids listed in 'travel_matrix', which may ", - "produce NAs in the final output.", - call. = FALSE + cli::cli_warn( + "{.arg land_use_data} is missing ids listed in {.arg travel_matrix}, + which may produce NAs in the final output." ) } @@ -180,10 +178,9 @@ assert_land_use_data <- function(land_use_data, for (col in cols_to_check) { if (any(is.na(land_use_data[[col]]))) { - warning( - "'land_use_data$", col, "' contains NA values, which may produce NAs ", - "in the final output.", - call. = FALSE + cli::cli_warn( + "{.arg land_use_data${col}} contains NA values, which may produce NAs + in the final output." ) } } @@ -217,11 +214,10 @@ assert_sociodemographic_data <- function(sociodemographic_data, ids <- sociodemographic_data$id duplicated_ids <- unique(ids[duplicated(ids)]) - stop( - "'sociodemographic_data' must not contain duplicated ids. ", - "Found duplicates: ", - paste0("'", duplicated_ids, "'", collapse = ", "), - call. = FALSE + cli::cli_abort( + "{.arg sociodemographic_data} must not contain duplicated ids. Found + duplicates: {.val {duplicated_ids}}.", + call = NULL ) } @@ -229,10 +225,9 @@ assert_sociodemographic_data <- function(sociodemographic_data, # merging the datasets will produce NAs, so we warn users about that if (!all(accessibility_data$id %in% sociodemographic_data$id)) { - warning( - "'sociodemographic_data' is missing ids listed in 'accessibility_data', ", - "which may produce NAs in the final output.", - call. = FALSE + cli::cli_warn( + "{.arg sociodemographic_data} is missing ids listed in + {.arg accessibility_data}, which may produce NAs in the final output." ) } @@ -247,10 +242,9 @@ assert_sociodemographic_data <- function(sociodemographic_data, for (col in non_income_cols) { if (any(is.na(sociodemographic_data[[col]]))) { - warning( - "'sociodemographic_data$", col, "' contains NA values, which may ", - "produce NAs in the final output.", - call. = FALSE + cli::cli_warn( + "{.arg sociodemographic_data${col}} contains NA values, which may + produce NAs in the final output." ) } } @@ -267,11 +261,10 @@ assert_sociodemographic_data <- function(sociodemographic_data, should_warn <- ifelse(is.na(should_warn), TRUE, should_warn) if (should_warn) { - warning( - "'sociodemographic_data$", income, "' contains NA values whose ", - "correspodent 'sociodemographic_data$population' is not 0, which ", - "may produce NAs in the final output.", - call. = FALSE + cli::cli_warn( + "{.arg sociodemographic_data${income}} contains NA values whose + correspodent {.arg sociodemographic_data$population} is not 0, + which may produce NAs in the final output." ) } } @@ -291,21 +284,20 @@ assert_summary_function <- function(summary_function) { ) if (inherits(result, "error")) { - stop( - "Assertion on 'summary_function' failed: Must be a function that ", - "takes a numeric vector as input. Instead, failed with message: ", - "'", result$message, "'", - call. = FALSE + cli::cli_abort( + "Assertion on {.arg summary_function} failed: Must be a function that + takes a numeric vector as input. Instead, failed with message: + {.val {conditionMessage(result)}}", + call = NULL ) } if (!is.numeric(result) || length(result) != 1) { - stop( - "Assertion on 'summary_function' failed: Must be a function that takes ", - "a numeric vector as input and returns a single numeric as output. ", - "Instead, returned: ", - "{", paste0(result, collapse = ","), "}", - call. = FALSE + cli::cli_abort( + "Assertion on {.arg summary_function} failed: Must be a function that + takes a numeric vector as input and returns a single numeric as output. + Instead, returned: {.val {result}}", + call = NULL ) } @@ -323,11 +315,11 @@ assert_decay_function <- function(decay_function) { ) if (inherits(result, "error")) { - stop( - "Assertion on 'decay_function' failed: Must be a function that ", - "takes a numeric vector as input. Instead, failed with message: ", - "'", result$message, "'", - call. = FALSE + cli::cli_abort( + "Assertion on {.arg decay_function} failed: Must be a function that + takes a numeric vector as input. Instead, failed with message: + {.val {conditionMessage(result)}}", + call = NULL ) } @@ -345,14 +337,15 @@ assert_decay_function <- function(decay_function) { right_lengths <- length(result) > 0 && all(elements_length == 100) if (!(right_numeric || (is.list(result) && right_classes && right_lengths))) { - stop( - "Assertion on 'decay_function' failed: Must be a function that takes ", - "a numeric vector as input and returns either:\n", - " - a numeric vector with the same length of input, or;\n", - " - a list of numeric vectors whose lengths are the same of input.\n", - "Instead, with input 1:100 it returned: ", - "{", paste0(result, collapse = ","), "}", - call. = FALSE + cli::cli_abort( + c( + "Assertion on {.arg decay_function} failed: Must be a function that + takes a numeric vector as input and returns either:", + "*" = "a numeric vector with the same length of input, or;", + "*" = "a list of numeric vectors whose lengths are the same of input.", + "i" = "Instead, with input {.code 1:100} it returned: {.val {result}}" + ), + call = NULL ) } @@ -365,9 +358,10 @@ assert_interval_increment <- function(interval_increment) { checkmate::assert_number(interval_increment, finite = TRUE) if (interval_increment <= 0) { - stop( - "Assertion on 'interval_increment' failed: ", - "Value must be greater than 0." + cli::cli_abort( + "Assertion on {.arg interval_increment} failed: Value must be greater + than 0.", + call = NULL ) } @@ -420,12 +414,15 @@ assert_and_assign_interval <- function(interval) { interval <- list(interval) names(interval) <- build_interval_name(interval[[1]]) } else { - stop( - "Assertion on 'interval' failed: Must be either:\n", - " - a numeric vector with 2 elements, indicating the start and end ", - "points of the travel cost interval;\n", - " - a list of numeric vectors with 2 elements, each one of them ", - "indicating the start and end points of a travel cost interval." + cli::cli_abort( + c( + "Assertion on {.arg interval} failed: Must be either:", + "*" = "a numeric vector with 2 elements, indicating the start and end + points of the travel cost interval;", + "*" = "a list of numeric vectors with 2 elements, each one of them + indicating the start and end points of a travel cost interval." + ), + call = NULL ) } @@ -439,11 +436,11 @@ assert_detailed_fill_missing_ids <- function(fill_missing_ids, checkmate::assert_logical(fill_missing_ids, len = 1, any.missing = FALSE) if (detailed_results && !fill_missing_ids) { - warning( - "Found non-default 'fill_missing_ids' (FALSE) with 'detailed_results = ", - "TRUE'. Please note that 'fill_missing_ids' does not affect the ", - "detailed spatial availability output.", - call. = FALSE + cli::cli_warn( + "Found non-default {.arg fill_missing_ids} (FALSE) with + {.code detailed_results = TRUE}. Please note that + {.arg fill_missing_ids} does not affect the detailed spatial + availability output." ) } diff --git a/R/balancing_cost.R b/R/balancing_cost.R index 4995f79..adea14b 100644 --- a/R/balancing_cost.R +++ b/R/balancing_cost.R @@ -61,9 +61,10 @@ balancing_cost <- function(travel_matrix, checkmate::assert_string(demand) checkmate::assert_number(cost_increment, finite = TRUE) if (cost_increment <= 0) { - stop( - "Assertion on 'cost_increment' failed: Value must be greater than 0.", - call. = FALSE + cli::cli_abort( + "Assertion on {.arg cost_increment} failed: Value must be greater + than 0.", + call = NULL ) } checkmate::assert_logical(fill_missing_ids, len = 1, any.missing = FALSE) @@ -76,8 +77,6 @@ balancing_cost <- function(travel_matrix, demand = demand ) - variable <- 3 - # if not a dt, keep original class to assign later when returning result if (!inherits(travel_matrix, "data.table")) { diff --git a/R/concentration_index.R b/R/concentration_index.R index 32aa34d..8d45716 100644 --- a/R/concentration_index.R +++ b/R/concentration_index.R @@ -161,9 +161,11 @@ concentration_index <- function(accessibility_data, by = .groups ] } else if (type == "corrected") { + # the bounds of the accessibility distribution are the same for all groups. # max()/min() may result in warnings if nrow(data) == 0 - data[, upper := suppressWarnings(max(get(.opp_colname))), by = .groups] - data[, lower := suppressWarnings(min(get(.opp_colname))), by = .groups] + + data[, upper := suppressWarnings(max(get(.opp_colname)))] + data[, lower := suppressWarnings(min(get(.opp_colname)))] # from Erreygers (2009), comparing equations 27 and 24, we have the # following correction factor diff --git a/R/constrained_accessibility.R b/R/constrained_accessibility.R index 2f50bbf..7df790a 100644 --- a/R/constrained_accessibility.R +++ b/R/constrained_accessibility.R @@ -293,7 +293,10 @@ constrained_accessibility <- function(travel_matrix, } # For 'total' and 'singly' we require active to be TRUE or FALSE if (is.null(active)) { - stop(sprintf("For '%s', active must be TRUE or FALSE.", constraint)) + cli::cli_abort( + "For {.val {constraint}}, active must be TRUE or FALSE.", + call = NULL + ) } # Validate the relevant side (opportunity column) only diff --git a/R/cumulative_interval.R b/R/cumulative_interval.R index 279ef1d..fa05c3a 100644 --- a/R/cumulative_interval.R +++ b/R/cumulative_interval.R @@ -162,18 +162,16 @@ cumulative_interval <- function(travel_matrix, access_list <- data.table::rbindlist(access_list, idcol = "cutoffs") access_list[, cutoffs := as.numeric(cutoffs)] - # the as.integer() call below makes sure that, even though the summary - # function may return doubles, the result is an integer (afterall when using - # a cumulative measure you cannot reach half of an opportunity) and that the - # float is rounded down to the integer (if you reach 2.8 opportunities you do - # reach 2, but you can't reach 3) + # the summary is returned as a double: summary functions such as median() + # may return integers for some groups and doubles for others, and the summary + # of opportunity counts doesn't have to be a whole number access_list <- lapply( interval_cutoffs, function(own_cutoffs) { access_list[ cutoffs %in% own_cutoffs, - .(access = as.integer(summary_function(access))), + .(access = as.numeric(summary_function(access))), by = eval(groups, envir = env) ] } diff --git a/R/decay_linear.R b/R/decay_linear.R index f5018e1..37cdca3 100644 --- a/R/decay_linear.R +++ b/R/decay_linear.R @@ -35,9 +35,9 @@ decay_linear <- function(cutoff) { # a cutoff of 0 would produce NaN weights when travel cost is 0 (0 / 0) if (any(cutoff <= 0)) { - stop( - "Assertion on 'cutoff' failed: Value must be greater than 0.", - call. = FALSE + cli::cli_abort( + "Assertion on {.arg cutoff} failed: Value must be greater than 0.", + call = NULL ) } diff --git a/R/decay_logistic.R b/R/decay_logistic.R index 65df84c..afd9449 100644 --- a/R/decay_logistic.R +++ b/R/decay_logistic.R @@ -49,10 +49,10 @@ decay_logistic <- function(cutoff, sd) { # are merged/double counted downstream if (anyDuplicated(data.frame(cutoff, sd))) { - stop( - "Assertion on 'cutoff' and 'sd' failed: ", - "Must not contain duplicated (cutoff, sd) pairs.", - call. = FALSE + cli::cli_abort( + "Assertion on {.arg cutoff} and {.arg sd} failed: Must not contain + duplicated (cutoff, sd) pairs.", + call = NULL ) } diff --git a/R/decay_stepped.R b/R/decay_stepped.R index bedc8cb..5fe5082 100644 --- a/R/decay_stepped.R +++ b/R/decay_stepped.R @@ -67,10 +67,10 @@ decay_stepped <- function(steps, weights) { ) if (anyDuplicated(step_weight_pairs)) { - stop( - "Assertion on 'steps' and 'weights' failed: ", - "Must not contain duplicated (steps, weights) pairs.", - call. = FALSE + cli::cli_abort( + "Assertion on {.arg steps} and {.arg weights} failed: Must not contain + duplicated (steps, weights) pairs.", + call = NULL ) } @@ -119,9 +119,10 @@ assert_steps <- function(steps) { is_numeric_like <- function(x) is.numeric(x) || is.integer(x) if (!(is_numeric_like(steps) || is.list(steps))) { - stop( - "Assertion on 'steps' failed: Must be either a numeric vector or a list ", - "of numeric vectors." + cli::cli_abort( + "Assertion on {.arg steps} failed: Must be either a numeric vector or a + list of numeric vectors.", + call = NULL ) } diff --git a/R/doubly_constrained.R b/R/doubly_constrained.R index 687f989..de55e60 100644 --- a/R/doubly_constrained.R +++ b/R/doubly_constrained.R @@ -68,17 +68,21 @@ doubly_constrained <- function(travel_matrix, D <- land_use_data[[supply]][match(destinations, land_use_data$id)] if (anyNA(O) || anyNA(D)) { - stop( - "For doubly-constrained, 'land_use_data' must list a non-NA demand for ", - "every origin and a non-NA supply for every destination in ", - "'travel_matrix'.", - call. = FALSE + cli::cli_abort( + "For doubly-constrained, {.arg land_use_data} must list a non-NA demand + for every origin and a non-NA supply for every destination in + {.arg travel_matrix}.", + call = NULL ) } # Validate totals match if (abs(sum(O) - sum(D)) > 1e-6) { - stop("For doubly-constrained, the sum of origins must equal the sum of destinations.") + cli::cli_abort( + "For doubly-constrained, the sum of origins must equal the sum of + destinations.", + call = NULL + ) } # Build impedance matrix @@ -139,13 +143,12 @@ doubly_constrained <- function(travel_matrix, } if (error >= error_threshold) { - warning( - "Doubly-constrained balancing did not converge (", stop_reason, " after ", - iteration_count, " iterations). Final error is ", signif(error, 3), - ", above 'error_threshold' (", error_threshold, "), so flows don't ", - "fully match the demand and supply totals. Consider increasing ", - "'max_iterations' or decreasing 'improvement_threshold'.", - call. = FALSE + cli::cli_warn( + "Doubly-constrained balancing did not converge ({stop_reason} after + {iteration_count} iterations). Final error is {signif(error, 3)}, above + {.arg error_threshold} ({error_threshold}), so flows don't fully match + the demand and supply totals. Consider increasing + {.arg max_iterations} or decreasing {.arg improvement_threshold}." ) } @@ -167,7 +170,10 @@ doubly_constrained <- function(travel_matrix, if (detailed_results) { return(detailed_dt[]) } else { - warning("Aggregated results equal marginals (O_i or D_j). Interpret with caution.") + cli::cli_warn( + "Aggregated results equal marginals (O_i or D_j). Interpret with + caution." + ) agg <- data.table::rbindlist(list( data.table::data.table(id = origins, flow = O), data.table::data.table(id = destinations, flow = D) diff --git a/R/fgt_poverty.R b/R/fgt_poverty.R index 0cdc43f..78b46f3 100644 --- a/R/fgt_poverty.R +++ b/R/fgt_poverty.R @@ -97,10 +97,10 @@ fgt_poverty <- function(accessibility_data, checkmate::assert_number(poverty_line, lower = 0, finite = TRUE) checkmate::assert_flag(poor_below_threshold) if (!poor_below_threshold && poverty_line == 0) { - stop( - "Assertion on 'poverty_line' failed: Value must be greater than 0 when ", - "'poor_below_threshold' is FALSE.", - call. = FALSE + cli::cli_abort( + "Assertion on {.arg poverty_line} failed: Value must be greater than 0 + when {.arg poor_below_threshold} is FALSE.", + call = NULL ) } assert_access_group_by(group_by) diff --git a/R/fill_missing_ids.R b/R/fill_missing_ids.R index 8356bdc..b489d7b 100644 --- a/R/fill_missing_ids.R +++ b/R/fill_missing_ids.R @@ -40,5 +40,10 @@ do_fill_missing_ids <- function(access_df, eval(access_col, envir = parent.frame()) := fill_value ] + # merge() returns a table keyed by the merge columns. results are returned + # unkeyed, whether missing ids were filled or not + + data.table::setkey(filled_access_df, NULL) + return(filled_access_df[]) } diff --git a/R/total_constrained.R b/R/total_constrained.R index 37a560c..6fed885 100644 --- a/R/total_constrained.R +++ b/R/total_constrained.R @@ -44,7 +44,11 @@ total_constrained <- function(travel_matrix, if (active) { #active accessibility if (is.null(supply) || !is.null(demand)) { - stop("For active = TRUE, supply must be specified and demand must be NULL.") + cli::cli_abort( + "For {.code active = TRUE}, {.arg supply} must be specified and + {.arg demand} must be NULL.", + call = NULL + ) } merge_id <- "to_id" group_id <- "from_id" @@ -56,7 +60,11 @@ total_constrained <- function(travel_matrix, } else { #passive accessibility if (is.null(demand) || !is.null(supply)) { - stop("For active = FALSE, demand must be specified and supply must be NULL.") + cli::cli_abort( + "For {.code active = FALSE}, {.arg demand} must be specified and + {.arg supply} must be NULL.", + call = NULL + ) } merge_id <- "from_id" group_id <- "to_id" diff --git a/R/warn_extra_cols.R b/R/warn_extra_cols.R index 9eed490..416e34f 100644 --- a/R/warn_extra_cols.R +++ b/R/warn_extra_cols.R @@ -9,14 +9,15 @@ warn_extra_cols <- function(travel_matrix, travel_cost, group_id, groups) { } if (!identical(travel_matrix_extra_cols, character(0))) { - warning( - "Found columns in 'travel_matrix' not listed in either ", - "'travel_cost' or 'group_by': {", - paste0("'", travel_matrix_extra_cols, "'", collapse = ","), - "}.\n", - "This can result in excessive aggregation when calculating ", - "accessibility, if these columns serve as group ids in 'travel_matrix'.", - call. = FALSE + cli::cli_warn( + c( + "Found columns in {.arg travel_matrix} not listed in either + {.arg travel_cost} or {.arg group_by}: + {.val {travel_matrix_extra_cols}}.", + "i" = "This can result in excessive aggregation when calculating + accessibility, if these columns serve as group ids in + {.arg travel_matrix}." + ) ) } @@ -31,15 +32,14 @@ warn_extra_access_cols <- function(accessibility_data, opportunity, group_id) { ) if (!identical(extra_cols, character(0))) { - warning( - "Found columns in 'accessibility_data' not listed in either ", - "'opportunity' or 'group_by': {", - paste0("'", extra_cols, "'", collapse = ","), - "}.\n", - "This can result in excessive aggregation when calculating ", - "inequality measures, if these columns serve as group ids in ", - "'accessibility_data'.", - call. = FALSE + cli::cli_warn( + c( + "Found columns in {.arg accessibility_data} not listed in either + {.arg opportunity} or {.arg group_by}: {.val {extra_cols}}.", + "i" = "This can result in excessive aggregation when calculating + inequality measures, if these columns serve as group ids in + {.arg accessibility_data}." + ) ) } diff --git a/codemeta.json b/codemeta.json index cef4e43..892858e 100644 --- a/codemeta.json +++ b/codemeta.json @@ -8,7 +8,7 @@ "codeRepository": "https://github.com/ipea/accessibility", "issueTracker": "https://github.com/ipea/accessibility/issues", "license": "https://spdx.org/licenses/MIT", - "version": "1.5.0", + "version": "1.5.0.9000", "programmingLanguage": { "@type": "ComputerLanguage", "name": "R", @@ -181,6 +181,19 @@ "sameAs": "https://CRAN.R-project.org/package=checkmate" }, "3": { + "@type": "SoftwareApplication", + "identifier": "cli", + "name": "cli", + "version": ">= 3.0.0", + "provider": { + "@id": "https://cran.r-project.org", + "@type": "Organization", + "name": "Comprehensive R Archive Network (CRAN)", + "url": "https://cran.r-project.org" + }, + "sameAs": "https://CRAN.R-project.org/package=cli" + }, + "4": { "@type": "SoftwareApplication", "identifier": "data.table", "name": "data.table", @@ -192,7 +205,7 @@ }, "sameAs": "https://CRAN.R-project.org/package=data.table" }, - "4": { + "5": { "@type": "SoftwareApplication", "identifier": "Hmisc", "name": "Hmisc", @@ -204,7 +217,7 @@ }, "sameAs": "https://CRAN.R-project.org/package=Hmisc" }, - "5": { + "6": { "@type": "SoftwareApplication", "identifier": "Rdpack", "name": "Rdpack", @@ -217,19 +230,32 @@ }, "sameAs": "https://CRAN.R-project.org/package=Rdpack" }, - "6": { + "7": { + "@type": "SoftwareApplication", + "identifier": "rlang", + "name": "rlang", + "version": ">= 1.0.0", + "provider": { + "@id": "https://cran.r-project.org", + "@type": "Organization", + "name": "Comprehensive R Archive Network (CRAN)", + "url": "https://cran.r-project.org" + }, + "sameAs": "https://CRAN.R-project.org/package=rlang" + }, + "8": { "@type": "SoftwareApplication", "identifier": "stats", "name": "stats" }, - "7": { + "9": { "@type": "SoftwareApplication", "identifier": "utils", "name": "utils" }, "SystemRequirements": null }, - "fileSize": "3272.323KB", + "fileSize": "3152.565KB", "releaseNotes": "https://github.com/ipea/accessibility/blob/main/NEWS.md", "readme": "https://github.com/ipea/accessibility/blob/main/README.md", "contIntegration": ["https://github.com/ipea/accessibility/actions", "https://app.codecov.io/gh/ipea/accessibility?branch=main"], diff --git a/tests/testthat/test-concentration_index.R b/tests/testthat/test-concentration_index.R index a33256c..d2b8405 100644 --- a/tests/testthat/test-concentration_index.R +++ b/tests/testthat/test-concentration_index.R @@ -194,7 +194,9 @@ test_that("concentration index is correctly calculated", { expected_result <- data.table::data.table( mode = c("transit", "transit2"), - concentration_index = c(-0.1314, -0.8282) + # the correction uses the bounds of the whole distribution, not of each + # group + concentration_index = c(-0.1314, -0.5003) ) expect_identical(result, expected_result) }) diff --git a/tests/testthat/test-constrained-essentials.R b/tests/testthat/test-constrained-essentials.R index fd30247..23e5ed2 100644 --- a/tests/testthat/test-constrained-essentials.R +++ b/tests/testthat/test-constrained-essentials.R @@ -168,7 +168,9 @@ test_that("doubly-constrained: errors when totals mismatch", { supply = "jobs", detailed_results = TRUE, active = NULL - ), "sum of origins must equal the sum of destinations" + ), + # cli wraps long messages, so spaces may be line breaks + "sum of origins must equal the sum of[[:space:]]+destinations" ) }) diff --git a/tests/testthat/test-cumulative_cutoff.R b/tests/testthat/test-cumulative_cutoff.R index c0c4923..d2b9a6b 100644 --- a/tests/testthat/test-cumulative_cutoff.R +++ b/tests/testthat/test-cumulative_cutoff.R @@ -450,3 +450,23 @@ test_that("raises a clear error if id types are incompatible", { NA ) }) + +test_that("result is not keyed, even when missing ids are filled", { + toy_matrix <- data.table::data.table( + from_id = c("a", "a", "b"), + to_id = c("a", "b", "a"), + travel_time = c(0, 10, 50) + ) + toy_land_use <- data.table::data.table(id = c("a", "b"), jobs = c(10, 20)) + + # b reaches nothing within 30 minutes, so it's filled with 0 + + result <- tester( + toy_matrix, + toy_land_use, + cutoff = 30, + group_by = character(0) + ) + expect_identical(result$jobs, c(30, 0)) + expect_null(data.table::key(result)) +}) diff --git a/tests/testthat/test-cumulative_interval.R b/tests/testthat/test-cumulative_interval.R index 2001735..c0acc87 100644 --- a/tests/testthat/test-cumulative_interval.R +++ b/tests/testthat/test-cumulative_interval.R @@ -126,18 +126,18 @@ test_that("result has correct structure", { expect_true(ncol(result) == 3) expect_is(result$id, "character") expect_is(result$mode, "character") - expect_is(result$jobs, "integer") + expect_is(result$jobs, "numeric") result <- tester(opportunity = "schools") expect_true(ncol(result) == 3) expect_is(result$id, "character") expect_is(result$mode, "character") - expect_is(result$schools, "integer") + expect_is(result$schools, "numeric") suppressWarnings(result <- tester(group_by = character(0))) expect_true(ncol(result) == 2) expect_is(result$id, "character") - expect_is(result$jobs, "integer") + expect_is(result$jobs, "numeric") result <- tester( data.table::data.table( @@ -151,14 +151,14 @@ test_that("result has correct structure", { expect_true(nrow(result) == 0) expect_is(result$id, "character") expect_is(result$mode, "character") - expect_is(result$jobs, "integer") + expect_is(result$jobs, "numeric") result <- tester(interval = list(c(10, 30), c(20, 30))) expect_true(ncol(result) == 4) expect_is(result$id, "character") expect_is(result$mode, "character") expect_is(result$interval, "character") - expect_is(result$jobs, "integer") + expect_is(result$jobs, "numeric") }) test_that("input data sets remain unchanged", { @@ -194,7 +194,7 @@ test_that("active and passive accessibility is correctly calculated", { expected_result <- data.table::data.table( id = rep(selected_ids, 2), mode = rep(c("transit", "transit2"), each = 5), - jobs = rep(as.integer(c(82, 517, 517, 304, 0)), 2) + jobs = rep(c(82, 517, 517, 304, 0), 2) ) expect_identical(result, expected_result) @@ -208,7 +208,7 @@ test_that("active and passive accessibility is correctly calculated", { expected_result <- data.table::data.table( id = rep(selected_ids, 2), mode = rep(c("transit", "transit2"), each = 5), - population = rep(as.integer(c(1946, 3457, 2363, 4552, 0)), 2) + population = rep(c(1946.5, 3457.5, 2363, 4552, 0), 2) ) expect_identical(result, expected_result) @@ -224,8 +224,8 @@ test_that("active and passive accessibility is correctly calculated", { mode = rep(rep(c("transit", "transit2"), each = 5), 2), interval = rep(c("[40,45]", "[40,50]"), each = 10), jobs = c( - rep(as.integer(c(82, 517, 517, 304, 0)), 2), - rep(as.integer(c(82, 599, 599, 499, 0)), 2) + rep(c(82, 517, 517, 304, 0), 2), + rep(c(82, 599, 599, 499, 0), 2) ) ) expect_identical(result, expected_result) @@ -242,8 +242,8 @@ test_that("active and passive accessibility is correctly calculated", { mode = rep(rep(c("transit", "transit2"), each = 5), 2), interval = rep(c("[40,45]", "[40,50]"), each = 10), population = c( - rep(as.integer(c(1946, 3457, 2363, 4552, 0)), 2), - rep(as.integer(c(5404, 4552, 2363, 4552, 0)), 2) + rep(c(1946.5, 3457.5, 2363, 4552, 0), 2), + rep(c(5404, 4552, 2363, 4552, 0), 2) ) ) expect_identical(result, expected_result) @@ -270,7 +270,7 @@ test_that("summarizes the result according to summary function", { expected_result <- data.table::data.table( id = rep(selected_ids, 2), mode = rep(c("transit", "transit2"), each = 5), - jobs = rep(as.integer(c(82, 408, 408, 109, 0)), 2) + jobs = rep(c(82, 408, 408, 109, 0), 2) ) expect_identical(result, expected_result) @@ -283,7 +283,7 @@ test_that("summarizes the result according to summary function", { expected_result <- data.table::data.table( id = rep(selected_ids, 2), mode = rep(c("transit", "transit2"), each = 5), - jobs = rep(as.integer(c(82, 512, 508, 304, 0)), 2) + jobs = rep(c(82, 512.5, 508, 304, 0), 2) ) expect_identical(result, expected_result) }) @@ -309,7 +309,7 @@ test_that("accessibility is correctly calculated when ids are missing", { expected_result <- data.table::data.table( id = rep(selected_ids[order(selected_ids)], each = 2), mode = rep(c("transit", "transit2"), 5), - jobs = rep(as.integer(c(0, 0, 0, 0, 0)), 2) + jobs = rep(c(0, 0, 0, 0, 0), 2) ) expect_identical(result, expected_result) }) @@ -395,8 +395,8 @@ test_that("each interval is summarised over its own cutoffs only", { # uses 5 and 15 (a: mean(10, 30) = 20), whether requested alone or together result <- toy_tester(list(c(0, 20), c(5, 15))) - expect_identical(result[interval == "[0,20]"]$jobs, c(33L, 26L, 33L)) - expect_identical(result[interval == "[5,15]"]$jobs, c(20L, 25L, 30L)) + expect_equal(result[interval == "[0,20]"]$jobs, c(100 / 3, 80 / 3, 100 / 3)) + expect_equal(result[interval == "[5,15]"]$jobs, c(20, 25, 30)) expect_identical( result[interval == "[0,20]"]$jobs, @@ -434,11 +434,11 @@ test_that("fill_missing_ids arg works correctly", { result <- toy_tester(interval = c(0, 20)) expect_identical(result$id, c("a", "b", "c", "d")) - expect_identical(result$jobs, c(33L, 26L, 33L, 0L)) + expect_equal(result$jobs, c(100 / 3, 80 / 3, 100 / 3, 0)) result <- toy_tester(interval = c(0, 20), fill_missing_ids = FALSE) expect_identical(result$id, c("a", "b", "c")) - expect_identical(result$jobs, c(33L, 26L, 33L)) + expect_equal(result$jobs, c(100 / 3, 80 / 3, 100 / 3)) }) test_that("the upper limit of the interval is always considered", { @@ -462,5 +462,5 @@ test_that("the upper limit of the interval is always considered", { summary_function = mean, group_by = character(0) ) - expect_identical(result$jobs, c(32L, 27L, 32L)) + expect_equal(result$jobs, c(32.5, 27.5, 32.5)) }) diff --git a/tests/testthat/test-decay_logistic.R b/tests/testthat/test-decay_logistic.R index a3fed4b..647b00e 100644 --- a/tests/testthat/test-decay_logistic.R +++ b/tests/testthat/test-decay_logistic.R @@ -59,7 +59,7 @@ test_that("output fn returns empty numeric if receives empty numeric/integer", { test_that("raises an error if (cutoff, sd) pairs are duplicated", { expect_error( tester(c(30, 30), c(5, 5)), - regexp = "duplicated .cutoff, sd. pairs" + regexp = "duplicated[[:space:]]+.cutoff,[[:space:]]+sd.[[:space:]]+pairs" ) expect_error(tester(c(30, 30), c(5, 10)), NA) }) diff --git a/tests/testthat/test-decay_stepped.R b/tests/testthat/test-decay_stepped.R index 05a91cc..ee08340 100644 --- a/tests/testthat/test-decay_stepped.R +++ b/tests/testthat/test-decay_stepped.R @@ -93,11 +93,11 @@ test_that("output fn returns empty numeric if receives empty numeric/integer", { test_that("raises an error if (steps, weights) pairs are duplicated", { expect_error( tester(list(c(10, 20), c(10, 20)), list(c(0.5, 0), c(0.5, 0))), - regexp = "duplicated .steps, weights. pairs" + regexp = "duplicated[[:space:]]+.steps,[[:space:]]+weights.[[:space:]]+pairs" ) expect_error( tester(list(c(10L, 20L), c(10, 20)), list(c(0.5, 0), c(0.5, 0))), - regexp = "duplicated .steps, weights. pairs" + regexp = "duplicated[[:space:]]+.steps,[[:space:]]+weights.[[:space:]]+pairs" ) expect_error( tester(list(c(10, 20), c(10, 20)), list(c(0.5, 0), c(0.6, 0))), diff --git a/tests/testthat/test-gravity.R b/tests/testthat/test-gravity.R index 4df5a46..13aba05 100644 --- a/tests/testthat/test-gravity.R +++ b/tests/testthat/test-gravity.R @@ -328,7 +328,7 @@ test_that("raises an error if travel_matrix contains negative travel costs", { expect_error( tester(travel_matrix = negative_matrix), - regexp = "Travel costs must not be negative" + regexp = "Travel costs must not[[:space:]]+be[[:space:]]+negative" ) # NA costs are still allowed diff --git a/tests/testthat/test-merge_by_reference.R b/tests/testthat/test-merge_by_reference.R new file mode 100644 index 0000000..03fcce5 --- /dev/null +++ b/tests/testthat/test-merge_by_reference.R @@ -0,0 +1,40 @@ +tester <- function(data, + land_use_data, + opportunity = "jobs", + left_df_idcol = "to_id") { + merge_by_reference(data, land_use_data, opportunity, left_df_idcol) +} + +test_that("matches character ids and gives NA to missing ids", { + data <- data.table::data.table( + from_id = c("a", "a", "b", "b"), + to_id = c("a", "c", "b", "z") + ) + land_use_data <- data.table::data.table( + id = c("c", "b", "a"), + jobs = c(30, 20, 10) + ) + original_land_use <- data.table::copy(land_use_data) + + tester(data, land_use_data) + expect_identical(data$jobs, c(10, 30, 20, NA)) + + tester(data, land_use_data, "jobs", left_df_idcol = "from_id") + expect_identical(data$jobs, c(10, 10, 20, 20)) + + expect_identical(land_use_data, original_land_use) +}) + +test_that("matches numeric and factor ids", { + data <- data.table::data.table(to_id = c(2L, 1L, 3L)) + land_use_data <- data.table::data.table(id = c(1, 2), jobs = c(5L, 7L)) + + tester(data, land_use_data) + expect_identical(data$jobs, c(7L, 5L, NA)) + + data <- data.table::data.table(to_id = factor(c("b", "a"))) + land_use_data <- data.frame(id = c("a", "b"), jobs = c(1, 2)) + + tester(data, land_use_data) + expect_identical(data$jobs, c(2, 1)) +}) From 1914c5a45cf5304ecbe4ea2bc51bf7515ea99019 Mon Sep 17 00:00:00 2001 From: rafapereirabr Date: Tue, 20 Oct 2026 20:41:04 -0300 Subject: [PATCH 17/31] optimize gravity --- R/decay_logistic.R | 9 ++------- R/decay_stepped.R | 21 +++++++++++---------- 2 files changed, 13 insertions(+), 17 deletions(-) diff --git a/R/decay_logistic.R b/R/decay_logistic.R index afd9449..a0ed528 100644 --- a/R/decay_logistic.R +++ b/R/decay_logistic.R @@ -14,8 +14,7 @@ #' that approximate binary decay, while higher values tend to linearize the #' decay. #' -#' @return A `function` that takes a generic travel cost vector (`numeric`) as -#' input and returns a vector of weights (`numeric`). +#' @template return_decay_function #' #' @details When using a function created with `decay_logistic()`, the output is #' named after the combination of cutoffs (`"c"`) and standard deviations @@ -65,11 +64,7 @@ decay_logistic <- function(cutoff, sd) { median = cutoff, std_dev = sd, FUN = function(median, std_dev) { - vapply( - travel_cost, - function(x) g(0, median, std_dev) / g(x, median, std_dev), - numeric(1) - ) + g(0, median, std_dev) / g(travel_cost, median, std_dev) }, SIMPLIFY = FALSE ) diff --git a/R/decay_stepped.R b/R/decay_stepped.R index 5fe5082..995e3b0 100644 --- a/R/decay_stepped.R +++ b/R/decay_stepped.R @@ -16,8 +16,7 @@ #' value at the actual step, not right after it. Please see the illustrative #' examples for effects of this assumption on the results. #' -#' @return A `function` that takes a generic travel cost vector (`numeric`) as -#' an input and returns a vector of weights (`numeric`). +#' @template return_decay_function #' #' @details When both `steps` and `weights` parameters are given `list`s, their #' content are matched element-wise to define each stepped weighting function @@ -82,14 +81,16 @@ decay_stepped <- function(steps, weights) { stps = steps, wghts = weights, FUN = function(stps, wghts) { - vapply( - travel_cost, - function(x) { - if (is.na(x)) return(NA_real_) - wghts[max(which(x >= stps))] - }, - numeric(1) - ) + # findInterval() returns the index of the last step lower than or equal + # to each cost. negative costs (index 0) get NA, as NA costs do + + step_index <- findInterval(travel_cost, stps) + step_index[step_index == 0L] <- NA_integer_ + + weights_by_cost <- wghts[step_index] + names(weights_by_cost) <- names(travel_cost) + + weights_by_cost }, SIMPLIFY = FALSE ) From ddc440b08a55e89b88e400311ee92ff8fd395da6 Mon Sep 17 00:00:00 2001 From: rafapereirabr Date: Tue, 20 Oct 2026 20:46:26 -0300 Subject: [PATCH 18/31] docs --- DESCRIPTION | 4 +- NEWS.md | 13 ++-- R/assert_and_assign.R | 3 +- R/balancing_cost.R | 23 ++++--- R/concentration_index.R | 46 +++++++------- R/constrained_accessibility.R | 4 +- R/cost_to_closest.R | 8 ++- R/cumulative_cutoff.R | 6 +- R/cumulative_interval.R | 4 +- R/fgt_poverty.R | 48 ++++++++------- R/floating_catchment_area.R | 3 +- R/gini_index.R | 36 +++++------ R/gravity.R | 2 + R/palma_ratio.R | 20 ++++--- R/spatial_availability.R | 7 ++- R/theil_t.R | 60 +++++++++---------- README.Rmd | 4 +- _pkgdown.yml | 4 +- codemeta.json | 4 +- inst/REFERENCES.bib | 4 +- man/accessibility.Rd | 2 +- man/balancing_cost.Rd | 34 +++++------ man/concentration_index.Rd | 5 +- man/constrained_accessibility.Rd | 15 ++--- man/cost_to_closest.Rd | 14 +++-- man/cumulative_cutoff.Rd | 8 +-- man/cumulative_interval.Rd | 6 +- man/decay_logistic.Rd | 3 +- man/decay_stepped.Rd | 3 +- man/fgt_poverty.Rd | 16 +++-- man/floating_catchment_area.Rd | 16 ++--- man/gini_index.Rd | 3 +- man/gravity.Rd | 19 ++++-- man/palma_ratio.Rd | 3 +- man/roxygen/templates/active.R | 2 +- man/roxygen/templates/arguments_fca.R | 4 -- man/roxygen/templates/decay_function.R | 7 ++- .../templates/fill_missing_ids_combinations.R | 4 +- man/roxygen/templates/input_data.R | 5 -- man/roxygen/templates/opportunity_access.R | 2 +- man/spatial_availability.Rd | 20 ++++--- man/theil_t.Rd | 5 +- vignettes/accessibility.Rmd | 4 +- vignettes/decay_functions.Rmd | 8 +-- vignettes/inequality_and_poverty.Rmd | 6 +- 45 files changed, 280 insertions(+), 237 deletions(-) delete mode 100644 man/roxygen/templates/arguments_fca.R delete mode 100644 man/roxygen/templates/input_data.R diff --git a/DESCRIPTION b/DESCRIPTION index 9d161ec..d57308f 100644 --- a/DESCRIPTION +++ b/DESCRIPTION @@ -12,13 +12,13 @@ Authors@R: c( person("Joey", "Reid", role = "ctb"), person("Ipea - Institute for Applied Economic Research", role = c("cph", "fnd")) ) -Description: A set of fast and convenient functions to help conducting +Description: A set of fast and convenient functions to help conduct accessibility analyses. Given a pre-computed travel cost matrix and a land use dataset (containing for example the location of jobs, healthcare and population), the package allows one to calculate accessibility levels, and accessibility poverty and inequality. The package covers the majority of the most commonly used accessibility measures (such as cumulative opportunities, - gravity-based and floating catchment areas methods), some cutting edge + gravity-based and floating catchment area methods), some cutting edge measures proposed in the literature (e.g. balancing cost and constrained accessibility) as well as the most frequently used inequality and poverty metrics (such as the Palma ratio, the concentration and Theil indices and diff --git a/NEWS.md b/NEWS.md index 6e00780..a081b94 100644 --- a/NEWS.md +++ b/NEWS.md @@ -124,6 +124,9 @@ large travel matrices, with identical results. If `travel_matrix` already has a column named like the opportunity column, that column is now fully replaced by the values in `land_use_data`. +- The functions created by `decay_logistic()` and `decay_stepped()` are now + vectorised instead of looping over each travel cost, which makes them about + 50 and 200 times faster on large travel matrices, with identical results. - Errors and warnings raised by the package now use `{cli}`, with consistent formatting of argument names and values. Long lists of ids or columns in messages are abbreviated. `{cli}` and `{rlang}` are new dependencies. @@ -144,8 +147,8 @@ constrained accessibility measures proposed in Soukhov et al. (2025). ## Minor changes -- The function `fgt_poverty()` has a new argument `poor_below_threshold`. when -set to `TRUE` (default,) the observations below the poverty line are considered +- The function `fgt_poverty()` has a new argument `poor_below_threshold`. When +set to `TRUE` (default) the observations below the poverty line are considered to be poor. This is the correct approach for primal accessibility measures (e.g. cumulative accessibility). If `FALSE`, then observations above the poverty line are considered to be poor. This is the correct approach for dual accessibility @@ -176,7 +179,7 @@ and 2 do not have an upper bound. Closes [#63](https://github.com/ipea/accessibi - New parameter `detailed_results` to `spatial_availability()`, used to specify whether results should be aggregated by origin-destination pair or by origin. - When aggregation by origin-destination pair, the output also includes the + When aggregating by origin-destination pair, the output also includes the demand, impedance and combined balancing factors used to calculate spatial availability. @@ -227,10 +230,10 @@ and 2 do not have an upper bound. Closes [#63](https://github.com/ipea/accessibi # accessibility 1.0.0 -The package has been to tremendous changes. Basically, there's not a single +The package has been through tremendous changes. Basically, there's not a single part of it that remained untouched: documentation, vignettes, function names, parameter names, extra functionality, performance improvements, etc. While it -is impossible to highlight everything that has been done, we'll try to summary +is impossible to highlight everything that has been done, we'll try to summarize some of the key points in the following topics. ## Breaking changes diff --git a/R/assert_and_assign.R b/R/assert_and_assign.R index 8f2cc44..e40478c 100644 --- a/R/assert_and_assign.R +++ b/R/assert_and_assign.R @@ -439,8 +439,7 @@ assert_detailed_fill_missing_ids <- function(fill_missing_ids, cli::cli_warn( "Found non-default {.arg fill_missing_ids} (FALSE) with {.code detailed_results = TRUE}. Please note that - {.arg fill_missing_ids} does not affect the detailed spatial - availability output." + {.arg fill_missing_ids} does not affect the detailed output." ) } diff --git a/R/balancing_cost.R b/R/balancing_cost.R index adea14b..e6d6747 100644 --- a/R/balancing_cost.R +++ b/R/balancing_cost.R @@ -14,26 +14,33 @@ #' @template demand #' @param cost_increment A number. The cost increment that should be used when #' defining the travel cost distribution from which the potential balancing -#' costs will be picked. For example, an increment of 1 tends to suitable for +#' costs will be picked. For example, an increment of 1 tends to be suitable for #' travel time distributions, meaning that the function will first check if #' any origins reach their balancing cost with a travel time of 0 minutes, -#' then 1 minute, 2 minutes, 3, 4, ..., etc. A increment of 1 might be too big +#' then 1 minute, 2 minutes, 3, 4, ..., etc. An increment of 1 might be too big #' for a distribution of monetary costs, on the other hand, which could #' possibly benefit from a smaller increment of 0.05, for example, resulting #' in the function looking for balancing costs first at a cost of 0, then #' 0.05, 0.10, ..., etc. Defaults to 1. #' @template group_by -#' @template fill_missing_ids_combinations +#' @param fill_missing_ids A `logical`. Origins that never reach as many +#' opportunities as there are people residing in them, as well as (when +#' `group_by` is not empty) combinations of groups and origins missing from +#' the travel matrix, would not be included in the output. When `TRUE` (the +#' default), the function identifies which combinations would be left out and +#' fills their respective balancing costs with `NA`, which incurs a +#' performance penalty. #' -#' @template return_accessibility -#' @return A data frame containing the accessibility estimates for each origin -#' in the travel matrix. Origins marked with a `NA` balancing cost never reach -#' as many opportunities as there is people residing in them, given the -#' specified travel matrix. +#' @return A data frame containing the balancing cost of each origin in the +#' travel matrix. Origins with a `NA` balancing cost never reach as many +#' opportunities as there are people residing in them, given the specified +#' travel matrix. #' #' @references #' \insertAllCited{} #' +#' @family accessibility measures +#' #' @examplesIf identical(tolower(Sys.getenv("NOT_CRAN")), "true") #' data_dir <- system.file("extdata", package = "accessibility") #' travel_matrix <- readRDS(file.path(data_dir, "travel_matrix.rds")) diff --git a/R/concentration_index.R b/R/concentration_index.R index 8d45716..d7e4361 100644 --- a/R/concentration_index.R +++ b/R/concentration_index.R @@ -1,7 +1,7 @@ #' Concentration Index #' #' Calculates the Concentration Index (CI) of a given accessibility -#' distribution. This measures estimates the extent to which accessibility +#' distribution. This measure estimates the extent to which accessibility #' inequalities are systematically associated with individuals' socioeconomic #' levels. CI values can theoretically vary between -1 and +1 (when all #' accessibility is concentrated in the most or in the least disadvantaged @@ -30,7 +30,7 @@ #' #' @template return_inequality #' -#' @family inequality +#' @family equity #' #' @references #' \insertAllCited{} @@ -59,13 +59,15 @@ #' ci #' #' @export -concentration_index <- function(accessibility_data, - sociodemographic_data, - opportunity, - population, - income, - type, - group_by = character(0)) { +concentration_index <- function( + accessibility_data, + sociodemographic_data, + opportunity, + population, + income, + type, + group_by = character(0) +) { checkmate::assert_string(opportunity) checkmate::assert_string(population) checkmate::assert_string(income) @@ -127,8 +129,7 @@ concentration_index <- function(accessibility_data, .income_colname <- income data[, .block_share := sum(pop_share), by = c(.groups, income)] - data[ - , + data[, fractional_rank := { first_in_block <- !duplicated(get(.income_colname)) cumsum(.block_share * first_in_block) - .block_share / 2 @@ -136,8 +137,7 @@ concentration_index <- function(accessibility_data, by = .groups ] - data[ - , + data[, avg_access := stats::weighted.mean( get(.opp_colname), w = get(.pop_colname) @@ -146,17 +146,17 @@ concentration_index <- function(accessibility_data, ] data[, diff_from_avg := get(.opp_colname) - avg_access] - data[ - , - cont_to_total := (fractional_rank - 0.5) * diff_from_avg * pop_share / + data[, + cont_to_total := (fractional_rank - 0.5) * + diff_from_avg * + pop_share / avg_access ] if (nrow(data) == 0 && identical(group_by, character(0))) { ci <- data.table::data.table(concentration_index = numeric(0)) } else if (type == "standard") { - ci <- data[ - , + ci <- data[, .(concentration_index = 2 * sum(cont_to_total)), by = .groups ] @@ -173,8 +173,7 @@ concentration_index <- function(accessibility_data, # when access is constant (upper == lower) there is no inequality, and the # factor would be Inf (0 * Inf = NaN). it's set to 0 so the index is 0 - data[ - , + data[, correction_factor := data.table::fifelse( upper > lower, 4 * avg_access / (upper - lower), @@ -182,14 +181,15 @@ concentration_index <- function(accessibility_data, ) ] - ci <- data[ - , + ci <- data[, .(concentration_index = 2 * sum(cont_to_total * correction_factor)), by = .groups ] } - if (exists("original_class")) class(ci) <- original_class + if (exists("original_class")) { + class(ci) <- original_class + } return(ci) } diff --git a/R/constrained_accessibility.R b/R/constrained_accessibility.R index 7df790a..8b23e8b 100644 --- a/R/constrained_accessibility.R +++ b/R/constrained_accessibility.R @@ -40,7 +40,7 @@ #' #' Allocates the system-wide total proportionally based on travel impedance #' between origins and destinations. This measure uses the logic of a -#' total ~(or 'unconstrained' by Wilson's terms)~ constraint. +#' total (or 'unconstrained' by Wilson's terms) constraint. #' #' Use this measure when the total quantity of **supply** OR **demand** in the #' system is known and representing accessibility as a proportion of this total @@ -156,7 +156,7 @@ #' to match; otherwise, the model will not converge. #' - `active` must be `NULL`. Since supply must match demand, their units are #' the same and there is no distinction between 'active' and 'passive' notions. -#' - Only accepts`detailed_results = TRUE`. +#' - Only accepts `detailed_results = TRUE`. #' #' ### Interpretation: #' - Results include OD-level flows (`flow`) along with balancing factors (`A_i`, diff --git a/R/cost_to_closest.R b/R/cost_to_closest.R index 3fb8afa..09c3e20 100644 --- a/R/cost_to_closest.R +++ b/R/cost_to_closest.R @@ -18,13 +18,15 @@ #' if they cannot reach the specified amount of opportunities across all #' destinations they can reach. For example, estimating the minimum travel #' time that an origin that can only reach 4 opportunities takes to reach 5 -#' opportunities resulting in such origin not being included in the output. +#' opportunities results in such origin not being included in the output. #' When `TRUE` (the default), the function identifies which ids would be left -#' out from the output and fill their respective minimum travel costs with -#' `Inf`, which incurs in a performance penalty. +#' out from the output and fills their respective minimum travel costs with +#' `Inf`, which incurs a performance penalty. #' #' @template return_accessibility #' +#' @family accessibility measures +#' #' @examplesIf identical(tolower(Sys.getenv("NOT_CRAN")), "true") #' data_dir <- system.file("extdata", package = "accessibility") #' travel_matrix <- readRDS(file.path(data_dir, "travel_matrix.rds")) diff --git a/R/cumulative_cutoff.R b/R/cumulative_cutoff.R index cf9d8fa..56c693f 100644 --- a/R/cumulative_cutoff.R +++ b/R/cumulative_cutoff.R @@ -13,7 +13,7 @@ #' @param cutoff Either a `numeric` vector or a list of `numeric` vectors, one #' for each cost specified in `travel_cost`. The travel cost cutoffs to #' consider when calculating accessibility levels. If a list, the function -#' finds every single possible cutoff combination and use them to calculate +#' finds every single possible cutoff combination and uses them to calculate #' accessibility (e.g. if one specifies that travel time cutoffs should be 30 #' and 60 minutes and that monetary cost cutoffs should be 5 and 10 dollars, #' the output includes accessibility estimates limited at 30 min & 5 dollars, @@ -26,13 +26,13 @@ #' @template group_by #' @template active #' @param fill_missing_ids A `logical`. Calculating cumulative accessibility may -#' result in missing ids if the they cannot reach any of the destinations +#' result in missing ids if they cannot reach any of the destinations #' within the specified travel cost cutoff. For example, using a travel time #' cutoff of 20 minutes, when estimating the accessibility of origin `A` that #' can only reach destinations with more than 40 minutes results in id `A` #' not being included in the output. When `TRUE` (the default), the function #' identifies which origins would be left out and fills their respective -#' accessibility values with 0, which incurs in a performance penalty. +#' accessibility values with 0, which incurs a performance penalty. #' #' @template return_accessibility #' diff --git a/R/cumulative_interval.R b/R/cumulative_interval.R index fa05c3a..bc15398 100644 --- a/R/cumulative_interval.R +++ b/R/cumulative_interval.R @@ -3,7 +3,7 @@ #' Calculates the average or median number of opportunities that can be reached #' considering multiple maximum travel cost thresholds within a given travel #' cost interval specified by the user. The time interval cumulative -#' accessibility measures was originally proposed by +#' accessibility measure was originally proposed by #' \insertCite{tomasiello2023time;textual}{accessibility}. #' @template description_generic_cost #' @@ -22,7 +22,7 @@ #' @param summary_function A function. This function is used to summarize a #' distribution of accessibility estimates within a travel cost interval as a #' single value. Can be any function that takes an arbitrary number of -#' numeric values as as input and returns a single number as output. Defaults +#' numeric values as input and returns a single number as output. Defaults #' to [stats::median()]. #' @template group_by #' @template active diff --git a/R/fgt_poverty.R b/R/fgt_poverty.R index 78b46f3..18b6dd7 100644 --- a/R/fgt_poverty.R +++ b/R/fgt_poverty.R @@ -13,16 +13,16 @@ #' others specified in `opportunity`. #' @template sociodem_data_without_income #' @param opportunity A string. The name of the column in `accessibility_data` -#' with the accessibility levels to be considerend when calculating +#' with the accessibility levels to be considered when calculating #' accessibility poverty. #' @param population A string. The name of the column in `sociodemographic_data` #' with the number of people in each cell. Used to weigh accessibility levels #' when calculating poverty. #' @param poverty_line A `numeric`. The poverty line below which individuals are #' considered to be in accessibility poverty. -#' @param poor_below_threshold Logic. If `TRUE` (default,) the observations below +#' @param poor_below_threshold A logical. If `TRUE` (default) the observations below #' the poverty line are considered to be poor. This is the correct approach -#' for primal accessibility measures (e.g.cumulative accessibility). If +#' for primal accessibility measures (e.g. cumulative accessibility). If #' `FALSE`, then observations above the poverty line are considered to be #' poor. This is the correct approach for dual accessibility measures (e.g. #' travel time to the closest facility). When set to `FALSE`, FGT 1 and 2 @@ -44,7 +44,7 @@ #' - with \eqn{\alpha = 0} (FGT0) the measure captures the *extent* of poverty #' as a simple headcount - i.e. the proportion of people below the poverty line; #' -#' - with \eqn{\alpha = 1} (FGT1) the measure, also know as the "poverty gap +#' - with \eqn{\alpha = 1} (FGT1) the measure, also known as the "poverty gap #' index", captures the *severity* of poverty as the average percentage distance #' between the poverty line and the accessibility of individuals below the #' poverty line; @@ -61,6 +61,8 @@ #' @references #' \insertAllCited{} #' +#' @family equity +#' #' @examplesIf identical(tolower(Sys.getenv("NOT_CRAN")), "true") #' data_dir <- system.file("extdata", package = "accessibility") #' travel_matrix <- readRDS(file.path(data_dir, "travel_matrix.rds")) @@ -84,14 +86,15 @@ #' poverty #' #' @export -fgt_poverty <- function(accessibility_data, - sociodemographic_data, - opportunity, - population, - poverty_line, - poor_below_threshold = TRUE, - group_by = character(0)) { - +fgt_poverty <- function( + accessibility_data, + sociodemographic_data, + opportunity, + population, + poverty_line, + poor_below_threshold = TRUE, + group_by = character(0) +) { checkmate::assert_string(opportunity) checkmate::assert_string(population) checkmate::assert_number(poverty_line, lower = 0, finite = TRUE) @@ -137,7 +140,6 @@ fgt_poverty <- function(accessibility_data, # primal accessibility --------------------------------------------------------- if (isTRUE(poor_below_threshold)) { - data[get(.opp_colname) >= poverty_line, c(".fgt0", ".fgt1", ".fgt2") := 0] data[ @@ -147,9 +149,9 @@ fgt_poverty <- function(accessibility_data, data[ get(.opp_colname) < poverty_line, `:=`( - .fgt0 = .norm_opp_shortfall ^ 0, - .fgt1 = .norm_opp_shortfall ^ 1, - .fgt2 = .norm_opp_shortfall ^ 2 + .fgt0 = .norm_opp_shortfall^0, + .fgt1 = .norm_opp_shortfall^1, + .fgt2 = .norm_opp_shortfall^2 ) ] } @@ -167,14 +169,13 @@ fgt_poverty <- function(accessibility_data, data[ get(.opp_colname) > poverty_line, `:=`( - .fgt0 = .norm_opp_shortfall ^ 0, - .fgt1 = .norm_opp_shortfall ^ 1, - .fgt2 = .norm_opp_shortfall ^ 2 + .fgt0 = .norm_opp_shortfall^0, + .fgt1 = .norm_opp_shortfall^1, + .fgt2 = .norm_opp_shortfall^2 ) ] } - if (nrow(data) == 0 && identical(group_by, character(0))) { fgt <- data.table::data.table( FGT0 = numeric(0), @@ -182,8 +183,7 @@ fgt_poverty <- function(accessibility_data, FGT2 = numeric(0) ) } else { - fgt <- data[ - , + fgt <- data[, .( FGT0 = stats::weighted.mean(x = .fgt0, w = get(.pop_colname)), FGT1 = stats::weighted.mean(x = .fgt1, w = get(.pop_colname)), @@ -193,7 +193,9 @@ fgt_poverty <- function(accessibility_data, ] } - if (exists("original_class")) class(fgt) <- original_class + if (exists("original_class")) { + class(fgt) <- original_class + } return(fgt[]) } diff --git a/R/floating_catchment_area.R b/R/floating_catchment_area.R index 40b701b..f79b1fc 100644 --- a/R/floating_catchment_area.R +++ b/R/floating_catchment_area.R @@ -17,7 +17,8 @@ #' @template group_by #' @template fill_missing_ids_combinations #' -#' @template return_accessibility +#' @return A data frame containing the accessibility estimates for each origin +#' in the travel matrix. #' #' @section Details: #' The package currently includes two built-in FCA measures: diff --git a/R/gini_index.R b/R/gini_index.R index a5f1955..adcd3cc 100644 --- a/R/gini_index.R +++ b/R/gini_index.R @@ -10,7 +10,7 @@ #' #' @template return_inequality #' -#' @family inequality +#' @family equity #' #' @examplesIf identical(tolower(Sys.getenv("NOT_CRAN")), "true") #' data_dir <- system.file("extdata", package = "accessibility") @@ -34,11 +34,13 @@ #' gini #' #' @export -gini_index <- function(accessibility_data, - sociodemographic_data, - opportunity, - population, - group_by = character(0)) { +gini_index <- function( + accessibility_data, + sociodemographic_data, + opportunity, + population, + group_by = character(0) +) { checkmate::assert_string(opportunity) checkmate::assert_string(population) assert_access_group_by(group_by) @@ -81,16 +83,15 @@ gini_index <- function(accessibility_data, data.table::setorderv(data, cols = c(opportunity, group_by)) - data[ - , + data[, .prop_pop := get(.pop_colname) / sum(get(.pop_colname)), by = .groups ] - data[ - , + data[, .cum_total_access := cumsum( as.numeric(get(.pop_colname) * get(.opp_colname)) - ) / sum(get(.pop_colname) * get(.opp_colname)), + ) / + sum(get(.pop_colname) * get(.opp_colname)), by = .groups ] @@ -102,8 +103,7 @@ gini_index <- function(accessibility_data, # (otherwise we would have a trapezoid in the first segment, whereas it should # always be a triangle) - data[ - , + data[, .small_edge := data.table::shift(.cum_total_access, fill = 0), by = .groups ] @@ -114,13 +114,15 @@ gini_index <- function(accessibility_data, if (nrow(data) == 0 && identical(group_by, character(0))) { gini <- data.table::data.table(gini_index = numeric(0)) } else { - gini <- data[ - , - .(gini_index = 1 - 2 * sum(.area_under_curve)), by = .groups + gini <- data[, + .(gini_index = 1 - 2 * sum(.area_under_curve)), + by = .groups ] } - if (exists("original_class")) class(gini) <- original_class + if (exists("original_class")) { + class(gini) <- original_class + } return(gini) } diff --git a/R/gravity.R b/R/gravity.R index 36972a0..0211afd 100644 --- a/R/gravity.R +++ b/R/gravity.R @@ -15,6 +15,8 @@ #' #' @template return_accessibility #' +#' @family accessibility measures +#' #' @examplesIf identical(tolower(Sys.getenv("NOT_CRAN")), "true") #' data_dir <- system.file("extdata", package = "accessibility") #' travel_matrix <- readRDS(file.path(data_dir, "travel_matrix.rds")) diff --git a/R/palma_ratio.R b/R/palma_ratio.R index 27993c3..b084d40 100644 --- a/R/palma_ratio.R +++ b/R/palma_ratio.R @@ -19,7 +19,7 @@ #' #' @template return_inequality #' -#' @family inequality +#' @family equity #' #' @examplesIf identical(tolower(Sys.getenv("NOT_CRAN")), "true") #' data_dir <- system.file("extdata", package = "accessibility") @@ -44,12 +44,14 @@ #' palma #' #' @export -palma_ratio <- function(accessibility_data, - sociodemographic_data, - opportunity, - population, - income, - group_by = character(0)) { +palma_ratio <- function( + accessibility_data, + sociodemographic_data, + opportunity, + population, + income, + group_by = character(0) +) { checkmate::assert_string(opportunity) checkmate::assert_string(population) checkmate::assert_string(income) @@ -159,7 +161,9 @@ palma_ratio <- function(accessibility_data, data.table::setkeyv(palma, NULL) - if (exists("original_class")) class(palma) <- original_class + if (exists("original_class")) { + class(palma) <- original_class + } return(palma[]) } diff --git a/R/spatial_availability.R b/R/spatial_availability.R index 4b2b6df..ea180f2 100644 --- a/R/spatial_availability.R +++ b/R/spatial_availability.R @@ -1,6 +1,6 @@ #' Spatial availability #' -#' Calculates spatial availability, an accessibility measured proposed by +#' Calculates spatial availability, an accessibility measure proposed by #' \insertCite{soukhov2023introducing;textual}{accessibility} that takes into #' account competition effects. The accessibility levels that result from using #' this measure are proportional both to the demand in each origin and to the @@ -27,7 +27,10 @@ #' availability. Please note that the argument `fill_missing_ids` #' does not affect the output when `detailed_results` is `TRUE`. #' -#' @template return_accessibility +#' @return A data frame containing the spatial availability estimates for +#' each origin in the travel matrix. When `detailed_results` is `TRUE`, the +#' estimates are given for each origin-destination pair instead, alongside the +#' balancing factors used to calculate them. #' #' @references #' \insertAllCited{} diff --git a/R/theil_t.R b/R/theil_t.R index 4185843..1c4f4f0 100644 --- a/R/theil_t.R +++ b/R/theil_t.R @@ -5,7 +5,7 @@ #' levels) to the natural log of *n*, in which *n* is the number of individuals #' in the accessibility dataset. If the individuals can be classified into #' mutually exclusive and completely exhaustive groups, the index can be -#' decomposed into a between-groups inequaliy component and a within-groups +#' decomposed into a between-groups inequality component and a within-groups #' component. #' #' Observations with no access contribute 0 to the index, following the usual @@ -34,7 +34,7 @@ #' between-groups component and another listing the contribution of each group #' to the within-groups component. #' -#' @family inequality +#' @family equity #' #' @examplesIf identical(tolower(Sys.getenv("NOT_CRAN")), "true") #' data_dir <- system.file("extdata", package = "accessibility") @@ -80,12 +80,14 @@ #' ti #' #' @export -theil_t <- function(accessibility_data, - sociodemographic_data, - opportunity, - population, - socioeconomic_groups = NULL, - group_by = character(0)) { +theil_t <- function( + accessibility_data, + sociodemographic_data, + opportunity, + population, + socioeconomic_groups = NULL, + group_by = character(0) +) { checkmate::assert_string(opportunity) checkmate::assert_string(population) checkmate::assert_string(socioeconomic_groups, null.ok = TRUE) @@ -153,8 +155,7 @@ theil_t <- function(accessibility_data, theil_without_groups <- function(data, opportunity, population, group_by) { .groups <- group_by - data[ - , + data[, avg_access := stats::weighted.mean( get(..opportunity), w = get(..population) @@ -176,12 +177,14 @@ theil_without_groups <- function(data, opportunity, population, group_by) { return(theil_index) } -theil_with_groups <- function(data, - sociodemographic_data, - opportunity, - population, - socioeconomic_groups, - group_by) { +theil_with_groups <- function( + data, + sociodemographic_data, + opportunity, + population, + socioeconomic_groups, + group_by +) { merge_by_reference( data, sociodemographic_data, @@ -192,16 +195,14 @@ theil_with_groups <- function(data, .groups <- group_by .socioecon_groups <- c(group_by, socioeconomic_groups) - data[ - , + data[, avg_access := stats::weighted.mean( get(..opportunity), w = get(..population) ), by = .groups ] - data[ - , + data[, group_avg_access := stats::weighted.mean( get(..opportunity), w = get(..population) @@ -209,8 +210,7 @@ theil_with_groups <- function(data, by = .socioecon_groups ] - summarized_data <- data[ - , + summarized_data <- data[, .( group_tot_access = as.numeric( sum(get(..opportunity) * get(..population)) @@ -226,8 +226,7 @@ theil_with_groups <- function(data, ), by = .socioecon_groups ] - summarized_data[ - , + summarized_data[, `:=`( group_avg_access = group_tot_access / group_tot_pop, access_share = group_tot_access / sum(group_tot_access), @@ -239,8 +238,7 @@ theil_with_groups <- function(data, # summary, containing total value and total within- and between-groups # inequality - summary <- summarized_data[ - , + summary <- summarized_data[, .( between_group = calc_theil_t( group_avg_access, @@ -267,13 +265,11 @@ theil_with_groups <- function(data, # decomposed within group within_group <- data.table::copy(summarized_data) - within_group[ - , + within_group[, value := within_contribution(group_theil_t, group_tot_access), by = .groups ] - within_group[ - , + within_group[, share_of_component := value / sum(value), by = .groups ] @@ -288,8 +284,7 @@ theil_with_groups <- function(data, # decomposed between group between_group <- data.table::copy(summarized_data) - between_group[ - , + between_group[, value := data.table::fifelse( access_share > 0, access_share * log(access_share / pop_share), @@ -328,4 +323,3 @@ within_contribution <- function(group_theil_t, group_tot_access) { data.table::fifelse(access_share > 0, group_theil_t * access_share, 0) } - diff --git a/README.Rmd b/README.Rmd index 452b360..4b09bc3 100644 --- a/README.Rmd +++ b/README.Rmd @@ -51,8 +51,8 @@ remotes::install_github("ipea/accessibility") ## Usage -This section aims to present a very brief overview of some of the packages' -features. Fore more details please read the vignettes: +This section aims to present a very brief overview of some of the package's +features. For more details please read the vignettes: - [Introduction to **accessibility**: calculating accessibility measures](https://ipea.github.io/accessibility/articles/accessibility.html) diff --git a/_pkgdown.yml b/_pkgdown.yml index 9744649..2849af9 100644 --- a/_pkgdown.yml +++ b/_pkgdown.yml @@ -21,12 +21,12 @@ reference: - title: "Decay functions" - contents: - starts_with("decay_") - - title: "Inequality" + - title: "Equity: Inequality" - contents: - concentration_index - gini_index - palma_ratio - theil_t - - title: "Poverty" + - title: "Equity: Poverty" - contents: - fgt_poverty diff --git a/codemeta.json b/codemeta.json index 892858e..e581bc0 100644 --- a/codemeta.json +++ b/codemeta.json @@ -2,7 +2,7 @@ "@context": "https://doi.org/10.5063/schema/codemeta-2.0", "@type": "SoftwareSourceCode", "identifier": "accessibility", - "description": "A set of fast and convenient functions to help conducting accessibility analyses. Given a pre-computed travel cost matrix and a land use dataset (containing for example the location of jobs, healthcare and population), the package allows one to calculate accessibility levels, and accessibility poverty and inequality. The package covers the majority of the most commonly used accessibility measures (such as cumulative opportunities, gravity-based and floating catchment areas methods), some cutting edge measures proposed in the literature (e.g. balancing cost and constrained accessibility) as well as the most frequently used inequality and poverty metrics (such as the Palma ratio, the concentration and Theil indices and the FGT family of measures).", + "description": "A set of fast and convenient functions to help conduct accessibility analyses. Given a pre-computed travel cost matrix and a land use dataset (containing for example the location of jobs, healthcare and population), the package allows one to calculate accessibility levels, and accessibility poverty and inequality. The package covers the majority of the most commonly used accessibility measures (such as cumulative opportunities, gravity-based and floating catchment area methods), some cutting edge measures proposed in the literature (e.g. balancing cost and constrained accessibility) as well as the most frequently used inequality and poverty metrics (such as the Palma ratio, the concentration and Theil indices and the FGT family of measures).", "name": "accessibility: Transport Accessibility Measures", "relatedLink": ["https://ipea.github.io/accessibility/", "https://CRAN.R-project.org/package=accessibility"], "codeRepository": "https://github.com/ipea/accessibility", @@ -255,7 +255,7 @@ }, "SystemRequirements": null }, - "fileSize": "3152.565KB", + "fileSize": "3154.045KB", "releaseNotes": "https://github.com/ipea/accessibility/blob/main/NEWS.md", "readme": "https://github.com/ipea/accessibility/blob/main/README.md", "contIntegration": ["https://github.com/ipea/accessibility/actions", "https://app.codecov.io/gh/ipea/accessibility?branch=main"], diff --git a/inst/REFERENCES.bib b/inst/REFERENCES.bib index 5661af4..b22839c 100644 --- a/inst/REFERENCES.bib +++ b/inst/REFERENCES.bib @@ -153,12 +153,13 @@ @article{tomasiello2023time @article{soukhov2025family, title={A family of accessibility measures derived from spatial interaction principles}, - author={Soukhov, Anastasia and Pereira, Rafael HM and Higgins, Christopher D and P{\'a}ez, Antonio}, + author={Soukhov, Anastasia and Pereira, Rafael H. M. and Higgins, Christopher D. and P{\'a}ez, Antonio}, journal={PLoS One}, volume={20}, number={11}, pages={e0335951}, year={2025}, + doi={10.1371/journal.pone.0335951}, publisher={Public Library of Science San Francisco, CA USA} } @@ -170,5 +171,6 @@ @article{wilson1971family number={1}, pages={1--32}, year={1971}, + doi={10.1068/a030001}, publisher={Sage Publications Sage UK: London, England} } diff --git a/man/accessibility.Rd b/man/accessibility.Rd index 910d851..7961609 100644 --- a/man/accessibility.Rd +++ b/man/accessibility.Rd @@ -8,7 +8,7 @@ \description{ \if{html}{\figure{logo.png}{options: style='float: right' alt='logo' width='120'}} -A set of fast and convenient functions to help conducting accessibility analyses. Given a pre-computed travel cost matrix and a land use dataset (containing for example the location of jobs, healthcare and population), the package allows one to calculate accessibility levels, and accessibility poverty and inequality. The package covers the majority of the most commonly used accessibility measures (such as cumulative opportunities, gravity-based and floating catchment areas methods), some cutting edge measures proposed in the literature (e.g. balancing cost and constrained accessibility) as well as the most frequently used inequality and poverty metrics (such as the Palma ratio, the concentration and Theil indices and the FGT family of measures). +A set of fast and convenient functions to help conduct accessibility analyses. Given a pre-computed travel cost matrix and a land use dataset (containing for example the location of jobs, healthcare and population), the package allows one to calculate accessibility levels, and accessibility poverty and inequality. The package covers the majority of the most commonly used accessibility measures (such as cumulative opportunities, gravity-based and floating catchment area methods), some cutting edge measures proposed in the literature (e.g. balancing cost and constrained accessibility) as well as the most frequently used inequality and poverty metrics (such as the Palma ratio, the concentration and Theil indices and the FGT family of measures). } \section{Usage}{ diff --git a/man/balancing_cost.Rd b/man/balancing_cost.Rd index 5a79e69..993c33a 100644 --- a/man/balancing_cost.Rd +++ b/man/balancing_cost.Rd @@ -40,10 +40,10 @@ considered.} \item{cost_increment}{A number. The cost increment that should be used when defining the travel cost distribution from which the potential balancing -costs will be picked. For example, an increment of 1 tends to suitable for +costs will be picked. For example, an increment of 1 tends to be suitable for travel time distributions, meaning that the function will first check if any origins reach their balancing cost with a travel time of 0 minutes, -then 1 minute, 2 minutes, 3, 4, ..., etc. A increment of 1 might be too big +then 1 minute, 2 minutes, 3, 4, ..., etc. An increment of 1 might be too big for a distribution of monetary costs, on the other hand, which could possibly benefit from a smaller increment of 0.05, for example, resulting in the function looking for balancing costs first at a cost of 0, then @@ -56,25 +56,19 @@ departure time column, that specifies the departure time of each entry in the data frame, passing \code{"departure_time"} to this parameter results in accessibility estimates grouped by origin and by departure time.} -\item{fill_missing_ids}{A \code{logical}. When calculating grouped accessibility -estimates (i.e. when \code{by_col} is not \code{NULL}), some combinations of groups -and origins may be missing. For example, if a single trip can depart from -origin \code{A} at 7:15am and reach destination \code{B} within 55 minutes, but no -trips departing from \code{A} at 7:30am can be completed at all, this second -combination will not be included in the output. When \code{TRUE} (the default), -the function identifies which combinations would be left out and fills -their respective accessibility values with 0, which incurs in a +\item{fill_missing_ids}{A \code{logical}. Origins that never reach as many +opportunities as there are people residing in them, as well as (when +\code{group_by} is not empty) combinations of groups and origins missing from +the travel matrix, would not be included in the output. When \code{TRUE} (the +default), the function identifies which combinations would be left out and +fills their respective balancing costs with \code{NA}, which incurs a performance penalty.} } \value{ -A data frame containing the accessibility estimates for each -origin/destination (depending if \code{active} is \code{TRUE} or \code{FALSE}) in the +A data frame containing the balancing cost of each origin in the +travel matrix. Origins with a \code{NA} balancing cost never reach as many +opportunities as there are people residing in them, given the specified travel matrix. - -A data frame containing the accessibility estimates for each origin -in the travel matrix. Origins marked with a \code{NA} balancing cost never reach -as many opportunities as there is people residing in them, given the -specified travel matrix. } \description{ Calculates the balancing cost measure, which is defined as the travel cost @@ -105,3 +99,9 @@ head(bc) \references{ \insertAllCited{} } +\seealso{ +Other accessibility measures: +\code{\link[=cost_to_closest]{cost_to_closest()}}, +\code{\link[=gravity]{gravity()}} +} +\concept{accessibility measures} diff --git a/man/concentration_index.Rd b/man/concentration_index.Rd index f9185ea..cf882df 100644 --- a/man/concentration_index.Rd +++ b/man/concentration_index.Rd @@ -25,7 +25,7 @@ Must contain the columns \code{id} and any others specified in \code{population} \code{income}.} \item{opportunity}{A string. The name of the column in \code{accessibility_data} -with the accessibility levels to be considerend when calculating inequality +with the accessibility levels to be considered when calculating inequality levels.} \item{population}{A string. The name of the column in \code{sociodemographic_data} @@ -59,7 +59,7 @@ A data frame containing the inequality estimates for the study area. } \description{ Calculates the Concentration Index (CI) of a given accessibility -distribution. This measures estimates the extent to which accessibility +distribution. This measure estimates the extent to which accessibility inequalities are systematically associated with individuals' socioeconomic levels. CI values can theoretically vary between -1 and +1 (when all accessibility is concentrated in the most or in the least disadvantaged @@ -98,6 +98,7 @@ ci } \seealso{ Other inequality: +\code{\link[=fgt_poverty]{fgt_poverty()}}, \code{\link[=gini_index]{gini_index()}}, \code{\link[=palma_ratio]{palma_ratio()}}, \code{\link[=theil_t]{theil_t()}} diff --git a/man/constrained_accessibility.Rd b/man/constrained_accessibility.Rd index 63d849d..b3fa46f 100644 --- a/man/constrained_accessibility.Rd +++ b/man/constrained_accessibility.Rd @@ -47,13 +47,14 @@ that will be considered.} \item{constraint}{A string. One of \code{"total"}, \code{"singly"}, or \code{"doubly"}. See Details section for more information.} -\item{decay_function}{A \code{fuction} that converts travel cost into an +\item{decay_function}{A \code{function} that converts travel cost into an impedance factor used to weight opportunities. This function should take a \code{numeric} vector and also return a \code{numeric} vector as output, with the same length as the input. For convenience, the package currently includes the following functions: \code{\link[=decay_binary]{decay_binary()}}, \code{\link[=decay_exponential]{decay_exponential()}}, -\code{\link[=decay_power]{decay_power()}} and \code{\link[=decay_stepped]{decay_stepped()}}. See the documentation of each decay -function for more details.} +\code{\link[=decay_linear]{decay_linear()}}, \code{\link[=decay_logistic]{decay_logistic()}}, \code{\link[=decay_power]{decay_power()}} and +\code{\link[=decay_stepped]{decay_stepped()}}. See the documentation of each decay function for more +details.} \item{active}{A logical. When \code{TRUE}, the function calculates active accessibility (the quantity of opportunities that can be reached from @@ -81,13 +82,13 @@ the data frame, passing \code{"departure_time"} to this parameter results in accessibility estimates grouped by origin and by departure time.} \item{fill_missing_ids}{A \code{logical}. When calculating grouped accessibility -estimates (i.e. when \code{by_col} is not \code{NULL}), some combinations of groups +estimates (i.e. when \code{group_by} is not empty), some combinations of groups and origins may be missing. For example, if a single trip can depart from origin \code{A} at 7:15am and reach destination \code{B} within 55 minutes, but no trips departing from \code{A} at 7:30am can be completed at all, this second combination will not be included in the output. When \code{TRUE} (the default), the function identifies which combinations would be left out and fills -their respective accessibility values with 0, which incurs in a +their respective accessibility values with 0, which incurs a performance penalty.} \item{detailed_results}{Logical. Whether to return detailed OD-level results.} @@ -106,7 +107,7 @@ proposed in \insertCite{soukhov2025family;textual}{accessibility}. Allocates the system-wide total proportionally based on travel impedance between origins and destinations. This measure uses the logic of a -total ~(or 'unconstrained' by Wilson's terms)~ constraint. +total (or 'unconstrained' by Wilson's terms) constraint. Use this measure when the total quantity of \strong{supply} OR \strong{demand} in the system is known and representing accessibility as a proportion of this total @@ -241,7 +242,7 @@ travel impedance. to match; otherwise, the model will not converge. \item \code{active} must be \code{NULL}. Since supply must match demand, their units are the same and there is no distinction between 'active' and 'passive' notions. -\item Only accepts\code{detailed_results = TRUE}. +\item Only accepts \code{detailed_results = TRUE}. } } diff --git a/man/cost_to_closest.Rd b/man/cost_to_closest.Rd index 27aa187..4270df2 100644 --- a/man/cost_to_closest.Rd +++ b/man/cost_to_closest.Rd @@ -48,7 +48,7 @@ accessibility estimates grouped by origin and by departure time.} \item{active}{A logical. When \code{TRUE}, the function calculates active accessibility (the quantity of opportunities that can be reached from -a given origin). when \code{FALSE}, it calculates passive accessibility (by +a given origin). When \code{FALSE}, it calculates passive accessibility (by how many people each destination can be reached), which is equivalent to the notion of market potential.} @@ -57,10 +57,10 @@ closest N number of opportunities may result in missing ids in the output if they cannot reach the specified amount of opportunities across all destinations they can reach. For example, estimating the minimum travel time that an origin that can only reach 4 opportunities takes to reach 5 -opportunities resulting in such origin not being included in the output. +opportunities results in such origin not being included in the output. When \code{TRUE} (the default), the function identifies which ids would be left -out from the output and fill their respective minimum travel costs with -\code{Inf}, which incurs in a performance penalty.} +out from the output and fills their respective minimum travel costs with +\code{Inf}, which incurs a performance penalty.} } \value{ A data frame containing the accessibility estimates for each @@ -98,3 +98,9 @@ df <- cost_to_closest( head(df) \dontshow{\}) # examplesIf} } +\seealso{ +Other accessibility measures: +\code{\link[=balancing_cost]{balancing_cost()}}, +\code{\link[=gravity]{gravity()}} +} +\concept{accessibility measures} diff --git a/man/cumulative_cutoff.Rd b/man/cumulative_cutoff.Rd index 3b95bd7..6e3a934 100644 --- a/man/cumulative_cutoff.Rd +++ b/man/cumulative_cutoff.Rd @@ -36,7 +36,7 @@ be considered in the calculation.} \item{cutoff}{Either a \code{numeric} vector or a list of \code{numeric} vectors, one for each cost specified in \code{travel_cost}. The travel cost cutoffs to consider when calculating accessibility levels. If a list, the function -finds every single possible cutoff combination and use them to calculate +finds every single possible cutoff combination and uses them to calculate accessibility (e.g. if one specifies that travel time cutoffs should be 30 and 60 minutes and that monetary cost cutoffs should be 5 and 10 dollars, the output includes accessibility estimates limited at 30 min & 5 dollars, @@ -56,18 +56,18 @@ accessibility estimates grouped by origin and by departure time.} \item{active}{A logical. When \code{TRUE}, the function calculates active accessibility (the quantity of opportunities that can be reached from -a given origin). when \code{FALSE}, it calculates passive accessibility (by +a given origin). When \code{FALSE}, it calculates passive accessibility (by how many people each destination can be reached), which is equivalent to the notion of market potential.} \item{fill_missing_ids}{A \code{logical}. Calculating cumulative accessibility may -result in missing ids if the they cannot reach any of the destinations +result in missing ids if they cannot reach any of the destinations within the specified travel cost cutoff. For example, using a travel time cutoff of 20 minutes, when estimating the accessibility of origin \code{A} that can only reach destinations with more than 40 minutes results in id \code{A} not being included in the output. When \code{TRUE} (the default), the function identifies which origins would be left out and fills their respective -accessibility values with 0, which incurs in a performance penalty.} +accessibility values with 0, which incurs a performance penalty.} } \value{ A data frame containing the accessibility estimates for each diff --git a/man/cumulative_interval.Rd b/man/cumulative_interval.Rd index d83df00..9bae7e1 100644 --- a/man/cumulative_interval.Rd +++ b/man/cumulative_interval.Rd @@ -49,7 +49,7 @@ within the interval. Defaults to 1.} \item{summary_function}{A function. This function is used to summarize a distribution of accessibility estimates within a travel cost interval as a single value. Can be any function that takes an arbitrary number of -numeric values as as input and returns a single number as output. Defaults +numeric values as input and returns a single number as output. Defaults to \code{\link[stats:median]{stats::median()}}.} \item{group_by}{A \code{character} vector. When not \code{character(0)} (the default), @@ -61,7 +61,7 @@ accessibility estimates grouped by origin and by departure time.} \item{active}{A logical. When \code{TRUE}, the function calculates active accessibility (the quantity of opportunities that can be reached from -a given origin). when \code{FALSE}, it calculates passive accessibility (by +a given origin). When \code{FALSE}, it calculates passive accessibility (by how many people each destination can be reached), which is equivalent to the notion of market potential.} @@ -79,7 +79,7 @@ travel matrix. Calculates the average or median number of opportunities that can be reached considering multiple maximum travel cost thresholds within a given travel cost interval specified by the user. The time interval cumulative -accessibility measures was originally proposed by +accessibility measure was originally proposed by \insertCite{tomasiello2023time;textual}{accessibility}. This function is generic over any kind of numeric travel cost, diff --git a/man/decay_logistic.Rd b/man/decay_logistic.Rd index b771691..fc40e81 100644 --- a/man/decay_logistic.Rd +++ b/man/decay_logistic.Rd @@ -17,7 +17,8 @@ decay.} } \value{ A \code{function} that takes a generic travel cost vector (\code{numeric}) as -input and returns a vector of weights (\code{numeric}). +an input and returns a \code{list} of weight vectors (a list of \code{numeric} +vectors, named after the arguments passed to the decay function). } \description{ Returns a logistic weighting function (in which the weights diff --git a/man/decay_stepped.Rd b/man/decay_stepped.Rd index 49c7cff..fedbb84 100644 --- a/man/decay_stepped.Rd +++ b/man/decay_stepped.Rd @@ -22,7 +22,8 @@ examples for effects of this assumption on the results.} } \value{ A \code{function} that takes a generic travel cost vector (\code{numeric}) as -an input and returns a vector of weights (\code{numeric}). +an input and returns a \code{list} of weight vectors (a list of \code{numeric} +vectors, named after the arguments passed to the decay function). } \description{ Returns a stepped weighting function to be used inside accessibility diff --git a/man/fgt_poverty.Rd b/man/fgt_poverty.Rd index 1fd73d8..71a678d 100644 --- a/man/fgt_poverty.Rd +++ b/man/fgt_poverty.Rd @@ -24,7 +24,7 @@ sociodemographic characteristics of the population in the study area cells. Must contain the columns \code{id} and any others specified in \code{population}.} \item{opportunity}{A string. The name of the column in \code{accessibility_data} -with the accessibility levels to be considerend when calculating +with the accessibility levels to be considered when calculating accessibility poverty.} \item{population}{A string. The name of the column in \code{sociodemographic_data} @@ -34,9 +34,9 @@ when calculating poverty.} \item{poverty_line}{A \code{numeric}. The poverty line below which individuals are considered to be in accessibility poverty.} -\item{poor_below_threshold}{Logic. If \code{TRUE} (default,) the observations below +\item{poor_below_threshold}{A logical. If \code{TRUE} (default) the observations below the poverty line are considered to be poor. This is the correct approach -for primal accessibility measures (e.g.cumulative accessibility). If +for primal accessibility measures (e.g. cumulative accessibility). If \code{FALSE}, then observations above the poverty line are considered to be poor. This is the correct approach for dual accessibility measures (e.g. travel time to the closest facility). When set to \code{FALSE}, FGT 1 and 2 @@ -69,7 +69,7 @@ used to calculate it: \itemize{ \item with \eqn{\alpha = 0} (FGT0) the measure captures the \emph{extent} of poverty as a simple headcount - i.e. the proportion of people below the poverty line; -\item with \eqn{\alpha = 1} (FGT1) the measure, also know as the "poverty gap +\item with \eqn{\alpha = 1} (FGT1) the measure, also known as the "poverty gap index", captures the \emph{severity} of poverty as the average percentage distance between the poverty line and the accessibility of individuals below the poverty line; @@ -111,3 +111,11 @@ poverty \references{ \insertAllCited{} } +\seealso{ +Other inequality: +\code{\link[=concentration_index]{concentration_index()}}, +\code{\link[=gini_index]{gini_index()}}, +\code{\link[=palma_ratio]{palma_ratio()}}, +\code{\link[=theil_t]{theil_t()}} +} +\concept{inequality} diff --git a/man/floating_catchment_area.Rd b/man/floating_catchment_area.Rd index e808819..e67b34a 100644 --- a/man/floating_catchment_area.Rd +++ b/man/floating_catchment_area.Rd @@ -43,13 +43,14 @@ considered.} Current available options are \code{"2sfca"} and \code{"bfca"}. More info in the details.} -\item{decay_function}{A \code{fuction} that converts travel cost into an +\item{decay_function}{A \code{function} that converts travel cost into an impedance factor used to weight opportunities. This function should take a \code{numeric} vector and also return a \code{numeric} vector as output, with the same length as the input. For convenience, the package currently includes the following functions: \code{\link[=decay_binary]{decay_binary()}}, \code{\link[=decay_exponential]{decay_exponential()}}, -\code{\link[=decay_power]{decay_power()}} and \code{\link[=decay_stepped]{decay_stepped()}}. See the documentation of each decay -function for more details.} +\code{\link[=decay_linear]{decay_linear()}}, \code{\link[=decay_logistic]{decay_logistic()}}, \code{\link[=decay_power]{decay_power()}} and +\code{\link[=decay_stepped]{decay_stepped()}}. See the documentation of each decay function for more +details.} \item{group_by}{A \code{character} vector. When not \code{character(0)} (the default), indicates the \code{travel_matrix} columns that should be used to group the @@ -59,19 +60,18 @@ the data frame, passing \code{"departure_time"} to this parameter results in accessibility estimates grouped by origin and by departure time.} \item{fill_missing_ids}{A \code{logical}. When calculating grouped accessibility -estimates (i.e. when \code{by_col} is not \code{NULL}), some combinations of groups +estimates (i.e. when \code{group_by} is not empty), some combinations of groups and origins may be missing. For example, if a single trip can depart from origin \code{A} at 7:15am and reach destination \code{B} within 55 minutes, but no trips departing from \code{A} at 7:30am can be completed at all, this second combination will not be included in the output. When \code{TRUE} (the default), the function identifies which combinations would be left out and fills -their respective accessibility values with 0, which incurs in a +their respective accessibility values with 0, which incurs a performance penalty.} } \value{ -A data frame containing the accessibility estimates for each -origin/destination (depending if \code{active} is \code{TRUE} or \code{FALSE}) in the -travel matrix. +A data frame containing the accessibility estimates for each origin +in the travel matrix. } \description{ Calculates accessibility accounting for the competition of resources using a diff --git a/man/gini_index.Rd b/man/gini_index.Rd index e3cba3d..2d1dfef 100644 --- a/man/gini_index.Rd +++ b/man/gini_index.Rd @@ -22,7 +22,7 @@ sociodemographic characteristics of the population in the study area cells. Must contain the columns \code{id} and any others specified in \code{population}.} \item{opportunity}{A string. The name of the column in \code{accessibility_data} -with the accessibility levels to be considerend when calculating inequality +with the accessibility levels to be considered when calculating inequality levels.} \item{population}{A string. The name of the column in \code{sociodemographic_data} @@ -69,6 +69,7 @@ gini \seealso{ Other inequality: \code{\link[=concentration_index]{concentration_index()}}, +\code{\link[=fgt_poverty]{fgt_poverty()}}, \code{\link[=palma_ratio]{palma_ratio()}}, \code{\link[=theil_t]{theil_t()}} } diff --git a/man/gravity.Rd b/man/gravity.Rd index d9ad5d3..404835a 100644 --- a/man/gravity.Rd +++ b/man/gravity.Rd @@ -34,13 +34,14 @@ the travel cost between origins and destinations. The notion of cost here is generic over any kind of numeric travel cost, such as distance, time and money.} -\item{decay_function}{A \code{fuction} that converts travel cost into an +\item{decay_function}{A \code{function} that converts travel cost into an impedance factor used to weight opportunities. This function should take a \code{numeric} vector and also return a \code{numeric} vector as output, with the same length as the input. For convenience, the package currently includes the following functions: \code{\link[=decay_binary]{decay_binary()}}, \code{\link[=decay_exponential]{decay_exponential()}}, -\code{\link[=decay_power]{decay_power()}} and \code{\link[=decay_stepped]{decay_stepped()}}. See the documentation of each decay -function for more details.} +\code{\link[=decay_linear]{decay_linear()}}, \code{\link[=decay_logistic]{decay_logistic()}}, \code{\link[=decay_power]{decay_power()}} and +\code{\link[=decay_stepped]{decay_stepped()}}. See the documentation of each decay function for more +details.} \item{group_by}{A \code{character} vector. When not \code{character(0)} (the default), indicates the \code{travel_matrix} columns that should be used to group the @@ -51,18 +52,18 @@ accessibility estimates grouped by origin and by departure time.} \item{active}{A logical. When \code{TRUE}, the function calculates active accessibility (the quantity of opportunities that can be reached from -a given origin). when \code{FALSE}, it calculates passive accessibility (by +a given origin). When \code{FALSE}, it calculates passive accessibility (by how many people each destination can be reached), which is equivalent to the notion of market potential.} \item{fill_missing_ids}{A \code{logical}. When calculating grouped accessibility -estimates (i.e. when \code{by_col} is not \code{NULL}), some combinations of groups +estimates (i.e. when \code{group_by} is not empty), some combinations of groups and origins may be missing. For example, if a single trip can depart from origin \code{A} at 7:15am and reach destination \code{B} within 55 minutes, but no trips departing from \code{A} at 7:30am can be completed at all, this second combination will not be included in the output. When \code{TRUE} (the default), the function identifies which combinations would be left out and fills -their respective accessibility values with 0, which incurs in a +their respective accessibility values with 0, which incurs a performance penalty.} } \value{ @@ -102,3 +103,9 @@ df_exp <- gravity( head(df_exp) \dontshow{\}) # examplesIf} } +\seealso{ +Other accessibility measures: +\code{\link[=balancing_cost]{balancing_cost()}}, +\code{\link[=cost_to_closest]{cost_to_closest()}} +} +\concept{accessibility measures} diff --git a/man/palma_ratio.Rd b/man/palma_ratio.Rd index c4ccd81..76d3334 100644 --- a/man/palma_ratio.Rd +++ b/man/palma_ratio.Rd @@ -24,7 +24,7 @@ Must contain the columns \code{id} and any others specified in \code{population} \code{income}.} \item{opportunity}{A string. The name of the column in \code{accessibility_data} -with the accessibility levels to be considerend when calculating inequality +with the accessibility levels to be considered when calculating inequality levels.} \item{population}{A string. The name of the column in \code{sociodemographic_data} @@ -82,6 +82,7 @@ palma \seealso{ Other inequality: \code{\link[=concentration_index]{concentration_index()}}, +\code{\link[=fgt_poverty]{fgt_poverty()}}, \code{\link[=gini_index]{gini_index()}}, \code{\link[=theil_t]{theil_t()}} } diff --git a/man/roxygen/templates/active.R b/man/roxygen/templates/active.R index 2e07c5a..aba01b9 100644 --- a/man/roxygen/templates/active.R +++ b/man/roxygen/templates/active.R @@ -1,5 +1,5 @@ #' @param active A logical. When `TRUE`, the function calculates active #' accessibility (the quantity of opportunities that can be reached from -#' a given origin). when `FALSE`, it calculates passive accessibility (by +#' a given origin). When `FALSE`, it calculates passive accessibility (by #' how many people each destination can be reached), which is equivalent #' to the notion of market potential. diff --git a/man/roxygen/templates/arguments_fca.R b/man/roxygen/templates/arguments_fca.R deleted file mode 100644 index 1de1930..0000000 --- a/man/roxygen/templates/arguments_fca.R +++ /dev/null @@ -1,4 +0,0 @@ -#' @param orig_col A `string` with the name of the column of origin ids. -#' @param dest_col A `string` with the name of the column of destination ids. -#' @param population_col A `string` with the name of the column of origin with -#' population count. diff --git a/man/roxygen/templates/decay_function.R b/man/roxygen/templates/decay_function.R index c8815b8..c249faf 100644 --- a/man/roxygen/templates/decay_function.R +++ b/man/roxygen/templates/decay_function.R @@ -1,8 +1,9 @@ -#' @param decay_function A `fuction` that converts travel cost into an +#' @param decay_function A `function` that converts travel cost into an #' impedance factor used to weight opportunities. This function should take a #' `numeric` vector and also return a `numeric` vector as output, with the #' same length as the input. For convenience, the package currently includes #' the following functions: [decay_binary()], [decay_exponential()], -#' [decay_power()] and [decay_stepped()]. See the documentation of each decay -#' function for more details. +#' [decay_linear()], [decay_logistic()], [decay_power()] and +#' [decay_stepped()]. See the documentation of each decay function for more +#' details. diff --git a/man/roxygen/templates/fill_missing_ids_combinations.R b/man/roxygen/templates/fill_missing_ids_combinations.R index e454597..3572a63 100644 --- a/man/roxygen/templates/fill_missing_ids_combinations.R +++ b/man/roxygen/templates/fill_missing_ids_combinations.R @@ -1,9 +1,9 @@ #' @param fill_missing_ids A `logical`. When calculating grouped accessibility -#' estimates (i.e. when `by_col` is not `NULL`), some combinations of groups +#' estimates (i.e. when `group_by` is not empty), some combinations of groups #' and origins may be missing. For example, if a single trip can depart from #' origin `A` at 7:15am and reach destination `B` within 55 minutes, but no #' trips departing from `A` at 7:30am can be completed at all, this second #' combination will not be included in the output. When `TRUE` (the default), #' the function identifies which combinations would be left out and fills -#' their respective accessibility values with 0, which incurs in a +#' their respective accessibility values with 0, which incurs a #' performance penalty. diff --git a/man/roxygen/templates/input_data.R b/man/roxygen/templates/input_data.R deleted file mode 100644 index 1b66fcc..0000000 --- a/man/roxygen/templates/input_data.R +++ /dev/null @@ -1,5 +0,0 @@ -#' @param data A `data.frame` with a travel cost matrix in long format, -#' containing the columns with origin ids, destination ids, the travel cost -#' between each origin-destination pair (this can be either in terms of -#' travel time, distance or monetary costs), and number of population and -#' opportunities located in each origin and destination. diff --git a/man/roxygen/templates/opportunity_access.R b/man/roxygen/templates/opportunity_access.R index 6ded7f5..a64ee71 100644 --- a/man/roxygen/templates/opportunity_access.R +++ b/man/roxygen/templates/opportunity_access.R @@ -1,3 +1,3 @@ #' @param opportunity A string. The name of the column in `accessibility_data` -#' with the accessibility levels to be considerend when calculating inequality +#' with the accessibility levels to be considered when calculating inequality #' levels. diff --git a/man/spatial_availability.Rd b/man/spatial_availability.Rd index 9809465..c4f6898 100644 --- a/man/spatial_availability.Rd +++ b/man/spatial_availability.Rd @@ -40,13 +40,14 @@ time and money.} number of opportunity-demanders at each origin (e.g., people) that will be considered.} -\item{decay_function}{A \code{fuction} that converts travel cost into an +\item{decay_function}{A \code{function} that converts travel cost into an impedance factor used to weight opportunities. This function should take a \code{numeric} vector and also return a \code{numeric} vector as output, with the same length as the input. For convenience, the package currently includes the following functions: \code{\link[=decay_binary]{decay_binary()}}, \code{\link[=decay_exponential]{decay_exponential()}}, -\code{\link[=decay_power]{decay_power()}} and \code{\link[=decay_stepped]{decay_stepped()}}. See the documentation of each decay -function for more details.} +\code{\link[=decay_linear]{decay_linear()}}, \code{\link[=decay_logistic]{decay_logistic()}}, \code{\link[=decay_power]{decay_power()}} and +\code{\link[=decay_stepped]{decay_stepped()}}. See the documentation of each decay function for more +details.} \item{alpha}{A \code{numeric}. A parameter used to modulate the effect of demand by population. When less than 1, opportunities are allocated more rapidly @@ -61,13 +62,13 @@ the data frame, passing \code{"departure_time"} to this parameter results in accessibility estimates grouped by origin and by departure time.} \item{fill_missing_ids}{A \code{logical}. When calculating grouped accessibility -estimates (i.e. when \code{by_col} is not \code{NULL}), some combinations of groups +estimates (i.e. when \code{group_by} is not empty), some combinations of groups and origins may be missing. For example, if a single trip can depart from origin \code{A} at 7:15am and reach destination \code{B} within 55 minutes, but no trips departing from \code{A} at 7:30am can be completed at all, this second combination will not be included in the output. When \code{TRUE} (the default), the function identifies which combinations would be left out and fills -their respective accessibility values with 0, which incurs in a +their respective accessibility values with 0, which incurs a performance penalty.} \item{detailed_results}{A \code{logical}. Whether to return spatial availability @@ -78,12 +79,13 @@ availability. Please note that the argument \code{fill_missing_ids} does not affect the output when \code{detailed_results} is \code{TRUE}.} } \value{ -A data frame containing the accessibility estimates for each -origin/destination (depending if \code{active} is \code{TRUE} or \code{FALSE}) in the -travel matrix. +A data frame containing the spatial availability estimates for +each origin in the travel matrix. When \code{detailed_results} is \code{TRUE}, the +estimates are given for each origin-destination pair instead, alongside the +balancing factors used to calculate them. } \description{ -Calculates spatial availability, an accessibility measured proposed by +Calculates spatial availability, an accessibility measure proposed by \insertCite{soukhov2023introducing;textual}{accessibility} that takes into account competition effects. The accessibility levels that result from using this measure are proportional both to the demand in each origin and to the diff --git a/man/theil_t.Rd b/man/theil_t.Rd index 5bbc36a..ffc0808 100644 --- a/man/theil_t.Rd +++ b/man/theil_t.Rd @@ -24,7 +24,7 @@ Must contain the columns \code{id} and any others specified in \code{population} \code{socioeconomic_groups}.} \item{opportunity}{A string. The name of the column in \code{accessibility_data} -with the accessibility levels to be considerend when calculating inequality +with the accessibility levels to be considered when calculating inequality levels.} \item{population}{A string. The name of the column in \code{sociodemographic_data} @@ -59,7 +59,7 @@ range from 0 (when all individuals have exactly the same accessibility levels) to the natural log of \emph{n}, in which \emph{n} is the number of individuals in the accessibility dataset. If the individuals can be classified into mutually exclusive and completely exhaustive groups, the index can be -decomposed into a between-groups inequaliy component and a within-groups +decomposed into a between-groups inequality component and a within-groups component. } \details{ @@ -116,6 +116,7 @@ ti \seealso{ Other inequality: \code{\link[=concentration_index]{concentration_index()}}, +\code{\link[=fgt_poverty]{fgt_poverty()}}, \code{\link[=gini_index]{gini_index()}}, \code{\link[=palma_ratio]{palma_ratio()}} } diff --git a/vignettes/accessibility.Rmd b/vignettes/accessibility.Rmd index 48915a3..5b74b4f 100644 --- a/vignettes/accessibility.Rmd +++ b/vignettes/accessibility.Rmd @@ -223,7 +223,7 @@ head(cum_interval) `gravity()` calculates gravity-based measures - i.e. measures in which the weight of opportunities is gradually discounted as the travel cost increases. -Of course, several different decay functions can be used to so, each one of +Of course, several different decay functions can be used to do so, each one of them with a range of possible different parameters. In order to accommodate such generalization, the function takes the decay function to be used as a parameter. @@ -255,7 +255,7 @@ supports two different methods: - 2-Step Floating Catchment Area (`"2sfca"`) - the first metric in the FCA family, originally proposed by @luo2003measures. -- Balanced Floating Catchment Area (`"bfca"`) - takes competition affects into +- Balanced Floating Catchment Area (`"bfca"`) - takes competition effects into account while correcting for issues of inflation of demand and service levels. Originally proposed by @paez2019demand and named in @pereira2021geographic. diff --git a/vignettes/decay_functions.Rmd b/vignettes/decay_functions.Rmd index c31721e..e613fcc 100644 --- a/vignettes/decay_functions.Rmd +++ b/vignettes/decay_functions.Rmd @@ -24,7 +24,7 @@ larger. For convenience, the package ships with some of the most frequently used functions in the accessibility literature, which are discussed below. Additionally, users can pass custom functions to convert travel costs into weights to be applied to the opportunities, which is discussed further down in -this the vignette. +this vignette. # Currently available decay functions @@ -96,7 +96,7 @@ $\beta$ is the parameter that tells the speed of decay. ## Stepped Similar to the binary function, but can take an arbitrary number of steps. The -current implementation assumes that values changes at each step, instead of +current implementation assumes that values change at each step, instead of right after it. $$ @@ -162,8 +162,8 @@ output_fn <- decay_exponential(c(0.2, 0.3)) output_fn(c(10, 15, 20)) ``` -`decay_stepped()`, as the only decay function that takes more than one argument -(both `steps` and `weights`) names the output elements after the combination of +`decay_stepped()`, which takes two arguments that can be lists of vectors +(both `steps` and `weights`), names the output elements after the combination of steps and weights: ```{r} diff --git a/vignettes/inequality_and_poverty.Rmd b/vignettes/inequality_and_poverty.Rmd index 9309583..217bac8 100644 --- a/vignettes/inequality_and_poverty.Rmd +++ b/vignettes/inequality_and_poverty.Rmd @@ -155,10 +155,10 @@ inequality component. The function includes a `socioeconomic_groups` parameter to indicate which variable from the sociodemographic dataset should be used identify the socioeconomic groups used to calculate these components. -Please note that the output `theil_t()` varies based on the value of +Please note that the output of `theil_t()` varies based on the value of `socioeconomic_groups`. If `NULL` (the default), the between- and within-groups components are not calculated, and the function returns a data frame containing -only the total aggregate inequality for the returned area. If +only the total aggregate inequality for the study area. If `socioeconomic_groups` is not `NULL`, however, the function returns a list containing three dataframes: one summarizing the total inequality and the between- and within-groups components, one listing the contribution of each @@ -206,7 +206,7 @@ them (either 0, 1 or 2) and which also changes their interpretation: - with $\alpha = 0$ (FGT0) the measure captures the extent of poverty as a simple headcount - i.e. the proportion of people below the poverty line; -- with $\alpha = 1$ (FGT1) the measure, also know as the "poverty gap index", +- with $\alpha = 1$ (FGT1) the measure, also known as the "poverty gap index", captures the severity of poverty as the average percentage distance between the poverty line and the accessibility of individuals below the poverty line; - with $\alpha = 2$ (FGT2) the measure simultaneously captures the extent and From 6691885ced719c1c0b5890c14de538650e419bf9 Mon Sep 17 00:00:00 2001 From: rafapereirabr Date: Tue, 20 Oct 2026 20:57:20 -0300 Subject: [PATCH 19/31] optimize decay functions --- NEWS.md | 4 +++ R/decay_exponential.R | 6 ++--- R/decay_logistic.R | 19 ++++++++----- R/decay_power.R | 17 +++++++----- R/weigh_unique_costs.R | 34 ++++++++++++++++++++++++ tests/testthat/test-weigh_unique_costs.R | 15 +++++++++++ 6 files changed, 79 insertions(+), 16 deletions(-) create mode 100644 R/weigh_unique_costs.R create mode 100644 tests/testthat/test-weigh_unique_costs.R diff --git a/NEWS.md b/NEWS.md index a081b94..bd7220c 100644 --- a/NEWS.md +++ b/NEWS.md @@ -127,6 +127,10 @@ - The functions created by `decay_logistic()` and `decay_stepped()` are now vectorised instead of looping over each travel cost, which makes them about 50 and 200 times faster on large travel matrices, with identical results. +- The functions created by `decay_exponential()`, `decay_power()` and + `decay_logistic()` now calculate each weight once per unique travel cost + when costs are repeated (e.g. rounded travel times), which makes them about + 2 to 6 times faster, with identical results. - Errors and warnings raised by the package now use `{cli}`, with consistent formatting of argument names and values. Long lists of ids or columns in messages are abbreviated. `{cli}` and `{rlang}` are new dependencies. diff --git a/R/decay_exponential.R b/R/decay_exponential.R index 6ed1476..5ab6bf6 100644 --- a/R/decay_exponential.R +++ b/R/decay_exponential.R @@ -33,9 +33,9 @@ decay_exponential <- function(decay_value) { ) weighting_function <- function(travel_cost) { - weights_list <- lapply( - decay_value, - function(x) exp(-x * travel_cost) + weights_list <- weigh_unique_costs( + travel_cost, + function(cost) lapply(decay_value, function(x) exp(-x * cost)) ) names(weights_list) <- decay_value diff --git a/R/decay_logistic.R b/R/decay_logistic.R index a0ed528..d607893 100644 --- a/R/decay_logistic.R +++ b/R/decay_logistic.R @@ -60,13 +60,18 @@ decay_logistic <- function(cutoff, sd) { 1 + exp(((cost - median) * base::pi) / (std_dev * sqrt(3))) } - sd_list <- mapply( - median = cutoff, - std_dev = sd, - FUN = function(median, std_dev) { - g(0, median, std_dev) / g(travel_cost, median, std_dev) - }, - SIMPLIFY = FALSE + sd_list <- weigh_unique_costs( + travel_cost, + function(cost) { + mapply( + median = cutoff, + std_dev = sd, + FUN = function(median, std_dev) { + g(0, median, std_dev) / g(cost, median, std_dev) + }, + SIMPLIFY = FALSE + ) + } ) list_names <- paste0("c", cutoff, ";sd", sd) diff --git a/R/decay_power.R b/R/decay_power.R index ca420ce..00b6cef 100644 --- a/R/decay_power.R +++ b/R/decay_power.R @@ -32,12 +32,17 @@ decay_power <- function(decay_value) { ) weighting_function <- function(travel_cost) { - weights_list <- lapply( - decay_value, - function(x) { - weights <- travel_cost ^ (-x) - weights[weights > 1] <- 1 - weights + weights_list <- weigh_unique_costs( + travel_cost, + function(cost) { + lapply( + decay_value, + function(x) { + weights <- cost ^ (-x) + weights[weights > 1] <- 1 + weights + } + ) } ) names(weights_list) <- decay_value diff --git a/R/weigh_unique_costs.R b/R/weigh_unique_costs.R new file mode 100644 index 0000000..925c25d --- /dev/null +++ b/R/weigh_unique_costs.R @@ -0,0 +1,34 @@ +#' @keywords internal +weigh_unique_costs <- function(travel_cost, weigh) { + # decay functions that use exp() or ^ are slow on large travel matrices. + # travel costs are usually rounded (e.g. integer minutes), so we calculate + # the weights once per unique cost and map them back to every cost. weigh() + # takes a cost vector and returns a list of weight vectors. finding the + # unique costs is slower than weighting all costs when costs are continuous, + # so we first check a sample of the costs + + sample_costs <- travel_cost[seq_len(min(length(travel_cost), 10000L))] + + if (length(unique(sample_costs)) > length(sample_costs) / 2) { + return(weigh(travel_cost)) + } + + unique_costs <- unique(travel_cost) + + if (length(unique_costs) > length(travel_cost) / 2) { + return(weigh(travel_cost)) + } + + cost_index <- match(travel_cost, unique_costs) + + weights_list <- lapply( + weigh(unique_costs), + function(weights) { + weights <- weights[cost_index] + names(weights) <- names(travel_cost) + weights + } + ) + + return(weights_list) +} diff --git a/tests/testthat/test-weigh_unique_costs.R b/tests/testthat/test-weigh_unique_costs.R new file mode 100644 index 0000000..a95a675 --- /dev/null +++ b/tests/testthat/test-weigh_unique_costs.R @@ -0,0 +1,15 @@ +weigh <- function(cost) list(a = exp(-0.1 * cost), b = cost ^ 2) + +test_that("weights repeated costs as if each cost was weighted separately", { + travel_cost <- c(10, NA, 0, 10, 20, NA, 0, 20, 10, Inf) + expect_identical(weigh_unique_costs(travel_cost, weigh), weigh(travel_cost)) + + names(travel_cost) <- letters[seq_along(travel_cost)] + expect_identical(weigh_unique_costs(travel_cost, weigh), weigh(travel_cost)) +}) + +test_that("weights mostly unique costs directly", { + travel_cost <- c(1.5, 2.5, 3.5, 1.5) + expect_identical(weigh_unique_costs(travel_cost, weigh), weigh(travel_cost)) + expect_identical(weigh_unique_costs(numeric(0), weigh), weigh(numeric(0))) +}) From f1cc59ba9c3612bac778a8e954d0b5ffcb0ada31 Mon Sep 17 00:00:00 2001 From: rafapereirabr Date: Tue, 20 Oct 2026 22:12:16 -0300 Subject: [PATCH 20/31] vignettes --- vignettes/accessibility.Rmd | 76 +++++++++++++++++++++++++++++++------ 1 file changed, 65 insertions(+), 11 deletions(-) diff --git a/vignettes/accessibility.Rmd b/vignettes/accessibility.Rmd index 5b74b4f..90800b2 100644 --- a/vignettes/accessibility.Rmd +++ b/vignettes/accessibility.Rmd @@ -23,8 +23,9 @@ pre-computed travel cost matrix and a land use dataset, the package allows one to calculate active and passive accessibility levels using multiple accessibility metrics, such as cumulative opportunities (using either a travel cost cutoff or a travel cost interval), minimum travel cost to closest N -number of activities, gravitational measures and different floating catchment -area methods. This vignette briefly overviews the package with a few +number of activities, gravitational measures, different floating catchment +area methods, spatial availability, balancing cost and constrained +accessibility measures. This vignette briefly overviews the package with a few reproducible examples. # Installation @@ -46,7 +47,7 @@ remotes::install_github("ipea/accessibility") # Overview of the package As of the time of writing this vignette, the **accessibility** package includes -five different functions to calculate accessibility: +eight different functions to calculate accessibility: 1. `cost_to_closest()` - calculates the minimum travel cost to the closest *n* number of opportunities. @@ -60,6 +61,14 @@ five different functions to calculate accessibility: 5. `floating_catchment_area()` - calculates accessibility taking into account the effects of competition for opportunities with different floating catchment area measures. +6. `spatial_availability()` - calculates accessibility taking into account + competition effects with the spatial availability measure, in which the + opportunities are allocated to origins proportionally to their demand and + travel cost. +7. `balancing_cost()` - calculates the travel cost required to reach as many + opportunities as the number of people in each origin. +8. `constrained_accessibility()` - calculates accessibility as potential + spatial interaction, using the total, singly or doubly constrained measures. You may have noticed that we've mentioned a few times that the functions calculate accessibility using travel *cost*, and not travel *time*. That's @@ -68,8 +77,8 @@ increases the impedance from an origin to a destination, such as travel time, monetary costs, distances, risk perception, etc., can be considered a generic *cost*. -The `gravity()` and `floating_catchment_area()` functions can use -different decay functions when estimating accessibility levels. These decay +The `gravity()`, `floating_catchment_area()`, `spatial_availability()` and +`constrained_accessibility()` functions can use different decay functions when estimating accessibility levels. These decay functions effectively weigh the number of opportunities in a destination by a factor that depends on the travel cost between the origin and the destination. For convenience, the package currently includes the following decay functions: @@ -79,8 +88,10 @@ For convenience, the package currently includes the following decay functions: 2. `decay_exponential()` - negative exponential decay function. 3. `decay_linear()` - linear decay function (weights decay linearly from 1 to 0 until a specific travel cost cutoff is reached). -4. `decay_power()` - inverse power decay function. -5. `decay_stepped()` - stepped decay function (similar to `decay_binary()`, but +4. `decay_logistic()` - logistic decay function (weights decay following a + reversed cumulative logistic curve around a travel cost cutoff). +5. `decay_power()` - inverse power decay function. +6. `decay_stepped()` - stepped decay function (similar to `decay_binary()`, but can take an arbitrary number of steps, instead of a single one). The users can also specify their own custom decay functions, if they need to @@ -193,12 +204,13 @@ head(passive_cum_cutoff) ``` The `active` parameter is available in most other accessibility functions as -well (with the exception of `floating_catchment_area()`), making it very easy -to calculate both active and passive accessibility. +well (with the exception of `floating_catchment_area()`, +`spatial_availability()` and `balancing_cost()`), making it very easy to +calculate both active and passive accessibility. ## Interval-based cumulative opportunities -`cumulative_time_interval()` calculates the interval-based cumulative +`cumulative_interval()` calculates the interval-based cumulative opportunities measure. This measure, developed by @tomasiello2023time, mitigates the impacts of arbitrary choices of cost cutoffs, one of the main disadvantages of the traditional threshold-based cumulative opportunities @@ -229,7 +241,7 @@ generalization, the function takes the decay function to be used as a parameter. In the example below, we calculate accessibility using a negative exponential -function with a `decay_value` (usually referred as the \eqn{\beta} in its +function with a `decay_value` (usually referred to as $\beta$ in its formulation) of 0.2. Please see [the vignette on decay functions](decay_functions.html) for more information on the decay functions shipped with the package and how to use custom functions. @@ -335,6 +347,48 @@ bal_cost <- balancing_cost( head(bal_cost) ``` +## Constrained accessibility + +`constrained_accessibility()` calculates the family of constrained +accessibility measures proposed by @soukhov2025family, in which accessibility +is conceptualised as potential spatial interaction. Borrowing the constraints +of the family of spatial interaction models proposed by @wilson1971family, the +function covers three cases, specified with the `constraint` parameter: + +- `"total"` - allocates the total supply (or demand) of the region according + to the travel impedance between origins and destinations; +- `"singly"` - allocates the supply at each destination (or the demand at each + origin) according to the travel impedance and the demand (or supply) on the + other side, thus accounting for competition; +- `"doubly"` - allocates flows so that they match both the demand at each + origin and the supply at each destination, which must sum to the same total. + +The results of the total and singly constrained measures are expressed in +units of supply (e.g. jobs) when `active = TRUE` and in units of demand (e.g. +population) when `active = FALSE`. In the example below, we calculate singly +constrained accessibility to jobs, considering the population competing for +them and a negative exponential decay function. Please note that the column +with the results is named `supply`, after the side being allocated, and not +after the opportunity column. + +```{r} +constrained <- constrained_accessibility( + travel_matrix, + land_use_data, + travel_cost = "travel_time", + demand = "population", + supply = "jobs", + constraint = "singly", + decay_function = decay_exponential(decay_value = 0.1), + active = TRUE +) +head(constrained) +``` + +Summing the results gives back the total number of jobs in the city, and, with +the same decay function, the results are equal to those of +`spatial_availability()`. + ## Visualize results If you have the spatial data of your origins/destinations, you can easily merge From c0ae940e7ea59a4d1774d06c733368372d5bdc39 Mon Sep 17 00:00:00 2001 From: rafapereirabr Date: Tue, 20 Oct 2026 22:26:46 -0300 Subject: [PATCH 21/31] optimize gravity --- R/accessibility-package.R | 6 +++++- R/apply_gravity_measure.R | 23 ++++++++++++++++------ R/floating_catchment_area.R | 39 ++++++++++++++++++++----------------- R/gravity.R | 6 +++++- 4 files changed, 48 insertions(+), 26 deletions(-) diff --git a/R/accessibility-package.R b/R/accessibility-package.R index 39c3752..1b995a7 100644 --- a/R/accessibility-package.R +++ b/R/accessibility-package.R @@ -95,7 +95,11 @@ utils::globalVariables( "share_of_total", ".block_share", "..groups", - "..group_by" + "..group_by", + ".weighted_opportunities", + ".weighted_demand", + ".weighted_ppr", + ".total_weight" ) ) diff --git a/R/apply_gravity_measure.R b/R/apply_gravity_measure.R index 13fa342..bfd5f50 100644 --- a/R/apply_gravity_measure.R +++ b/R/apply_gravity_measure.R @@ -19,12 +19,23 @@ apply_gravity_measure <- function(data, decay_function, travel_cost) { } else { opp_weights <- decay_function(data[[travel_cost]]) - access <- lapply( - opp_weights, - function(opp_weight) cbind(data, opp_weight) - ) - access <- data.table::rbindlist(access, idcol = "decay_function_arg") - access[, decay_function_arg := try_to_convert_to_num(decay_function_arg)] + decay_function_args <- names(opp_weights) + if (is.null(decay_function_args)) { + decay_function_args <- seq_along(opp_weights) + } + decay_function_args <- try_to_convert_to_num(decay_function_args) + + # stacking the data once and then assigning the weights is much cheaper + # than binding the weights to a copy of the data for each decay function + # argument + + access <- data.table::rbindlist(rep(list(data), length(opp_weights))) + access[, opp_weight := unlist(opp_weights, use.names = FALSE)] + access[ + , + decay_function_arg := rep(decay_function_args, each = nrow(data)) + ] + data.table::setcolorder(access, "decay_function_arg") } return(access[]) diff --git a/R/floating_catchment_area.R b/R/floating_catchment_area.R index f79b1fc..f5b97a1 100644 --- a/R/floating_catchment_area.R +++ b/R/floating_catchment_area.R @@ -156,11 +156,16 @@ fca_2sfca <- function(data, opportunity, demand, group_by) { # step 1a - calculate the demand to each destination as the weight between # each od pair multiplied by the number of people at the origin + # products are calculated before aggregating, so the sums are optimized by + # data.table (GForce) + + data[, .weighted_demand := get(.demand_colname) * opp_weight] data[ , - pop_served := sum(get(.demand_colname) * opp_weight, na.rm = TRUE), + pop_served := sum(.weighted_demand, na.rm = TRUE), by = c("to_id", group_by) ] + data[, .weighted_demand := NULL] # step 1b - calculate the provider-to-population ratio (ppr) of a destination # as number of opportunities/resources located at it divided by the @@ -172,9 +177,10 @@ fca_2sfca <- function(data, opportunity, demand, group_by) { # multiplication between the ppr of destination j and the weight between od # pair ij + data[, .weighted_ppr := ppr * opp_weight] access <- data[ , - .(access = sum(ppr * opp_weight, na.rm = TRUE)), + .(access = sum(.weighted_ppr, na.rm = TRUE)), by = c("from_id", group_by) ] @@ -187,30 +193,26 @@ fca_bfca <- function(data, opportunity, demand, group_by) { .opportunity_colname <- opportunity .demand_colname <- demand - # calculate balanced (normalized) opp_weight by origin and by destination + # calculate balanced (normalized) opp_weight by origin and by destination. + # products and totals are calculated in separate steps, so the sums are + # optimized by data.table (GForce) - data[ - , - balanced_opp_weight_i := opp_weight / sum(opp_weight), - by = c("from_id", group_by) - ] - data[ - , - balanced_opp_weight_j := opp_weight / sum(opp_weight), - by = c("to_id", group_by) - ] + data[, .total_weight := sum(opp_weight), by = c("from_id", group_by)] + data[, balanced_opp_weight_i := opp_weight / .total_weight] + data[, .total_weight := sum(opp_weight), by = c("to_id", group_by)] + data[, balanced_opp_weight_j := opp_weight / .total_weight] + data[, .total_weight := NULL] # step 1a - calculate the balanced demand to each destination as the balanced # weight by origin multiplied by the number of people at the origin + data[, .weighted_demand := get(.demand_colname) * balanced_opp_weight_i] data[ , - balanced_pop_served := sum( - get(.demand_colname) * balanced_opp_weight_i, - na.rm = TRUE - ), + balanced_pop_served := sum(.weighted_demand, na.rm = TRUE), by = c("to_id", group_by) ] + data[, .weighted_demand := NULL] # step 1b - calculate the balanced provider-to-population ratio # (balanced_ppr) of a destination as number of opportunities/resources @@ -222,9 +224,10 @@ fca_bfca <- function(data, opportunity, demand, group_by) { # multiplication between the balanced_ppr of destination j and the balanced # weight by destination between od pair ij + data[, .weighted_ppr := balanced_ppr * balanced_opp_weight_j] access <- data[ , - .(access = sum(balanced_ppr * balanced_opp_weight_j, na.rm = TRUE)), + .(access = sum(.weighted_ppr, na.rm = TRUE)), by = c("from_id", group_by) ] diff --git a/R/gravity.R b/R/gravity.R index 0211afd..6f35966 100644 --- a/R/gravity.R +++ b/R/gravity.R @@ -94,10 +94,14 @@ gravity <- function(travel_matrix, warn_extra_cols(travel_matrix, travel_cost, group_id, groups) + # the weighted opportunities are calculated before aggregating, so the sum + # below is optimized by data.table (GForce) + .opp_colname <- opportunity + data[, .weighted_opportunities := get(.opp_colname) * opp_weight] access <- data[ , - .(access = sum(get(.opp_colname) * opp_weight)), + .(access = sum(.weighted_opportunities)), by = eval(groups, envir = env) ] From 51e82c9a28585692b3c987aee9757468211a2dda Mon Sep 17 00:00:00 2001 From: rafapereirabr Date: Tue, 20 Oct 2026 22:27:06 -0300 Subject: [PATCH 22/31] docs --- NEWS.md | 8 ++++++++ man/concentration_index.Rd | 4 ++-- man/gini_index.Rd | 4 ++-- man/palma_ratio.Rd | 4 ++-- man/theil_t.Rd | 4 ++-- 5 files changed, 16 insertions(+), 8 deletions(-) diff --git a/NEWS.md b/NEWS.md index bd7220c..3d0063a 100644 --- a/NEWS.md +++ b/NEWS.md @@ -131,6 +131,14 @@ `decay_logistic()` now calculate each weight once per unique travel cost when costs are repeated (e.g. rounded travel times), which makes them about 2 to 6 times faster, with identical results. +- `gravity()` and `floating_catchment_area()` are faster, especially with + several decay function values (e.g. `gravity()` with three values is about 4 + times faster on large travel matrices), because the travel matrix is no + longer copied once per value and sums are optimized by `{data.table}`. + Results may differ from previous versions by floating point rounding only + (relative differences below 1e-14). `gravity()` with integer opportunities + and `decay_binary()` no longer fails when accessibility exceeds the largest + integer. - Errors and warnings raised by the package now use `{cli}`, with consistent formatting of argument names and values. Long lists of ids or columns in messages are abbreviated. `{cli}` and `{rlang}` are new dependencies. diff --git a/man/concentration_index.Rd b/man/concentration_index.Rd index cf882df..ee1a240 100644 --- a/man/concentration_index.Rd +++ b/man/concentration_index.Rd @@ -97,10 +97,10 @@ ci \insertAllCited{} } \seealso{ -Other inequality: +Other equity: \code{\link[=fgt_poverty]{fgt_poverty()}}, \code{\link[=gini_index]{gini_index()}}, \code{\link[=palma_ratio]{palma_ratio()}}, \code{\link[=theil_t]{theil_t()}} } -\concept{inequality} +\concept{equity} diff --git a/man/gini_index.Rd b/man/gini_index.Rd index 2d1dfef..4fbc2cc 100644 --- a/man/gini_index.Rd +++ b/man/gini_index.Rd @@ -67,10 +67,10 @@ gini \dontshow{\}) # examplesIf} } \seealso{ -Other inequality: +Other equity: \code{\link[=concentration_index]{concentration_index()}}, \code{\link[=fgt_poverty]{fgt_poverty()}}, \code{\link[=palma_ratio]{palma_ratio()}}, \code{\link[=theil_t]{theil_t()}} } -\concept{inequality} +\concept{equity} diff --git a/man/palma_ratio.Rd b/man/palma_ratio.Rd index 76d3334..0e46ea2 100644 --- a/man/palma_ratio.Rd +++ b/man/palma_ratio.Rd @@ -80,10 +80,10 @@ palma \dontshow{\}) # examplesIf} } \seealso{ -Other inequality: +Other equity: \code{\link[=concentration_index]{concentration_index()}}, \code{\link[=fgt_poverty]{fgt_poverty()}}, \code{\link[=gini_index]{gini_index()}}, \code{\link[=theil_t]{theil_t()}} } -\concept{inequality} +\concept{equity} diff --git a/man/theil_t.Rd b/man/theil_t.Rd index ffc0808..1871804 100644 --- a/man/theil_t.Rd +++ b/man/theil_t.Rd @@ -114,10 +114,10 @@ ti \dontshow{\}) # examplesIf} } \seealso{ -Other inequality: +Other equity: \code{\link[=concentration_index]{concentration_index()}}, \code{\link[=fgt_poverty]{fgt_poverty()}}, \code{\link[=gini_index]{gini_index()}}, \code{\link[=palma_ratio]{palma_ratio()}} } -\concept{inequality} +\concept{equity} From 233f96436600b8e49cb0af966f2b3f5963d1602a Mon Sep 17 00:00:00 2001 From: rafapereirabr Date: Tue, 20 Oct 2026 22:31:40 -0300 Subject: [PATCH 23/31] docs of fgt_poverty --- R/fgt_poverty.R | 38 ++++++++++++++++++++++++-------------- man/fgt_poverty.Rd | 42 ++++++++++++++++++++++++++---------------- 2 files changed, 50 insertions(+), 30 deletions(-) diff --git a/R/fgt_poverty.R b/R/fgt_poverty.R index 18b6dd7..9aab462 100644 --- a/R/fgt_poverty.R +++ b/R/fgt_poverty.R @@ -38,25 +38,35 @@ #' FGT2) for the study area. #' #' @section Interpretation of FGT measures: -#' The interpretation of each FGT measure depends on the \eqn{\alpha} parameter -#' used to calculate it: +#' With a poverty line \eqn{z}, the FGT measures are calculated as the +#' population-weighted average of \eqn{g_i^\alpha}, in which \eqn{g_i} is the +#' normalised shortfall of each observation \eqn{i} with accessibility +#' \eqn{y_i}: \eqn{g_i = (z - y_i) / z} if \eqn{y_i < z} and 0 otherwise. When +#' `poor_below_threshold` is `FALSE`, \eqn{g_i} is instead the normalised +#' excess \eqn{(y_i - z) / z} if \eqn{y_i > z}, and 0 otherwise. The +#' interpretation of each measure depends on the \eqn{\alpha} parameter: #' -#' - with \eqn{\alpha = 0} (FGT0) the measure captures the *extent* of poverty -#' as a simple headcount - i.e. the proportion of people below the poverty line; +#' - with \eqn{\alpha = 0} (FGT0) the measure captures the *incidence* (or +#' extent) of poverty as a simple headcount - i.e. the proportion of people in +#' poverty; #' #' - with \eqn{\alpha = 1} (FGT1) the measure, also known as the "poverty gap -#' index", captures the *severity* of poverty as the average percentage distance -#' between the poverty line and the accessibility of individuals below the -#' poverty line; +#' index", captures the *depth* (or intensity) of poverty as the average +#' normalised shortfall from the poverty line, calculated over the whole +#' population (people not in poverty have a shortfall of 0); #' -#' - with \eqn{\alpha = 2} (FGT2) the measure simultaneously captures the -#' *extent* and the *severity* of poverty by calculating the number of people -#' below the poverty line weighted by the size of the accessibility shortfall -#' relative to the poverty line. +#' - with \eqn{\alpha = 2} (FGT2) the measure, also known as the "squared +#' poverty gap index", captures the *severity* of poverty. Squaring the +#' shortfalls gives more weight to the people furthest from the poverty line, +#' which makes the measure sensitive to inequality among the poor. #' -#' FGT values range from 0 to 1. A value of 0 indicates that every individual is -#' above the poverty line. When every individual is below the poverty line, -#' however, FGT0 value is 1 and FGT1 and FGT2 values approach 1. +#' A value of 0 indicates that no one is in poverty. When `poor_below_threshold` +#' is `TRUE`, FGT values range from 0 to 1: FGT0 is 1 when everyone is below the +#' poverty line, and FGT1 and FGT2 are 1 only when everyone has an +#' accessibility of 0. When `poor_below_threshold` is `FALSE`, FGT0 still ranges +#' from 0 to 1, but FGT1 and FGT2 have no upper bound: they can be greater than +#' 1 (e.g. when travel costs are more than twice the poverty line) and are +#' `Inf` if any accessibility value is `Inf`. #' #' @references #' \insertAllCited{} diff --git a/man/fgt_poverty.Rd b/man/fgt_poverty.Rd index 71a678d..1b27759 100644 --- a/man/fgt_poverty.Rd +++ b/man/fgt_poverty.Rd @@ -64,24 +64,34 @@ the interpretation of the measures. } \section{Interpretation of FGT measures}{ -The interpretation of each FGT measure depends on the \eqn{\alpha} parameter -used to calculate it: +With a poverty line \eqn{z}, the FGT measures are calculated as the +population-weighted average of \eqn{g_i^\alpha}, in which \eqn{g_i} is the +normalised shortfall of each observation \eqn{i} with accessibility +\eqn{y_i}: \eqn{g_i = (z - y_i) / z} if \eqn{y_i < z} and 0 otherwise. When +\code{poor_below_threshold} is \code{FALSE}, \eqn{g_i} is instead the normalised +excess \eqn{(y_i - z) / z} if \eqn{y_i > z}, and 0 otherwise. The +interpretation of each measure depends on the \eqn{\alpha} parameter: \itemize{ -\item with \eqn{\alpha = 0} (FGT0) the measure captures the \emph{extent} of poverty -as a simple headcount - i.e. the proportion of people below the poverty line; +\item with \eqn{\alpha = 0} (FGT0) the measure captures the \emph{incidence} (or +extent) of poverty as a simple headcount - i.e. the proportion of people in +poverty; \item with \eqn{\alpha = 1} (FGT1) the measure, also known as the "poverty gap -index", captures the \emph{severity} of poverty as the average percentage distance -between the poverty line and the accessibility of individuals below the -poverty line; -\item with \eqn{\alpha = 2} (FGT2) the measure simultaneously captures the -\emph{extent} and the \emph{severity} of poverty by calculating the number of people -below the poverty line weighted by the size of the accessibility shortfall -relative to the poverty line. +index", captures the \emph{depth} (or intensity) of poverty as the average +normalised shortfall from the poverty line, calculated over the whole +population (people not in poverty have a shortfall of 0); +\item with \eqn{\alpha = 2} (FGT2) the measure, also known as the "squared +poverty gap index", captures the \emph{severity} of poverty. Squaring the +shortfalls gives more weight to the people furthest from the poverty line, +which makes the measure sensitive to inequality among the poor. } -FGT values range from 0 to 1. A value of 0 indicates that every individual is -above the poverty line. When every individual is below the poverty line, -however, FGT0 value is 1 and FGT1 and FGT2 values approach 1. +A value of 0 indicates that no one is in poverty. When \code{poor_below_threshold} +is \code{TRUE}, FGT values range from 0 to 1: FGT0 is 1 when everyone is below the +poverty line, and FGT1 and FGT2 are 1 only when everyone has an +accessibility of 0. When \code{poor_below_threshold} is \code{FALSE}, FGT0 still ranges +from 0 to 1, but FGT1 and FGT2 have no upper bound: they can be greater than +1 (e.g. when travel costs are more than twice the poverty line) and are +\code{Inf} if any accessibility value is \code{Inf}. } \examples{ @@ -112,10 +122,10 @@ poverty \insertAllCited{} } \seealso{ -Other inequality: +Other equity: \code{\link[=concentration_index]{concentration_index()}}, \code{\link[=gini_index]{gini_index()}}, \code{\link[=palma_ratio]{palma_ratio()}}, \code{\link[=theil_t]{theil_t()}} } -\concept{inequality} +\concept{equity} From 196fc4fd409b71a4323abddee72efdfa4fb59d8b Mon Sep 17 00:00:00 2001 From: rafapereirabr Date: Tue, 20 Oct 2026 22:35:03 -0300 Subject: [PATCH 24/31] optimize doubly_constrained --- NEWS.md | 5 +++++ R/doubly_constrained.R | 31 ++++++++++++++++--------------- 2 files changed, 21 insertions(+), 15 deletions(-) diff --git a/NEWS.md b/NEWS.md index 3d0063a..d48e8ae 100644 --- a/NEWS.md +++ b/NEWS.md @@ -139,6 +139,11 @@ (relative differences below 1e-14). `gravity()` with integer opportunities and `decay_binary()` no longer fails when accessibility exceeds the largest integer. +- `constrained_accessibility(constraint = "doubly")` is about 2 to 4 times + faster and uses much less memory on large travel matrices, because the + balancing factors are now calculated with matrix products instead of loops + over origins and destinations. Results may differ from previous versions by + floating point rounding only. - Errors and warnings raised by the package now use `{cli}`, with consistent formatting of argument names and values. Long lists of ids or columns in messages are abbreviated. `{cli}` and `{rlang}` are new dependencies. diff --git a/R/doubly_constrained.R b/R/doubly_constrained.R index de55e60..0ceaa0d 100644 --- a/R/doubly_constrained.R +++ b/R/doubly_constrained.R @@ -96,7 +96,7 @@ doubly_constrained <- function(travel_matrix, dimnames = list(origins, destinations) ) impedance_matrix[ - cbind(as.character(data$from_id), as.character(data$to_id)) + cbind(match(data$from_id, origins), match(data$to_id, destinations)) ] <- data$opp_weight # Iterative proportional fitting @@ -111,25 +111,23 @@ doubly_constrained <- function(travel_matrix, # Update Ai # a null denominator means the zone reaches (or is reached by) no one, so - # its factor is 0 and it gets no flow + # its factor is 0 and it gets no flow. the denominators of all zones are + # calculated at once with matrix products - for (i in seq_along(O)) { - denominator <- sum(Bj * D * impedance_matrix[i, ]) - Ai[i] <- if (denominator > 0) 1 / denominator else 0 - } + denominator_i <- drop(impedance_matrix %*% (Bj * D)) + Ai <- data.table::fifelse(denominator_i > 0, 1 / denominator_i, 0) # Update Bj - Bj_new <- numeric(length(D)) - for (j in seq_along(D)) { - denominator <- sum(Ai * O * impedance_matrix[, j]) - Bj_new[j] <- if (denominator > 0) 1 / denominator else 0 - } - - # Compute flows - Tij <- outer(Ai * O, Bj_new * D) * impedance_matrix + denominator_j <- drop(crossprod(impedance_matrix, Ai * O)) + Bj_new <- data.table::fifelse(denominator_j > 0, 1 / denominator_j, 0) # Compute error - error <- (sum(abs(O - rowSums(Tij))) + sum(abs(D - colSums(Tij)))) / sum(O) + # the row and column sums of the flows (Tij = Ai * O_i * Bj * D_j * f_ij) + # are calculated from the factors, without building the flow matrix + + flow_from <- Ai * O * drop(impedance_matrix %*% (Bj_new * D)) + flow_to <- Bj_new * D * denominator_j + error <- (sum(abs(O - flow_from)) + sum(abs(D - flow_to))) / sum(O) error_change <- abs(previous_error - error) if (error < error_threshold || error_change < improvement_threshold || iteration_count >= max_iterations) { @@ -152,6 +150,9 @@ doubly_constrained <- function(travel_matrix, ) } + # Compute flows + Tij <- outer(Ai * O, Bj * D) * impedance_matrix + # Compute kappa_doubly # kappa_ij^D = A_i * B_j * O_i * f(c_ij) kappa_matrix <- outer(Ai * O, Bj) * impedance_matrix From ff72154cebeba755506c0e4b1633372385cd876d Mon Sep 17 00:00:00 2001 From: rafapereirabr Date: Tue, 20 Oct 2026 22:43:38 -0300 Subject: [PATCH 25/31] fgt vignette --- vignettes/inequality_and_poverty.Rmd | 28 +++++++++++++++------------- 1 file changed, 15 insertions(+), 13 deletions(-) diff --git a/vignettes/inequality_and_poverty.Rmd b/vignettes/inequality_and_poverty.Rmd index 217bac8..da82c5a 100644 --- a/vignettes/inequality_and_poverty.Rmd +++ b/vignettes/inequality_and_poverty.Rmd @@ -199,20 +199,22 @@ theil_with_groups ## Foster-Greer-Thorbecke (FGT) poverty measures `fgt_poverty()` calculates the FGT metrics, a family of poverty measures -originally proposed by @foster1984class that capture the extent and severity of -poverty within an accessibility distribution. The FGT family is composed of -three measures that differ based on the $\alpha$ parameter used to calculate -them (either 0, 1 or 2) and which also changes their interpretation: - -- with $\alpha = 0$ (FGT0) the measure captures the extent of poverty as a - simple headcount - i.e. the proportion of people below the poverty line; +originally proposed by @foster1984class that capture the incidence, depth and +severity of poverty within an accessibility distribution. The FGT family is +composed of three measures that differ based on the $\alpha$ parameter used to +calculate them (either 0, 1 or 2) and which also changes their interpretation: + +- with $\alpha = 0$ (FGT0) the measure captures the incidence (or extent) of + poverty as a simple headcount - i.e. the proportion of people below the + poverty line; - with $\alpha = 1$ (FGT1) the measure, also known as the "poverty gap index", - captures the severity of poverty as the average percentage distance between - the poverty line and the accessibility of individuals below the poverty line; -- with $\alpha = 2$ (FGT2) the measure simultaneously captures the extent and - the severity of poverty by calculating the number of people below the poverty - line weighted by the size of the accessibility shortfall relative to the - poverty line. + captures the depth of poverty as the average shortfall from the poverty line, + relative to the poverty line, calculated over the whole population (people + above the poverty line have a shortfall of 0); +- with $\alpha = 2$ (FGT2) the measure, also known as the "squared poverty gap + index", captures the severity of poverty. By squaring the shortfalls, it gives + more weight to the people furthest below the poverty line, which makes it + sensitive to inequality among the poor. This function includes an additional `poverty_line` parameter, used to define the poverty line below which individuals are considered to be in accessibility From 4de9265041c6e5c19bc967decb332f2c5e34a375 Mon Sep 17 00:00:00 2001 From: rafapereirabr Date: Tue, 20 Oct 2026 23:03:06 -0300 Subject: [PATCH 26/31] optimize cost_to_closest and spatial_availability --- NEWS.md | 7 ++++ R/accessibility-package.R | 4 ++- R/cost_to_closest.R | 71 ++++++++++++++++++++++++--------------- R/spatial_availability.R | 34 +++++++++++-------- 4 files changed, 74 insertions(+), 42 deletions(-) diff --git a/NEWS.md b/NEWS.md index d48e8ae..8004598 100644 --- a/NEWS.md +++ b/NEWS.md @@ -144,6 +144,13 @@ balancing factors are now calculated with matrix products instead of loops over origins and destinations. Results may differ from previous versions by floating point rounding only. +- `spatial_availability()` is about 2 times faster on large travel matrices, + because its balancing factors are now calculated with sums optimized by + `{data.table}`. Results may differ from previous versions by floating point + rounding only. +- `cost_to_closest()` is about 1.3 to 2.3 times faster, especially with + several values of `n`, because the travel matrix is now filtered and sorted + only once, with identical results. - Errors and warnings raised by the package now use `{cli}`, with consistent formatting of argument names and values. Long lists of ids or columns in messages are abbreviated. `{cli}` and `{rlang}` are new dependencies. diff --git a/R/accessibility-package.R b/R/accessibility-package.R index 1b995a7..16e1f33 100644 --- a/R/accessibility-package.R +++ b/R/accessibility-package.R @@ -99,7 +99,9 @@ utils::globalVariables( ".weighted_opportunities", ".weighted_demand", ".weighted_ppr", - ".total_weight" + ".total_weight", + ".opp", + ".cost" ) ) diff --git a/R/cost_to_closest.R b/R/cost_to_closest.R index 09c3e20..26f856b 100644 --- a/R/cost_to_closest.R +++ b/R/cost_to_closest.R @@ -104,10 +104,7 @@ cost_to_closest <- function(travel_matrix, groups <- c(group_id, group_by) warn_extra_cols(travel_matrix, travel_cost, group_id, groups) - access <- lapply( - n, - function(.x) calculate_cost(data, groups, opportunity, travel_cost, .x) - ) + access <- calculate_cost(data, groups, opportunity, travel_cost, n) names(access) <- n access <- data.table::rbindlist(access, idcol = "n") access[, n := as.numeric(n)] @@ -149,32 +146,52 @@ cost_to_closest <- function(travel_matrix, } -calculate_cost <- function(data, groups, opportunity, travel_cost, .x) { +calculate_cost <- function(data, groups, opportunity, travel_cost, n) { env <- environment() .opportunity_colname <- opportunity - .cost_colname <- travel_cost - - if (.x == 1) { - cost <- data[ - get(.opportunity_colname) > 0, - .(min_cost = suppressWarnings(min(get(.cost_colname)))), - by = eval(groups, envir = env) - ] - } else { - opport_cumsum <- data[get(.opportunity_colname) > 0, ] - data.table::setorderv(opport_cumsum, c(groups, travel_cost)) - opport_cumsum[ - , - cum_opport := cumsum(get(.opportunity_colname)), - by = eval(groups, envir = env) - ] - - cost <- opport_cumsum[ - cum_opport >= .x, - .(min_cost = suppressWarnings(min(get(.cost_colname)))), - by = eval(groups, envir = env) - ] + + # only destinations (or origins) with opportunities are relevant. the costs + # and opportunities get fixed names, so that min() is optimized by + # data.table (GForce) + + data <- data[ + get(.opportunity_colname) > 0, + c(groups, opportunity, travel_cost), + with = FALSE + ] + data.table::setnames(data, c(opportunity, travel_cost), c(".opp", ".cost")) + + # n = 1 is calculated before sorting the data, to keep the order of the + # results. for larger values of n, the data is sorted and the cumulative + # number of opportunities is calculated only once. warnings are suppressed + # outside of j (so min() is still optimized), because data.table evaluates + # min() on an empty vector when no rows are selected + + if (1 %in% n) { + cost_n1 <- suppressWarnings( + data[, .(min_cost = min(.cost)), by = eval(groups, envir = env)] + ) + } + + if (any(n != 1)) { + data.table::setorderv(data, c(groups, ".cost")) + data[, cum_opport := cumsum(.opp), by = eval(groups, envir = env)] } + cost <- lapply( + n, + function(.x) { + if (.x == 1) return(cost_n1) + + suppressWarnings( + data[ + cum_opport >= .x, + .(min_cost = min(.cost)), + by = eval(groups, envir = env) + ] + ) + } + ) + return(cost) } diff --git a/R/spatial_availability.R b/R/spatial_availability.R index ea180f2..73acf5b 100644 --- a/R/spatial_availability.R +++ b/R/spatial_availability.R @@ -138,32 +138,38 @@ spatial_availability <- function(travel_matrix, # destinations that no one reaches (or that are only reached by origins # without demand) have null denominators. their opportunities are not - # allocated to any origin, so their factors are set to 0 instead of NaN + # allocated to any origin, so their factors are set to 0 instead of NaN. + # the denominators are calculated in a separate step, so the sums are + # optimized by data.table (GForce). the temporary column is dropped after + # each use, because the first sum may be an integer and the second a double + data[, .total_weight := sum(opp_weight), by = c("to_id", group_by)] data[ , - impedance_bal_fac := { - denominator <- sum(opp_weight) - if (isTRUE(denominator == 0)) 0 else opp_weight / denominator - }, - by = c("to_id", group_by) + impedance_bal_fac := data.table::fifelse( + !is.na(.total_weight) & .total_weight == 0, + 0, + opp_weight / .total_weight + ) ] + data[, .total_weight := NULL] + data[, combined_bal_fac := demand_bal_fac * impedance_bal_fac] + data[, .total_weight := sum(combined_bal_fac), by = c("to_id", group_by)] data[ , - combined_bal_fac := { - numerator <- demand_bal_fac * impedance_bal_fac - denominator <- sum(numerator) - if (isTRUE(denominator == 0)) 0 else numerator / denominator - }, - by = c("to_id", group_by) + combined_bal_fac := data.table::fifelse( + !is.na(.total_weight) & .total_weight == 0, + 0, + combined_bal_fac / .total_weight + ) ] + data[, .total_weight := NULL] .opportunity_colname <- opportunity data[ , - spatial_availability := combined_bal_fac * get(.opportunity_colname), - by = c("to_id", group_by) + spatial_availability := combined_bal_fac * get(.opportunity_colname) ] if (detailed_results) { From 150aba1ce5ec3c60e58c45ce01766a551c3e2a91 Mon Sep 17 00:00:00 2001 From: rafapereirabr Date: Tue, 20 Oct 2026 23:15:40 -0300 Subject: [PATCH 27/31] fgt_poverty corner case --- NEWS.md | 7 ++++ R/fgt_poverty.R | 34 ++++++++++++++++++- man/fgt_poverty.Rd | 6 +++- tests/testthat/test-fgt_poverty.R | 49 ++++++++++++++++++++++++++++ vignettes/inequality_and_poverty.Rmd | 37 +++++++++++++++++++++ 5 files changed, 131 insertions(+), 2 deletions(-) diff --git a/NEWS.md b/NEWS.md index 8004598..0667efd 100644 --- a/NEWS.md +++ b/NEWS.md @@ -155,6 +155,13 @@ formatting of argument names and values. Long lists of ids or columns in messages are abbreviated. `{cli}` and `{rlang}` are new dependencies. +- `fgt_poverty()` now warns when `poor_below_threshold = FALSE` and + `accessibility_data` contains `Inf` values (e.g. from `cost_to_closest()` for + origins that can't reach the required opportunities), which make FGT1 and + FGT2 infinite. The documentation and the inequality and poverty vignette now + explain this case, and the vignette includes an example with + `poor_below_threshold = FALSE`. + ## Notes - Updated the package website and GitHub URLs after the repository moved from diff --git a/R/fgt_poverty.R b/R/fgt_poverty.R index 9aab462..1751155 100644 --- a/R/fgt_poverty.R +++ b/R/fgt_poverty.R @@ -66,7 +66,11 @@ #' accessibility of 0. When `poor_below_threshold` is `FALSE`, FGT0 still ranges #' from 0 to 1, but FGT1 and FGT2 have no upper bound: they can be greater than #' 1 (e.g. when travel costs are more than twice the poverty line) and are -#' `Inf` if any accessibility value is `Inf`. +#' `Inf` if any accessibility value is `Inf`. This happens, for example, with +#' the travel costs of origins that can't reach the required opportunities, as +#' returned by [cost_to_closest()]. The function raises a warning in this case, +#' and these values should be replaced by a maximum travel cost (e.g. the +#' highest cost in the travel matrix) before calculating the FGT measures. #' #' @references #' \insertAllCited{} @@ -144,6 +148,34 @@ fgt_poverty <- function( warn_extra_access_cols(accessibility_data, opportunity, group_by) + # with dual measures, infinite costs (e.g. from cost_to_closest() when an + # origin can't reach the required opportunities) make FGT1 and FGT2 infinite + + if (!poor_below_threshold) { + inf_rows <- is.infinite(data[[opportunity]]) & + data[[opportunity]] > 0 & + data[[population]] > 0 + inf_rows <- inf_rows & !is.na(inf_rows) + + if (any(inf_rows)) { + n_inf <- sum(inf_rows) + inf_population <- sum(data[[population]][inf_rows]) + + cli::cli_warn( + c( + "{.arg accessibility_data${opportunity}} contains {.val {Inf}} values + in {n_inf} observation{?s} with population greater than 0 (total + population of {inf_population}).", + "i" = "With {.code poor_below_threshold = FALSE}, FGT1 and FGT2 are + {.val {Inf}} wherever these observations are included.", + "i" = "Consider replacing {.val {Inf}} values with a maximum travel + cost (e.g. the highest cost in the travel matrix) before calculating + the FGT measures." + ) + ) + } + } + .opp_colname <- opportunity .pop_colname <- population .groups <- group_by diff --git a/man/fgt_poverty.Rd b/man/fgt_poverty.Rd index 1b27759..261d51f 100644 --- a/man/fgt_poverty.Rd +++ b/man/fgt_poverty.Rd @@ -91,7 +91,11 @@ poverty line, and FGT1 and FGT2 are 1 only when everyone has an accessibility of 0. When \code{poor_below_threshold} is \code{FALSE}, FGT0 still ranges from 0 to 1, but FGT1 and FGT2 have no upper bound: they can be greater than 1 (e.g. when travel costs are more than twice the poverty line) and are -\code{Inf} if any accessibility value is \code{Inf}. +\code{Inf} if any accessibility value is \code{Inf}. This happens, for example, with +the travel costs of origins that can't reach the required opportunities, as +returned by \code{\link[=cost_to_closest]{cost_to_closest()}}. The function raises a warning in this case, +and these values should be replaced by a maximum travel cost (e.g. the +highest cost in the travel matrix) before calculating the FGT measures. } \examples{ diff --git a/tests/testthat/test-fgt_poverty.R b/tests/testthat/test-fgt_poverty.R index d140572..4e13de6 100644 --- a/tests/testthat/test-fgt_poverty.R +++ b/tests/testthat/test-fgt_poverty.R @@ -270,3 +270,52 @@ test_that("fgt measures match hand-computed values on a toy example", { ) ) }) + +test_that("warns about Inf values when poor_below_threshold = FALSE", { + toy_access <- data.table::data.table( + id = c("a", "b", "c"), + travel_time = c(10, 30, Inf) + ) + toy_sociodem <- data.table::data.table( + id = c("a", "b", "c"), + population = c(1, 1, 1) + ) + + expect_warning( + result <- tester( + toy_access, + toy_sociodem, + opportunity = "travel_time", + poverty_line = 20, + poor_below_threshold = FALSE, + group_by = character(0) + ), + regexp = "Inf" + ) + expect_identical(result$FGT1, Inf) + + # no warning when Inf is not a problem: with primal measures, and when the + # observation has no population + + expect_silent( + tester( + toy_access, + toy_sociodem, + opportunity = "travel_time", + poverty_line = 20, + group_by = character(0) + ) + ) + + toy_sociodem[id == "c", population := 0] + expect_silent( + tester( + toy_access, + toy_sociodem, + opportunity = "travel_time", + poverty_line = 20, + poor_below_threshold = FALSE, + group_by = character(0) + ) + ) +}) diff --git a/vignettes/inequality_and_poverty.Rmd b/vignettes/inequality_and_poverty.Rmd index da82c5a..88c1178 100644 --- a/vignettes/inequality_and_poverty.Rmd +++ b/vignettes/inequality_and_poverty.Rmd @@ -232,4 +232,41 @@ poverty <- fgt_poverty( poverty ``` +By default, `fgt_poverty()` considers that people with accessibility *below* +the poverty line are poor, which is the right approach for primal accessibility +measures, such as the number of accessible jobs. For dual measures, such as the +travel time to the closest facility calculated with `cost_to_closest()`, higher +values indicate worse access, so people *above* the poverty line should be +considered poor. This is done by setting `poor_below_threshold = FALSE`. In +this case, FGT1 and FGT2 measure the excess travel cost relative to the poverty +line, so they are not bounded by 1. In the example below, we consider that +people who take more than 20 minutes to reach the closest school are in +accessibility poverty. Note that `cost_to_closest()` returns `Inf` for origins +that can't reach any school in the travel matrix, which would make FGT1 and +FGT2 infinite (`fgt_poverty()` warns about it), so we replace these values by +the longest travel time in the travel matrix (120 minutes) beforehand. + +```{r} +time_to_school <- cost_to_closest( + travel_matrix, + land_use_data, + opportunity = "schools", + travel_cost = "travel_time" +) + +unreachable <- is.infinite(time_to_school$travel_time) +time_to_school$travel_time[unreachable] <- 120 + +poverty_dual <- fgt_poverty( + time_to_school, + sociodemographic_data = land_use_data, + opportunity = "travel_time", + population = "population", + poverty_line = 20, + poor_below_threshold = FALSE +) + +poverty_dual +``` + # References From 8f58141dd7e6073b5bebf1f2f842ca5d431ff632 Mon Sep 17 00:00:00 2001 From: rafapereirabr Date: Tue, 20 Oct 2026 23:36:51 -0300 Subject: [PATCH 28/31] accessibility measures now treat NA values and missing ids in land_use_data as 0 --- NEWS.md | 16 +- R/assert_and_assign.R | 16 +- R/balancing_cost.R | 28 ++- R/cost_to_closest.R | 3 +- R/cumulative_cutoff.R | 3 +- R/cumulative_interval.R | 3 +- R/doubly_constrained.R | 37 +++- R/floating_catchment_area.R | 16 +- R/gravity.R | 3 +- R/merge_by_reference.R | 17 +- R/singly_constrained.R | 16 +- R/spatial_availability.R | 16 +- R/total_constrained.R | 8 +- man/balancing_cost.Rd | 3 +- man/constrained_accessibility.Rd | 3 +- man/cost_to_closest.Rd | 3 +- man/cumulative_cutoff.Rd | 3 +- man/cumulative_interval.Rd | 3 +- man/floating_catchment_area.Rd | 3 +- man/gravity.Rd | 3 +- man/roxygen/templates/land_use_data.R | 3 +- man/spatial_availability.Rd | 3 +- tests/testthat/test-assert_land_use_data.R | 15 +- tests/testthat/test-balancing_cost.R | 44 ++++ tests/testthat/test-constrained-essentials.R | 68 ++++++- tests/testthat/test-merge_by_reference.R | 17 ++ tests/testthat/test-na_land_use.R | 199 +++++++++++++++++++ 27 files changed, 488 insertions(+), 64 deletions(-) create mode 100644 tests/testthat/test-na_land_use.R diff --git a/NEWS.md b/NEWS.md index 0667efd..e212350 100644 --- a/NEWS.md +++ b/NEWS.md @@ -16,6 +16,13 @@ factor with the bounds of the whole accessibility distribution, instead of the bounds of each `group_by` group, so groups are corrected on a common scale. Results only change when `group_by` is used. +- Accessibility functions now treat `NA` values in `land_use_data` + (opportunities, demand and supply) and ids of `travel_matrix` missing from + `land_use_data` as 0, and their warnings say so. Previously, such a value + made the results of every id that reached it `NA`, except when + `fill_missing_ids = TRUE` had to fill other missing ids, in which case + those `NA` results became 0. When every value is `NA`, results are now the + same as when every value is 0 (which can be `NaN`). ## Bug fixes @@ -35,8 +42,7 @@ position instead of by id. Results were only correct when `travel_matrix` listed every origin-destination pair ordered by destination and `land_use_data` was ordered like the travel matrix. Pairs missing from - `travel_matrix` are now treated as unreachable, and ids missing from - `land_use_data` raise an informative error. + `travel_matrix` are now treated as unreachable. - Fixed a bug in `theil_t()` in which observations with no access were wrongly handled: without `socioeconomic_groups` they were excluded from the index (but not from the average accessibility), and with @@ -44,14 +50,14 @@ the index, following the usual `0 * log(0) = 0` convention. `group_by` levels in which no one has access are now kept in the output with `NA`, instead of being dropped. +- Fixed a bug in `spatial_availability()` in which an `NA` demand in + `land_use_data` made every result `NA`, even when the id with the `NA` was + not in `travel_matrix`. - Fixed two bugs in `balancing_cost()`: origins only balanced when they reached more opportunities than people (instead of as many opportunities as people), and origins that balanced at a cost greater than the last multiple of `cost_increment` were reported as `NA`. `cost_increment` must now be greater than 0. -- Fixed a bug in `spatial_availability()` in which an `NA` demand in - `land_use_data` made every result `NA`, even when the id with the `NA` was - not in `travel_matrix`. - Fixed a bug in `constrained_accessibility(constraint = "doubly")` in which a small constant added to the balancing factors distorted the results when impedance weights were very small (e.g. with steep decay functions or long diff --git a/R/assert_and_assign.R b/R/assert_and_assign.R index e40478c..e258a18 100644 --- a/R/assert_and_assign.R +++ b/R/assert_and_assign.R @@ -135,8 +135,8 @@ assert_land_use_data <- function(land_use_data, ) } - # if an id is present in travel_matrix but not in land_use_data, merging the - # datasets will produce NAs, so we warn users about that + # if an id is present in travel_matrix but not in land_use_data, its values + # are treated as 0, which may hide data problems, so we warn users about that travel_matrix_ids <- if (is.null(active)) { c(travel_matrix$from_id, travel_matrix$to_id) @@ -163,13 +163,13 @@ assert_land_use_data <- function(land_use_data, if (!all(travel_matrix_ids %in% land_use_data$id)) { cli::cli_warn( - "{.arg land_use_data} is missing ids listed in {.arg travel_matrix}, - which may produce NAs in the final output." + "{.arg land_use_data} is missing ids listed in {.arg travel_matrix}. + Their values are treated as 0." ) } - # if either land_use_data$opportunity or $demand contains NA values, the final - # result may contain NAs as well, so we warn about that too + # NA values in land_use_data$opportunity or $demand are treated as 0, which + # may hide data problems, so we warn about that too cols_to_check <- c( opportunity, @@ -179,8 +179,8 @@ assert_land_use_data <- function(land_use_data, for (col in cols_to_check) { if (any(is.na(land_use_data[[col]]))) { cli::cli_warn( - "{.arg land_use_data${col}} contains NA values, which may produce NAs - in the final output." + "{.arg land_use_data${col}} contains NA values, which are treated as + 0." ) } } diff --git a/R/balancing_cost.R b/R/balancing_cost.R index e6d6747..efd8624 100644 --- a/R/balancing_cost.R +++ b/R/balancing_cost.R @@ -97,13 +97,23 @@ balancing_cost <- function(travel_matrix, land_use_data <- data.table::as.data.table(land_use_data) } - merge_by_reference(data, land_use_data, opportunity, left_df_idcol = "to_id") + merge_by_reference( + data, + land_use_data, + opportunity, + left_df_idcol = "to_id", + na_as_zero = TRUE + ) groups <- c("from_id", group_by) warn_extra_cols(travel_matrix, travel_cost, "from_id", groups) - lookup_vec <- land_use_data[[demand]] - names(lookup_vec) <- land_use_data$id + # demand of each origin, matched by id. NA values and ids missing from + # land_use_data count as 0 people + + origins <- unique(data$from_id) + origin_demand <- land_use_data[[demand]][match(origins, land_use_data$id)] + origin_demand[is.na(origin_demand)] <- 0 max_lookup_cost <- if (length(data[[travel_cost]]) > 0) { max(data[[travel_cost]], na.rm = TRUE) @@ -122,9 +132,7 @@ balancing_cost <- function(travel_matrix, # origins with 0 people should always have a balancing time of 0 - .demand_colname <- demand - no_pop_origins <- land_use_data[get(.demand_colname) == 0]$id - no_pop_origins_in_ttm <- no_pop_origins[no_pop_origins %in% data$from_id] + no_pop_origins_in_ttm <- origins[origin_demand == 0] grouped_no_pop_origins <- lapply( groups, @@ -143,7 +151,13 @@ balancing_cost <- function(travel_matrix, for (i in lookup_costs) { cum_opps <- calc_cum_cutoff(data, groups, opportunity, travel_cost, i) - cum_opps[, did_balance := access >= lookup_vec[as.character(from_id)]] + # set() keeps origin_demand from being shadowed by a column of cum_opps + + data.table::set( + cum_opps, + j = "did_balance", + value = cum_opps$access >= origin_demand[match(cum_opps$from_id, origins)] + ) balanced <- cum_opps[did_balance == TRUE] if (nrow(balanced) > 0) { diff --git a/R/cost_to_closest.R b/R/cost_to_closest.R index 26f856b..84f6563 100644 --- a/R/cost_to_closest.R +++ b/R/cost_to_closest.R @@ -97,7 +97,8 @@ cost_to_closest <- function(travel_matrix, data, land_use_data, opportunity, - left_df_idcol = ifelse(active, "to_id", "from_id") + left_df_idcol = ifelse(active, "to_id", "from_id"), + na_as_zero = TRUE ) group_id <- ifelse(active, "from_id", "to_id") diff --git a/R/cumulative_cutoff.R b/R/cumulative_cutoff.R index 56c693f..060ec3a 100644 --- a/R/cumulative_cutoff.R +++ b/R/cumulative_cutoff.R @@ -131,7 +131,8 @@ cumulative_cutoff <- function(travel_matrix, data, land_use_data, opportunity, - left_df_idcol = ifelse(active, "to_id", "from_id") + left_df_idcol = ifelse(active, "to_id", "from_id"), + na_as_zero = TRUE ) group_id <- ifelse(active, "from_id", "to_id") diff --git a/R/cumulative_interval.R b/R/cumulative_interval.R index bc15398..1e88037 100644 --- a/R/cumulative_interval.R +++ b/R/cumulative_interval.R @@ -125,7 +125,8 @@ cumulative_interval <- function(travel_matrix, data, land_use_data, opportunity, - left_df_idcol = ifelse(active, "to_id", "from_id") + left_df_idcol = ifelse(active, "to_id", "from_id"), + na_as_zero = TRUE ) group_id <- ifelse(active, "from_id", "to_id") diff --git a/R/doubly_constrained.R b/R/doubly_constrained.R index 0ceaa0d..b149893 100644 --- a/R/doubly_constrained.R +++ b/R/doubly_constrained.R @@ -55,8 +55,20 @@ doubly_constrained <- function(travel_matrix, } # Merge demand and supply - merge_by_reference(data, land_use_data, demand, left_df_idcol = "from_id") - merge_by_reference(data, land_use_data, supply, left_df_idcol = "to_id") + merge_by_reference( + data, + land_use_data, + demand, + left_df_idcol = "from_id", + na_as_zero = TRUE + ) + merge_by_reference( + data, + land_use_data, + supply, + left_df_idcol = "to_id", + na_as_zero = TRUE + ) # Apply decay data <- apply_gravity_measure(data, decay_function, travel_cost) @@ -67,20 +79,25 @@ doubly_constrained <- function(travel_matrix, O <- land_use_data[[demand]][match(origins, land_use_data$id)] D <- land_use_data[[supply]][match(destinations, land_use_data$id)] - if (anyNA(O) || anyNA(D)) { + # NA values and ids missing from land_use_data count as 0 demand/supply + + O[is.na(O)] <- 0 + D[is.na(D)] <- 0 + + # Validate totals match + if (abs(sum(O) - sum(D)) > 1e-6) { cli::cli_abort( - "For doubly-constrained, {.arg land_use_data} must list a non-NA demand - for every origin and a non-NA supply for every destination in - {.arg travel_matrix}.", + "For doubly-constrained, the sum of origins must equal the sum of + destinations. NA values and ids missing from {.arg land_use_data} count + as 0.", call = NULL ) } - # Validate totals match - if (abs(sum(O) - sum(D)) > 1e-6) { + if (sum(O) <= 0) { cli::cli_abort( - "For doubly-constrained, the sum of origins must equal the sum of - destinations.", + "For doubly-constrained, the total demand and supply must be greater + than 0.", call = NULL ) } diff --git a/R/floating_catchment_area.R b/R/floating_catchment_area.R index f5b97a1..bd4f732 100644 --- a/R/floating_catchment_area.R +++ b/R/floating_catchment_area.R @@ -111,8 +111,20 @@ floating_catchment_area <- function(travel_matrix, land_use_data <- data.table::as.data.table(land_use_data) } - merge_by_reference(data, land_use_data, opportunity, left_df_idcol = "to_id") - merge_by_reference(data, land_use_data, demand, left_df_idcol = "from_id") + merge_by_reference( + data, + land_use_data, + opportunity, + left_df_idcol = "to_id", + na_as_zero = TRUE + ) + merge_by_reference( + data, + land_use_data, + demand, + left_df_idcol = "from_id", + na_as_zero = TRUE + ) data <- apply_gravity_measure(data, decay_function, travel_cost) diff --git a/R/gravity.R b/R/gravity.R index 6f35966..e99f971 100644 --- a/R/gravity.R +++ b/R/gravity.R @@ -80,7 +80,8 @@ gravity <- function(travel_matrix, data, land_use_data, opportunity, - left_df_idcol = ifelse(active, "to_id", "from_id") + left_df_idcol = ifelse(active, "to_id", "from_id"), + na_as_zero = TRUE ) data <- apply_gravity_measure(data, decay_function, travel_cost) diff --git a/R/merge_by_reference.R b/R/merge_by_reference.R index b99821f..437dfda 100644 --- a/R/merge_by_reference.R +++ b/R/merge_by_reference.R @@ -2,7 +2,8 @@ merge_by_reference <- function(data, land_use_data, opportunity, - left_df_idcol) { + left_df_idcol, + na_as_zero = FALSE) { # when calculating active accessibility, we want the number of opportunities # in the destination. if passive, the number of opportunities in the origin @@ -20,9 +21,13 @@ merge_by_reference <- function(data, match(left_ids, right_ids) } - data.table::set( - data, - j = opportunity, - value = land_use_data[[opportunity]][match_index] - ) + values <- land_use_data[[opportunity]][match_index] + + # in accessibility measures, NA values and ids missing from land_use_data + # count as 0 opportunities (or 0 demand/supply). the inequality and poverty + # functions keep the NAs + + if (na_as_zero && anyNA(values)) values[is.na(values)] <- 0L + + data.table::set(data, j = opportunity, value = values) } diff --git a/R/singly_constrained.R b/R/singly_constrained.R index 9b9d5d4..cc2bf58 100644 --- a/R/singly_constrained.R +++ b/R/singly_constrained.R @@ -47,8 +47,20 @@ singly_constrained <- function(travel_matrix, } # Merge demand and supply - merge_by_reference(data, land_use_data, demand, left_df_idcol = "from_id") - merge_by_reference(data, land_use_data, supply, left_df_idcol = "to_id") + merge_by_reference( + data, + land_use_data, + demand, + left_df_idcol = "from_id", + na_as_zero = TRUE + ) + merge_by_reference( + data, + land_use_data, + supply, + left_df_idcol = "to_id", + na_as_zero = TRUE + ) # Apply decay data <- apply_gravity_measure(data, decay_function, travel_cost) diff --git a/R/spatial_availability.R b/R/spatial_availability.R index 73acf5b..e829d56 100644 --- a/R/spatial_availability.R +++ b/R/spatial_availability.R @@ -110,8 +110,20 @@ spatial_availability <- function(travel_matrix, land_use_data <- data.table::as.data.table(land_use_data) } - merge_by_reference(data, land_use_data, opportunity, left_df_idcol = "to_id") - merge_by_reference(data, land_use_data, demand, left_df_idcol = "from_id") + merge_by_reference( + data, + land_use_data, + opportunity, + left_df_idcol = "to_id", + na_as_zero = TRUE + ) + merge_by_reference( + data, + land_use_data, + demand, + left_df_idcol = "from_id", + na_as_zero = TRUE + ) data <- apply_gravity_measure(data, decay_function, travel_cost) diff --git a/R/total_constrained.R b/R/total_constrained.R index 6fed885..6f89364 100644 --- a/R/total_constrained.R +++ b/R/total_constrained.R @@ -93,7 +93,13 @@ total_constrained <- function(travel_matrix, } # Merge land use data - merge_by_reference(data, land_use_data, opportunity_col, left_df_idcol = merge_id) + merge_by_reference( + data, + land_use_data, + opportunity_col, + left_df_idcol = merge_id, + na_as_zero = TRUE + ) # Apply decay data <- apply_gravity_measure(data, decay_function, travel_cost) diff --git a/man/balancing_cost.Rd b/man/balancing_cost.Rd index 993c33a..ed54a55 100644 --- a/man/balancing_cost.Rd +++ b/man/balancing_cost.Rd @@ -23,7 +23,8 @@ and any others specified in \code{travel_cost}.} \item{land_use_data}{A data frame. The distribution of opportunities within the study area cells. Must contain the columns \code{id} and any others -specified in \code{opportunity}.} +specified in \code{opportunity}. \code{NA} values, and ids in \code{travel_matrix} that +are missing from this table, are treated as 0.} \item{opportunity}{A string. The name of the column in \code{land_use_data} with the number of opportunities/resources/services to be considered when diff --git a/man/constrained_accessibility.Rd b/man/constrained_accessibility.Rd index b3fa46f..5dba3a4 100644 --- a/man/constrained_accessibility.Rd +++ b/man/constrained_accessibility.Rd @@ -29,7 +29,8 @@ and any others specified in \code{travel_cost}.} \item{land_use_data}{A data frame. The distribution of opportunities within the study area cells. Must contain the columns \code{id} and any others -specified in \code{opportunity}.} +specified in \code{opportunity}. \code{NA} values, and ids in \code{travel_matrix} that +are missing from this table, are treated as 0.} \item{travel_cost}{A string. The name of the column in \code{travel_matrix} with the travel cost between origins and destinations. The notion of cost diff --git a/man/cost_to_closest.Rd b/man/cost_to_closest.Rd index 4270df2..0545740 100644 --- a/man/cost_to_closest.Rd +++ b/man/cost_to_closest.Rd @@ -23,7 +23,8 @@ and any others specified in \code{travel_cost}.} \item{land_use_data}{A data frame. The distribution of opportunities within the study area cells. Must contain the columns \code{id} and any others -specified in \code{opportunity}.} +specified in \code{opportunity}. \code{NA} values, and ids in \code{travel_matrix} that +are missing from this table, are treated as 0.} \item{opportunity}{A string. The name of the column in \code{land_use_data} with the number of opportunities/resources/services to be considered when diff --git a/man/cumulative_cutoff.Rd b/man/cumulative_cutoff.Rd index 6e3a934..02023c5 100644 --- a/man/cumulative_cutoff.Rd +++ b/man/cumulative_cutoff.Rd @@ -23,7 +23,8 @@ and any others specified in \code{travel_cost}.} \item{land_use_data}{A data frame. The distribution of opportunities within the study area cells. Must contain the columns \code{id} and any others -specified in \code{opportunity}.} +specified in \code{opportunity}. \code{NA} values, and ids in \code{travel_matrix} that +are missing from this table, are treated as 0.} \item{opportunity}{A string. The name of the column in \code{land_use_data} with the number of opportunities/resources/services to be considered when diff --git a/man/cumulative_interval.Rd b/man/cumulative_interval.Rd index 9bae7e1..7ec74f1 100644 --- a/man/cumulative_interval.Rd +++ b/man/cumulative_interval.Rd @@ -25,7 +25,8 @@ and any others specified in \code{travel_cost}.} \item{land_use_data}{A data frame. The distribution of opportunities within the study area cells. Must contain the columns \code{id} and any others -specified in \code{opportunity}.} +specified in \code{opportunity}. \code{NA} values, and ids in \code{travel_matrix} that +are missing from this table, are treated as 0.} \item{opportunity}{A string. The name of the column in \code{land_use_data} with the number of opportunities/resources/services to be considered when diff --git a/man/floating_catchment_area.Rd b/man/floating_catchment_area.Rd index e67b34a..6d05407 100644 --- a/man/floating_catchment_area.Rd +++ b/man/floating_catchment_area.Rd @@ -24,7 +24,8 @@ and any others specified in \code{travel_cost}.} \item{land_use_data}{A data frame. The distribution of opportunities within the study area cells. Must contain the columns \code{id} and any others -specified in \code{opportunity}.} +specified in \code{opportunity}. \code{NA} values, and ids in \code{travel_matrix} that +are missing from this table, are treated as 0.} \item{opportunity}{A string. The name of the column in \code{land_use_data} with the number of opportunities/resources/services to be considered when diff --git a/man/gravity.Rd b/man/gravity.Rd index 404835a..0fac893 100644 --- a/man/gravity.Rd +++ b/man/gravity.Rd @@ -23,7 +23,8 @@ and any others specified in \code{travel_cost}.} \item{land_use_data}{A data frame. The distribution of opportunities within the study area cells. Must contain the columns \code{id} and any others -specified in \code{opportunity}.} +specified in \code{opportunity}. \code{NA} values, and ids in \code{travel_matrix} that +are missing from this table, are treated as 0.} \item{opportunity}{A string. The name of the column in \code{land_use_data} with the number of opportunities/resources/services to be considered when diff --git a/man/roxygen/templates/land_use_data.R b/man/roxygen/templates/land_use_data.R index aa199fd..0758313 100644 --- a/man/roxygen/templates/land_use_data.R +++ b/man/roxygen/templates/land_use_data.R @@ -1,3 +1,4 @@ #' @param land_use_data A data frame. The distribution of opportunities within #' the study area cells. Must contain the columns `id` and any others -#' specified in `opportunity`. +#' specified in `opportunity`. `NA` values, and ids in `travel_matrix` that +#' are missing from this table, are treated as 0. diff --git a/man/spatial_availability.Rd b/man/spatial_availability.Rd index c4f6898..6b96f57 100644 --- a/man/spatial_availability.Rd +++ b/man/spatial_availability.Rd @@ -25,7 +25,8 @@ and any others specified in \code{travel_cost}.} \item{land_use_data}{A data frame. The distribution of opportunities within the study area cells. Must contain the columns \code{id} and any others -specified in \code{opportunity}.} +specified in \code{opportunity}. \code{NA} values, and ids in \code{travel_matrix} that +are missing from this table, are treated as 0.} \item{opportunity}{A string. The name of the column in \code{land_use_data} with the number of opportunities/resources/services to be considered when diff --git a/tests/testthat/test-assert_land_use_data.R b/tests/testthat/test-assert_land_use_data.R index b111f4f..18c40ce 100644 --- a/tests/testthat/test-assert_land_use_data.R +++ b/tests/testthat/test-assert_land_use_data.R @@ -27,7 +27,8 @@ test_that("warns if travel_matrix contains ids that land_use_data does not", { travel_time = 50 ) ) - ) + ), + "treated as 0" ) expect_warning( @@ -42,7 +43,8 @@ test_that("warns if travel_matrix contains ids that land_use_data does not", { ) ), active = FALSE - ) + ), + "treated as 0" ) expect_warning( @@ -57,7 +59,8 @@ test_that("warns if travel_matrix contains ids that land_use_data does not", { ) ), active = NULL - ) + ), + "treated as 0" ) expect_warning( @@ -72,7 +75,8 @@ test_that("warns if travel_matrix contains ids that land_use_data does not", { ) ), active = NULL - ) + ), + "treated as 0" ) }) @@ -81,7 +85,8 @@ test_that("warns if either opportunity or demand cols contain NAs", { tester( rbind(land_use_data, data.table::data.table(id = "hehe"), fill = TRUE), demand = "population" - ) + ), + "treated as 0" ) }) diff --git a/tests/testthat/test-balancing_cost.R b/tests/testthat/test-balancing_cost.R index 95e8772..277525d 100644 --- a/tests/testthat/test-balancing_cost.R +++ b/tests/testthat/test-balancing_cost.R @@ -292,3 +292,47 @@ test_that("works even if travel_matrix and land_use has specific colnames", { land_use_data[, demand := NULL] }) + +test_that("NA demand and missing ids count as 0 people", { + # "a" has NA population, "b" has 0 and "d" is missing from land_use_data, so + # they all balance at cost 0. "c" (30 people) reaches 5 jobs at cost 0, 25 at + # 15 and 35 at 20, so it balances at 20 (and at 40 when costs are doubled) + + travel_matrix <- data.table::data.table( + from_id = rep(c("a", "b", "c", "d"), each = 4), + to_id = rep(c("a", "b", "c", "d"), times = 4), + travel_time = c(0, 10, 20, 30, 10, 0, 15, 25, 20, 15, 0, 10, 30, 25, 10, 0) + ) + land_use_data <- data.table::data.table( + id = c("a", "b", "c"), + jobs = c(10, 20, 5), + population = c(NA, 0, 30) + ) + + result <- suppressWarnings( + tester(travel_matrix, land_use_data, group_by = character(0)) + ) + data.table::setkey(result, NULL) + data.table::setorderv(result, "id") + expect_identical(result$id, c("a", "b", "c", "d")) + expect_identical(result$travel_time, c(0, 0, 20, 0)) + + doubled_matrix <- data.table::copy(travel_matrix)[ + , + travel_time := travel_time * 2 + ] + two_modes <- rbind( + data.table::copy(travel_matrix)[, mode := "walk"], + doubled_matrix[, mode := "slow_walk"] + ) + + result <- suppressWarnings( + tester(two_modes, land_use_data, fill_missing_ids = FALSE) + ) + expect_identical(nrow(result), 8L) + expect_identical( + result[id == "c"][order(mode)]$travel_time, + c(40, 20) + ) + expect_true(all(result[id != "c"]$travel_time == 0)) +}) diff --git a/tests/testthat/test-constrained-essentials.R b/tests/testthat/test-constrained-essentials.R index 23e5ed2..9bba5d2 100644 --- a/tests/testthat/test-constrained-essentials.R +++ b/tests/testthat/test-constrained-essentials.R @@ -289,8 +289,9 @@ test_that("doubly-constrained: results don't depend on row order or sparsity", { expect_identical(result[from_id == "1" & to_id == "3"]$flow, 0) }) -test_that("doubly-constrained: errors when land_use_data misses an id", { - # the validators also warn about the missing id +test_that("doubly-constrained: errors when totals don't match", { + # the missing id counts as 0, so the totals no longer match. the validators + # also warn about the missing id expect_error( suppressWarnings(constrained_accessibility( @@ -303,7 +304,68 @@ test_that("doubly-constrained: errors when land_use_data misses an id", { supply = "jobs", detailed_results = TRUE )), - regexp = "must list a non-NA demand" + regexp = "sum[[:space:]]+of[[:space:]]+origins[[:space:]]+must" + ) +}) + +test_that("doubly-constrained: NA values and missing ids count as 0", { + # id "4" is in travel_matrix but has NA demand and supply (or is missing from + # land_use_data). it counts as 0, so it gets no flows and the flows between + # the other ids are the same as without it + + tm_4 <- data.table::data.table( + expand.grid(from_id = c("1", "2", "3", "4"), to_id = c("1", "2", "3", "4")) + ) + tm_4[, travel_time := 10 + 5 * abs(as.integer(from_id) - as.integer(to_id))] + tm_3 <- tm_4[from_id != "4" & to_id != "4"] + + lu_na <- rbind( + make_lu_match20(), + data.table::data.table(id = "4", population = NA, jobs = NA) + ) + + run <- function(tm, lu) { + suppressWarnings(constrained_accessibility( + constraint = "doubly", + travel_matrix = tm, + land_use_data = lu, + travel_cost = "travel_time", + decay_function = exp_decay_01, + demand = "population", + supply = "jobs", + detailed_results = TRUE + )) + } + + expected <- run(tm_3, make_lu_match20()) + + for (lu in list(lu_na, make_lu_match20())) { + result <- run(tm_4, lu) + + expect_identical(result[from_id == "4" | to_id == "4"]$flow, rep(0, 7)) + expect_equal( + result[from_id != "4" & to_id != "4"]$flow, + expected$flow, + tolerance = 1e-6 + ) + } +}) + +test_that("doubly-constrained: errors when total demand and supply are 0", { + lu_zero <- make_lu_match20()[, `:=`(population = NA, jobs = 0)] + + expect_error( + suppressWarnings(constrained_accessibility( + constraint = "doubly", + travel_matrix = make_tm_3x3(), + land_use_data = lu_zero, + travel_cost = "travel_time", + decay_function = exp_decay_01, + demand = "population", + supply = "jobs", + detailed_results = TRUE + )), + regexp = "total[[:space:]]+demand[[:space:]]+and[[:space:]]+supply" ) }) diff --git a/tests/testthat/test-merge_by_reference.R b/tests/testthat/test-merge_by_reference.R index 03fcce5..3c0b3b1 100644 --- a/tests/testthat/test-merge_by_reference.R +++ b/tests/testthat/test-merge_by_reference.R @@ -38,3 +38,20 @@ test_that("matches numeric and factor ids", { tester(data, land_use_data) expect_identical(data$jobs, c(2, 1)) }) + +test_that("na_as_zero turns NA values and missing ids into 0", { + data <- data.table::data.table(to_id = c("a", "b", "c", "z")) + land_use_data <- data.table::data.table( + id = c("a", "b", "c"), + jobs = c(10L, NA, 5L) + ) + original_land_use <- data.table::copy(land_use_data) + + merge_by_reference(data, land_use_data, "jobs", "to_id", na_as_zero = TRUE) + expect_identical(data$jobs, c(10L, 0L, 5L, 0L)) + + merge_by_reference(data, land_use_data, "jobs", "to_id") + expect_identical(data$jobs, c(10L, NA, 5L, NA)) + + expect_identical(land_use_data, original_land_use) +}) diff --git a/tests/testthat/test-na_land_use.R b/tests/testthat/test-na_land_use.R new file mode 100644 index 0000000..fc78bdd --- /dev/null +++ b/tests/testthat/test-na_land_use.R @@ -0,0 +1,199 @@ +# NA values and ids missing from land_use_data count as 0 in every +# accessibility measure. each measure is run with a land_use_data that has NAs +# and misses an id, and with one in which those values are explicitly 0. the +# second one has no NAs and no missing ids, so it never goes through the NA +# replacement. results are calculated with fill_missing_ids = FALSE, because +# filling would turn NA results into 0 and hide the difference + +na_travel_matrix <- data.table::data.table( + from_id = rep(c("a", "b", "c", "d"), each = 4), + to_id = rep(c("a", "b", "c", "d"), times = 4), + travel_time = c(0, 10, 20, 30, 10, 0, 15, 25, 20, 15, 0, 10, 30, 25, 10, 0) +) + +# "b" has NA jobs, "a" has NA population and "d" is missing + +na_land_use_data <- data.table::data.table( + id = c("a", "b", "c"), + jobs = c(10, NA, 5), + population = c(NA, 20, 30) +) + +zero_land_use_data <- data.table::data.table( + id = c("a", "b", "c", "d"), + jobs = c(10, 0, 5, 0), + population = c(0, 20, 30, 0) +) + +na_calls <- list( + cumulative_cutoff = function(lu) { + cumulative_cutoff( + na_travel_matrix, + lu, + "jobs", + "travel_time", + cutoff = 20, + fill_missing_ids = FALSE + ) + }, + cumulative_interval = function(lu) { + cumulative_interval( + na_travel_matrix, + lu, + "jobs", + "travel_time", + interval = c(10, 30), + fill_missing_ids = FALSE + ) + }, + gravity = function(lu) { + gravity( + na_travel_matrix, + lu, + "jobs", + "travel_time", + decay_function = decay_exponential(0.1), + fill_missing_ids = FALSE + ) + }, + gravity_passive = function(lu) { + gravity( + na_travel_matrix, + lu, + "population", + "travel_time", + decay_function = decay_exponential(0.1), + active = FALSE, + fill_missing_ids = FALSE + ) + }, + fca_2sfca = function(lu) { + floating_catchment_area( + na_travel_matrix, + lu, + "jobs", + "travel_time", + demand = "population", + method = "2sfca", + decay_function = decay_exponential(0.1), + fill_missing_ids = FALSE + ) + }, + fca_bfca = function(lu) { + floating_catchment_area( + na_travel_matrix, + lu, + "jobs", + "travel_time", + demand = "population", + method = "bfca", + decay_function = decay_exponential(0.1), + fill_missing_ids = FALSE + ) + }, + spatial_availability = function(lu) { + spatial_availability( + na_travel_matrix, + lu, + "jobs", + "travel_time", + demand = "population", + decay_function = decay_exponential(0.1), + fill_missing_ids = FALSE + ) + }, + spatial_availability_detailed = function(lu) { + spatial_availability( + na_travel_matrix, + lu, + "jobs", + "travel_time", + demand = "population", + decay_function = decay_exponential(0.1), + detailed_results = TRUE + ) + }, + cost_to_closest = function(lu) { + cost_to_closest( + na_travel_matrix, + lu, + "jobs", + "travel_time", + n = c(1, 12), + fill_missing_ids = FALSE + ) + }, + total = function(lu) { + constrained_accessibility( + na_travel_matrix, + lu, + "travel_time", + supply = "jobs", + constraint = "total", + decay_function = decay_exponential(0.1), + active = TRUE, + fill_missing_ids = FALSE + ) + }, + total_detailed = function(lu) { + constrained_accessibility( + na_travel_matrix, + lu, + "travel_time", + supply = "jobs", + constraint = "total", + decay_function = decay_exponential(0.1), + active = TRUE, + detailed_results = TRUE + ) + }, + singly = function(lu) { + constrained_accessibility( + na_travel_matrix, + lu, + "travel_time", + demand = "population", + supply = "jobs", + constraint = "singly", + decay_function = decay_exponential(0.1), + active = TRUE, + fill_missing_ids = FALSE + ) + }, + singly_detailed = function(lu) { + constrained_accessibility( + na_travel_matrix, + lu, + "travel_time", + demand = "population", + supply = "jobs", + constraint = "singly", + decay_function = decay_exponential(0.1), + active = TRUE, + detailed_results = TRUE + ) + } +) + +test_that("NA values and missing ids in land_use_data count as 0", { + for (measure in names(na_calls)) { + with_na <- suppressWarnings(na_calls[[measure]](na_land_use_data)) + with_zero <- na_calls[[measure]](zero_land_use_data) + + expect_equal(with_na, with_zero, info = measure) + expect_false(anyNA(with_na), info = measure) + } +}) + +test_that("warns that NA values and missing ids are treated as 0", { + expect_warning( + gravity( + na_travel_matrix, + na_land_use_data, + "jobs", + "travel_time", + decay_function = decay_exponential(0.1) + ), + "treated as 0" + ) +}) From 5e85126b9187ff075ec23b355fc89e0ae5140f17 Mon Sep 17 00:00:00 2001 From: rafapereirabr Date: Tue, 20 Oct 2026 23:45:34 -0300 Subject: [PATCH 29/31] docs --- NEWS.md | 4 ++++ R/constrained_accessibility.R | 4 +++- R/doubly_constrained.R | 2 ++ man/constrained_accessibility.Rd | 4 +++- tests/testthat/test-constrained-wrapper.R | 29 +++++++++++++++++++++++ 5 files changed, 41 insertions(+), 2 deletions(-) diff --git a/NEWS.md b/NEWS.md index e212350..9acb8bb 100644 --- a/NEWS.md +++ b/NEWS.md @@ -104,6 +104,10 @@ - Fixed a bug in `constrained_accessibility(constraint = "total")` that added a data.table index attribute to the user's `land_use_data`. +- Fixed a bug in `constrained_accessibility(constraint = "doubly")` that + returned a `data.table` when `travel_matrix` was a `data.frame` (or another + class), instead of keeping the class of the input. + ## Minor changes - Accessibility, inequality and poverty functions now raise an error if diff --git a/R/constrained_accessibility.R b/R/constrained_accessibility.R index 8b23e8b..da90c19 100644 --- a/R/constrained_accessibility.R +++ b/R/constrained_accessibility.R @@ -47,7 +47,9 @@ #' is meaningful. #' #' ### Requirement: -#' - Either `demand` or `supply` must be provided (cannot provide both). +#' - `supply` must be provided when `active = TRUE`, and `demand` when +#' `active = FALSE`. The other argument is ignored, so the same call can be +#' used with `constraint = "singly"`, which requires both. #' #' ### Interpretation: #' - `active = TRUE` (*active accessibility*): Results represent the total diff --git a/R/doubly_constrained.R b/R/doubly_constrained.R index b149893..b02013c 100644 --- a/R/doubly_constrained.R +++ b/R/doubly_constrained.R @@ -186,6 +186,7 @@ doubly_constrained <- function(travel_matrix, ) if (detailed_results) { + if (exists("original_class")) class(detailed_dt) <- original_class return(detailed_dt[]) } else { cli::cli_warn( @@ -196,6 +197,7 @@ doubly_constrained <- function(travel_matrix, data.table::data.table(id = origins, flow = O), data.table::data.table(id = destinations, flow = D) )) + if (exists("original_class")) class(agg) <- original_class return(agg[]) } diff --git a/man/constrained_accessibility.Rd b/man/constrained_accessibility.Rd index 5dba3a4..af2ddb6 100644 --- a/man/constrained_accessibility.Rd +++ b/man/constrained_accessibility.Rd @@ -115,7 +115,9 @@ system is known and representing accessibility as a proportion of this total is meaningful. \subsection{Requirement:}{ \itemize{ -\item Either \code{demand} or \code{supply} must be provided (cannot provide both). +\item \code{supply} must be provided when \code{active = TRUE}, and \code{demand} when +\code{active = FALSE}. The other argument is ignored, so the same call can be +used with \code{constraint = "singly"}, which requires both. } } diff --git a/tests/testthat/test-constrained-wrapper.R b/tests/testthat/test-constrained-wrapper.R index c52cb14..8be4d6c 100644 --- a/tests/testthat/test-constrained-wrapper.R +++ b/tests/testthat/test-constrained-wrapper.R @@ -148,3 +148,32 @@ test_that("raises a clear error if demand or supply has length > 1", { regexp = "Must have length 1" ) }) + +test_that("returns a dataframe whose class is the same as travel_matrix's", { + tm_df <- as.data.frame(tm) + + for (constraint in c("total", "singly", "doubly")) { + result <- tester(constraint, travel_matrix = tm_df, detailed_results = TRUE) + expect_false(inherits(result, "data.table")) + expect_is(result, "data.frame") + + result <- tester(constraint, detailed_results = TRUE) + expect_is(result, "data.table") + } + + result <- suppressWarnings(tester("doubly", travel_matrix = tm_df)) + expect_false(inherits(result, "data.table")) +}) + +test_that("total constraint ignores the side not used by active", { + # supply is used when active = TRUE, and demand when active = FALSE + + expect_identical( + tester("total", active = TRUE), + tester("total", active = TRUE, demand = NULL) + ) + expect_identical( + tester("total", active = FALSE), + tester("total", active = FALSE, supply = NULL) + ) +}) From 01cdf19f1b14e0fb767c0c2d47bd91acd7a5d969 Mon Sep 17 00:00:00 2001 From: rafapereirabr Date: Wed, 21 Oct 2026 09:32:03 -0300 Subject: [PATCH 30/31] optimize cumulative_cutoff --- NEWS.md | 17 +++ R/accessibility-package.R | 9 +- R/balancing_cost.R | 65 +++++------ R/cumulative_cutoff.R | 129 +++++++++++++++++++--- R/cumulative_interval.R | 79 +++++-------- R/sum_by_cost_bin.R | 77 +++++++++++++ tests/testthat/test-balancing_cost.R | 28 +++++ tests/testthat/test-cumulative_cutoff.R | 79 +++++++++++++ tests/testthat/test-cumulative_interval.R | 71 ++++++++++++ 9 files changed, 449 insertions(+), 105 deletions(-) create mode 100644 R/sum_by_cost_bin.R diff --git a/NEWS.md b/NEWS.md index 9acb8bb..0b8bba5 100644 --- a/NEWS.md +++ b/NEWS.md @@ -26,6 +26,10 @@ ## Bug fixes +- Fixed a bug in `cumulative_interval()` in which some cutoffs were left out + of the summary when `interval_increment` produced cutoffs with floating + point noise (e.g. `interval_increment = 0.1`, whose fourth cutoff is + `0.30000000000000004`). - Fixed a bug in `balancing_cost()` in which numeric ids (e.g. `101`, `102`) were used as positions to look up each origin's population, which resulted in `NA` or wrong balancing costs. @@ -161,6 +165,19 @@ - `cost_to_closest()` is about 1.3 to 2.3 times faster, especially with several values of `n`, because the travel matrix is now filtered and sorted only once, with identical results. +- `cumulative_cutoff()` (with several cutoffs of a single travel cost), + `cumulative_interval()` and `balancing_cost()` are much faster on large + travel matrices (e.g. about 10 times with 30 cutoffs, 4 times with an + interval of 40 minutes and 6 to 20 times with `cost_increment = 0.1`), + because the opportunities are summed once by travel cost instead of once + per cutoff, so their running time and memory barely depend on the number of + cutoffs. With non-integer opportunities, results may differ from previous + versions by floating point rounding. In `cumulative_cutoff()` and + `cumulative_interval()`, integer accessibility levels larger than the + largest integer are now returned as doubles, instead of raising an error + when only some ids exceeded it. + `cumulative_interval()` now returns its rows sorted by interval, id and + `group_by` columns in every case. - Errors and warnings raised by the package now use `{cli}`, with consistent formatting of argument names and values. Long lists of ids or columns in messages are abbreviated. `{cli}` and `{rlang}` are new dependencies. diff --git a/R/accessibility-package.R b/R/accessibility-package.R index 16e1f33..48e6869 100644 --- a/R/accessibility-package.R +++ b/R/accessibility-package.R @@ -51,7 +51,6 @@ utils::globalVariables( "combined_bal_fac", "decay_function_arg", "cost", - "did_balance", "avg_access", "i.avg_access", ".palma_group", @@ -101,7 +100,13 @@ utils::globalVariables( ".weighted_ppr", ".total_weight", ".opp", - ".cost" + ".cost", + ".bin", + ".n", + "i..opp", + "i..n", + ".cum_opp", + ".demand" ) ) diff --git a/R/balancing_cost.R b/R/balancing_cost.R index efd8624..5a993b3 100644 --- a/R/balancing_cost.R +++ b/R/balancing_cost.R @@ -148,29 +148,41 @@ balancing_cost <- function(travel_matrix, data <- data[!(from_id %in% no_pop_origins_in_ttm)] - for (i in lookup_costs) { - cum_opps <- calc_cum_cutoff(data, groups, opportunity, travel_cost, i) - - # set() keeps origin_demand from being shadowed by a column of cum_opps + # the opportunities reached by each origin are summed by cost bin (the first + # lookup cost that each travel cost is lower than or equal to) and + # accumulated. an origin balances at the first lookup cost at which it + # reaches as many opportunities as people. the cumulative sum only changes + # at bins that contain some travel cost, so the first of those bins that + # balances is the balancing cost + + binned_sums <- sum_by_cost_bin( + data, + groups, + opportunity, + travel_cost, + lookup_costs + ) + data.table::setorderv(binned_sums, c(groups, ".bin")) + binned_sums[, .cum_opp := cumsum(.opp), by = c(groups)] - data.table::set( - cum_opps, - j = "did_balance", - value = cum_opps$access >= origin_demand[match(cum_opps$from_id, origins)] - ) - balanced <- cum_opps[did_balance == TRUE] + # set() keeps origin_demand from being shadowed by a column of binned_sums - if (nrow(balanced) > 0) { - balanced[, c("access", "did_balance") := NULL] - balanced[, cost := i] + data.table::set( + binned_sums, + j = ".demand", + value = origin_demand[match(binned_sums$from_id, origins)] + ) - access <- rbind(access, balanced) + # warnings are suppressed outside of j (so min() is still optimized), because + # data.table evaluates min() on an empty vector when no rows are selected - data <- data[!balanced, on = groups] + balanced <- suppressWarnings( + binned_sums[.cum_opp >= .demand, .(.bin = min(.bin)), by = c(groups)] + ) + data.table::set(balanced, j = "cost", value = lookup_costs[balanced$.bin]) + balanced[, .bin := NULL] - if (nrow(data) == 0) break - } - } + if (nrow(balanced) > 0) access <- rbind(access, balanced) if (fill_missing_ids) { unique_values <- lapply(groups, function(x) unique(travel_matrix[[x]])) @@ -200,20 +212,3 @@ balancing_cost <- function(travel_matrix, return(access[]) } - - -calc_cum_cutoff <- function(data, groups, opportunity, travel_cost, .cutoff) { - env <- environment() - .opportunity_colname <- opportunity - .cost_colname <- travel_cost - - data <- data[get(.cost_colname) <= .cutoff] - - cum_cutoff_access <- data[ - , - .(access = sum(get(.opportunity_colname))), - by = eval(groups, envir = env) - ] - - return(cum_cutoff_access) -} diff --git a/R/cumulative_cutoff.R b/R/cumulative_cutoff.R index 060ec3a..3a6c8e2 100644 --- a/R/cumulative_cutoff.R +++ b/R/cumulative_cutoff.R @@ -147,6 +147,112 @@ cumulative_cutoff <- function(travel_matrix, names(cutoff_list) <- travel_cost cutoff_df <- do.call(data.table::CJ, cutoff_list) + # with several cutoffs of a single travel cost, the opportunities are summed + # by cost bin once and accumulated. with a single cutoff (or several travel + # costs), filtering the travel matrix is faster + + if (length(travel_cost) == 1 && nrow(cutoff_df) > 1) { + access <- cumulative_by_cutoff( + data, + travel_matrix, + groups, + opportunity, + travel_cost, + cutoffs = sort(cutoff_list[[1]]), + fill_missing_ids = fill_missing_ids + ) + } else { + access <- cumulative_by_filtering( + data, + travel_matrix, + groups, + group_id, + opportunity, + travel_cost, + cutoff_df, + cutoff_list, + fill_missing_ids + ) + } + + data.table::setnames(access, c(group_id, "access"), c("id", opportunity)) + data.table::setcolorder( + access, + c("id", setdiff(groups, group_id), travel_cost, opportunity) + ) + data.table::setorderv(access, c("id", setdiff(groups, group_id), travel_cost)) + + env <- environment() + if (length(travel_cost) == 1 && length(cutoff) == 1) { + access[, get("travel_cost", envir = env) := NULL] + } + + # the table of all id and cutoff combinations is keyed, but results are + # returned unkeyed + + data.table::setkey(access, NULL) + + if (exists("original_class")) class(access) <- original_class + + return(access[]) +} + + +#' @keywords internal +cumulative_by_cutoff <- function(data, + travel_matrix, + groups, + opportunity, + travel_cost, + cutoffs, + fill_missing_ids) { + binned_sums <- sum_by_cost_bin( + data, + groups, + opportunity, + travel_cost, + cutoffs + ) + + # with fill_missing_ids, every combination of ids and groups gets a row for + # each cutoff. otherwise, only the combinations that reach some destination + # within the cutoff, even if it has no opportunities + + if (fill_missing_ids) { + unique_values <- lapply(groups, function(x) unique(travel_matrix[[x]])) + names(unique_values) <- groups + grid_groups <- do.call(data.table::CJ, unique_values) + } else { + grid_groups <- unique(binned_sums[, ..groups]) + } + + access <- cumulative_sums_grid( + binned_sums, + grid_groups, + groups, + n_cutoffs = length(cutoffs), + integer_opportunity = is.integer(data[[opportunity]]) + ) + + if (!fill_missing_ids) access <- access[.n > 0] + + data.table::set(access, j = travel_cost, value = cutoffs[access$.bin]) + access[, c(".bin", ".n") := NULL] + + return(access[]) +} + + +#' @keywords internal +cumulative_by_filtering <- function(data, + travel_matrix, + groups, + group_id, + opportunity, + travel_cost, + cutoff_df, + cutoff_list, + fill_missing_ids) { env <- environment() .opportunity_colname <- opportunity @@ -178,9 +284,11 @@ cumulative_cutoff <- function(travel_matrix, cum_opps <- cum_opps[cum_opps[, .I[1], by = .od_group]$V1] } + # opportunities are summed as doubles, so integer sums don't overflow + cum_opps <- cum_opps[ , - .(access = sum(get(.opportunity_colname))), + .(access = sum(as.numeric(get(.opportunity_colname)))), by = eval(groups, envir = env) ] @@ -195,6 +303,11 @@ cumulative_cutoff <- function(travel_matrix, access <- data.table::rbindlist(access) + is_integer_opportunity <- is.integer(data[[opportunity]]) + if (is_integer_opportunity && all(access$access <= .Machine$integer.max)) { + data.table::set(access, j = "access", value = as.integer(access$access)) + } + if (fill_missing_ids) { unique_values <- lapply(groups, function(x) unique(travel_matrix[[x]])) names(unique_values) <- groups @@ -211,19 +324,5 @@ cumulative_cutoff <- function(travel_matrix, } } - data.table::setnames(access, c(group_id, "access"), c("id", opportunity)) - data.table::setcolorder( - access, - c("id", setdiff(groups, group_id), travel_cost, opportunity) - ) - data.table::setorderv(access, c("id", setdiff(groups, group_id), travel_cost)) - - env <- environment() - if (length(travel_cost) == 1 && length(cutoff) == 1) { - access[, get("travel_cost", envir = env) := NULL] - } - - if (exists("original_class")) class(access) <- original_class - return(access[]) } diff --git a/R/cumulative_interval.R b/R/cumulative_interval.R index 1e88037..dd3daa6 100644 --- a/R/cumulative_interval.R +++ b/R/cumulative_interval.R @@ -114,7 +114,6 @@ cumulative_interval <- function(travel_matrix, ) cutoffs <- unique(unlist(interval_cutoffs)) cutoffs <- cutoffs[order(cutoffs)] - names(cutoffs) <- as.character(cutoffs) # small optimization: we can ditch anything that costs more than the highest # cost cutoff value, since it won't affect the results anyway @@ -135,33 +134,38 @@ cumulative_interval <- function(travel_matrix, warn_extra_cols(travel_matrix, travel_cost, group_id, groups) - .opportunity_colname <- opportunity - access_list <- lapply( - cutoffs, - function(x) { - data[ - get(.cost_colname) <= x, - .(access = sum(get(.opportunity_colname))), - by = eval(groups, envir = env) - ] - } - ) + # accessibility is calculated for every cutoff at once from the sums of + # opportunities by cost bin. every combination of ids and groups gets a row + # for each cutoff, with 0 opportunities where nothing is reached, otherwise + # the summary measure would be miscalculated (e.g. think of median of 1 3 5 + # and median of 0 1 3 5) - # some of the estimates for lower thresholds may have less number of rows - # because some origins may not reach any destinations when the travel cost - # restrictions are harder. in such case, we have to fill these datasets with - # the missing combinations having accessibility 0, otherwise the summary - # measure would be miscalculated (e.g. think of median of 1 3 5 and median of - # 0 1 3 5) + binned_sums <- sum_by_cost_bin( + data, + groups, + opportunity, + travel_cost, + cutoffs + ) # ids that can't reach any opportunity at any cutoff are filled as well, and # only dropped at the end if fill_missing_ids is FALSE - reached_ids <- unique(data.table::rbindlist(access_list)[, ..groups]) + reached_ids <- unique(binned_sums[, ..groups]) + + unique_values <- lapply(groups, function(x) unique(travel_matrix[[x]])) + names(unique_values) <- groups - access_list <- fill_access_list(access_list, travel_matrix, groups) - access_list <- data.table::rbindlist(access_list, idcol = "cutoffs") - access_list[, cutoffs := as.numeric(cutoffs)] + grid_groups <- do.call(data.table::CJ, unique_values) + + access_list <- cumulative_sums_grid( + binned_sums, + grid_groups, + groups, + n_cutoffs = length(cutoffs), + integer_opportunity = is.integer(data[[opportunity]]) + ) + data.table::set(access_list, j = "cutoffs", value = cutoffs[access_list$.bin]) # the summary is returned as a double: summary functions such as median() # may return integers for some groups and doubles for others, and the summary @@ -193,34 +197,3 @@ cumulative_interval <- function(travel_matrix, return(access[]) } - - -#' @keywords internal -fill_access_list <- function(access, travel_matrix, groups) { - unique_values <- lapply(groups, function(x) unique(travel_matrix[[x]])) - names(unique_values) <- groups - possible_combinations <- do.call(data.table::CJ, unique_values) - - access_nrows <- vapply(access, nrow, integer(1)) - possible_nrows <- nrow(possible_combinations) - should_fill <- access_nrows < possible_nrows - - filled_access <- mapply( - do_fill = should_fill, - access_df = access, - SIMPLIFY = FALSE, - FUN = function(do_fill, access_df) { - if (!do_fill) return(access_df) - - filled_access_df <- do_fill_missing_ids( - access_df, - possible_combinations, - groups - ) - - return(filled_access_df[]) - } - ) - - return(filled_access) -} diff --git a/R/sum_by_cost_bin.R b/R/sum_by_cost_bin.R new file mode 100644 index 0000000..8216bd0 --- /dev/null +++ b/R/sum_by_cost_bin.R @@ -0,0 +1,77 @@ +#' @keywords internal +sum_by_cost_bin <- function(data, groups, opportunity, travel_cost, cutoffs) { + # sums the opportunities of each group by cost "bin": the index of the first + # cutoff that the travel cost is lower than or equal to. cumulative sums over + # the bins then give the opportunities within every cutoff with a single pass + # over the travel matrix, instead of one pass per cutoff. cutoffs must be + # sorted + + bin <- findInterval(data[[travel_cost]], cutoffs, left.open = TRUE) + 1L + + # rows with NA costs or costs above the last cutoff can't be reached. the + # subset with keep always copies the columns, so the columns of the caller's + # table are never sorted or updated by reference + + keep <- which(bin <= length(cutoffs)) + + binned <- lapply(groups, function(col) data[[col]][keep]) + names(binned) <- groups + binned$.bin <- bin[keep] + + # opportunities are summed as doubles, so integer sums don't overflow + + binned$.opp <- as.numeric(data[[opportunity]][keep]) + binned <- data.table::setDT(binned) + + binned_sums <- binned[ + , + .(.opp = sum(.opp), .n = .N), + by = c(groups, ".bin") + ] + + return(binned_sums) +} + + +#' @keywords internal +cumulative_sums_grid <- function(binned_sums, + grid_groups, + groups, + n_cutoffs, + integer_opportunity) { + # one row per group (from grid_groups) and cutoff. bins that no row falls + # into get 0 opportunities and 0 rows. the cumulative sums are doubles, so + # they don't overflow, and are converted back to integers when the + # opportunities are integers and the sums fit + + grid <- grid_groups[rep(seq_len(nrow(grid_groups)), each = n_cutoffs)] + data.table::set( + grid, + j = ".bin", + value = rep.int(seq_len(n_cutoffs), nrow(grid_groups)) + ) + + grid[ + binned_sums, + on = c(groups, ".bin"), + `:=`(.opp = i..opp, .n = i..n) + ] + + # the NAs are replaced on the whole columns, outside of by, which would + # otherwise call fcoalesce() once per group + + data.table::set( + grid, + j = ".opp", + value = data.table::fcoalesce(grid$.opp, 0) + ) + data.table::set(grid, j = ".n", value = data.table::fcoalesce(grid$.n, 0L)) + grid[, `:=`(access = cumsum(.opp), .n = cumsum(.n)), by = c(groups)] + grid[, .opp := NULL] + + if (integer_opportunity && all(grid$access <= .Machine$integer.max)) { + data.table::set(grid, j = "access", value = as.integer(grid$access)) + } + + return(grid[]) +} diff --git a/tests/testthat/test-balancing_cost.R b/tests/testthat/test-balancing_cost.R index 277525d..8408d81 100644 --- a/tests/testthat/test-balancing_cost.R +++ b/tests/testthat/test-balancing_cost.R @@ -336,3 +336,31 @@ test_that("NA demand and missing ids count as 0 people", { ) expect_true(all(result[id != "c"]$travel_time == 0)) }) + +test_that("origins that never balance are filled with NA or dropped", { + # a has 100 people but only reaches 30 jobs + + toy_matrix <- data.table::data.table( + from_id = c("a", "a", "b"), + to_id = c("a", "b", "b"), + travel_time = c(0, 10, 0) + ) + toy_land_use <- data.table::data.table( + id = c("a", "b"), + jobs = c(10, 20), + population = c(100, 5) + ) + + result <- tester(toy_matrix, toy_land_use, group_by = character(0)) + expect_identical(result$id, c("a", "b")) + expect_identical(result$travel_time, c(NA, 0)) + + result <- tester( + toy_matrix, + toy_land_use, + group_by = character(0), + fill_missing_ids = FALSE + ) + expect_identical(result$id, "b") + expect_identical(result$travel_time, 0) +}) diff --git a/tests/testthat/test-cumulative_cutoff.R b/tests/testthat/test-cumulative_cutoff.R index d2b9a6b..dc86e6f 100644 --- a/tests/testthat/test-cumulative_cutoff.R +++ b/tests/testthat/test-cumulative_cutoff.R @@ -469,4 +469,83 @@ test_that("result is not keyed, even when missing ids are filled", { ) expect_identical(result$jobs, c(30, 0)) expect_null(data.table::key(result)) + + for (fill in c(TRUE, FALSE)) { + result <- tester( + as.data.frame(toy_matrix), + toy_land_use, + cutoff = c(30, 60), + group_by = character(0), + fill_missing_ids = fill + ) + expect_null(attr(result, "sorted")) + } +}) + +test_that("edge cases of cutoffs and travel costs are handled", { + # costs equal to infinity or NA are never reached. b reaches a destination + # within 30 minutes, but it has no opportunities. c reaches nothing + + toy_matrix <- data.table::data.table( + from_id = c("a", "a", "a", "a", "b", "b", "c"), + to_id = c("a", "b", "c", "d", "b", "a", "a"), + travel_time = c(0, 30, NA, Inf, 0, 45, 100) + ) + toy_land_use <- data.table::data.table( + id = c("a", "b", "c", "d"), + jobs = c(10, 0, 7, 5) + ) + + result <- tester( + toy_matrix, + toy_land_use, + cutoff = c(60, 30), + group_by = character(0), + fill_missing_ids = FALSE + ) + expect_identical(result$id, c("a", "a", "b", "b")) + expect_identical(result$travel_time, c(30, 60, 30, 60)) + expect_identical(result$jobs, c(10, 10, 0, 10)) + + result <- tester( + toy_matrix, + toy_land_use, + cutoff = c(60, 30), + group_by = character(0) + ) + expect_identical(result$id, c("a", "a", "b", "b", "c", "c")) + expect_identical(result$jobs, c(10, 10, 0, 10, 0, 0)) +}) + +test_that("large integer sums don't overflow", { + # a reaches 10 jobs and b reaches more jobs than the largest integer, which + # used to raise an error when sums of different ids had different types + + toy_matrix <- data.table::data.table( + from_id = c("a", "b", "b"), + to_id = c("a", "a", "b"), + travel_time = c(0, 0, 20) + ) + toy_land_use <- data.table::data.table( + id = c("a", "b"), + jobs = c(10L, .Machine$integer.max) + ) + + result <- tester( + toy_matrix, + toy_land_use, + cutoff = c(30, 60), + group_by = character(0), + fill_missing_ids = FALSE + ) + expect_identical(result$jobs, c(10, 10, rep(.Machine$integer.max + 10, 2))) + + result <- tester( + toy_matrix, + toy_land_use, + cutoff = 30, + group_by = character(0), + fill_missing_ids = FALSE + ) + expect_identical(result$jobs, c(10, .Machine$integer.max + 10)) }) diff --git a/tests/testthat/test-cumulative_interval.R b/tests/testthat/test-cumulative_interval.R index c0acc87..44f5896 100644 --- a/tests/testthat/test-cumulative_interval.R +++ b/tests/testthat/test-cumulative_interval.R @@ -196,6 +196,7 @@ test_that("active and passive accessibility is correctly calculated", { mode = rep(c("transit", "transit2"), each = 5), jobs = rep(c(82, 517, 517, 304, 0), 2) ) + data.table::setorderv(expected_result, c("id", "mode")) expect_identical(result, expected_result) result <- tester( @@ -210,6 +211,7 @@ test_that("active and passive accessibility is correctly calculated", { mode = rep(c("transit", "transit2"), each = 5), population = rep(c(1946.5, 3457.5, 2363, 4552, 0), 2) ) + data.table::setorderv(expected_result, c("id", "mode")) expect_identical(result, expected_result) # with more than one interval @@ -228,6 +230,7 @@ test_that("active and passive accessibility is correctly calculated", { rep(c(82, 599, 599, 499, 0), 2) ) ) + data.table::setorderv(expected_result, c("interval", "id", "mode")) expect_identical(result, expected_result) result <- tester( @@ -246,6 +249,7 @@ test_that("active and passive accessibility is correctly calculated", { rep(c(5404, 4552, 2363, 4552, 0), 2) ) ) + data.table::setorderv(expected_result, c("interval", "id", "mode")) expect_identical(result, expected_result) }) @@ -272,6 +276,7 @@ test_that("summarizes the result according to summary function", { mode = rep(c("transit", "transit2"), each = 5), jobs = rep(c(82, 408, 408, 109, 0), 2) ) + data.table::setorderv(expected_result, c("id", "mode")) expect_identical(result, expected_result) result <- tester( @@ -285,6 +290,7 @@ test_that("summarizes the result according to summary function", { mode = rep(c("transit", "transit2"), each = 5), jobs = rep(c(82, 512.5, 508, 304, 0), 2) ) + data.table::setorderv(expected_result, c("id", "mode")) expect_identical(result, expected_result) }) @@ -439,6 +445,71 @@ test_that("fill_missing_ids arg works correctly", { result <- toy_tester(interval = c(0, 20), fill_missing_ids = FALSE) expect_identical(result$id, c("a", "b", "c")) expect_equal(result$jobs, c(100 / 3, 80 / 3, 100 / 3)) + expect_identical(result, result[order(id)]) + expect_null(data.table::key(result)) +}) + +test_that("float cutoffs are all used in the summary", { + # the cutoffs are seq(0, 0.5, 0.1), whose 4th element is + # 0.30000000000000004. a reaches 0, 1, 3, 7, 7 and 7 jobs + + toy_matrix <- data.table::data.table( + from_id = c("a", "a", "a"), + to_id = c("a", "b", "c"), + travel_time = c(0.1, 0.2, 0.3) + ) + toy_land_use <- data.table::data.table( + id = c("a", "b", "c"), + jobs = c(1, 2, 4) + ) + + result <- tester( + toy_matrix, + toy_land_use, + interval = c(0, 0.5), + interval_increment = 0.1, + summary_function = length, + group_by = character(0), + fill_missing_ids = FALSE + ) + expect_identical(result$jobs, 6) + + result <- tester( + toy_matrix, + toy_land_use, + interval = c(0, 0.5), + interval_increment = 0.1, + group_by = character(0), + fill_missing_ids = FALSE + ) + expect_identical(result$jobs, 5) +}) + +test_that("works when nothing is reached within the interval", { + toy_matrix <- data.table::data.table( + from_id = c("a", "b"), + to_id = c("b", "a"), + travel_time = c(70, 80) + ) + toy_land_use <- data.table::data.table(id = c("a", "b"), jobs = c(1, 2)) + + result <- tester( + toy_matrix, + toy_land_use, + interval = c(10, 20), + group_by = character(0) + ) + expect_identical(result$id, c("a", "b")) + expect_identical(result$jobs, c(0, 0)) + + result <- tester( + toy_matrix, + toy_land_use, + interval = c(10, 20), + group_by = character(0), + fill_missing_ids = FALSE + ) + expect_identical(nrow(result), 0L) }) test_that("the upper limit of the interval is always considered", { From 24a79eaed27d44dab3ff939d16823c8398efe275 Mon Sep 17 00:00:00 2001 From: rafapereirabr Date: Wed, 21 Oct 2026 09:47:34 -0300 Subject: [PATCH 31/31] news --- NEWS.md | 259 +++++++++++++++-------------------------------- cran-comments.md | 5 +- 2 files changed, 84 insertions(+), 180 deletions(-) diff --git a/NEWS.md b/NEWS.md index 0b8bba5..c477895 100644 --- a/NEWS.md +++ b/NEWS.md @@ -3,198 +3,101 @@ ## New features - `cumulative_interval()` has a new argument `fill_missing_ids`. When `TRUE` - (the default), origins (or destinations) that cannot reach any opportunity - within the interval are included in the output with an accessibility of 0, - as in `cumulative_cutoff()`. Previously they were dropped from the output. + (the default), ids that reach no opportunity are kept with accessibility 0, + as in `cumulative_cutoff()`. ## Breaking changes -- `cumulative_interval()` now returns the summary of the accessibility levels - as a `numeric` without truncation (e.g. the median of 26 and 27 - opportunities is 26.5). Previously, results were truncated to integers. -- `concentration_index(type = "corrected")` now calculates the correction - factor with the bounds of the whole accessibility distribution, instead of - the bounds of each `group_by` group, so groups are corrected on a common - scale. Results only change when `group_by` is used. -- Accessibility functions now treat `NA` values in `land_use_data` - (opportunities, demand and supply) and ids of `travel_matrix` missing from - `land_use_data` as 0, and their warnings say so. Previously, such a value - made the results of every id that reached it `NA`, except when - `fill_missing_ids = TRUE` had to fill other missing ids, in which case - those `NA` results became 0. When every value is `NA`, results are now the - same as when every value is 0 (which can be `NaN`). +- Accessibility functions now treat `NA` opportunity, demand and supply + values, and `travel_matrix` ids missing from `land_use_data`, as 0. + Previously, they could make results `NA`. +- `concentration_index(type = "corrected")` now uses the bounds of the whole + accessibility distribution, not of each `group_by` group, to calculate the + correction factor. +- `cumulative_interval()` no longer truncates results to integers (e.g. a + median of 26.5). ## Bug fixes -- Fixed a bug in `cumulative_interval()` in which some cutoffs were left out - of the summary when `interval_increment` produced cutoffs with floating - point noise (e.g. `interval_increment = 0.1`, whose fourth cutoff is - `0.30000000000000004`). -- Fixed a bug in `balancing_cost()` in which numeric ids (e.g. `101`, `102`) - were used as positions to look up each origin's population, which resulted - in `NA` or wrong balancing costs. -- Fixed a bug in `spatial_availability()` in which a destination that no one - could reach (or that was only reached by origins with no demand) made the - results of every origin `NaN`. The opportunities in such destinations are - now left unallocated. -- Fixed a bug in `gini_index()`, `concentration_index()` and `theil_t()` in - which large integer opportunity and population counts overflowed when - multiplied, returning `NA` (e.g. with long travel time cutoffs or large - cities). -- Fixed a bug in `constrained_accessibility(constraint = "doubly")` in which - the impedance matrix and the demand and supply totals were aligned by row - position instead of by id. Results were only correct when `travel_matrix` - listed every origin-destination pair ordered by destination and - `land_use_data` was ordered like the travel matrix. Pairs missing from - `travel_matrix` are now treated as unreachable. -- Fixed a bug in `theil_t()` in which observations with no access were - wrongly handled: without `socioeconomic_groups` they were excluded from the - index (but not from the average accessibility), and with - `socioeconomic_groups` they made the index `NaN`. They now contribute 0 to - the index, following the usual `0 * log(0) = 0` convention. `group_by` - levels in which no one has access are now kept in the output with `NA`, - instead of being dropped. -- Fixed a bug in `spatial_availability()` in which an `NA` demand in - `land_use_data` made every result `NA`, even when the id with the `NA` was - not in `travel_matrix`. -- Fixed two bugs in `balancing_cost()`: origins only balanced when they reached - more opportunities than people (instead of as many opportunities as people), - and origins that balanced at a cost greater than the last multiple of - `cost_increment` were reported as `NA`. `cost_increment` must now be greater - than 0. -- Fixed a bug in `constrained_accessibility(constraint = "doubly")` in which - a small constant added to the balancing factors distorted the results when - impedance weights were very small (e.g. with steep decay functions or long - travel costs). The function now also warns when the balancing procedure - stops before reaching `error_threshold`. -- Fixed a bug in `constrained_accessibility()` (`constraint = "total"` or - `"singly"`) in which `fill_missing_ids = TRUE` raised an error ("Object - 'access' not found") when `group_by` left an origin/destination and group - combination missing from the results. -- Fixed a bug in `concentration_index()` in which observations with the same - income received different fractional ranks, so the result depended on the - order of the rows (especially with ordinal variables such as income - deciles). Tied observations now share the same rank, the midpoint of their - block in the cumulative population distribution. -- Fixed a bug in `cumulative_interval()` in which, when several intervals - were passed to `interval`, each interval was summarised over the cutoffs of - every interval within its range, so its result depended on the other - intervals requested (e.g. `c(0, 20)` together with `c(5, 15)`). -- Fixed a bug in `cumulative_interval()` in which the upper limit of the - interval was not considered when the interval width was not a multiple of - `interval_increment` (e.g. `interval = c(0, 25)` with - `interval_increment = 10`). -- Fixed a bug in `cumulative_cutoff()` in which cutoffs with floating point - noise (e.g. created with `seq(0, 0.5, by = 0.1)`) could have their results - replaced by 0 when filling missing ids. Travel cost columns with - non-syntactic names (e.g. `"bus fare"`) now also work. - -- Fixed a bug in `palma_ratio()` in which the income cutoffs depended on the - scale of the population weights, which resulted in `NA` when population was - given as shares (e.g. summing to 1). -- Fixed a bug in `concentration_index(type = "corrected")` in which the index - was `NaN` when everyone had the same accessibility. It's now 0, as with - `type = "standard"`. -- `fgt_poverty()` now raises an error if `poverty_line` is not a single finite - number (or is 0 when `poor_below_threshold = FALSE`) and if - `poor_below_threshold` is not a single `TRUE` or `FALSE`. Previously these - inputs silently returned recycled values, `NA` or `Inf`, or a cryptic error. -- Fixed a bug in `palma_ratio()` in which, with more than one `group_by` - column, the output included rows with `NA` for group combinations that were - not present in the data. The output is also no longer keyed. - -- Fixed a bug in `decay_stepped()` that emitted a warning for each `NA` - travel cost (the weight is still `NA`). -- Fixed a bug in `constrained_accessibility(constraint = "total")` that added - a data.table index attribute to the user's `land_use_data`. - -- Fixed a bug in `constrained_accessibility(constraint = "doubly")` that - returned a `data.table` when `travel_matrix` was a `data.frame` (or another - class), instead of keeping the class of the input. +- `balancing_cost()`: numeric ids were used as positions to look up + population, returning `NA` or wrong costs; origins balanced only when + opportunities exceeded (instead of matched) population; and costs above the + last multiple of `cost_increment` were `NA`. `cost_increment` must now be + greater than 0. +- `concentration_index()`: tied incomes now share the same fractional rank, + so results no longer depend on row order. With `type = "corrected"`, the + index is now 0, instead of `NaN`, when everyone has the same accessibility. +- `concentration_index()`, `gini_index()` and `theil_t()` no longer return + `NA` due to integer overflow with large opportunity and population counts. +- `constrained_accessibility()`: + - `constraint = "doubly"` aligned impedances and totals by row position + instead of by id; pairs missing from `travel_matrix` are now unreachable. + - `constraint = "doubly"` no longer adds a constant to the balancing + factors, which distorted results with very small weights, and now warns + when balancing stops before reaching `error_threshold`. + - `constraint = "doubly"` now returns the same class as `travel_matrix`. + - `constraint = "total"` or `"singly"` no longer errors when + `fill_missing_ids = TRUE` and `group_by` leave combinations missing. + - `constraint = "total"` no longer adds an index to the user's + `land_use_data`. +- `cumulative_cutoff()`: cutoffs with floating point noise (e.g. from + `seq(0, 0.5, by = 0.1)`) could be zeroed when filling missing ids. Travel + cost columns with non-syntactic names now work. +- `cumulative_interval()`: cutoffs with floating point noise (e.g. from + `interval_increment = 0.1`) were left out of the summary. +- `decay_stepped()` no longer warns for each `NA` travel cost. +- `palma_ratio()`: income cutoffs depended on the scale of population (`NA` + with population shares), and with several `group_by` columns the output + included `NA` rows for absent combinations. The output is no longer keyed. +- `spatial_availability()`: a destination no one could reach made all + results `NaN`. Its opportunities are now left unallocated. +- `theil_t()`: observations with no access are now included and contribute 0 + (`0 * log(0) = 0`), instead of being excluded or making the index `NaN`. + `group_by` levels where no one has access are kept with `NA`. ## Minor changes +- Errors and warnings now use `{cli}`. `{cli}` and `{rlang}` are new + dependencies. - Accessibility, inequality and poverty functions now raise an error if - `land_use_data` or `sociodemographic_data` contain duplicated ids. - Previously, only one of the duplicated rows was silently used. -- Accessibility functions now raise an error if a `travel_cost` column of - `travel_matrix` contains negative values, which previously produced weights - above 1 and inflated accessibility levels. -- Accessibility functions now raise a clear error when the ids of - `travel_matrix` and `land_use_data` have incompatible types (e.g. integer - and character), instead of an obscure join error. -- `decay_linear()` now requires `cutoff` to be greater than 0 (a cutoff of 0 - produced `NaN` weights). -- `decay_logistic()` and `decay_stepped()` now raise an error when given - duplicated parameter combinations, which produced duplicated results. -- `constrained_accessibility()` now checks that `demand` and `supply` are - single strings. -- Results of accessibility functions are no longer returned as keyed - data.tables when missing ids are filled (previously, they were keyed only in - this case). -- Opportunities, demand, supply, population and income are now matched to - the input data by id with `match()` instead of an update join. Accessibility - functions are about 1.2 to 3.3 times faster and use about half the memory on - large travel matrices, with identical results. If `travel_matrix` already - has a column named like the opportunity column, that column is now fully - replaced by the values in `land_use_data`. -- The functions created by `decay_logistic()` and `decay_stepped()` are now - vectorised instead of looping over each travel cost, which makes them about - 50 and 200 times faster on large travel matrices, with identical results. -- The functions created by `decay_exponential()`, `decay_power()` and - `decay_logistic()` now calculate each weight once per unique travel cost - when costs are repeated (e.g. rounded travel times), which makes them about - 2 to 6 times faster, with identical results. -- `gravity()` and `floating_catchment_area()` are faster, especially with - several decay function values (e.g. `gravity()` with three values is about 4 - times faster on large travel matrices), because the travel matrix is no - longer copied once per value and sums are optimized by `{data.table}`. - Results may differ from previous versions by floating point rounding only - (relative differences below 1e-14). `gravity()` with integer opportunities - and `decay_binary()` no longer fails when accessibility exceeds the largest + `land_use_data` or `sociodemographic_data` have duplicated ids. +- Accessibility functions: + - raise an error on negative travel costs and a clear error when the id + types of `travel_matrix` and `land_use_data` are incompatible. + - match land use data by id with `match()`, making them about 1.2 to 3.3 + times faster with half the memory. A `travel_matrix` column named like + the opportunity column is now replaced. + - no longer return keyed data.tables when filling missing ids. +- Several functions are faster on large travel matrices. Results may differ + from previous versions by floating point rounding only. +- `balancing_cost()`, `cumulative_cutoff()` and `cumulative_interval()` are + about 4 to 20 times faster, because opportunities are summed once per travel + cost instead of once per cutoff. +- `constrained_accessibility()` checks that `demand` and `supply` are single + strings. With `constraint = "doubly"`, it is about 2 to 4 times faster and + uses much less memory. +- `cost_to_closest()` is about 1.3 to 2.3 times faster. +- `cumulative_cutoff()` and `cumulative_interval()` return accessibility + above the largest integer as doubles instead of erroring. + `cumulative_interval()` always sorts rows by interval, id and `group_by`. +- `decay_exponential()`, `decay_logistic()`, `decay_power()` and + `decay_stepped()` are about 2 to 200 times faster. +- `decay_linear()` requires `cutoff` greater than 0. +- `decay_logistic()` and `decay_stepped()` raise an error on duplicated + parameter combinations. +- `fgt_poverty()` raises an error on invalid `poverty_line` or + `poor_below_threshold`, and warns when `poor_below_threshold = FALSE` and + `accessibility_data` has `Inf` values (e.g. from `cost_to_closest()`). +- `floating_catchment_area()` and `gravity()` are faster with several decay + values (about 4 times with three). `gravity()` with integer opportunities and + `decay_binary()` no longer fails when accessibility exceeds the largest integer. -- `constrained_accessibility(constraint = "doubly")` is about 2 to 4 times - faster and uses much less memory on large travel matrices, because the - balancing factors are now calculated with matrix products instead of loops - over origins and destinations. Results may differ from previous versions by - floating point rounding only. -- `spatial_availability()` is about 2 times faster on large travel matrices, - because its balancing factors are now calculated with sums optimized by - `{data.table}`. Results may differ from previous versions by floating point - rounding only. -- `cost_to_closest()` is about 1.3 to 2.3 times faster, especially with - several values of `n`, because the travel matrix is now filtered and sorted - only once, with identical results. -- `cumulative_cutoff()` (with several cutoffs of a single travel cost), - `cumulative_interval()` and `balancing_cost()` are much faster on large - travel matrices (e.g. about 10 times with 30 cutoffs, 4 times with an - interval of 40 minutes and 6 to 20 times with `cost_increment = 0.1`), - because the opportunities are summed once by travel cost instead of once - per cutoff, so their running time and memory barely depend on the number of - cutoffs. With non-integer opportunities, results may differ from previous - versions by floating point rounding. In `cumulative_cutoff()` and - `cumulative_interval()`, integer accessibility levels larger than the - largest integer are now returned as doubles, instead of raising an error - when only some ids exceeded it. - `cumulative_interval()` now returns its rows sorted by interval, id and - `group_by` columns in every case. -- Errors and warnings raised by the package now use `{cli}`, with consistent - formatting of argument names and values. Long lists of ids or columns in - messages are abbreviated. `{cli}` and `{rlang}` are new dependencies. - -- `fgt_poverty()` now warns when `poor_below_threshold = FALSE` and - `accessibility_data` contains `Inf` values (e.g. from `cost_to_closest()` for - origins that can't reach the required opportunities), which make FGT1 and - FGT2 infinite. The documentation and the inequality and poverty vignette now - explain this case, and the vignette includes an example with - `poor_below_threshold = FALSE`. +- `spatial_availability()` is about 2 times faster. ## Notes -- Updated the package website and GitHub URLs after the repository moved from - the `ipeaGIT` to the `ipea` organization. -- Vignettes and tests now only use the suggested packages `{sf}`, - `{ggplot2}` and `{tibble}` when they are installed. +- Updated URLs after the repository moved from `ipeaGIT` to `ipea`. + # accessibility 1.5.0 diff --git a/cran-comments.md b/cran-comments.md index 2c50cef..49bfbc2 100644 --- a/cran-comments.md +++ b/cran-comments.md @@ -1,8 +1,9 @@ -── R CMD check results ─────────────────────────────────────────────────────────── accessibility 1.5.0 ──── -Duration: 3m 1.3s +── R CMD check results ──────────────────────────────── accessibility 2.0.0 ──── +Duration: 1m 57.1s 0 errors ✔ | 0 warnings ✔ | 0 notes ✔ + ## New features - New function `constrained_accessibility()`, which implements the family of