Fix HAP model bias correction, redesign by_gr, and validate against real data

.github/workflows/build.yaml

@@ -38,6 +38,7 @@ jobs:
 
       - name: Install dependencies
         run: |
+          options(timeout = 600)
           install.packages(c("remotes","rcmdcheck","stringi"), repos = "https://cloud.r-project.org")
           install.packages(c("evaluate"), repos = "https://cloud.r-project.org")
           install.packages(c("knitr","rmarkdown","htmlwidgets", "plm"), repos = "https://cloud.r-project.org")

.github/workflows/test_coverage.yaml

@@ -39,10 +39,12 @@ jobs:
           sudo apt-get install -y \
             libcurl4-openssl-dev libssl-dev libxml2-dev \
             libudunits2-dev libgdal-dev libgeos-dev libproj-dev \
-            libmagick++-dev libavfilter-dev libharfbuzz-dev libfribidi-dev
+            libmagick++-dev libavfilter-dev libharfbuzz-dev libfribidi-dev \
+            libuv1-dev
 
       - name: Install R dependencies
         run: |
+          options(timeout = 600)
           install.packages(c("remotes", "rcmdcheck", "covr"), repos = "https://cloud.r-project.org")
           remotes::install_deps(dependencies = TRUE)
         shell: Rscript {0}

NAMESPACE

@@ -2,7 +2,6 @@
 
 export(calc_ResidEm_grp)
 export(calc_hap_impacts)
-export(calc_hia_adder)
 export(create_panel)
 export(fit_model)
 export(get_gdp_ctry)

NEWS.md

@@ -7,4 +7,21 @@
 
 ## New features
 
-## Bug Fixes
+* `fit_model()` re-specified: linear year trend replaces year fixed effects, NMVOC dropped (non-significant under every specification/SE estimator tested), and Driscoll-Kraay standard errors are now computed and returned (`vcov_dk`), justified by confirmed serial correlation and cross-sectional dependence in the panel.
+* The country-level bias-correction term is now computed internally by `fit_model()` from the fitted model itself (`bias_adder$bias.adder`), replacing the static `rhap::hia_adder` dataset, which has been removed along with `calc_hia_adder()`.
+* `by_gr = TRUE` in `calc_hap_impacts()` now reports within-country income-decile group *shares* of population, PM2.5+NOx exposure, and GDP (table + pie chart), instead of a fabricated group-level health-impact estimate. The regression model is fit on between-country variation only, so applying it to within-country decile inputs was an unvalidated extrapolation; shares of GCAM's own group-level data carry no such risk.
+* `fit_model()`'s `bias_adder` now also returns `reliability_ratio` (and `calc_hap_impacts()`'s output a derived `reliability` "high"/"medium"/"low" label), flagging countries whose absolute-level prediction rests mostly on the bias correction rather than the regression's own covariates -- computed already, previously discarded.
+* `calc_hap_impacts()` gains a `fit_result` argument: pass a previously-computed `fit_model(HIA_var)` result to skip refitting the national regression (including its Driscoll-Kraay vcov) on every call, e.g. when comparing many GCAM scenarios for the same `HIA_var`.
+* Added a new "Validating the econometric model" vignette: a reproducible record of the panel/inference diagnostics, the (rejected) GDP x PM2.5 interaction test, the additive-vs-multiplicative bias-adder investigation, and the GBD real-data validation methodology.
+
+## Bug Fixes
+
+* Fixed the country-level bias correction silently clamping a negative "predicted" death/YLL/DALY rate to exactly 0 (occurred for ~12% of observations on the model's own training data). It now floors at a small fraction of the naive, uncorrected prediction instead. (A multiplicative bias correction, which prevents negative values by construction, was also tried; it was rejected after validation against real GBD data showed it performing far worse than the additive form -- see `fit_model()`'s documentation and the econometric-model vignette.)
+* Fixed `fit_model(HIA_var = "yll")`: valid input silently resolved to `NULL` due to a mismatched lookup key (`"ylls"` vs. `"yll"`).
+* Fixed a rerun of `calc_hap_impacts(map = TRUE)` against a not-freshly-cleared `output/` directory failing because the map-output file mover matched, and tried to rename, its own destination folder.
+* Fixed several `.RData` test snapshots (`calc_hap_impacts`, `calc_ResidEm_grp` x2, `fit_model`) being compared vacuously: `load()` injects an object under whatever name it was *saved* as, which for these files happened to match the local variable holding the freshly-computed test result (`testOutput`), silently overwriting it before `expect_equal()` ran -- so the test was comparing the stale snapshot to itself, regardless of what the function under test actually returned. Also removed the now-redundant, structurally-brittle `fit_model` snapshot in favor of targeted structural/sign/parameter checks.
+
+## Package/performance
+
+* Removed dead code (`flsp_pc_gr`, `flsp_pc_ctry_gr`, `gdp_pc_ctry_gr`) left over from the `by_gr` redesign.
+* GCAM group-level queries (`subregional population`, `subregional income`) and their downstream processing now only run when `by_gr = TRUE` (the shares they feed are otherwise never consumed) -- skips two extra GCAM queries and their processing on every default-mode (`by_gr = FALSE`) call.