Recovering interaction structure from rich, abundant features.
raf implements the RAF estimator from Chandrasekhar, Jackson, McCormick,
Rohe, Shane and Xu (2026), On Tubes; or, Model-Building from Rich Data by
Isolating Concepts. Use it when an outcome is observed for pairs of units
from two populations (villages x program designs, settings x objects,
rankees x rankers) and each unit is described by many characteristics. It finds a few latent concepts on each
side and estimates the matrix H of interactions between them.
# install.packages("remotes")
remotes::install_github("jshane1/raf", build_vignettes = TRUE)build_vignettes = TRUE installs the two walkthroughs (vignette("raf"),
vignette("llm-labeling")). It needs knitr, rmarkdown and pandoc
(bundled with RStudio). Leave it out for a quicker install.
library(raf)
dat <- raf_data(my_data, outcome = "y",
row_vars = "village_id", col_vars = "design_id",
row_covariates = village_vars, col_covariates = design_vars)
raf_scree(dat, max_rank = 8, seed = 1) # choose the rank
fit <- raf_fit(dat, rank = 4, seed = 1) # split -> soft-impute SVD -> varimax -> WLS for H
summary(fit) # pure-unit and weight diagnostics
fit <- raf_set_labels(fit, raf_label_dictionary(fit)) # or raf_label_llm(), or by hand
fit <- raf_orient(fit) # axes point towards their labels
raf_labeled_H(fit)
plot(fit)| Step | Function | What it does |
|---|---|---|
| Build | raf_data(), raf_matrix() |
Aggregate to a row unit x column unit matrix (NA where unobserved); double-center to remove main effects |
| Rank | raf_scree() |
Singular values for choosing k |
| Fit | raf_fit() |
Split cells into folds; rank-k soft-impute SVD on fold 1; varimax each side; sign-anchor; purity-weighted FWL least squares for H on fold 2 |
| Pieces | raf_H_wls(), raf_purity() |
The estimator and its weights on any loadings you supply |
| Label | raf_label_dictionary(), raf_label_llm(), raf_label_prompt() + raf_labels_from_json(), raf_set_labels(), raf_orient() |
Name each axis from its correlations with the units' characteristics; orient it towards its label |
| Read | raf_labeled_H(), plot() |
The labeled interaction matrix |
| Simulate | raf_simulate() |
Data from the model with known truth |
vignette("raf") walks through a complete application and explains every
step. vignette("llm-labeling") covers LLM labeling with a hosted API, a
local model, or no key at all.
The core package needs no API key and makes no network calls.
raf_label_llm() supports the Claude API, the OpenAI API, and any
OpenAI-compatible server, including local models (Ollama, LM Studio, vLLM).
Local models need no key and keep everything on the machine. Without any
model you can print the prompt with raf_label_prompt(), paste it into a
chat interface, and read the reply with raf_labels_from_json(). Only
characteristic names and correlations are sent, never unit-level data.
raf_fit(seed = )seeds both random steps, the split of cells into folds and the soft-impute SVD's random start, so a seeded fit is reproducible. It leaves the global RNG state untouched. Without a seed,Hcan move noticeably between runs because the soft-impute start changes.raf_data()aggregates repeated observations of a cell by their mean. Useaggregate = "sum"to sum them instead.- The matrix is double-centered once, before the split. Use
center_folds = TRUEto center each fold on its own cells. - A custom rotation function may return either a rotation object or just
the rotated loadings; in the second case the rotation matrix is recovered
so
Hstays consistent with the rotated axes.
Version 0.1.0, under development alongside the working paper. Standard
errors for H are not provided yet.
If you use raf, please cite the paper (citation("raf") prints this):
Chandrasekhar, A. G., M. O. Jackson, T. H. McCormick, K. Rohe, J. Shane, and B. Xu (2026). On Tubes; or, Model-Building from Rich Data by Isolating Concepts. Working paper.
MIT. See LICENSE.md.