Skip to content

Launch the model executable with the TBB its build resolved #1261

Description

@jgabry

Part of the design described in #1254. I asked Claude to write the text below based on that.


Part of the v1.0 compilation-state work (#1258). Lands any time after Stage 3, since it
reads the tbb_dir field the build record carries (dev-notes/compilation-state.md, §4).
Nothing in the rebuild assessment turns on it.

The problem

The executable carries a hard reference to the TBB it was built against. Stan Math
bakes the path in absolutely, and which path depends on the build's configuration
(stan/lib/stan_math/make/compiler_flags:277-330): TBB_LIB where the build named an
external TBB, and $(abspath $(TBB_BIN)) otherwise, where TBB_BIN defaults to the
installation's own lib/tbb.

The two platforms fail in opposite directions. Both branches bake an -rpath into the
binary (compiler_flags:303, :329) and both guard it out on Windows (:302, :328).
Measured on a default-layout build, otool reports a single LC_RPATH into the
builder's stan/lib/stan_math/lib/tbb with no fallback entry, so on macOS and Linux
tbb_path() returns NULL (R/run.R:1238-1247), cmdstanr supplies nothing, and the
loader either finds that directory or refuses the binary. On Windows there is no rpath,
tbb_path() defaults dir to cmdstan_path(), and every call site takes it bare, so
the session's currently selected installation supplies the TBB whatever built the binary.

That is wrong whenever the selected installation is not the builder, and it is reached
without any record involved: build a model, call set_cmdstan_path(), then sample. On
Windows a 2.39 binary runs against 2.40's TBB in released cmdstanr. instantiate reaches
the same state by design: stan_package_model() sets the CmdStan path, constructs the
object, and restores the previous path on.exit, so by the time the user samples the
session points at a third installation.

The rule

cmdstanr supplies the TBB directory the build's call named, recorded at build time
as tbb_dir, rather than the selected installation's. At each launch site, what goes on
PATH is this table and nowhere else:

the record in hand what goes on PATH
usable, and the directory it names exists that directory
usable, and the directory it names is gone nothing
no usable record the selected installation's own

Usable means hash-bound to this executable (§4), not merely present: a record that
does not bind names the TBB of some other binary.

A gone directory is not a reason to substitute another. For a default-layout build
the recorded directory sits inside the builder tree, so gone and builder-gone are one
event, and substituting the selected installation is the 2.39-on-2.40 case above by a
second route. For a build that named its own TBB the substitute either lacks the library
the binary imports, changing nothing, or supplies a different build of it under the same
name, which loads. withr::with_path() prefixes (R/run.R:657), so it would also
outrank a working TBB the user already has on PATH. Supplying nothing is what macOS
and Linux do on every call today, and what it leaves behind is the launch error §6
already specifies for a recorded TBB that is gone.

Deriving the directory from the builder path would be wrong.
<builder>/stan/lib/stan_math/lib/tbb is the answer only for a build that left
TBB_BIN alone and named no TBB_LIB. A Windows user who built against their own TBB
would get that directory prepended to PATH ahead of the one their binary is linked
against. The record holds the directory the call named, tbb_lib or else tbb_bin, as
make receives it, resolved against the installation when relative, and the
installation's own lib/tbb when the call named neither. It is written at build time, so a later set_cmdstan_path() cannot move it.

Where it applies

Bare tbb_path() is already right wherever the program comes out of the selected
installation, because there the session's installation owns the binary:
bin/stansummary and bin/diagnose (R/run.R:336), the make that builds them on
demand (:422), the model build (R/model.R:859) and the stanc invocations (:1146,
:1272, R/utils.R:1046).

It is wrong only where the model executable is launched, and that is four sites, not
the two that sample: R/run.R:660 and :782, run_info_cli() (R/cpp_opts.R:11),
which is the <exe> info call §7 hydrates an adopted executable with, and
parse_cmdstan_args() (R/model.R:2750) behind $cmdstan_defaults(). Those four are
every invocation in the package whose command is the model binary.

run_info_cli() is reached only when there is no usable record, since §7 adopts a
usable record without launching the binary, so it takes the table's last row by
construction, except straight after a build, where the directory that build resolved
is in hand and has not been written yet.

tbb_path() is not the helper for this. Its dir means an installation root and
it appends stan/lib/stan_math/lib/tbb (R/run.R:1238-1247), so a resolved
C:/opt/tbb/lib comes back as C:/opt/tbb/lib/stan/lib/stan_math/lib/tbb, and
R/install.R:485, :510 and :532 depend on the root meaning. The four launch sites
take a second helper that accepts the recorded directory and holds the three rows above,
including the Windows guard. The nine other call sites keep tbb_path() untouched.

Three routes to the TBB stay untracked

A TBB_LIB or TBB_BIN set in make/local or in ~/.config/stan/make.local; the
same two arriving from the environment; and a direct LDFLAGS_TBB override, which wins
because compiler_flags:306 assigns it with ?= while TBB_LIB reads empty and
TBB_BIN_ABSOLUTE_PATH stays at its default, so recovering the directory from it would
mean parsing linker flags. Make reads all three, so the build links against the TBB
they name, but tbb_dir sees only the call's cpp_options and records the
installation's own directory for those builds, which is what every launch gets today.
Asking make for the resolved value at build time was tried in Stage 3 and dropped: it
needs a fragment makefile and per-platform path conversion for a corner the launch
code has never handled. All three are low-level configuration, out of scope for 1.0 on
the same terms as the rest of §6's untracked list.

Checklist

  • Second helper: takes a recorded directory, returns the table's answer, Windows-guarded.
  • The four launch sites call it; the nine installation-program sites are untouched.
  • Tests: usable record with existing directory prepends it; usable record with gone
    directory prepends nothing; no usable record falls back to the selected installation;
    a non-default TBB_LIB build is launched with the recorded directory, not
    <builder>/stan/lib/stan_math/lib/tbb.
  • The post-build launch (run_info_cli() before the record is written) uses the
    directory in hand.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions