Reorder compilation flag composition

[avehtari]

cmdstanr composes <make/local STANCFLAGS>, which means the explicit stanc_options can be overruled by make/local. I propose we change the order to <make/local STANCFLAGS> , so that make/local provides the defaults which the user can override with explicit stanc_options argument. For example, if make/local has --O1 and the user wants to test --Oexperimental they could the latter with stanc_options without need to edit make/local.

I can make a PR, if it is agreed that is a good idea, ping @jgabry

[WardBrian]

Does cmdstanr actually read make/local back in to compose a make command, or does it just invoke make and let the cmdstan makefile handle reading make/local?

[avehtari]

It turns out that --Oexperimental loses to --O1 and --O0 now matter which order they have. To fix this in cmdstanr would then require dropping make/local --O... flag if other flag is given by the user in stanc_options. Now the question is whether the logic should be fixed in stanc instead`

[WardBrian]

If I changed the logic in stanc I would probably making passing multiple of these an error, which is the most “intuitive” but makes these things more annoying.

We’re also strongly considering removing Oexperimental or just making it equivalent to O1, because of the number of known issues with it

[WardBrian]

Relevant logic in stanc: https://github.com/stan-dev/stanc3/blob/67a036e2815dfe131686275d01b920562bbcaea5/src/stanc/CLI.ml#L362

[jgabry]

On the v1.0 branch (which isn't fully functional yet, very much a WIP) this general issue is already handled, but not by reordering. We read make/local's STANCFLAGS and drop any flag the user also passed via stanc_options so stanc_options takes precedence. I've had several requests for something like this, not just this issue.

But that doesn't actually fix this specific example with the optimization flags, because --O0, --O1 and --Oexperimental are different flag names and apparently stanc has logic that picks one regardless of order. We could maybe extend the v1.0 branch to treat these optimization flags in a such a way that the user specifying --Oexperimental via stanc_options overrides --O1 from make/local, but I would probably prefer changing the logic in stanc itself like @WardBrian mentioned. That would make this an error and force the user to edit make/local, but would simplify things for cmdstanr. If an error -> make/local edit is really a pain point I'm open to reconsidering and including this in cmdstanr though.