Conversation
6c69f3c to
7ee0a6c
Compare
| {{/ntfy}} | ||
|
|
||
|
|
||
| {{#parallel}} |
There was a problem hiding this comment.
why did this lose {{ bbi_exe_path }}?
There was a problem hiding this comment.
I believe the refactored methods stopped filling in bbi_exe_path, and giving {{bbi_exe_path}} here would have been empty, causing a failure on node after submission. But it is templated again now (in re to your other comment), so i can re-insert it.
| #SBATCH --output={{log_path}} | ||
|
|
||
| {{#parallel}} | ||
| exec bbi nonmem run local {{model_path}}.mod --parallel --threads={{ncpu}} --config {{config_path}} |
There was a problem hiding this comment.
Hmmm not sure how I feel about exec bbi seems like it needs to be templated...
There was a problem hiding this comment.
assuming you want the same for all the other templates (exec pharos nonmem, exec rscript etc)
…ompute and login node have same container image
|
What's the justification for making this an s3 method at all. I do not generally like the boilerplate unless we have a good reason for it. I'm also not sure the way this is designed is the right mental model of if we were to do a generic interface where things would register. Eg we don't want a bunch of concrete implementations consolidated here, more likely we want to provide a set of patterns and functions that makes it easier to use slurm from the tool you care about without necessarily pulling in too much of the ecosystem. |
|
It was inspired by a client meeting, not a direct request, just thought it would make the process simpler for the next client(s). I will work on de-generalizing the function, and make a template/pattern that can be adopted by multiple tools. I am doing this on a new branch, and will push after doing some proofing. |












Summary
Turns
submit_slurm_job()into an S3 generic. What runs on the compute node isdecided by what you pass as
input, and the command itself lives in the jobtemplate, not in R. Each method's only job is to assemble the variables its
template needs and hand them to one shared renderer.
inputbbi_nonmem_modelnonmem-bbitemplate (bbi nonmem run local)hyperion_nonmem_modelhyperion::submit_model_to_slurm(), which carries its own template and finds its own pharos config.R/.qmdpathrscript/quartotemplate by extension; a character vector submits one job per pathtemplate+template_optssubmit_slurm_job.default()errors with what it does acceptNew
submit_rendered_job()is the shared worker: fill the whisker template,write the script under
submission_root,chmod,sbatch. It knows nothingabout what runs on the node, and paths flow through untouched — pharos-style,
the template owns any path handling.
Templates
Four templates now ship in
inst/templates/(based on the pharos slurmtemplate):
nonmem-bbi,nonmem-pharos,quarto,rscript. Beyond your owntemplate_opts, a template can use{{job_name}},{{partition}},{{ncpu}},{{parallel}},{{num_mpi_cpus}},{{account}},{{log_path}}, and per method{{model_path}},{{config_path}}or{{script_path}}.Breaking changes
submit_slurm_job():.mod→input,slurm_job_template_path→template,bbi_config_path→config_path,slurm_template_opts→template_opts.slurmtools.slurm_job_template_pathandslurmtools.bbi_config_pathare no longer read —templatenow defaults tothe shipped template for the input type, and
config_pathdefaults tooptions('slurmtools.config_path'). The.onAttachmessage was updated tomatch.
.mod/.ctlpath is now refused with a pointer tobbr::read_model()/hyperion::read_model(). Previously a bare path wassilently coerced into a
list(absolute_model_path = ...)and run through thebbi path, which guesses the engine from a filename. Choosing the engine is the
caller's decision, so it now requires a model object.
submit_nonmem_model()andR/submit-model.Rareremoved, along with the
lifecycledependency. (It was@keywords internaland never exported, so no public surface goes away.)
Partition handling
Split the resolve/validate step out of the cpu-fit check, and put the cache
lookup in one place:
cached_partition_table()— the cache-or-lookup step, shared.validate_partition()— resolves thepartition = get_slurm_partitions()default (takes the first/smallest) and confirms it's a real partition.
check_slurm_partitions()— unchanged behaviour, now expects analready-resolved partition and returns it invisibly.
This replaces the old
match.arg(partition), which only worked because thedefault happened to be the full vector.
Vignette
Running-nonmem.Rmdexecutes live during check, and was still written againstthe old API —
R CMD checkfailed at vignette build. Ported it and its twolocal template files:
slurm_job_template_path→template,slurm_template_opts→template_opts,options('slurmtools.bbi_config_path')→options('slurmtools.config_path'); the "default template variables" sectionnow documents what the methods actually inject.
(
{{bbi_exe_path}},{{bbi_config_path}},{{project_name}},{{project_path}}) — whisker renders those empty, i.e. a job that submitsfine and dies on the node. They now invoke
bbifrom PATH and use{{config_path}}/{{#account}}/{{log_path}}.user = "matthews", and the cleanup chunk now cancels only the two job idsthe vignette submitted (parsed from
submission$stdout) instead of everyRUNNING job of a hardcoded user. The old cleanup also called
cancel_slurm_job(job, confirm = FALSE)— the argument isauto_confirm, soit had been silently erroring inside
tryCatchand never cancelled anything.Tests
New
tests/testthat/test-submit-slurm-job.Rcovers template selection perextension, path pass-through, vectorised submission, script write + chmod,
relative
submission_root/template,template_optsoverridingmethod-built values, the bare-control-stream refusal, unsupported extensions,
user-supplied templates for arbitrary file types, missing files, the
.defaulterror, the bbi template render, and partition validation.
devtools::test(): 39 passing, 0 failures, 1 skip (thehyperion-not-installed branch, which can't fire where hyperion is installed).
devtools::check(): 0 errors, 0 notes, 1 warning (qpdfnot installed onthe check machine — no PDFs in the package, so environmental only). Note the
vignette build really submits (and then cancels) two tiny
1001bbi runs onthe cluster.
Housekeeping
rproject.toml: project name washyperion→slurmtools; prism snapshotbumped to
2026-06-30; dropdependencies_onlyso the package itself installs.hyperionadded toSuggests.air.tomladded (excludesrv/scripts/from formatting)..gitignore: root-level accessory.qmd/.htmldesign notes and their renderartifacts, plus
manual-test/scratch..Rbuildignore: same accessory files — they're out of git butR CMD checkbuilds from the working tree, and they caused 3 of the 4 check notes.
heremoved Imports → Suggests: its lastR/user was the removedsubmit-model.R; the vignette still uses it.Note for reviewers
The
.onAttachmessage reportsslurmtools.submission_rootbut notslurmtools.config_path, even though the bbi method defaults to it. Happy toadd it if we want that surfaced at load.