Repository navigation
Conversation
This fixes the anchoring of data timeframes for MC-DATA embedding and adds unit tests. - An explicit timeframe ID is counted from the run's first orbit, as in the data, not from the start moved past the ITS ramp-up. - A data timeframe may start after the end of run recorded in CCDB; it is no longer refused. - Its interaction rate is read at the last time the CTP counted collisions. - Its conditions are read just before the end of run, where the run's objects are valid. - Sampled anchored jobs are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This switches off the TPC looper generator and the QED background for --data-anchoring and stamps the production's interaction rate into imported contexts. - The data already carries both backgrounds. - The looper generator aborted on contexts taken from data, which carried no interaction rate. - --with-qed and an explicit GenTPCLoopers.loopersVeto still win. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This extends the MC-DATA embedding utils for production use and adds unit tests. - --dfs restricts the collision contexts to given DF folders, and --expect-tfs stops with exit code 3 if the timeframes differ from the plan. - --list-units writes DF folder -> timeframe IDs as JSON. - Collisions are ordered by BC time plus collision time; the AO2D order made the ITS/MFT digitizer fail with "deduced ROFrame precedes already processed one". - DF folders without triggered collisions no longer crash, and files with more than 1000 DF folders are no longer truncated. - --set-interaction-rate stores a rate in existing contexts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This provides new tools in MC/run/ANCHOR/dataembedding to run MC-DATA embedding as a production.
- plan_embedding.py samples a fraction of a run's data AO2Ds and packs whole DF folders into jobs (embedding_table.json); a DF folder is never split.
- submit_embedding.py submits the table via grid_submit.sh: one split-on-collection masterjob per cycle, so jobs run close to their data, or one production split.
- check_embedding_output.py merges per parent file and checks the DF folders, the parentFiles links and that MC collisions sit at the data collision BCs; excluded timeframes are taken into account.
- embedding_row.py finds a job's row from its input file and cycle, or from a row index.
- Seeds are a hash of the job id (ALIEN_PROC_ID), DF folder and timeframe; distinct timeframes share a seed only by chance.
plan_embedding.py --run 544653 --data-path /alice/data/2023/LHC23zzm/544653/apass4 \
--fraction 0.01 --seed 1 --tfs-per-job 200 --out plan
submit_embedding.py --plan plan --anchor-env anchor.env --table-dir <alien dir> --jobname emb --submit
check_embedding_output.py check --plan plan --mc <MC AO2D> --parent <data AO2D>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This lets anchorMC_DataEmbedding.sh treat one row of a work table and makes the job robust at scale. - With ALIEN_JDL_MC_DATA_EMBEDDING_TABLE the job takes its data file and DF folders from the table. - Each timeframe gets its own seed from the job id (ALIEN_PROC_ID), its DF folder and its timeframe ID. - Failed tasks are retried twice, and workflow creation is tried twice. - Each parallel workflow gets its share of the job's CPU and memory. - The final merge orders inputs by DF folder and timeframe and cannot split a folder on size. - Timeframes are reported done, failed or excluded in embedding_status.tsv; a failed one makes the job exit non-zero, excluded ones do not. - The excluded-timestamp and workflow-creation-only paths no longer nest the workspaces. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This extends the MC-DATA embedding of b83568d to productions. A planner samples a fraction of a run's data AO2Ds and packs whole DF folders into jobs. Each job embeds exactly its DF folders and links every MC DF folder to its data parent via
parentFiles. A DF folder is never split across jobs, so no index offsets are needed.Submission goes through
grid_submit.sh, with one split-on-collection masterjob per cycle, so that jobs run close to their data. The job script only reads JDL variables and one input file, the work table, which should also allow submitting the same jobs via LPM.plan_embedding.py --run 544653 --data-path /alice/data/2023/LHC23zzm/544653/apass4 \ --fraction 0.01 --seed 1 --tfs-per-job 200 --out plan submit_embedding.py --plan plan --anchor-env anchor.env --table-dir <alien dir> --jobname emb --submit check_embedding_output.py check --plan plan --mc <MC AO2D> --parent <data AO2D>A 1 % production of run 544653 (LHC23zzm apass4, 89 files, 7828 timeframes,
pythia8ppsignals) exposed problems that only appear at this scale: data timeframes starting after the recorded end of run, collisions not ordered in time within a timeframe, and the TPC looper generator aborting on contexts taken from data. These are fixed here, and failed tasks are now retried, which covers transient CCDB failures. All 89 outputs passcheck_embedding_output.py: the DF folders, theparentFileslinks and the BCs of the MC collisions match the data. With--data-anchoring, TPC loopers and QED are now off by default, which reduced the CPU time per timeframe from 85 to 31–67 minutes;--with-qedandGenTPCLoopers.loopersVeto=falsebring them back. Each timeframe gets a seed hashed fromALIEN_PROC_ID, its DF folder and its timeframe ID; distinct timeframes share a seed only by chance.Note that sampled anchored MC is not affected: the anchoring changes apply only to explicit timeframe IDs.
A job still copies the BC and collision tables of its whole data file, also when it treats only some of its DF folders; this is small (about 8 MB per file) and left for later.
Reading MC and data together in an analysis, with
--aod-origin-level-mapping EMB:1as ino2-testworkflows-analysis-emb, does not work yet on these files. The AOD reader opens the correct parent file and then fails; this is left for a separate investigation.Assisted by Claude Code.