Skip to content

Commit 6f7e15c

Browse files
authored
GPU: document the contents of a standalone dataset and how to produce one (#16135)
1 parent 5cc7845 commit 6f7e15c

1 file changed

Lines changed: 60 additions & 4 deletions

File tree

‎GPU/documentation/build-standalone.md‎

Lines changed: 60 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -81,23 +81,79 @@ The standalone benchmark supports running on Run2 data exported from AliRoot, or
8181
In o2, `o2-tpc-reco-workflow` and the `o2-gpu-reco-workflow` can dump event data with the `configKeyValue` `GPU_global.dump=1;`.
8282
This will dump the event data to the local folder, all dumped files have a `.dump` file extension. If there are multiple TFs/events processed, there will be multiple `event.i.dump` files. In order to create a standalone dataset out of these, just copy all the `.dump` files to a subfolder in `~/standalone/events/[FOLDERNAME]`.
8383

84+
## What a dataset consists of
85+
86+
A dataset folder is read by `GPUReconstruction::ReadSettings` and `GPUChainTracking::ReadSettings`, and holds:
87+
88+
- `settings.dump` - the GRP settings (solenoid field, continuous max time bin). Without it the benchmark aborts with `Error reading event config file`.
89+
- `event.<i>.dump` - one file per time frame / event.
90+
- the calibration objects, each written only if it was present when dumping: `tpctransform.dump`, `tpcpadgaincalib.dump`, `tpczslinkmapping.dump`, `dEdxCalibContainer.dump`, `matlut.dump`, `trdgeometry.dump`, `trdrecoparam.dump`.
91+
92+
`tpctransform.dump` is needed for everything except already-transformed clusters; without it the run stops at `Need cluster native data for on-the-fly TPC transform`. `dEdxCalibContainer.dump` is needed unless dE/dx is switched off with `--rundEdx 0`.
93+
94+
The folder is looked up under `./events/` relative to the working directory, so `-e mydata` reads `./events/mydata/`. Use `--absoluteEventsDir` to pass a path outside of `./events/`.
95+
96+
## Updating the calibration objects of a dataset
97+
98+
The calibration data formats in O2 change often. When they do, the `event.<i>.dump` files of a dataset stay readable, but the calibration objects next to them have to be updated. There are two ways:
99+
100+
- Dump the dataset again with `dpl-workflow.sh`, as described below. This works for any data.
101+
- For MC without distortions, where all calibration objects hold nominal values, recreate them with `--recreateTrivialCalibObjects`. It writes the objects with their default contents to the current folder, and exits.
102+
103+
`--recreateTrivialCalibObjects` exists in both builds of the benchmark, which differ in what they can write:
104+
105+
- `o2-gpu-standalone-benchmark`, built as part of O2, writes all of them: `tpctransform.dump`, `tpcpadgaincalib.dump`, `dEdxCalibContainer.dump`, `trdgeometry.dump`, `trdrecoparam.dump`, `tpczslinkmapping.dump`, and `matlut.dump`, which it fetches from the CCDB. The material LUT is taken for the current time, or for the timestamp in ms given as the option's value (`--recreateTrivialCalibObjects <timestamp>`).
106+
- `ca` of the standalone build has no CCDB access, and does not write `matlut.dump` and `tpczslinkmapping.dump`.
107+
108+
`o2-gpu-standalone-benchmark` needs `O2_ROOT` set, since it reads the TPC mapping files; `ca` is completely standalone and needs nothing. Both also write a `settings.dump` with default values, which would replace the one of the dataset: run them in an empty folder and copy only the calibration objects into the dataset, e.g.
109+
```
110+
mkdir -p calib && cd calib
111+
o2-gpu-standalone-benchmark --recreateTrivialCalibObjects
112+
cp $(ls *.dump | grep -v settings.dump) ~/standalone/events/mydata/
113+
```
114+
115+
## From simulated data
116+
84117
Data can be dumped from raw data, or from MC data, e.g. generated by the Full System Test. In case of MC data, also MC labels are dumped, such that they are used in the `./ca --qa` mode.
85118

86-
To get a dump from simulated data, please run e.g. the FST simulation as described in [full-system-test-setup.md](https://github.com/AliceO2Group/AliceO2/blob/dev/prodtests/full-system-test/documentation/full-system-test-setup.md).
87-
A simple run as
119+
First simulate, digitize and convert to raw, as described in [full-system-test-setup.md](https://github.com/AliceO2Group/AliceO2/blob/dev/prodtests/full-system-test/documentation/full-system-test-setup.md). Run it in a scratch folder, since it writes a lot of intermediate files:
88120
```
121+
mkdir -p test && cd test
122+
ulimit -n 1024
89123
DISABLE_PROCESSING=1 NEvents=5 NEventsQED=100 SHMSIZE=16000000000 $O2_ROOT/prodtests/full_system_test.sh
90124
```
91-
should be enough.
125+
`ulimit -n 1024` avoids the workflow running out of file descriptors. `NEvents` is the number of collisions in the time frame, and sets the TPC occupancy. `NEventsQED` is the number of QED background events, which are simulated only for ITS, MFT and the forward detectors, not for the TPC.
92126

93-
Afterwards run the following command to dump the data:
127+
`DISABLE_PROCESSING=1` stops after the raw data is written, since the reconstruction is run separately below with the dump enabled. Afterwards run the following command in the same folder to dump the data:
94128
```
95129
SYNCMODE=1 CONFIG_EXTRA_PROCESS_o2_gpu_reco_workflow="GPU_global.dump=1;" WORKFLOW_DETECTORS=TPC SHMSIZE=16000000000 $O2_ROOT/prodtests/full-system-test/dpl-workflow.sh
96130
```
131+
This reads the raw files written by the simulation directly. `CONFIG_EXTRA_PROCESS_o2_gpu_reco_workflow="GPU_global.dump=1;"` is what writes the `.dump` files.
132+
133+
To take the data from DataDistribution instead, as on the EPNs, add `EXTINPUT=1`: the workflow then reads from the `readout-proxy` channel, and `datadistribution.sh` from the same folder has to run in a second shell to feed it (it needs the `DataDistribution` package).
134+
135+
Finally collect the result into a dataset:
136+
```
137+
mkdir -p ~/standalone/events/fst-tpc && cp *.dump ~/standalone/events/fst-tpc/
138+
```
97139

98140
To dump standalone data from CTF raw data in `myctf.root`, you can use the same script, e.g.:
99141
```
100142
CTFINPUT=1 INPUT_FILE_LIST=myctf.root CONFIG_EXTRA_PROCESS_o2_gpu_reco_workflow="GPU_global.dump=1;" WORKFLOW_DETECTORS=TPC SHMSIZE=16000000000 $O2_ROOT/prodtests/full-system-test/dpl-workflow.sh
101143
```
102144

103145
On the EPNs, you can find some reference data sets at `/home/drohr/standalone/events`.
146+
147+
## Using the built-in event generator
148+
149+
**Unsupported:** the built-in generator was written for particular studies of how the track fit behaves, and is not maintained. Use it only for such special tests, not to produce datasets; for anything else use the simulated or raw data above.
150+
151+
With `--eventGenerator` the benchmark generates simple synthetic events itself, without any simulation or raw data. The parent `events/` folder has to exist, since the generator only creates the dataset folder itself:
152+
```
153+
mkdir -p events
154+
./ca --eventGenerator -e gen --setMaxTimeBin 0 --rundEdx 0
155+
```
156+
`--setMaxTimeBin` has to be given, because the GRP settings are not read from a dataset in this mode and their default is the invalid sentinel `-2`.
157+
158+
This writes `settings.dump` and the `event.<i>.dump` files, but no calibration objects. The events hold the legacy cluster format, not native clusters, so they can be inspected and used for the generator's own QA, but tracking on them stops at `Need cluster native data for on-the-fly TPC transform`, also with calibration objects present.
159+

0 commit comments

Comments
 (0)