HGCAL simulation using law

Resources

law
luigi

Setup

This repository has submodules, so you should clone it with

git clone --recursive https://github.com/riga/hgcalsim.git

If you want to write files into the common hgcalsim output directory on EOS at /eos/cms/store/cmst3/group/hgcal/CMG_studies/hgcalsim (given that you have the necessary permissions), export

export HGC_STORE_EOS="/eos/cms/store/cmst3/group/hgcal/CMG_studies/hgcalsim"

or put the above line in your bashrc.

After cloning, run

source setup.sh

This will install CMSSW and a few python packages once. You should source the setup script everytime you start with a new session.

Luigi scheduler

Task statuses and dependency trees can be visualized live using a central luigi scheduler. This is optional and no strict requirement to run tasks.

In order to let the tasks communicate with a central luigi scheduler, you should set

export HGC_SCHEDULER_HOST="..."
export HGC_SCHEDULER_PORT="..."

most probably in your bashrc file. Otherwise, you should add --local-scheduler to all law run commands.

Storage on EOS

By default, most of the tasks write their output files to EOS. To prevent that (either because you don't have the necessary permissions on in case you want to test your code), add --eos False to the law run commands.

Example commands

Re-compile CMSSW with 2 cores after making some updates to the code:

law run CompileCMSSW --n-cores 2

Run GSD, RECO, NTUP and conversion steps:

law run ConverterTask --n-events 2 --branch 0 --version dev

Run the above steps for 10 tasks on HTCondor:

law run ConverterTask --n-events 2 --n-tasks 10 --version dev1_converter --pilot --workflow htcondor

Merge the converted files into a configurable number of files (--n-merged-files):

law run MergeConvertedFiles --n-events 2 --n-tasks 10 --n-merged-files 1 --version dev1_converter \
    --ConverterTask-pilot --ConverterTask-workflow htcondor

sitongan / hgcalsim