Analysis Getting Started » History » Revision 15
Revision 14 (Sean Jeffas, 05/11/2023 10:32 AM) → Revision 15/25 (Sean Jeffas, 05/11/2023 10:33 AM)
h1. Analysis Getting Started
These instructions are specific to the analyzer and SBS-offline installations existing under /work/halla/sbs, maintained by Andrew Puckett.
{{toc}}
h1. How to Reach the SBS Work Directory
* Log in to the JLab ifarm
** https://scicomp.jlab.org/docs/getting_started
* The SBS work directory is located at /work/halla/sbs
** Created a directory here with @mkdir username@
** If you do not have permission contact Ole Hansen (ole@jlab.org) and ask to be added to the SBS user group.
h1. Setting up Environments
* If you want to set up your own personal analyzer you can look here https://github.com/JeffersonLab/analyzer
h1. Getting Files from Cache
* All raw EVIO files from GMn are on tape at @/mss/halla/sbs@
* Cached EVIO files are located at @/cache/halla/sbs@
* To write files from tape to cache see documentation here, https://scicomp.jlab.org/docs/node/586
** For example, to get all EVIO splits from run runnumber in GMn to cache execute @jcache get /mss/halla/sbs/raw/*runnumber*@
h1. Setting up the SBS Replay
* If you do not plan to make changes to the replay code you can simply use the version located at @/work/halla/sbs/SBS_OFFLINE@ and @/work/halla/sbs/SBS_REPLAY@
* The SBS analysis is located at https://github.com/JeffersonLab/SBS-offline and https://github.com/JeffersonLab/SBS-replay
** You can copy the github versions to your own work directory, if you plan to make your own changes to the analysis
h2. SBS Installation
* Follow the README instructions on https://github.com/JeffersonLab/SBS-offline to install SBS-offline.
* After installing, there should be a directory, @install/run_replay_here@.
* Inside there should be one file named @.rootrc@ (it is a hidden file).
** Wherever you run the replay, this file must be there to load the SBS-offline libraries. Either run your replays here, or move the @.rootrc@ file to the new destination.
h2. SBS Replay Environments
* The following lines should be used in a script to define where the data and output files should be located
<pre>
setenv SBS_REPLAY path-to-your-replay/SBS-replay
setenv DB_DIR $SBS_REPLAY/DB
setenv DATA_DIR /cache/mss/halla/sbs/raw
setenv OUT_DIR path-to-your-volatile/rootfiles
setenv LOG_DIR path-to-your-volatile/logs
setenv ANALYZER_CONFIGPATH $SBS_REPLAY/replay
</code></pre>
* @DATA_DIR@ tells the replay where the EVIO files are.
* @OUT_DIR@ tells the replay where to put the output ROOT files.
h2. Running the SBS Replay
* The main replay script, with all detectors, is @replay_gmn.C@ located at https://github.com/JeffersonLab/SBS-replay/tree/master/replay
* An example of running this using a shell script can be found here, @/work/halla/sbs/puckett/GMN_ANALYSIS/run_GMN_swif2.csh@
h1. Working example scripts for GMN analysis on the batch farm
The most efficient and convenient way to analyze GMN data on the batch farm is using the swif2 system. A general overview of the swif2 system is available from the computer center's documentation page here
The first step in using the swif2 system is setting up a "workflow" under your CUE account using "swif2 create", as documented here.
Once you have created a workflow, it can be used to launch jobs on the batch farm. The general command-line reference for using swif2 can be found here
Working example scripts to launch GMN replay jobs on the batch farm can be found at
/work/halla/sbs/puckett/GMN_ANALYSIS/launch_GMN_replay_swif2.sh
/work/halla/sbs/puckett/GMN_ANALYSIS/run_GMN_swif2.sh
These scripts both refer to directories and workflows that are specific to the "puckett" user account on the farm. They should be viewed as templates and examples for you to copy to your own work disk area and develop your own scripts and workflows. The first of these two scripts takes just two arguments: a run number and a maximum number of segments. The proper usage would be:
./launch_GMN_replay_swif2.sh runnum maxsegment
Here "runnum" refers to the CODA run number and "maxsegment" is the number of EVIO file segments to be replayed.
The script will create one batch job per EVIO file (assuming the file exists in /mss/halla/sbs/raw) and add it the workflow "puckett_GMN_analysis".
So for example, if run 99999 has the following file segments:
/mss/halla/sbs/raw/e1209019_99999.evio.0.0
/mss/halla/sbs/raw/e1209019_99999.evio.0.1
/mss/halla/sbs/raw/e1209019_99999.evio.0.2
/mss/halla/sbs/raw/e1209019_99999.evio.0.3
/mss/halla/sbs/raw/e1209019_99999.evio.0.4
/mss/halla/sbs/raw/e1209019_99999.evio.0.5
/mss/halla/sbs/raw/e1209019_99999.evio.0.6
/mss/halla/sbs/raw/e1209019_99999.evio.0.7
/mss/halla/sbs/raw/e1209019_99999.evio.0.8
/mss/halla/sbs/raw/e1209019_99999.evio.0.9
Then ./launch_GMN_replay_swif2.sh 99999 9 will create one job for each file in the list above from 0 to 9 inclusive. In other words, the 2nd command line argument refers to the segment number of the last file we want to replay, NOT the total number of segments, which is generally equal to Njobs = maxsegment + 1, since the segment number always starts at zero.
After executing this script, if the workflow is not already running, I would tell it to start releasing jobs to the batch farm using the command:
swif2 run puckett_GMN_analysis
For your own workflow you would replace "puckett_GMN_analysis" with the name of your swif2 workflow.
The second script (run_GMN_swif2.sh) takes care of setting up the environment on the farm node and actually executing the analyzer, and copying the output files of the replay job to an appropriate directory on /volatile. You don't need to call this script directly, but it is called with appropriate arguments by the swif2 jobs created by the first script.