Project

General

Profile

Analysis How-To » History » Revision 3

Revision 2 (Richard Trotta, 04/30/2019 01:25 PM) → Revision 3/18 (Richard Trotta, 04/30/2019 01:28 PM)

h1. Analysis How-To 

 {{>toc}} 

 h2. How should I analyze data? 

 * Doing things locally will always be your best option for actual analysis.  
 ** Fork a repo of hallc_replay_kaonlt %{color:red}and% UTIL_KAONLT for your own custom version that you can play with 
 ** !! 

 * Once you have forked the repo, clone hallc_replay_kaonlt to a local directory 
 ** !! 

 * Now the tricky part, UTIL_KAONLT is a submodule of hallc_replay_kaonlt so some intermediate steps will need to be made 
 ** !! 
 ** Check .gitmodules to make sure submod is listed 
 ** !! 
 ** !! 

 * If HEAD is detached (check with git branch -a) 
 ** !! 
 ** !! 

 * Check if head is really detached 
 ** !! 
 ** !! 

 * Change to master branch 
 ** !! 

 * Pull and check branch again, everything should be set! 
 ** !! 

 * Now that we have our repo locally we should set it up to pull from the “main” JeffersonLab version 

 * First check your remote “origin” repo (this is where you will push to) 
 ** !! 

 * Now that we have our repo locally we should set it up to pull from the “main” JeffersonLab version 

 * First check your remote “origin” repo (this is where you will push to) 
 ** !! 

 * Next lets set up the “upstream” which is the JeffersonLab repo (DO NOT push HERE) 
 ** !! 

 * You will not be able to push to upstream unless you’re Stephen or me so don’t worry too much. Just be cautious. 

 * A similar procedure can be performed with UTIL_KAONLT 

 * Finally, let's talk about branches. Let’s add the develop branch to our local system… 
 ** First create a branch locally called develop and change to it 
 ** !! 

 * Now simply pull develop 
 ** !! 

 * To create a new branch you must first create it in github 
 ** !! 

 * Then simply repeat the steps for setting up a branch from the previous slide 

 h2. Replaying 

 * Before we can analyze we must replay. This should be done in the farm to save yourself time and local cpu effort.    The easiest way is to do a batch job submission, but this comes with some prep work. 

 * Before a batch submission, I highly encourage two preliminary steps 
 ## Do all debugging of replays locally, once this works move to the farm 
 ## Once on the farm you have two options; your ifarm version or our group (discussed soon).    This is for final debugging purposes to assure everything works in the farm, then you can submit a batch job. Save the root files in /volatile/hallc/c-kaonlt/<USER> (note: volatile is NOT backed up) 

 * There is a batch script I have created and Stephen as changed with the help of Brad to assure it will not mess things up.    Again, I highly recommend the two above steps before moving onto this script or you will be wasting time and resources. 

 * Your final batch job submissions can be saved directly to tape. 

 h2. Group environment 

 * You can do replays under your farm directory or you can use our group environment. 

 * We have set up a group environment with a version of our repo that currently mimics the cdaq as close as possible (although an updated hcana is used). 
 ** This group environment is under /u/group/c-kaonlt 
 ** I have made a directory USERS which you can use for person replay scripts and environments. DO NOT change any replays that are not under USERS without contacting Stephen or me first. 
 ** There is an hcana already set up here, use this for any group replays. If you would like to use a different version of hcana please use your farm directory. If I find a hcana in USERS I will destroy it. 

 * You may have issues with hcana, make sure you are in the JLab software environment version 2.1 
 ** source /site/12gev_phys/softenv.csh 2.1 (or .sh if using bash) 

 * The group environment has a 100 gb quota and is backed up. This means two important things… 
 ## DO NOT save root files here! Ever! 
 ## It’s backed up so its good for important calibration work (*wink *wink) 

 * Upon the request of Stephen, any improper use of this environment will incur a penalty of one beer/bottle of single malt or an owl shift (depending upon severity). 

 h2. Writing to tape 

 * Writing to tape info, read - https://scicomp.jlab.org/docs/write-through-cache. 

 * In your batch script, specify OUTPUT_FILE:/cache/hallc/kaonlt/USER/ROOTfiles/FILE. 
 ** Material in /cache is automatically copied to tape after some time if it is static 
 ** Small files (~1 MB) will not be backed up on tape 
 ** Once copied to tape, you can view the tape stub (NOT the file itself) under /mss/hallc/kaonlt/… 
 ** The tape does not handle overwriting well so if submit a job you must create a new "pass" directory… 
 *** -->jput ... file.root /mss/hallc/kaonlt/USER/ROOTfiles/pass1/ 
 ** The tape has FAR more space than we could get through so do not worry about "filling" it 
 ** Write to tape once you're happy with your code... just do it correctly