Project

General

Profile

Analysis How-To » History » Version 15

Richard Trotta, 04/30/2019 03:53 PM

1 1 Richard Trotta
h1. Analysis How-To
2
3
{{>toc}}
4 2 Richard Trotta
5
h2. How should I analyze data?
6
7
* Doing things locally will always be your best option for actual analysis. 
8 15 Richard Trotta
** Fork a repo of hallc_replay_kaonlt %{color:red}and% UTIL_KAONLT for your own custom version that you can play with
9 10 Richard Trotta
!fork.png!
10 2 Richard Trotta
11
* Once you have forked the repo, clone hallc_replay_kaonlt to a local directory
12 5 Richard Trotta
<pre><code class="bash">
13
$USER> git clone https://github.com/USER/hallc_replay_kaonlt.git
14
</code></pre>
15 1 Richard Trotta
16
* Now the tricky part, UTIL_KAONLT is a submodule of hallc_replay_kaonlt so some intermediate steps will need to be made
17 5 Richard Trotta
<pre><code class="bash">
18 8 Richard Trotta
$USER/hallc_replay> git submodule --init --recursive
19 5 Richard Trotta
</code></pre>
20 1 Richard Trotta
** Check .gitmodules to make sure submod is listed
21 5 Richard Trotta
<pre><code class="bash">
22
[submodule "UTIL_KAONLT"]
23
     path = UTIL_KAONLT
24
     url = https://github.com/USER/UTIL_KAONLT
25
     branch = <branchname>
26
</code></pre>
27
<pre><code class="bash">
28 8 Richard Trotta
$USER/hallc_replay> git submodule update --recursive --remote
29 5 Richard Trotta
</code></pre>
30 2 Richard Trotta
31 1 Richard Trotta
* If HEAD is detached (check with git branch -a)
32
<pre><code class="bash">
33 8 Richard Trotta
$USER/hallc_replay> cd UTIL_KAONLT
34
35
$USER/hallc_replay/UTIL_KAONLT> git branch -a
36 6 Richard Trotta
* (HEAD detached at ###)
37 1 Richard Trotta
</code></pre>
38
39 6 Richard Trotta
* Check if head is really detached
40 1 Richard Trotta
<pre><code class="bash">
41 8 Richard Trotta
$USER/hallc_replay/UTIL_KAONLT> git symbolic-ref HEAD
42 6 Richard Trotta
fatal: ref HEAD is not a symbolic ref
43
</code></pre>
44
<pre><code class="bash">
45 8 Richard Trotta
$USER/hallc_replay/UTIL_KAONLT> git remote update
46 6 Richard Trotta
Fetching origin
47
</code></pre>
48 1 Richard Trotta
49
* Change to master branch
50 6 Richard Trotta
<pre><code class="bash">
51 8 Richard Trotta
$USER/hallc_replay/UTIL_KAONLT> git checkout master
52 1 Richard Trotta
Switched to branch 'master'
53
Your branch is up-to-date with 'origin/master'
54 6 Richard Trotta
</code></pre>
55
56 1 Richard Trotta
* Pull and check branch again, everything should be set!
57
<pre><code class="bash">
58 8 Richard Trotta
$USER/hallc_replay/UTIL_KAONLT> git pull
59 6 Richard Trotta
Already up-to-date
60 8 Richard Trotta
61
$USER/hallc_replay/UTIL_KAONLT> git branch -a
62 6 Richard Trotta
* master
63
</code></pre>
64 1 Richard Trotta
65
* Now that we have our repo locally we should set it up to pull from the “main” JeffersonLab version
66
67
* First check your remote “origin” repo (this is where you will push to)
68 6 Richard Trotta
<pre><code class="bash">
69 8 Richard Trotta
$USER/hallc_replay> git remote -v
70 6 Richard Trotta
origin https://github.com/USER/hallc_replay_kaonlt.git (fetch)
71
origin https://github.com/USER/hallc_replay_kaonlt.git (push)
72
</code></pre>
73 2 Richard Trotta
74
* Next lets set up the “upstream” which is the JeffersonLab repo (DO NOT push HERE)
75 6 Richard Trotta
<pre><code class="bash">
76 8 Richard Trotta
$USER/hallc_replay> git remote add upstream https://github.com/JeffersonLab/hallc_replay_kaonlt.git
77 6 Richard Trotta
78 8 Richard Trotta
$USER/hallc_replay> git remote -v
79 6 Richard Trotta
origin https://github.com/USER/hallc_replay_kaonlt.git (fetch)
80 1 Richard Trotta
origin https://github.com/USER/hallc_replay_kaonlt.git (push)
81
upstream https://github.com/JeffersonLab/hallc_replay_kaonlt.git (fetch)
82 6 Richard Trotta
upstream https://github.com/JeffersonLab/hallc_replay_kaonlt.git (push)
83
</code></pre>
84 2 Richard Trotta
85
* You will not be able to push to upstream unless you’re Stephen or me so don’t worry too much. Just be cautious.
86
87
* A similar procedure can be performed with UTIL_KAONLT
88 1 Richard Trotta
89 2 Richard Trotta
* Finally, let's talk about branches. Let’s add the develop branch to our local system…
90
** First create a branch locally called develop and change to it
91 7 Richard Trotta
<pre><code class="bash">
92 8 Richard Trotta
$USER/hallc_replay> git branch develop
93 1 Richard Trotta
94 8 Richard Trotta
$USER/hallc_replay> git checkout develop
95 7 Richard Trotta
M  UTIL_KAONLT
96
M  UTIL_OL
97
Switched to branch 'develop'
98
</code></pre>
99
100 1 Richard Trotta
* Now simply pull develop
101 7 Richard Trotta
<pre><code class="bash">
102 8 Richard Trotta
$USER/hallc_replay> git pull origin develop
103 7 Richard Trotta
</code></pre>
104 2 Richard Trotta
105
* To create a new branch you must first create it in github
106 10 Richard Trotta
!newbranch.png!
107 2 Richard Trotta
108
* Then simply repeat the steps for setting up a branch from the previous slide
109
110 11 Richard Trotta
h2. Getting .dat files from tape
111
112
* If *.dat for a particular run is not in /cache/hallc/spring17/raw follow the instructions below...
113
** In /cache/hallc.spring17/raw type: 
114
<pre>
115
> jcache get /mss/hallc/spring17/raw/<YourRawFile>.dat
116
</pre>
117
** This will take a little while to process. You can check the status of your process by typing:
118
<pre>
119
> jcache pendingRequest <JlabUserName>
120
</pre>
121
*->More information on using jcache can be found at* https://scicomp.jlab.org/docs/%20
122
123 2 Richard Trotta
h2. Replaying
124
125
* Before we can analyze we must replay. This should be done in the farm to save yourself time and local cpu effort.  The easiest way is to do a batch job submission, but this comes with some prep work.
126
127
* Before a batch submission, I highly encourage two preliminary steps
128
## Do all debugging of replays locally, once this works move to the farm
129 3 Richard Trotta
## Once on the farm you have two options; your ifarm version or our group (discussed soon).  This is for final debugging purposes to assure everything works in the farm, then you can submit a batch job. Save the root files in /volatile/hallc/c-kaonlt/<USER> (note: volatile is NOT backed up)
130
131
* There is a batch script I have created and Stephen as changed with the help of Brad to assure it will not mess things up.  Again, I highly recommend the two above steps before moving onto this script or you will be wasting time and resources.
132
133
* Your final batch job submissions can be saved directly to tape.
134
135
h2. Group environment
136
137
* You can do replays under your farm directory or you can use our group environment.
138
139
* We have set up a group environment with a version of our repo that currently mimics the cdaq as close as possible (although an updated hcana is used).
140
** This group environment is under /u/group/c-kaonlt
141
** I have made a directory USERS which you can use for person replay scripts and environments. DO NOT change any replays that are not under USERS without contacting Stephen or me first.
142
** There is an hcana already set up here, use this for any group replays. If you would like to use a different version of hcana please use your farm directory. If I find a hcana in USERS I will destroy it.
143
144
* You may have issues with hcana, make sure you are in the JLab software environment version 2.1
145
** source /site/12gev_phys/softenv.csh 2.1 (or .sh if using bash)
146
147
* The group environment has a 100 gb quota and is backed up. This means two important things…
148
## DO NOT save root files here! Ever!
149
## It’s backed up so its good for important calibration work (*wink *wink)
150
151
* Upon the request of Stephen, any improper use of this environment will incur a penalty of one beer/bottle of single malt or an owl shift (depending upon severity).
152
153
h2. Writing to tape
154
155
* Writing to tape info, read - https://scicomp.jlab.org/docs/write-through-cache.
156
157 13 Richard Trotta
* In your batch script, specify OUTPUT_FILE:/cache/hallc/kaonlt/USER/ROOTfiles/
158 3 Richard Trotta
** Material in /cache is automatically copied to tape after some time if it is static
159
** Small files (~1 MB) will not be backed up on tape
160 4 Richard Trotta
** Once copied to tape, you can view the tape stub (NOT the file itself) under /mss/hallc/kaonlt/…
161
** The tape does not handle overwriting well so if submit a job you must create a new "pass" directory…
162
*** -->jput ... file.root /mss/hallc/kaonlt/USER/ROOTfiles/pass1/
163
** The tape has FAR more space than we could get through so do not worry about "filling" it
164
** Write to tape once you're happy with your code... just do it correctly
165
166
h2. Few more words of warning
167
168
* Do not write analysis to tape unless you are 100% certain it works correctly (and you don't want to repeat it very soon).
169
170
* For farm jobs some info is included below -
171
** See https://scicomp.jlab.org/docs/text_command_file for info on commands	
172
** Do not set CPU above 1 (it will slow your job down in the queue and hcana is single threaded anyway so you gain nothing)
173
** Farm/Auger project: c-kaonlt
174 1 Richard Trotta
** For TEMPORARY output, write to volatile - /volatile/hallc/c-kaonlt/USER, this space is NOT backed up!
175
** Specify the FULL path to this in your symbolic link
176
** Make sure relvant directories are created
177
178
* You can use our work environment (/work/hallc/kaon), but this is not backed up and I will no be setting up an environment similar to group there. It’s a good place to put personal scripts if you don’t want to take up space in your farm directory.