Project

General

Profile

Analysis How-To » History » Version 16

Richard Trotta, 11/24/2019 03:09 PM

1 1 Richard Trotta
h1. Analysis How-To
2
3
{{>toc}}
4 2 Richard Trotta
5
h2. How should I analyze data?
6
7
* Doing things locally will always be your best option for actual analysis. 
8 15 Richard Trotta
** Fork a repo of hallc_replay_kaonlt %{color:red}and% UTIL_KAONLT for your own custom version that you can play with
9 10 Richard Trotta
!fork.png!
10 2 Richard Trotta
11
* Once you have forked the repo, clone hallc_replay_kaonlt to a local directory
12 5 Richard Trotta
<pre><code class="bash">
13
$USER> git clone https://github.com/USER/hallc_replay_kaonlt.git
14
</code></pre>
15 1 Richard Trotta
16
* Now the tricky part, UTIL_KAONLT is a submodule of hallc_replay_kaonlt so some intermediate steps will need to be made
17 5 Richard Trotta
<pre><code class="bash">
18 8 Richard Trotta
$USER/hallc_replay> git submodule --init --recursive
19 5 Richard Trotta
</code></pre>
20 1 Richard Trotta
** Check .gitmodules to make sure submod is listed
21 5 Richard Trotta
<pre><code class="bash">
22
[submodule "UTIL_KAONLT"]
23
     path = UTIL_KAONLT
24
     url = https://github.com/USER/UTIL_KAONLT
25
     branch = <branchname>
26
</code></pre>
27
<pre><code class="bash">
28 8 Richard Trotta
$USER/hallc_replay> git submodule update --recursive --remote
29 5 Richard Trotta
</code></pre>
30 2 Richard Trotta
31 1 Richard Trotta
* If HEAD is detached (check with git branch -a)
32
<pre><code class="bash">
33 8 Richard Trotta
$USER/hallc_replay> cd UTIL_KAONLT
34
35
$USER/hallc_replay/UTIL_KAONLT> git branch -a
36 6 Richard Trotta
* (HEAD detached at ###)
37 1 Richard Trotta
</code></pre>
38
39 6 Richard Trotta
* Check if head is really detached
40 1 Richard Trotta
<pre><code class="bash">
41 8 Richard Trotta
$USER/hallc_replay/UTIL_KAONLT> git symbolic-ref HEAD
42 6 Richard Trotta
fatal: ref HEAD is not a symbolic ref
43
</code></pre>
44
<pre><code class="bash">
45 8 Richard Trotta
$USER/hallc_replay/UTIL_KAONLT> git remote update
46 6 Richard Trotta
Fetching origin
47
</code></pre>
48 1 Richard Trotta
49
* Change to master branch
50 6 Richard Trotta
<pre><code class="bash">
51 8 Richard Trotta
$USER/hallc_replay/UTIL_KAONLT> git checkout master
52 1 Richard Trotta
Switched to branch 'master'
53
Your branch is up-to-date with 'origin/master'
54 6 Richard Trotta
</code></pre>
55
56 1 Richard Trotta
* Pull and check branch again, everything should be set!
57
<pre><code class="bash">
58 8 Richard Trotta
$USER/hallc_replay/UTIL_KAONLT> git pull
59 6 Richard Trotta
Already up-to-date
60 8 Richard Trotta
61
$USER/hallc_replay/UTIL_KAONLT> git branch -a
62 6 Richard Trotta
* master
63
</code></pre>
64 1 Richard Trotta
65
* Now that we have our repo locally we should set it up to pull from the “main” JeffersonLab version
66
67
* First check your remote “origin” repo (this is where you will push to)
68 6 Richard Trotta
<pre><code class="bash">
69 8 Richard Trotta
$USER/hallc_replay> git remote -v
70 6 Richard Trotta
origin https://github.com/USER/hallc_replay_kaonlt.git (fetch)
71
origin https://github.com/USER/hallc_replay_kaonlt.git (push)
72
</code></pre>
73 2 Richard Trotta
74
* Next lets set up the “upstream” which is the JeffersonLab repo (DO NOT push HERE)
75 6 Richard Trotta
<pre><code class="bash">
76 8 Richard Trotta
$USER/hallc_replay> git remote add upstream https://github.com/JeffersonLab/hallc_replay_kaonlt.git
77 6 Richard Trotta
78 8 Richard Trotta
$USER/hallc_replay> git remote -v
79 6 Richard Trotta
origin https://github.com/USER/hallc_replay_kaonlt.git (fetch)
80 1 Richard Trotta
origin https://github.com/USER/hallc_replay_kaonlt.git (push)
81
upstream https://github.com/JeffersonLab/hallc_replay_kaonlt.git (fetch)
82 6 Richard Trotta
upstream https://github.com/JeffersonLab/hallc_replay_kaonlt.git (push)
83
</code></pre>
84 2 Richard Trotta
85
* You will not be able to push to upstream unless you’re Stephen or me so don’t worry too much. Just be cautious.
86
87
* A similar procedure can be performed with UTIL_KAONLT
88 1 Richard Trotta
89 2 Richard Trotta
* Finally, let's talk about branches. Let’s add the develop branch to our local system…
90
** First create a branch locally called develop and change to it
91 7 Richard Trotta
<pre><code class="bash">
92 8 Richard Trotta
$USER/hallc_replay> git branch develop
93 1 Richard Trotta
94 8 Richard Trotta
$USER/hallc_replay> git checkout develop
95 7 Richard Trotta
M  UTIL_KAONLT
96
M  UTIL_OL
97
Switched to branch 'develop'
98
</code></pre>
99
100 1 Richard Trotta
* Now simply pull develop
101 7 Richard Trotta
<pre><code class="bash">
102 8 Richard Trotta
$USER/hallc_replay> git pull origin develop
103 7 Richard Trotta
</code></pre>
104 2 Richard Trotta
105
* To create a new branch you must first create it in github
106 10 Richard Trotta
!newbranch.png!
107 2 Richard Trotta
108
* Then simply repeat the steps for setting up a branch from the previous slide
109
110 16 Richard Trotta
h2. Navigating hallc_replay_lt
111
112
* The GitHub can be found "here":https://github.com/JeffersonLab/hallc_replay_lt
113
* Attached is a summary for all directories in files in hallc_replay_lt [%{background:yellow}ONGOING%] attachment:hallc_replay_outline.pdf
114
115 11 Richard Trotta
h2. Getting .dat files from tape
116
117
* If *.dat for a particular run is not in /cache/hallc/spring17/raw follow the instructions below...
118
** In /cache/hallc.spring17/raw type: 
119
<pre>
120
> jcache get /mss/hallc/spring17/raw/<YourRawFile>.dat
121
</pre>
122
** This will take a little while to process. You can check the status of your process by typing:
123
<pre>
124
> jcache pendingRequest <JlabUserName>
125
</pre>
126
*->More information on using jcache can be found at* https://scicomp.jlab.org/docs/%20
127
128 2 Richard Trotta
h2. Replaying
129
130
* Before we can analyze we must replay. This should be done in the farm to save yourself time and local cpu effort.  The easiest way is to do a batch job submission, but this comes with some prep work.
131
132
* Before a batch submission, I highly encourage two preliminary steps
133
## Do all debugging of replays locally, once this works move to the farm
134 3 Richard Trotta
## Once on the farm you have two options; your ifarm version or our group (discussed soon).  This is for final debugging purposes to assure everything works in the farm, then you can submit a batch job. Save the root files in /volatile/hallc/c-kaonlt/<USER> (note: volatile is NOT backed up)
135
136
* There is a batch script I have created and Stephen as changed with the help of Brad to assure it will not mess things up.  Again, I highly recommend the two above steps before moving onto this script or you will be wasting time and resources.
137
138
* Your final batch job submissions can be saved directly to tape.
139
140
h2. Group environment
141
142
* You can do replays under your farm directory or you can use our group environment.
143
144
* We have set up a group environment with a version of our repo that currently mimics the cdaq as close as possible (although an updated hcana is used).
145
** This group environment is under /u/group/c-kaonlt
146
** I have made a directory USERS which you can use for person replay scripts and environments. DO NOT change any replays that are not under USERS without contacting Stephen or me first.
147
** There is an hcana already set up here, use this for any group replays. If you would like to use a different version of hcana please use your farm directory. If I find a hcana in USERS I will destroy it.
148
149
* You may have issues with hcana, make sure you are in the JLab software environment version 2.1
150
** source /site/12gev_phys/softenv.csh 2.1 (or .sh if using bash)
151
152
* The group environment has a 100 gb quota and is backed up. This means two important things…
153
## DO NOT save root files here! Ever!
154
## It’s backed up so its good for important calibration work (*wink *wink)
155
156
* Upon the request of Stephen, any improper use of this environment will incur a penalty of one beer/bottle of single malt or an owl shift (depending upon severity).
157
158
h2. Writing to tape
159
160
* Writing to tape info, read - https://scicomp.jlab.org/docs/write-through-cache.
161
162 13 Richard Trotta
* In your batch script, specify OUTPUT_FILE:/cache/hallc/kaonlt/USER/ROOTfiles/
163 3 Richard Trotta
** Material in /cache is automatically copied to tape after some time if it is static
164
** Small files (~1 MB) will not be backed up on tape
165 4 Richard Trotta
** Once copied to tape, you can view the tape stub (NOT the file itself) under /mss/hallc/kaonlt/…
166
** The tape does not handle overwriting well so if submit a job you must create a new "pass" directory…
167
*** -->jput ... file.root /mss/hallc/kaonlt/USER/ROOTfiles/pass1/
168
** The tape has FAR more space than we could get through so do not worry about "filling" it
169
** Write to tape once you're happy with your code... just do it correctly
170
171
h2. Few more words of warning
172
173
* Do not write analysis to tape unless you are 100% certain it works correctly (and you don't want to repeat it very soon).
174
175
* For farm jobs some info is included below -
176
** See https://scicomp.jlab.org/docs/text_command_file for info on commands	
177
** Do not set CPU above 1 (it will slow your job down in the queue and hcana is single threaded anyway so you gain nothing)
178
** Farm/Auger project: c-kaonlt
179 1 Richard Trotta
** For TEMPORARY output, write to volatile - /volatile/hallc/c-kaonlt/USER, this space is NOT backed up!
180
** Specify the FULL path to this in your symbolic link
181
** Make sure relvant directories are created
182
183
* You can use our work environment (/work/hallc/kaon), but this is not backed up and I will no be setting up an environment similar to group there. It’s a good place to put personal scripts if you don’t want to take up space in your farm directory.
184 16 Richard Trotta
185
h2. Analysis Starting Points
186
187
* Starting from this page: https://redmine.jlab.org/projects/podd/wiki/Workshop2018
188
* In general, items with a (*) on that page denotes an interactive tutorial component.  
189
** If you login to Bluejeans via: https://jlab.bluejeans.com/ then you can view the recorded video sessions.
190
191
* Review the following presentations first:
192
** Farm Use and Computing Resources Tips and Tricks -- Brad Sawatzky
193
** Overview & Update of the Hall C Analyzer -- Eric Pooser
194
** Eric's talk from the 2019 Hall C Winter Collab meeting is also quite
195
    useful: https://www.jlab.org/indico/event/296/session/11/contribution/12/material/slides/0.pdf
196
197
* Then move on to the Tuesday 'Hall C' sessions starting with the git howto:
198
** Effective Git use (*) -- Steve Wood
199
** Folks really do need to understand how to work with git.  They should follow up on the tutorials/howtos Steve mentions in his talk before moving on.
200
201
* If you want a space to work, log on to the ifarm and create a directory under /group/c-kaonlt/Users/.  For example:
202
<pre><code class="bash">
203
  > cd /group/c-kaonlt/Users/
204
  > mkdir trottar   ## change 'trottar' to your username
205
  > cd trottar      ## change 'trottar' to your username
206
</code></pre>
207
208
* If you want to follow along with the interactive sessions, the you can get the VM (virtualbox) setup working.  Most of the interactive 'howtos' are in the Tuesday afternoon session.  Ole's talks are likely a good place to start with the VM.  The video recordings may be of particular use here.
209
** 14:30 Reading and processing trees (part2) (*) -- Ole Hansen
210
211
* Next level analysis would include working through the 'Hall C Analysis' talks in the Tuesday morning session.  They are on calibration tasks that we will need to perform ourselves. They also outline how the various detector maps, cut, and def files interact.  (This is the kind of information I would like to get consolidated in a set of new 'howto' pages on the wiki.)
212
213
* All the software and steps outlined on the "old" pages will match what we will do very closely, so it is *not* wasted time to start with the above documentation.
214
215
* For those of you who can get started on a Detector Calibration procedure, you can start editing tasks in: https://redmine.jlab.org/projects/kltexp/wiki/Analysis_Tasks 
216
** Start by selecting the task (i.e. detector) of interest
217
** If you have files/scripts that are not already online as part of a git repo, or similar, then you can upload a copy in "tasks":https://redmine.jlab.org/projects/kltexp/wiki/Analysis_Tasks (as stated above)
218
** Be sure you put a description in there too that says who you are, and where you got the scripts from.
219
220
* Note that all of the calibrations steps have already been done by existing groups in Hall C!  Talk to folks on the previous experiments (if that isn't you) and have them show you the documentation and notes they already have.  Start with just getting those notes linked to this page.  If different groups have different procedures, you can provide links/references to both.
221
222
* Clean up/consolidation will be the next step.