Project

General

Profile

Analysis How-To » History » Version 18

Richard Trotta, 11/26/2019 11:31 AM

1 1 Richard Trotta
h1. Analysis How-To
2
3
{{>toc}}
4 2 Richard Trotta
5 17 Richard Trotta
h2. Analysis Starting Points
6
7
* Starting from this page: https://redmine.jlab.org/projects/podd/wiki/Workshop2018
8
* In general, items with a (*) on that page denotes an interactive tutorial component.  
9
** If you login to Bluejeans via: https://jlab.bluejeans.com/ then you can view the recorded video sessions.
10
11
* Review the following presentations first:
12
** Farm Use and Computing Resources Tips and Tricks -- Brad Sawatzky
13
** Overview & Update of the Hall C Analyzer -- Eric Pooser
14
** Eric's talk from the 2019 Hall C Winter Collab meeting is also quite
15
    useful: https://www.jlab.org/indico/event/296/session/11/contribution/12/material/slides/0.pdf
16
17
* Then move on to the Tuesday 'Hall C' sessions starting with the git howto:
18
** Effective Git use (*) -- Steve Wood
19
** Folks really do need to understand how to work with git.  They should follow up on the tutorials/howtos Steve mentions in his talk before moving on.
20
21
* If you want a space to work, log on to the ifarm and create a directory under /group/c-kaonlt/Users/.  For example:
22
<pre><code class="bash">
23
  > cd /group/c-kaonlt/Users/
24
  > mkdir trottar   ## change 'trottar' to your username
25
  > cd trottar      ## change 'trottar' to your username
26
</code></pre>
27
28
* If you want to follow along with the interactive sessions, the you can get the VM (virtualbox) setup working.  Most of the interactive 'howtos' are in the Tuesday afternoon session.  Ole's talks are likely a good place to start with the VM.  The video recordings may be of particular use here.
29
** 14:30 Reading and processing trees (part2) (*) -- Ole Hansen
30
31
* Next level analysis would include working through the 'Hall C Analysis' talks in the Tuesday morning session.  They are on calibration tasks that we will need to perform ourselves. They also outline how the various detector maps, cut, and def files interact.  (This is the kind of information I would like to get consolidated in a set of new 'howto' pages on the wiki.)
32
33
* All the software and steps outlined on the "old" pages will match what we will do very closely, so it is *not* wasted time to start with the above documentation.
34
35
* For those of you who can get started on a Detector Calibration procedure, you can start editing tasks in: https://redmine.jlab.org/projects/kltexp/wiki/Analysis_Tasks 
36
** Start by selecting the task (i.e. detector) of interest
37
** If you have files/scripts that are not already online as part of a git repo, or similar, then you can upload a copy in "tasks":https://redmine.jlab.org/projects/kltexp/wiki/Analysis_Tasks (as stated above)
38
** Be sure you put a description in there too that says who you are, and where you got the scripts from.
39
40
* Note that all of the calibrations steps have already been done by existing groups in Hall C!  Talk to folks on the previous experiments (if that isn't you) and have them show you the documentation and notes they already have.  Start with just getting those notes linked to this page.  If different groups have different procedures, you can provide links/references to both.
41
42
* Clean up/consolidation will be the next step.
43
44 2 Richard Trotta
h2. How should I analyze data?
45
46
* Doing things locally will always be your best option for actual analysis. 
47 15 Richard Trotta
** Fork a repo of hallc_replay_kaonlt %{color:red}and% UTIL_KAONLT for your own custom version that you can play with
48 10 Richard Trotta
!fork.png!
49 2 Richard Trotta
50
* Once you have forked the repo, clone hallc_replay_kaonlt to a local directory
51 5 Richard Trotta
<pre><code class="bash">
52
$USER> git clone https://github.com/USER/hallc_replay_kaonlt.git
53
</code></pre>
54 1 Richard Trotta
55
* Now the tricky part, UTIL_KAONLT is a submodule of hallc_replay_kaonlt so some intermediate steps will need to be made
56 5 Richard Trotta
<pre><code class="bash">
57 8 Richard Trotta
$USER/hallc_replay> git submodule --init --recursive
58 5 Richard Trotta
</code></pre>
59 1 Richard Trotta
** Check .gitmodules to make sure submod is listed
60 5 Richard Trotta
<pre><code class="bash">
61
[submodule "UTIL_KAONLT"]
62
     path = UTIL_KAONLT
63
     url = https://github.com/USER/UTIL_KAONLT
64
     branch = <branchname>
65
</code></pre>
66
<pre><code class="bash">
67 8 Richard Trotta
$USER/hallc_replay> git submodule update --recursive --remote
68 5 Richard Trotta
</code></pre>
69 2 Richard Trotta
70 1 Richard Trotta
* If HEAD is detached (check with git branch -a)
71
<pre><code class="bash">
72 8 Richard Trotta
$USER/hallc_replay> cd UTIL_KAONLT
73
74
$USER/hallc_replay/UTIL_KAONLT> git branch -a
75 6 Richard Trotta
* (HEAD detached at ###)
76 1 Richard Trotta
</code></pre>
77
78 6 Richard Trotta
* Check if head is really detached
79 1 Richard Trotta
<pre><code class="bash">
80 8 Richard Trotta
$USER/hallc_replay/UTIL_KAONLT> git symbolic-ref HEAD
81 6 Richard Trotta
fatal: ref HEAD is not a symbolic ref
82
</code></pre>
83
<pre><code class="bash">
84 8 Richard Trotta
$USER/hallc_replay/UTIL_KAONLT> git remote update
85 6 Richard Trotta
Fetching origin
86
</code></pre>
87 1 Richard Trotta
88
* Change to master branch
89 6 Richard Trotta
<pre><code class="bash">
90 8 Richard Trotta
$USER/hallc_replay/UTIL_KAONLT> git checkout master
91 1 Richard Trotta
Switched to branch 'master'
92
Your branch is up-to-date with 'origin/master'
93 6 Richard Trotta
</code></pre>
94
95 1 Richard Trotta
* Pull and check branch again, everything should be set!
96
<pre><code class="bash">
97 8 Richard Trotta
$USER/hallc_replay/UTIL_KAONLT> git pull
98 6 Richard Trotta
Already up-to-date
99 8 Richard Trotta
100
$USER/hallc_replay/UTIL_KAONLT> git branch -a
101 6 Richard Trotta
* master
102
</code></pre>
103 1 Richard Trotta
104
* Now that we have our repo locally we should set it up to pull from the “main” JeffersonLab version
105
106
* First check your remote “origin” repo (this is where you will push to)
107 6 Richard Trotta
<pre><code class="bash">
108 8 Richard Trotta
$USER/hallc_replay> git remote -v
109 6 Richard Trotta
origin https://github.com/USER/hallc_replay_kaonlt.git (fetch)
110
origin https://github.com/USER/hallc_replay_kaonlt.git (push)
111
</code></pre>
112 2 Richard Trotta
113
* Next lets set up the “upstream” which is the JeffersonLab repo (DO NOT push HERE)
114 6 Richard Trotta
<pre><code class="bash">
115 8 Richard Trotta
$USER/hallc_replay> git remote add upstream https://github.com/JeffersonLab/hallc_replay_kaonlt.git
116 6 Richard Trotta
117 8 Richard Trotta
$USER/hallc_replay> git remote -v
118 6 Richard Trotta
origin https://github.com/USER/hallc_replay_kaonlt.git (fetch)
119 1 Richard Trotta
origin https://github.com/USER/hallc_replay_kaonlt.git (push)
120
upstream https://github.com/JeffersonLab/hallc_replay_kaonlt.git (fetch)
121 6 Richard Trotta
upstream https://github.com/JeffersonLab/hallc_replay_kaonlt.git (push)
122
</code></pre>
123 2 Richard Trotta
124
* You will not be able to push to upstream unless you’re Stephen or me so don’t worry too much. Just be cautious.
125
126
* A similar procedure can be performed with UTIL_KAONLT
127 1 Richard Trotta
128 2 Richard Trotta
* Finally, let's talk about branches. Let’s add the develop branch to our local system…
129
** First create a branch locally called develop and change to it
130 7 Richard Trotta
<pre><code class="bash">
131 8 Richard Trotta
$USER/hallc_replay> git branch develop
132 1 Richard Trotta
133 8 Richard Trotta
$USER/hallc_replay> git checkout develop
134 7 Richard Trotta
M  UTIL_KAONLT
135
M  UTIL_OL
136
Switched to branch 'develop'
137
</code></pre>
138
139 1 Richard Trotta
* Now simply pull develop
140 7 Richard Trotta
<pre><code class="bash">
141 8 Richard Trotta
$USER/hallc_replay> git pull origin develop
142 7 Richard Trotta
</code></pre>
143 2 Richard Trotta
144
* To create a new branch you must first create it in github
145 10 Richard Trotta
!newbranch.png!
146 2 Richard Trotta
147
* Then simply repeat the steps for setting up a branch from the previous slide
148
149 16 Richard Trotta
h2. Navigating hallc_replay_lt
150
151
* The GitHub can be found "here":https://github.com/JeffersonLab/hallc_replay_lt
152 18 Richard Trotta
* Attached is a summary for all directories in files in hallc_replay_lt [ %{background:yellow}ONGOING% ] attachment:hallc_replay_outline.pdf
153 16 Richard Trotta
154 11 Richard Trotta
h2. Getting .dat files from tape
155
156
* If *.dat for a particular run is not in /cache/hallc/spring17/raw follow the instructions below...
157
** In /cache/hallc.spring17/raw type: 
158
<pre>
159
> jcache get /mss/hallc/spring17/raw/<YourRawFile>.dat
160
</pre>
161
** This will take a little while to process. You can check the status of your process by typing:
162
<pre>
163
> jcache pendingRequest <JlabUserName>
164
</pre>
165
*->More information on using jcache can be found at* https://scicomp.jlab.org/docs/%20
166
167 2 Richard Trotta
h2. Replaying
168
169
* Before we can analyze we must replay. This should be done in the farm to save yourself time and local cpu effort.  The easiest way is to do a batch job submission, but this comes with some prep work.
170
171
* Before a batch submission, I highly encourage two preliminary steps
172
## Do all debugging of replays locally, once this works move to the farm
173 3 Richard Trotta
## Once on the farm you have two options; your ifarm version or our group (discussed soon).  This is for final debugging purposes to assure everything works in the farm, then you can submit a batch job. Save the root files in /volatile/hallc/c-kaonlt/<USER> (note: volatile is NOT backed up)
174
175
* There is a batch script I have created and Stephen as changed with the help of Brad to assure it will not mess things up.  Again, I highly recommend the two above steps before moving onto this script or you will be wasting time and resources.
176
177
* Your final batch job submissions can be saved directly to tape.
178
179
h2. Group environment
180
181
* You can do replays under your farm directory or you can use our group environment.
182
183
* We have set up a group environment with a version of our repo that currently mimics the cdaq as close as possible (although an updated hcana is used).
184 16 Richard Trotta
** This group environment is under /u/group/c-kaonlt
185
** I have made a directory USERS which you can use for person replay scripts and environments. DO NOT change any replays that are not under USERS without contacting Stephen or me first.
186
** There is an hcana already set up here, use this for any group replays. If you would like to use a different version of hcana please use your farm directory. If I find a hcana in USERS I will destroy it.
187
188
* You may have issues with hcana, make sure you are in the JLab software environment version 2.1
189
** source /site/12gev_phys/softenv.csh 2.1 (or .sh if using bash)
190
191
* The group environment has a 100 gb quota and is backed up. This means two important things…
192
## DO NOT save root files here! Ever!
193
## It’s backed up so its good for important calibration work (*wink *wink)
194
195
* Upon the request of Stephen, any improper use of this environment will incur a penalty of one beer/bottle of single malt or an owl shift (depending upon severity).
196
197
h2. Writing to tape
198
199
* Writing to tape info, read - https://scicomp.jlab.org/docs/write-through-cache.
200
201
* In your batch script, specify OUTPUT_FILE:/cache/hallc/kaonlt/USER/ROOTfiles/
202
** Material in /cache is automatically copied to tape after some time if it is static
203
** Small files (~1 MB) will not be backed up on tape
204
** Once copied to tape, you can view the tape stub (NOT the file itself) under /mss/hallc/kaonlt/…
205
** The tape does not handle overwriting well so if submit a job you must create a new "pass" directory…
206
*** -->jput ... file.root /mss/hallc/kaonlt/USER/ROOTfiles/pass1/
207
** The tape has FAR more space than we could get through so do not worry about "filling" it
208
** Write to tape once you're happy with your code... just do it correctly
209
210
h2. Few more words of warning
211
212
* Do not write analysis to tape unless you are 100% certain it works correctly (and you don't want to repeat it very soon).
213
214
* For farm jobs some info is included below -
215
** See https://scicomp.jlab.org/docs/text_command_file for info on commands	
216
** Do not set CPU above 1 (it will slow your job down in the queue and hcana is single threaded anyway so you gain nothing)
217
** Farm/Auger project: c-kaonlt
218
** For TEMPORARY output, write to volatile - /volatile/hallc/c-kaonlt/USER, this space is NOT backed up!
219
** Specify the FULL path to this in your symbolic link
220
** Make sure relvant directories are created
221
222
* You can use our work environment (/work/hallc/kaon), but this is not backed up and I will no be setting up an environment similar to group there. It’s a good place to put personal scripts if you don’t want to take up space in your farm directory.