Project

General

Profile

Daily Reports » History » Version 35

Jixie Zhang, 01/21/2021 05:06 AM

1 1 Xiaochao Zheng
h1. Daily Reports
2
3 24 Xiaochao Zheng
*run plan document on [[https://docs.google.com/document/d/1Bmag_YPrl1szxLhYyA4eLUP2PV_pM0x51pHbeCj2hL4/edit?usp=sharing]]*
4 21 Xiaochao Zheng
5 1 Xiaochao Zheng
Current status:
6 32 Jixie Zhang
Beam energy scan:      done (804-822)
7
1GeV   24 hours:       done (823-847)
8
6GeV   16 hours:       done (848-857)
9
12GeV  24 hours:       done (867-902)
10
16GeV   8 hours:       done (924-944)
11 34 Jixie Zhang
2GeV   24 hours:       done (945-969)
12
4GeV   24 hours:       expect to be done by Jan 22, 23:30 
13 1 Xiaochao Zheng
8GeV   20 hours:       not yet
14
10GeV the rest of the beam time
15
16 32 Jixie Zhang
*daily report 1/20/2021 -- by Jixie*
17 1 Xiaochao Zheng
18 35 Jixie Zhang
# taking 4 GeV data  
19
# MWPC2 still have trouble, Evan will look into it in day time during beam is off
20
# Beam is off from 6 AM, 
21
# MWPC decode program is ready to write raw data and reconstructed hit into root tree
22
# Possible impact from the ATLAS group, who are taking day beam time to test their 'telescope detectors'. 
23
## Their setup include 3 silicon detectors in a row then one center detector then another 3 silicon detectors in a row. 
24
## The silicon detectors have a 4x4 cm window. The window frame is 3/4 inches thick aluminum. 
25
## Their center detector is an aluminum box (about 1cm for the wall thickness) has a 2x2 inches window to allow beam pass through. 
26
## The sensor of the center detector is 1x1 mm only, therefore they have to align the sensor to the beam line carefully. They want to stay in the beam line for the whole week in order to avoid position alignment (cost ~2 hours each time). If they push their setup away from the beam line, they need to redo the position alignment. 
27
## We are taking 4, 8 and 10 GeV data this week. The beam spot size is as large as 12 x 12 cm for 4 GeV. We have to ask ATLAS group to push their setup away from beam line during 4 GeV data taking. From 8 and 10 GeV, we might let them keep only the center detector in the beam line, removing all other 6 silicon detectors.
28
29
30
31
*daily report 1/20/2021 -- by Jixie*
32
33 32 Jixie Zhang
# Our beam time changed to night time from today: from 8PM to 8AM next day.
34 34 Jixie Zhang
# MWPC2 will be swapped in day time
35
# Beam was off from 7 AM, expect to be back by 6 PM
36
# Ecal decode program was modified to remove unused branches
37
# Finished taking 2 GeV data by 23:35
38
# Started taking 4 GeV data, plan to take 24 hours
39 32 Jixie Zhang
40
41
*daily report 1/19/2021 -- by Jixie*
42
43
# Production data of 16 GeV started from run 930, which is launched at 00:30 (AM).
44
# Finished taking 16 GeV data. 
45
# Started taking 2 GeV data from 08:45 AM, Plan to take 24 hours at this energy
46
# 2GeV beam spot is much worse than 16 GeV. MCR can do very little at this low energy  
47
# Requested MCR to push number of particle at MT6SC1, it can reach 40K.  Trigger counts per spill is about 1300
48
# Got confirmed from Evan that MWPC2 will be swapped in Wednesday day 
49 25 Jixie Zhang
50 26 Jixie Zhang
*daily report 1/18/2021 -- by Jixie*
51 1 Xiaochao Zheng
52 26 Jixie Zhang
# Finished taking 12 GeV data including calibration data(903,904,905). Run range is 867-905
53 28 Jixie Zhang
# Started taking 16 GeV data. HVs for this energy were determined. We plan to take 8 hours of data, plus 1 hour calibration. 
54 26 Jixie Zhang
# Calibration data for 16 GeV are ready(run 924-929). 
55
# Requested MCR to control the beam spot size to 6 cm diameter. After that we determine the new coordinate of the center of the 3 Ecal modules.
56 1 Xiaochao Zheng
57 26 Jixie Zhang
*daily report 1/17/2021 -- by Jixie*
58 1 Xiaochao Zheng
59 26 Jixie Zhang
# Automation on logentry works now. More and more analysis scripts are ready to use
60
# Finished taking 6 GeV data, run number range is 848 - 866
61
# All beam energy of <=8 GeV will use 8GeV HV setting. Finihsed taking calibration data (run 853,854,855) for this setting.  
62
# 16 GeV beam is available, and its trigger rate can reach 15K per spill, which is almost 10 times at all lower energies. We could not take this beam energy until a 3rd HV module is set up. 
63
# Started taking 12 GeV data from ~2PM. The plan is to take 24 hours of data, plus 2 hours of calibration data. 
64
# Motion table position is optimized to let the beam hit the center of 3 blocks.
65
# scripts for backing up data and transfer data to JLab volitile is ready. Data will be backed up several time a day.
66
67 1 Xiaochao Zheng
*daily report 1/16/2021 -- by Jixie*
68
69 26 Jixie Zhang
# Beam energy scan was finished on Jan 15, HV for each beam energy are determined 
70
# Finished 24 hours of 1 GeV data taking using 8GeV HV setting. We forgot to change it back. Since detector is not saturated, do not bother to change it. Run number range is 823 - 847.
71
# Start 6 GeV data taking by run number 848. Plan to take 18 hours of data.
72
# Working on making automatic start|end of run log entry.  
73
# MWPC decoder and analysis program works. beam spot size is huge (9cm x 8 cm). 
74
75 21 Xiaochao Zheng
*daily report 1/15/2021 -- by Xiaochao from far away*
76
77
# good gracious, we keep getting 24 hours a day
78 22 Xiaochao Zheng
# we should keep our dialy work hours below 16 and stretch shifts to 8 hours  
79
# rate scan is done up to 12 GeV energy. And we are now taking 1 GeV data
80
# the trigger is from FTBF's 3 scintillator paddles coincidence with the spill signal. FTBF's 2-chamber Cherenkov signals are not in the trigger but are in our FADCs.
81
# MWPC has problems: 1 was fixed today (hardware) and both are working and online, daq is working and synced with our daq. But the software is missing a library so we can't decode the data. 
82 1 Xiaochao Zheng
# According to David's post on Slack:
83 23 Xiaochao Zheng
84
> David Flay 2:20 PM
85
> ok, using the rates above, I find for our remaining 11 days:
86
> 2:20
87
> E = 12.0 GeV, numShifts = 1.3, totEvt = 16356343
88
> E = 10.0 GeV, numShifts = 0.8, totEvt = 16356343
89
> E = 8.0 GeV, numShifts = 0.7, totEvt = 16356343
90
> E = 6.0 GeV, numShifts = 0.8, totEvt = 16356343
91
> E = 1.0 GeV, numShifts = 14.7, totEvt = 16356343
92
> E = 2.0 GeV, numShifts = 2.5, totEvt = 16356343
93
> E = 4.0 GeV, numShifts = 1.2, totEvt = 16356343
94
> 2:21
95
> clearly that 14.7 shifts doesn’t make sense.
96
> David Flay 2:27 PM
97
> I think it checks out;
98
> 2:27
99
> Total time: 264.0 hrs
100
> E = 12.0 GeV, weight = 0.061, time = 16.1 hrs, numShifts = 1.3, totEvt = 16356343
101
> E = 10.0 GeV, weight = 0.038, time = 10.0 hrs, numShifts = 0.8, totEvt = 16356343
102
> E = 8.0 GeV, weight = 0.032, time = 8.5 hrs, numShifts = 0.7, totEvt = 16356343
103
> E = 6.0 GeV, weight = 0.038, time = 9.9 hrs, numShifts = 0.8, totEvt = 16356343
104
> E = 1.0 GeV, weight = 0.666, time = 175.9 hrs, numShifts = 14.7, totEvt = 16356343
105
> E = 2.0 GeV, weight = 0.112, time = 29.5 hrs, numShifts = 2.5, totEvt = 16356343
106
> E = 4.0 GeV, weight = 0.053, time = 14.0 hrs, numShifts = 1.2, totEvt = 16356343
107
> Totals:
108
> Time = 264.0 hrs
109 1 Xiaochao Zheng
> Shifts = 22.0
110 21 Xiaochao Zheng
> 
111 20 Xiaochao Zheng
112
*daily report 1/14/2021 -- by Xiaochao from far away*
113
# beam started sometime in the evening. 
114
# we got the night time beam as a bonus - the other group does not need it - causing us to scramble the shift schedule. Paul Reimer came in for the night shift, until 3am - thanks!
115
# by midnight, we are sending beam to center of each block, trying to adjust HV/gain-match;
116 26 Jixie Zhang
# Jixie adjusted Cherenkov pressure around 10pm. Previously the two chambers were set at 4.0 and 0.2 psia, respectively. I think Jixie lowered one of them from 4.0 to 0.8
117 20 Xiaochao Zheng
118
_To do:_
119
# complete gain/HV-matching of the 3 shashlyk modules
120
# center beam at the middle of the 3-block (a little off-center may be ideal, better than at the exact center) and do energy scan over night to figure out rates
121
# David will rework the runplan during the day
122
# One of the MWPC is broken, will be installed during the day
123 1 Xiaochao Zheng
124
*Jixie  9:04 PM
125
daily report 1/13/2021*
126
A busy and tired day! At least our DAQ is ready to take beam with our own trigger!
127
# Start installation from 7AM. Finished installing detectors and cabling by 1PM
128
# Put the rack in the control room so no access needed for daq work. Our daq machines are also in the control room.
129
# David came to site. Help a lot in setting up computers. Xinzhan and I are working on detectors and daq cabling. Both desktops are online. CODA runs well.
130
# Operational Readiness Clearance (ORC) meeting started from 1 PM. Detectors are all good. However, JLab made VME cards were suggested to change fuse at the beginning...... After some long negotiation, we were requested to put kapton tape to cover the power interface. We also received some suggestions to run the experiment safely. We might get all signature by tomorrow
131
# We connected HV and signal cables to the patch panel. All PMT signals have been checked using an oscilloscope. All detectors work well. We do not have enough time to modify daq to take the FTBF trigger. FTBF has 3 channels for Cherenkov, none of them add into our FADC yet. (Need to find cable and do fan in fan out...) MWPC daq does not have a veto input, it can not take our busy status immediately. We have to form a new trigger then feed it. We will do this in the morning of Thursday.
132
# Accelerator is down. One magnet is not functioning. They are waiting for the radiation to cool down so they can fix it tomorrow morning. Beam might be available in the afternoon.
133
134
_To do:_
135
# Do FTBF controlled access leader training in Thursday morning 9AM
136 33 Jixie Zhang
# MWPC daq is a well integrated machine (looks like a laptop). It does not provide L1A signal. We need to learn deep about this system to find out a possible way to take their L1A. We will then veto it using out busy status. Need to spend time here Thursday.
137 1 Xiaochao Zheng
# connect cherenkov into our FADC.
138
139 2 Alexandre Camsonne
Pictures under useful pictures with prefix 20200113_xx
140 8 Alexandre Camsonne
141 12 Xiaochao Zheng
142 1 Xiaochao Zheng
!{width:20%}https://redmine.jlab.org/attachments/download/1159/IMG_20210113_085123.jpg! !{width:20%}https://redmine.jlab.org/attachments/download/1160/IMG_20210113_160026.jpg! !{width:20%}https://redmine.jlab.org/attachments/download/1161/IMG_20210113_141747.jpg! !{width:20%}https://redmine.jlab.org/attachments/download/1162/IMG_20210113_141726.jpg! 
143 13 Xiaochao Zheng
144 12 Xiaochao Zheng
!{width:20%}https://redmine.jlab.org/attachments/download/1163/IMG_20210113_121019.jpg! !{width:20%}https://redmine.jlab.org/attachments/download/1164/IMG_20210113_120704.jpg! !{width:20%}https://redmine.jlab.org/attachments/download/1165/IMG_20210113_120825.jpg! 
145 13 Xiaochao Zheng
146 9 Xiaochao Zheng
!{width:20%}https://redmine.jlab.org/attachments/download/1166/IMG_20210113_120735.jpg! !{width:20%}https://redmine.jlab.org/attachments/download/1167/IMG_20210113_115807.jpg! !{width:20%}https://redmine.jlab.org/attachments/download/1168/IMG_20210113_090032.jpg! 
147
148
*Jixie  8:02 PM daily update 1/12/2021*
149
# Took radiation practical training, both of Jixie and Xinzhan
150
# Met Lorenzo, have a long and deep conversation with him. He helped us setting up the gateway machine, which is soliddaq. Right now solidaq has 2 network interfaces, one of them is borrowed from FTBF.  soliddaq can access the public network, it also serves as the gateway of the private network. Because IP forward function is not enabled, right now private network members can not access public networks yet. (I need help from an expert to enable this.)
151
# If you want to access to clrlpc, you have to access to solidaq using fermilab linux account, then ssh from soliddaq into clrlpc using a-compton account. If you want to do that, send me your fermilab username. I can add your username into the name lists that allows you to login soliddaq using a-compton account.
152
# David brought the hard drive and pcie to usb3.1 convert card here. Two 5T hard drive (2.5") were installed into the usb hub provided by Alex. Tested the data transfer speed. It can reach 130MB/s when the CPU is idle. Data transfer speed changes a lot when coda is running. 
153
# JLab experts are working on DSC issues. The Fan-In-Fan-out module degrades the input signal too much, which is part of the reason that DSC lost signal.  We borrowed one from FTBF.  JLab experts are also checking and fixing the software.  Right now we can see TDC in the output file. Alex is still working on it to make sure it does not lose data.
154
# David is working on ORC form.
155
       
156
_Here is how we transfer data back to jlab:_
157
# make a copy of data into soliddaq, this copy will also be our backup of all raw data
158 10 Xiaochao Zheng
# sent data from soliddaq to jlab
159 9 Xiaochao Zheng
160
_To Do:_
161
# The beam will be off by 6AM Wednesday. We will start installation from 7 AM. We aim to finish cabling and installation by 1 PM.
162
# ROC inspection is scheduled at 1 PM Wednesday. I will show in person. David will join remotely.
163 14 Xiaochao Zheng
# The beam is scheduled to be delivered by 5:30 PM.  Look forward to taking beam! (edited) 
164
165
166
167
*Jixie  9:03 PM daily report, 1/11/2021*
168
# Put all our computers into a private network switch, use ip 192.168.1.x, reserve IP 192.168.1.1 for the gateway. All computers can talk to each other. Still waiting for FTBF to provide a gateway. Talked to Evan and Lorenzo. They understanded our request and will set up one tomorrow.
169
# Rebooted vme crate.. The hostname 'intelvmeha8' came back. Start and stop coda smoothly, no need to manually start ROC2.
170
# DSC did not write data to the output file. Experts at Jlab are working on this.
171
# Talked to Evan about MWPC daq. Learned that we can easily take their trigger, which is SC1&SC2 (both stay at FTBF beam line). We decided to take the L1A signal from MWPC daq to be our trigger. In this way the events from our daq are already synchronized with MWPC daq. We will also put our SCs into the FADC, they just do not serve as triggers anymore. We will also plug Cherenkov signal into our FADC. We will also put a copy of MWPC daq's L1A into our TDC.
172
# David is preparing the ORC document
173
# Had a meeting with Lorenzo, we agreed to start shift at 8 AM|PM.
174
# install kerberos kit in clrlpc. We will not use it during data taking but can use it if needed. Only people with a Fermilab linux account can access it.
175
# taped the lead block.
176
177
_ To Do:_
178
# get a gateway
179
# solve DSC
180
# look at raw data file of MWPC daq, try to understand their daq
181
182 15 Xiaochao Zheng
183 14 Xiaochao Zheng
*Jixie  10:15 PM daily report 1/10/2021*
184
# pause daq data taking to test 3 preshowers.  They are all good
185
# continued testing TDC  configuration. Looked at TDC signal in raw data file.   improving/updating coda and decode program under the guidance of Alex.
186
# taking cosmic data.  right now daq machine clrlpc is offline
187
188
_To Do:_
189
# finish setting up private network
190
# tape the lead block
191
# practice sending data back to jlab
192 16 Xiaochao Zheng
# take training on Tuesday
193 17 Xiaochao Zheng
194
*Jixie  6:59 PM daily report 1/9/2021*
195
# Finished installing kerberos kit into soliddaq desktop (the backup one)
196
# zoom meeting with Alex, we are now able to run coda only if we do not follow Fermilab kerberos ssh requirements. A trick is that we used jlab configuration to start coda and all rocs, when coda runs we switch to fermilab configuration. This is not a solution! We have to put daq into a private network.
197
# Found a 1000Mbps switch and several cat5e cables. Cat5e cable's speed can reach 1000 Mbps, while cat6's is 10Gbps. Will be better if there are enough cat6 cables. Plug 2 laptops into the switch, manually specify IP and gateway, 2 laptops can talk to each other. I tried to set up my laptop (windows10) as the gateway and using WIFI to connect to the fermilab public network. Failed!
198
199
Our solution will be: soliddaq, clrlpc, intelvmeha8 will be in a private network.
200
* Ask Fermilab to provide a gateway. (The backup daq machine must be identical to the main daq machine.
201
* To transfer data to JLab, I will send data using the backup daq to Jlab work disk.
202
* Alex, please help me to find out which disk I can write raw data to, total space no more than 14T.
203
204
_To Do:_
205
# Request FTBF to provide a gateway, finish setting up the private network
206
# CODA roc2 need to be start manually, need to fix that
207
# work out a script to transfer data
208
# waiting for HA approval then tape the lead
209 18 Xiaochao Zheng
# test preshowers when a 2nd HV module is available. (edited) 
210
211
212
*Jixie  10:22 PM daily update 1/8/2021*
213
# both desktops got blocked because password authority is not allowed.   Solved it  after reconfigured sshd_config file.
214
# applied host principles for both desktops.  Got password in the afternoon but have typed a wrong host name for the main daq machine.  Reapplied and now waiting for supervisor's approval. The backup machine (soliddaq) is ready to install kerberos kit but do not have time to do it.
215
# connected daq system.  hooked SCs and Ecals into daq.  viewed trigger signals.  ready to take cosmic data.  Preshowers are not connected yet.
216
# hooked vme directly to clrlpc,  configured them to talk to each other.  2 machines can talk now.
217
# Alex helped thought zoom meeting.  After some modifications,  Alex can run coda to take events.  But I could not run it by myself because some configuration changes that  Alex did might not be saved.
218
# Paul came to help.  We are trying to solve the network issue.   Desktop clrlpc has a puppet service to overwrite sshd_config frequently.  Therefore all my modifications is gone.  We have to solve it ASAP otherwise this machine will be blocked again once the Fermilab robot catches it.
219
Service puppet was temporary disabled. 
220
# Rearranged detectors such that 2 SCs and Ecals are in the same line.
221
#  My laptop got blocked due to use X11 service......
222
223
_To Do:_
224
# Install kerberos kit
225
# configure coda
226
# test preshowers
227 19 Xiaochao Zheng
# tape the lead plate (edited) 
228
229
*Jixie  8:42 PM daily report 1/7/2021*
230
# installed both the nim and the vme crates onto a movable rack
231
# learned how to use FTBF's HV module.  This module can only supply HV in range from the set point subtract 580V  to the set point.  For example,  If a set point is 2500V,  the HV range is 1920-2500.  In our case,  we need to set ecal at 900V,  SC at 1300V, and preshowers at 2100V.  We have to use two HV module.
232
# We look at the signals for all 3 ecal and 2 SC.  We didn't damage them during the bumpy driving.  The signals looks good.
233
# We installed 3 ecal modules into the tray. Carefully addjust their positions to make sure them attach to each other tightly.
234
# registered both desktops, clrlpc and soliddaq, to Fermilab public network.  The latter is a back up for daq machine.   Both of them are now online.   If you have a computer account of Fermilab,  you can ssh to it using account a-compton.
235
236
_To do:_
237
# test preshower detectors
238
# applying a computer account so I can access daq machine from apartment
239
# connect daq system to detectors
240 1 Xiaochao Zheng
# tape the lead plate after HA got approved (edited)