Project

General

Profile

ML beam test PID Meetings » History » Version 31

Richard Trotta, 05/30/2024 12:13 PM

1 1 Richard Trotta
{{>toc}}
2
3
h1. ML beam test PID Meetings
4
5 3 Richard Trotta
---
6
7 1 Richard Trotta
h2. Summer 2024
8 4 Richard Trotta
9
---
10
11 25 Richard Trotta
h3. May 28th, 2024
12 1 Richard Trotta
13
* Darren ran with Python3.9
14
* Docker (containerization) Definition
15
** Containerization is a technology that allows developers to package and run applications along with all their dependencies in isolated environments called containers. This ensures that the application runs consistently across different computing environments, from a developer's laptop to testing, staging, and production.
16
** Docker is a popular platform that simplifies containerization. It provides tools to create, deploy, and manage containers. With Docker, developers can write code locally, share their work with colleagues, and deploy to production in a seamless and efficient manner. Docker containers are lightweight, fast, and portable, making them ideal for modern software development and deployment.
17 9 Richard Trotta
18 17 Richard Trotta
* "A Crash Course for Summer Research":https://drive.google.com/file/d/1-TPLo5VGSwCUDjqj_j4g--p4FbBgYHbw/view?usp=sharing\
19
* "GitHub and Python Introduction":https://github.com/trottar/UVA_summer_students
20 18 Richard Trotta
** Navigate to "python_tutorials":https://github.com/trottar/UVA_summer_students/tree/master/python_tutorials and read through the two html files ("Introduction":https://github.com/trottar/UVA_summer_students/blob/master/python_tutorials/Introduction.ipynb and "python_tutorial_2":https://github.com/trottar/UVA_summer_students/blob/master/python_tutorials/python_tutorial_2.ipynb)
21
*** Note, these are html files so you'll need 
22 8 Richard Trotta
23
h4. Near-term Goals
24
25
# Setup Linux Subsystem for Windows
26
# Fork the ML Beam Testing GitHub repository
27
# Containerize ML Beam Testing GitHub repository with Docker
28 5 Richard Trotta
29 14 Richard Trotta
h4. Homework
30
31 16 Richard Trotta
* "Read chapters 0-3 and finish exercises":https://drive.google.com/file/d/1-TPLo5VGSwCUDjqj_j4g--p4FbBgYHbw/view?usp=sharing
32 14 Richard Trotta
33 3 Richard Trotta
---
34 19 Richard Trotta
35 25 Richard Trotta
h3. May 29th, 2024
36 19 Richard Trotta
37
h4. Setting Up Jupyter Notebooks in Docker
38
39
# Create a directory to store, build, and run Docker container
40
	<pre><code class="bash">
41
cd /path/to/directory
42
mkdir beamtest_dir
43 20 Richard Trotta
cd beamtest_dir
44
</code></pre>
45 21 Richard Trotta
# Once in the new directory, create a Dockerfile
46 20 Richard Trotta
	<pre><code class="bash">
47
touch Dockerfile
48
</code></pre>
49 21 Richard Trotta
# Using the text editor (e.g., vim, gedit, or emacs) of your choice, edit the Dockerfile with the following information
50 20 Richard Trotta
	<pre><code class="bash">
51
# Use the official Python 3.9 image as the base image
52
FROM python:3.9
53
54
# Install Jupyter Notebook and other dependencies
55
RUN pip install --no-cache-dir jupyter
56
57
# Create a working directory
58
WORKDIR /workspace
59
60
# Expose the Jupyter Notebook port
61
EXPOSE 8888
62
63
# Set the default command to start Jupyter Notebook
64 1 Richard Trotta
CMD ["jupyter", "notebook", "--ip=0.0.0.0", "--port=8888", "--no-browser", "--allow-root"]
65
</code></pre>
66 21 Richard Trotta
# Build the Docker image (e.g., helloworld)
67
	<pre><code class="bash">
68
docker build -t helloworld .
69
</code></pre>
70
# Run the Docker container
71
</code></pre>
72
	<pre><code class="bash">
73
docker run -p 8888:8888 helloworld
74
</code></pre>
75
# Jupyter Notebook is now running. Navigate to a browser and type in either
76 23 Richard Trotta
77 21 Richard Trotta
* The custom token authorization screen</code></pre>
78
	<pre><code class="bash">
79
http://localhost:8888
80
</code></pre>
81 22 Richard Trotta
82 21 Richard Trotta
* To bypass the token screen, copy/paste the URL that splashes in the terminal into your browser
83 24 Richard Trotta
84
h4. Near-term Goals
85
86
# Try and get the "python tutorial":https://github.com/trottar/UVA_summer_students Jupyter notebooks working in a container.
87
88
h4. Homework
89
90
* "Read chapters 0-3 and finish exercises":https://drive.google.com/file/d/1-TPLo5VGSwCUDjqj_j4g--p4FbBgYHbw/view?usp=sharing
91 27 Richard Trotta
92
---
93
94
h3. May 30th, 2024
95
96
h4. Setting Up Bash Script for Running Docker
97
98 28 Richard Trotta
# Create a bash script
99
	<pre><code class="bash">
100
touch run_docker.sh
101
</code></pre>
102 29 Richard Trotta
# Using the text editor (e.g., vim, gedit, or emacs) of your choice, edit run_docker.sh
103
	<pre><code class="bash">
104
#!/bin/bash
105 1 Richard Trotta
106 29 Richard Trotta
# Build docker image ($1 is the first bash script argument)
107
docker build -t $1
108
109
# Run the docker container (renaming it with the '_container' string)
110
docker run -p 8888:8888 --name "${1}_container" $1
111
</code></pre>
112
# Adjust permissions with "chmod":https://www.theserverside.com/blog/Coffee-Talk-Java-News-Stories-and-Opinions/how-permissions-chmod-with-numbers-command-explained-777-rwx-unix to allow script execution
113
	<pre><code class="bash">
114
chmod 755 run_docker.sh
115
</code></pre>
116 31 Richard Trotta
# Execute run_docker.sh with container name (e.g., helloworld)
117
	<pre><code class="bash">
118
./run_docker.sh helloworld
119
</code></pre>
120 1 Richard Trotta
121 31 Richard Trotta
The docker image should be properly built and the container should be running. Follow instructions from "yesterday":https://redmine.jlab.org/projects/uva-phys-zheng/wiki/ML_beam_test_PID_Meetings#May-29th-2024 to continue.
122
123
h4. Setting Up Bash Script for committing Docker container
124
125
# Create a bash script
126
	<pre><code class="bash">
127
touch commit_docker.sh
128
</code></pre>
129
# Using the text editor (e.g., vim, gedit, or emacs) of your choice, edit commit_docker.sh
130
	<pre><code class="bash">
131
#!/bin/bash
132
133
# Create a formatted string with date to organize committed containers
134
# - `date` is a command that prints or sets the system date and time.
135
# - `+%H` extracts the hour in 24-hour format.
136
# - `+%M` extracts the minute.
137
# - `+%S` extracts the second.
138
# - `+%Y` extracts the year.
139
# - `+%m` extracts the month.
140
# - `%d` extracts the day.
141
f_date=$(date +%Y-%m-%d_h%Hm%Ms%S)
142
143
# While a docker container is running (with root name given as first argument), 
144
# commit to a dated container so that changes in the container are saved
145
docker commit "${1}_container "${1}_${f_date}"
146
</code></pre>
147 28 Richard Trotta
148
h4. Useful Docker Commands
149 27 Richard Trotta
150
* To see all docker images
151
	<pre><code class="bash">
152
docker images
153
</code></pre>
154
* To see all docker containers running
155
	<pre><code class="bash">
156
docker ps
157
</code></pre>
158
* To end specific docker container
159
	<pre><code class="bash">
160
docker end <container_id>
161
</code></pre>
162
* To end all docker container
163
	<pre><code class="bash">
164
docker end $(docker ps -q)
165
</code></pre>