Actions
ML beam test PID Meetings » History » Revision 44
« Previous |
Revision 44/71
(diff)
| Next »
Richard Trotta, 05/31/2024 11:14 AM
- Table of contents
- ML beam test PID Meetings
ML beam test PID Meetings¶
Summer 2024¶
May 28th, 2024¶
- Darren ran with Python3.9
- Docker (containerization) Definition
- Containerization is a technology that allows developers to package and run applications along with all their dependencies in isolated environments called containers. This ensures that the application runs consistently across different computing environments, from a developer's laptop to testing, staging, and production.
- Docker is a popular platform that simplifies containerization. It provides tools to create, deploy, and manage containers. With Docker, developers can write code locally, share their work with colleagues, and deploy to production in a seamless and efficient manner. Docker containers are lightweight, fast, and portable, making them ideal for modern software development and deployment.
- A Crash Course for Summer Research
- GitHub and Python Introduction
- Navigate to python_tutorials and read through the two html files (Introduction and python_tutorial_2)
- Note, these are html files so you'll need
- Navigate to python_tutorials and read through the two html files (Introduction and python_tutorial_2)
Near-term Goals¶
- Setup Linux Subsystem for Windows
- Fork the ML Beam Testing GitHub repository
- Containerize ML Beam Testing GitHub repository with Docker
Homework¶
May 29th, 2024¶
Setting Up Jupyter Notebooks in Docker¶
- Create a directory to store, build, and run Docker container
cd /path/to/directory mkdir beamtest_dir cd beamtest_dir
- Once in the new directory, create a Dockerfile
touch Dockerfile
- Using the text editor (e.g., vim, gedit, or emacs) of your choice, edit the Dockerfile with the following information
# Use the official Python 3.9 image as the base image FROM python:3.9 # Install Jupyter Notebook and other dependencies RUN pip install --no-cache-dir jupyter # Create a working directory WORKDIR /workspace # Expose the Jupyter Notebook port EXPOSE 8888 # Set the default command to start Jupyter Notebook CMD ["jupyter", "notebook", "--ip=0.0.0.0", "--port=8888", "--no-browser", "--allow-root"]
- Build the Docker image (e.g., helloworld)
docker build -t helloworld .
- Run the Docker container
docker run -p 8888:8888 helloworld
- Jupyter Notebook is now running. Navigate to a browser and type in either
- The custom token authorization screen
http://localhost:8888
- To bypass the token screen, copy/paste the URL that splashes in the terminal into your browser
Near-term Goals¶
- Try and get the python tutorial Jupyter notebooks working in a container.
Homework¶
May 30th, 2024¶
Setting Up Bash Script for Running Docker¶
- Create a bash script
touch run_docker.sh
- Using the text editor (e.g., vim, gedit, or emacs) of your choice, edit run_docker.sh
#!/bin/bash # Build docker image ($1 is the first bash script argument) docker build -t $1 . # Run the docker container (renaming it with the '_container' string) docker run -p 8888:8888 --name "${1}_container" $1
- Adjust permissions with chmod to allow script execution
chmod 755 run_docker.sh
- Execute run_docker.sh with container name (e.g., helloworld)
./run_docker.sh helloworld
The docker image should be properly built and the container should be running. Follow instructions from yesterday to continue.
Setting Up Bash Script for committing Docker container¶
This should be done while a docker container is running and performed in another terminal.
- Create a bash script
touch commit_docker.sh
- Using the text editor (e.g., vim, gedit, or emacs) of your choice, edit commit_docker.sh
#!/bin/bash # Create a formatted string with date to organize committed containers # - `date` is a command that prints or sets the system date and time. # - `+%H` extracts the hour in 24-hour format. # - `+%M` extracts the minute. # - `+%S` extracts the second. # - `+%Y` extracts the year. # - `+%m` extracts the month. # - `%d` extracts the day. f_date=$(date +%Y-%m-%d_h%Hm%Ms%S) # While a docker container is running (with root name given as first argument), # commit to a dated container so that changes in the container are saved docker commit "${1}_container" "${1}_${f_date}"
- Adjust permissions with chmod to allow script execution
chmod 755 commit_docker.sh
- Execute run_docker.sh with container name (e.g., helloworld)
./commit_docker.sh helloworld
Useful Docker Commands¶
- To see all docker images
docker images
- To see all docker containers running
docker ps
- To end specific docker container
docker stop <container_id>
- To end all docker container
docker stop $(docker ps -q)
Near-term Goals¶
- Try and get the python tutorial Jupyter notebooks working in a container.
- Take a look at the GitHub ML Beam Test Repository
- In particular, look at the script solid_beamtest_hallc_2022/aiml/Pencil_Beam/Pencil_Cher_ML.ipynb
- Try to get this notebook working in a container by creating a requirements.txt file (use this site as a general guide)
- We will cover this in detail tomorrow
Homework¶
May 31st, 2024¶
Clone a git repository into the Docker container by adding a line to the Dockerfile¶
- Using the text editor (e.g., vim, gedit, or emacs) of your choice, edit the Dockerfile with the following information (the python tutorial GitHub repo is used as an example)
# Use the official Python 3.9 image as the base image FROM python:3.9 # Install Jupyter Notebook and other dependencies RUN pip install --no-cache-dir jupyter # Create a working directory WORKDIR /workspace # Clone GitHub repo RUN git clone https://github.com/trottar/UVA_summer_students.git # Expose the Jupyter Notebook port EXPOSE 8888 # Set the default command to start Jupyter Notebook CMD ["jupyter", "notebook", "--ip=0.0.0.0", "--port=8888", "--no-browser", "--allow-root"]
Quick requirements.txt introduction¶
- requirements.txt is a way to easily install many python packages by recursively reading through a text file
- This text file can either be generated...
- Manually, by writing the required python packages into the text file.
- Note: it is good practice to specify the version of the package. Such as...
matplotlib==3.3.4
- Note: it is good practice to specify the version of the package. Such as...
- Automatically, by running the following command
pip freeze > requirements.txt
- Note: This command will include all packages installed.
- Manually, by writing the required python packages into the text file.
- A Docker container can install all these packages upon building its image by adding a few lines to the Dockerfile
- Using the text editor (e.g., vim, gedit, or emacs) of your choice, edit the Dockerfile with the following information
# Use the official Python 3.9 image as the base image FROM python:3.9 # Install Jupyter Notebook and other dependencies RUN pip install --no-cache-dir jupyter # Create a working directory WORKDIR /workspace # Clone GitHub repo RUN git clone https://github.com/trottar/UVA_summer_students.git # copy the dependencies file to the working directory COPY requirements.txt . # install dependencies RUN pip install -r requirements.txt # Expose the Jupyter Notebook port EXPOSE 8888 # Set the default command to start Jupyter Notebook CMD ["jupyter", "notebook", "--ip=0.0.0.0", "--port=8888", "--no-browser", "--allow-root"]
Useful Docker Commands¶
- Remove all stopped containers (use with caution)
docker system prune -f
Near-term Goals¶
- Take a look at the GitHub ML Beam Test Repository
- In particular, look at the script solid_beamtest_hallc_2022/aiml/Pencil_Beam/Pencil_Cher_ML.ipynb
- Try to get this notebook working in a container by creating a requirements.txt file (use this site as a general guide)
Homework¶
Updated by Richard Trotta 3 months ago · 44 revisions