Data Engineering Workshop 1

One-day workshop on Docker, web scraping, regular expressions, PostgreSQL, and Git.

Prerequisites

Operating system and tools

Use Ubuntu 20.04 LTS (or a similar Linux desktop) with:

• Python 3.9 or above
• Docker
• Docker Compose
• pip3
• Git (any recent version)

GitHub account

1. Create an account on GitHub if you do not already have one.
2. Fork the DataEngineering-Workshop1 repository. See how to fork a repository.
3. Clone your fork using SSH:
   • Set up an SSH key using the GitHub SSH documentation if needed.
   • Open your forked repository in the browser.
   • Click Code, choose SSH, and copy the URL.
   • Clone the repository (replace YOUR-GIT-ID with your GitHub username):

git clone git@github.com:<YOUR-GIT-ID>/DataEngineering-Workshop1.git

Install Docker

From your cloned repository, run:

sudo prerequisites/install_docker.sh

Confirm your environment

git --version
docker --version
docker-compose --version

Example versions (yours may be newer):

• git version 2.25.1
• Docker version 20.10.17
• docker-compose version 1.25.0

Learning outcomes

By the end of this workshop, you will be able to:

• Build and use Docker images
• Scrape a website using requests / urllib and BeautifulSoup
• Use regular expressions in scraping workflows
• Use key PostgreSQL features
• Dockerize a small project

Schedule

Time	Topics
09:00–11:00	Introduction to Docker
11:00–01:00	Introduction to Web Scraping
01:00–02:00	Break
02:00–03:00	Dockerizing a project
03:00–04:00	Introduction to PostgreSQL
04:00–04:30	Introduction to GitHub
04:30–04:45	Q & A
04:45–05:00	Wrapping Up