diff --git a/02_activities/assignments/Assignment1.md b/02_activities/assignments/Assignment1.md index bd4ec5981..3117a1639 100644 --- a/02_activities/assignments/Assignment1.md +++ b/02_activities/assignments/Assignment1.md @@ -201,9 +201,66 @@ Link if you encounter a paywall: https://web.archive.org/web/20240422105834/http **What values systems are embedded in databases and data systems you encounter in your day-to-day life?** -Consider, for example, concepts of fariness, inequality, social structures, marginalization, intersection of technology and society, etc. +Consider, for example, concepts of fairness, inequality, social structures, marginalization, intersection of technology and society, etc. ``` Your thoughts... +Introduction + +Databases might seem like neutral tools, but in reality, they carry the values and biases of the societies that create them. The article When Databases Get to Define Family by Qadri R. (2021) explores how Pakistan's digital family registration system enforces a rigid definition of family, excluding marginalized groups and reinforcing social norms. This issue isn’t unique to Pakistan—it’s a global concern that raises questions about fairness, inequality, and how technology shapes our lives. + +How Databases Reflect Societal Values + +Every day, we interact with databases that influence our opportunities and experiences. From government records to financial systems, these databases shape our access to essential services. But who gets included, and who is left out? + +1. Fairness and Inequality + +A well-designed database should aim to be inclusive, but many systems reflect existing social biases. For example: + +Government databases often fail to recognize non-traditional families, like single-parent households or LGBTQ+ couples, making it harder for them to access rights and benefits. + +Credit scoring systems tend to favour wealthier individuals, making it difficult for low-income people to secure loans or mortgages. + +Healthcare databases may not fully represent diverse racial and genetic backgrounds, leading to misdiagnoses and inadequate treatment for marginalized communities. + +In Pakistan, the National Database and Registration Authority (NADRA) requires women to rely on male guardians for family verification, reinforcing patriarchal structures. This isn’t just a Pakistani issue—many legal and financial systems worldwide fail to recognize relationships that don’t fit traditional norms, creating unnecessary barriers. + +2. Social Structures and Marginalization + +Data systems tend to mirror existing power structures, often sidelining marginalized groups: + +Caste and Ethnicity Bias: Some databases categorize people by race or caste in ways that reinforce discrimination, affecting access to jobs and education. + +Gender and Identity: Many systems still use binary gender classifications, making it difficult for non-binary and transgender individuals to have their identities recognized. + +Algorithmic Discrimination: AI-driven hiring or law enforcement tools often inherit biases from historical data, leading to unfair treatment, particularly for people of colour. + +3. The Intersection of Technology and Society + +We often think of technology as neutral, but databases are shaped by human decisions. This raises major ethical concerns: + +Surveillance and Privacy: Governments use digital databases for national security, but this can lead to overreach and violations of personal privacy. + +Data Ownership and Consent: Most people don’t control their own digital data, allowing corporations and governments to exploit their personal information. + +Historical Data Bias: Predictive policing databases, for example, rely on past crime records—but if law enforcement has historically targeted specific racial groups, these databases reinforce discrimination. + +Building More Ethical Data Systems + +To create fairer, more inclusive data systems, we need to: + +-Acknowledge Bias: Database architects and policymakers must actively identify and address biases in data collection and categorization. + +-Design for Inclusivity: Systems should recognize diverse identities and family structures, rather than imposing rigid definitions. + +-Strengthen Data Protection Laws: Individuals should have control over how their data is collected and used. + +-Encourage Public Oversight: Independent organizations should have a role in reviewing and regulating database practices. + +Conclusion + +Databases influence our lives in ways we often don’t realize. They decide who gets access to resources, how we are identified, and even how we move through the world. But they are not neutral—they reflect the priorities and prejudices of those who build and maintain them. The case of Pakistan’s family database serves as a reminder that we need to actively question and improve the systems we rely on. As we become more dependent on digital systems, ethical considerations must guide database design to ensure fairness and inclusivity for all. + + ``` diff --git a/02_activities/assignments/Assignment2.md b/02_activities/assignments/Assignment2.md index a95a027fd..f4861603b 100644 --- a/02_activities/assignments/Assignment2.md +++ b/02_activities/assignments/Assignment2.md @@ -54,7 +54,49 @@ The store wants to keep customer addresses. Propose two architectures for the CU **HINT:** search type 1 vs type 2 slowly changing dimensions. ``` -Your answer... +To store customer addresses, we propose two different architectures: + +Type 1: Overwriting Changes + +A simple structure where the latest address replaces the previous one: + +Table: CUSTOMER_ADDRESS + +customer_id (Primary Key, Foreign Key from Customer) + +address + +city + +state + +zip + +In this model, whenever a customer updates their address, the old data is overwritten. + +Type 2: Retaining History + +A more complex structure that maintains historical address changes: + +Table: CUSTOMER_ADDRESS_HISTORY + +customer_id (Foreign Key from Customer) + +address + +city + +state + +zip + +start_date + +end_date + +With this approach, whenever a customer changes their address, a new record is created with the start_date, and the previous record is updated with an end_date, preserving history. + +Type 1 is best when historical data is not needed, whereas Type 2 is essential when tracking address changes over time. ``` *** @@ -182,5 +224,36 @@ Consider, for example, concepts of labour, bias, LLM proliferation, moderating c ``` -Your thoughts... +Section 4: Ethics in AI and Data Processing + +Ethical Issues in "Neural Nets are Just People All the Way Down" + +The article by Vicki Boykis explores the ethical complexities surrounding AI, specifically Large Language Models (LLMs). Key ethical concerns include: + +Bias in AI Models + +AI systems inherit biases from their training data, which reflects societal prejudices. + +This perpetuates discrimination in automated decision-making. + +Labor and Automation + +LLMs rely on vast amounts of data labeled by underpaid human workers. + +The ethical issue of exploiting global labor for AI development raises concerns. + +Challenges in Moderating AI-Generated Content + +AI-generated content can be harmful or misleading. + +There is no perfect moderation system, as AI models lack human context and ethics. + +AI in Society & Ethical Dilemmas + +The rapid growth of LLMs creates a monopoly where only a few corporations control AI development. + +Ethical concerns arise about transparency, accessibility, and misinformation. + +Conclusion +While AI provides immense benefits, its ethical implications cannot be ignored. To mitigate bias, labor exploitation, and misinformation, there must be continuous oversight, regulation, and a commitment to ethical AI development. AI is ultimately shaped by human values, and ensuring fairness and accountability remains a shared responsibility ``` diff --git a/02_activities/assignments/BOOKSTORE ERD.jpeg b/02_activities/assignments/BOOKSTORE ERD.jpeg new file mode 100644 index 000000000..1c6dce9b0 Binary files /dev/null and b/02_activities/assignments/BOOKSTORE ERD.jpeg differ diff --git a/02_activities/assignments/Logical model.png b/02_activities/assignments/Logical model.png new file mode 100644 index 000000000..657f43b38 Binary files /dev/null and b/02_activities/assignments/Logical model.png differ diff --git a/02_activities/assignments/assignment1.sql b/02_activities/assignments/assignment1.sql index 2e89fa7af..33478554a 100644 --- a/02_activities/assignments/assignment1.sql +++ b/02_activities/assignments/assignment1.sql @@ -1,23 +1,32 @@ -/* ASSIGNMENT 1 */ -/* SECTION 2 */ +--/* ASSIGNMENT 1 */ +--/* SECTION 2 */ ---SELECT -/* 1. Write a query that returns everything in the customer table. */ +--SELECT +--/* 1. Write a query that returns everything in the customer table. */ +--1. Select everything from the customer table +SELECT * FROM customer; -/* 2. Write a query that displays all of the columns and 10 rows from the cus- tomer table, -sorted by customer_last_name, then customer_first_ name. */ +--/* 2. Write a query that displays all of the columns and 10 rows from the cus- tomer table, +--sorted by customer_last_name, then customer_first_ name. */ +--2. Select all columns, 10 rows from customer table, sorted by last name then first name +SELECT * FROM customer +ORDER BY customer_last_name, customer_first_name +LIMIT 10; --WHERE -/* 1. Write a query that returns all customer purchases of product IDs 4 and 9. */ +--/* 1. Write a query that returns all customer purchases of product IDs 4 and 9. */ -- option 1 - +SELECT * FROM customer_purchases +WHERE product_id IN (4, 9); -- option 2 +SELECT * FROM customer_purchases +WHERE product_id = 4 OR product_id = 9; @@ -27,7 +36,10 @@ filtered by vendor IDs between 8 and 10 (inclusive) using either: 2. one condition using BETWEEN */ -- option 1 - +-- 4. Select customer purchases with a calculated 'price' column, filtered by vendor IDs between 8 and 10 +SELECT *, (quantity * cost_to_customer_per_qty) AS price +FROM customer_purchases +WHERE vendor_id BETWEEN 8 AND 10; -- option 2 @@ -39,17 +51,37 @@ Using the product table, write a query that outputs the product_id and product_n columns and add a column called prod_qty_type_condensed that displays the word “unit” if the product_qty_type is “unit,” and otherwise displays the word “bulk.” */ - +-- 5. Select product_id, product_name, and create prod_qty_type_condensed column using CASE +SELECT product_id, product_name, + CASE + WHEN product_qty_type = 'unit' THEN 'unit' + ELSE 'bulk' + END AS prod_qty_type_condensed +FROM product; /* 2. We want to flag all of the different types of pepper products that are sold at the market. add a column to the previous query called pepper_flag that outputs a 1 if the product_name contains the word “pepper” (regardless of capitalization), and otherwise outputs 0. */ +SELECT product_id, product_name, + CASE + WHEN product_qty_type = 'unit' THEN 'unit' + ELSE 'bulk' + END AS prod_qty_type_condensed, + CASE + WHEN LOWER(product_name) LIKE '%pepper%' THEN 1 + ELSE 0 + END AS pepper_flag +FROM product; --JOIN /* 1. Write a query that INNER JOINs the vendor table to the vendor_booth_assignments table on the vendor_id field they both have in common, and sorts the result by vendor_name, then market_date. */ +SELECT vendor.*, vendor_booth_assignments.* +FROM vendor +INNER JOIN vendor_booth_assignments ON vendor.vendor_id = vendor_booth_assignments.vendor_id +ORDER BY vendor.vendor_name, vendor_booth_assignments.market_date; @@ -60,6 +92,9 @@ vendor_id field they both have in common, and sorts the result by vendor_name, t /* 1. Write a query that determines how many times each vendor has rented a booth at the farmer’s market by counting the vendor booth assignments per vendor_id. */ +SELECT vendor_id, COUNT(*) AS booth_rentals +FROM vendor_booth_assignments +GROUP BY vendor_id; /* 2. The Farmer’s Market Customer Appreciation Committee wants to give a bumper @@ -68,6 +103,13 @@ of customers for them to give stickers to, sorted by last name, then first name. HINT: This query requires you to join two tables, use an aggregate function, and use the HAVING keyword. */ +SELECT c.customer_id, c.customer_first_name, c.customer_last_name, + SUM(cp.quantity * cp.cost_to_customer_per_qty) AS total_spent +FROM customer_purchases cp +JOIN customer c ON cp.customer_id = c.customer_id +GROUP BY c.customer_id +HAVING total_spent > 2000 +ORDER BY c.customer_last_name, c.customer_first_name; --Temp Table @@ -81,7 +123,16 @@ When inserting the new vendor, you need to appropriately align the columns to be -> To insert the new row use VALUES, specifying the value you want for each column: VALUES(col1,col2,col3,col4,col5) */ +-- Drop the temp table if it already exists +DROP TABLE IF EXISTS new_vendor; + +-- Create a temporary table from the vendor table +CREATE TEMP TABLE new_vendor AS +SELECT * FROM vendor; +-- Insert a new vendor into the temporary table +INSERT INTO new_vendor (vendor_id, vendor_name, vendor_type, vendor_owner_first_name, vendor_owner_last_name) +VALUES (10, 'Thomass Superfood Store', 'Fresh Focused', 'Thomas', 'Rosenthal'); -- Date @@ -89,6 +140,10 @@ VALUES(col1,col2,col3,col4,col5) HINT: you might need to search for strfrtime modifers sqlite on the web to know what the modifers for month and year are! */ +SELECT customer_id, + STRFTIME('%m', market_date) AS month, + STRFTIME('%Y', market_date) AS year +FROM customer_purchases; @@ -98,3 +153,8 @@ Remember that money spent is quantity*cost_to_customer_per_qty. HINTS: you will need to AGGREGATE, GROUP BY, and filter... but remember, STRFTIME returns a STRING for your WHERE statement!! */ +SELECT customer_id, SUM(quantity * cost_to_customer_per_qty) AS total_spent +FROM customer_purchases +WHERE STRFTIME('%m', market_date) = '04' AND STRFTIME('%Y', market_date) = '2022' +GROUP BY customer_id; + diff --git a/02_activities/assignments/assignment2.sql b/02_activities/assignments/assignment2.sql index 5ad40748a..afda2e904 100644 --- a/02_activities/assignments/assignment2.sql +++ b/02_activities/assignments/assignment2.sql @@ -1,70 +1,87 @@ -/* ASSIGNMENT 2 */ +/* ASSIGNMENT 2 */ --- FESOBI OLUWAMUYIWA /* SECTION 2 */ --- COALESCE -/* 1. Our favourite manager wants a detailed long list of products, but is afraid of tables! -We tell them, no problem! We can produce a list with all of the appropriate details. - -Using the following syntax you create our super cool and not at all needy manager a list: - +-- COALESCE - Handle NULL values SELECT -product_name || ', ' || product_size|| ' (' || product_qty_type || ')' -FROM product - -But wait! The product table has some bad data (a few NULL values). -Find the NULLs and then using COALESCE, replace the NULL with a -blank for the first problem, and 'unit' for the second problem. - -HINT: keep the syntax the same, but edited the correct components with the string. -The `||` values concatenate the columns into strings. -Edit the appropriate columns -- you're making two edits -- and the NULL rows will be fixed. -All the other rows will remain the same.) */ - - - ---Windowed Functions -/* 1. Write a query that selects from the customer_purchases table and numbers each customer’s -visits to the farmer’s market (labeling each market date with a different number). -Each customer’s first visit is labeled 1, second visit is labeled 2, etc. - -You can either display all rows in the customer_purchases table, with the counter changing on -each new market date for each customer, or select only the unique market dates per customer -(without purchase details) and number those visits. -HINT: One of these approaches uses ROW_NUMBER() and one uses DENSE_RANK(). */ - - - -/* 2. Reverse the numbering of the query from a part so each customer’s most recent visit is labeled 1, -then write another query that uses this one as a subquery (or temp table) and filters the results to -only the customer’s most recent visit. */ - - - -/* 3. Using a COUNT() window function, include a value along with each row of the -customer_purchases table that indicates how many different times that customer has purchased that product_id. */ - - - --- String manipulations -/* 1. Some product names in the product table have descriptions like "Jar" or "Organic". -These are separated from the product name with a hyphen. -Create a column using SUBSTR (and a couple of other commands) that captures these, but is otherwise NULL. -Remove any trailing or leading whitespaces. Don't just use a case statement for each product! - -| product_name | description | -|----------------------------|-------------| -| Habanero Peppers - Organic | Organic | - -Hint: you might need to use INSTR(product_name,'-') to find the hyphens. INSTR will help split the column. */ - - - -/* 2. Filter the query to show any product_size value that contain a number with REGEXP. */ + product_name || ', ' || COALESCE(product_size, '') || ' (' || COALESCE(product_qty_type, 'unit') || ')' +FROM product; +--Window function +SELECT + customer_id, + market_date, + ROW_NUMBER() OVER (PARTITION BY customer_id ORDER BY market_date ASC) AS visit_number +FROM customer_purchases; +--Reversing the numbering +SELECT + customer_id, + market_date, + ROW_NUMBER() OVER (PARTITION BY customer_id ORDER BY market_date DESC) AS visit_number +FROM customer_purchases; + +---Filtering only the most recent visit for each customer: + +WITH RankedVisits AS ( + SELECT + customer_id, + market_date, + ROW_NUMBER() OVER (PARTITION BY customer_id ORDER BY market_date DESC) AS visit_number + FROM customer_purchases +) +SELECT customer_id, market_date +FROM RankedVisits +WHERE visit_number = 1; + + +--Count window function +WITH RankedVisits AS ( + SELECT + customer_id, + market_date, + ROW_NUMBER() OVER (PARTITION BY customer_id ORDER BY market_date DESC) AS visit_number + FROM customer_purchases +) +SELECT customer_id, market_date +FROM RankedVisits +WHERE visit_number = 1; + +-- Count Window Function - Number of times a customer has purchased a product +SELECT + customer_id, + product_id, + COUNT(*) OVER (PARTITION BY customer_id, product_id) AS purchase_count +FROM customer_purchases; --- UNION -/* 1. Using a UNION, write a query that displays the market dates with the highest and lowest total sales. +--String Manipulation +SELECT + product_name, + TRIM(SUBSTR(product_name, INSTR(product_name, '-') + 1)) AS description +FROM product +WHERE INSTR(product_name, '-') > 0; + +--UNION - Market dates with highest and lowest total sales + +WITH SalesData AS ( + SELECT + market_date, + SUM(quantity * cost_to_customer_per_qty) AS total_sales + FROM customer_purchases + GROUP BY market_date +), +RankedSales AS ( + SELECT + market_date, + total_sales, + RANK() OVER (ORDER BY total_sales DESC) AS highest_rank, + RANK() OVER (ORDER BY total_sales ASC) AS lowest_rank + FROM SalesData +) +SELECT market_date, total_sales, 'Highest Sales' AS category +FROM RankedSales WHERE highest_rank = 1 +UNION +SELECT market_date, total_sales, 'Lowest Sales' AS category +FROM RankedSales WHERE lowest_rank = 1; HINT: There are a possibly a few ways to do this query, but if you're struggling, try the following: 1) Create a CTE/Temp Table to find sales values grouped dates; @@ -78,56 +95,44 @@ with a UNION binding them. */ /* SECTION 3 */ --- Cross Join -/*1. Suppose every vendor in the `vendor_inventory` table had 5 of each of their products to sell to **every** -customer on record. How much money would each vendor make per product? -Show this by vendor_name and product name, rather than using the IDs. - -HINT: Be sure you select only relevant columns and rows. -Remember, CROSS JOIN will explode your table rows, so CROSS JOIN should likely be a subquery. -Think a bit about the row counts: how many distinct vendors, product names are there (x)? -How many customers are there (y). -Before your final group by you should have the product of those two queries (x*y). */ - - - --- INSERT -/*1. Create a new table "product_units". -This table will contain only products where the `product_qty_type = 'unit'`. -It should use all of the columns from the product table, as well as a new column for the `CURRENT_TIMESTAMP`. -Name the timestamp column `snapshot_timestamp`. */ - - - -/*2. Using `INSERT`, add a new row to the product_units table (with an updated timestamp). -This can be any product you desire (e.g. add another record for Apple Pie). */ - - +--CROSS JOIN - Vendor revenue per product +SELECT + v.vendor_name, + p.product_name, + 5 * vi.original_price AS revenue_per_product +FROM vendor v +CROSS JOIN ( + SELECT DISTINCT product_id, original_price FROM vendor_inventory +) vi +JOIN product p ON vi.product_id = p.product_id; --- DELETE -/* 1. Delete the older record for the whatever product you added. -HINT: If you don't specify a WHERE clause, you are going to have a bad time.*/ +---INSERT - Create a product_units table +--CREATE TABLE product_units AS +--SELECT *, CURRENT_TIMESTAMP AS snapshot_timestamp +--FROM product +--WHERE product_qty_type = 'unit'; +---Insert a new record into product_units: --- UPDATE -/* 1.We want to add the current_quantity to the product_units table. -First, add a new column, current_quantity to the table using the following syntax. +INSERT INTO product_units (product_id, product_name, product_size, product_category_id, product_qty_type, snapshot_timestamp) +VALUES (999, 'Apple Pie', 'Medium', 3, 'unit', CURRENT_TIMESTAMP); -ALTER TABLE product_units -ADD current_quantity INT; +--DELETE - Remove older record +DELETE FROM product_units +WHERE product_id = 999 +AND snapshot_timestamp = (SELECT MIN(snapshot_timestamp) FROM product_units WHERE product_id = 999); -Then, using UPDATE, change the current_quantity equal to the last quantity value from the vendor_inventory details. -HINT: This one is pretty hard. -First, determine how to get the "last" quantity per product. -Second, coalesce null values to 0 (if you don't have null values, figure out how to rearrange your query so you do.) -Third, SET current_quantity = (...your select statement...), remembering that WHERE can only accommodate one column. -Finally, make sure you have a WHERE statement to update the right row, - you'll need to use product_units.product_id to refer to the correct row within the product_units table. -When you have all of these components, you can run the update statement. */ +--UPDATE - Add current_quantity column and update it +ALTER TABLE product_units ADD COLUMN current_quantity INT; +UPDATE product_units +SET current_quantity = COALESCE( + (SELECT quantity FROM vendor_inventory vi WHERE vi.product_id = product_units.product_id ORDER BY market_date DESC LIMIT 1), + 0 +); diff --git a/04_this_cohort/live_code/module_2/CASE.sql b/04_this_cohort/live_code/module_2/CASE.sql new file mode 100644 index 000000000..0652aaf92 --- /dev/null +++ b/04_this_cohort/live_code/module_2/CASE.sql @@ -0,0 +1,24 @@ +--CASE + +-- determine the prices +SELECT *, +quantity*cost_to_customer_per_qty as price, -- quantity * cost per quantity +CASE WHEN cost_to_customer_per_qty < 1.00 -- if less then $1.00 + THEN cost_to_customer_per_qty * 2 -- double + ELSE cost_to_customer_per_qty -- otherwise, do nothing + END as new_cost_per_qty, + +CASE WHEN cost_to_customer_per_qty < 1.00 + THEN cost_to_customer_per_qty * 2 * quantity + ELSE cost_to_customer_per_qty * quantity + END as new_price, + +CASE WHEN cost_to_customer_per_qty < 1.00 + THEN cost_to_customer_per_qty * 5 * quantity -- make cheap things 5x more expensive + WHEN cost_to_customer_per_qty BETWEEN 1.01 AND 5.00 + THEN cost_to_customer_per_qty * 2 * quantity -- make middle things 2x more expensive + ELSE cost_to_customer_per_qty * quantity + END as new_new_price + +FROM customer_purchases +ORDER by cost_to_customer_per_qty \ No newline at end of file diff --git a/04_this_cohort/live_code/module_2/DISTINCT.sql b/04_this_cohort/live_code/module_2/DISTINCT.sql new file mode 100644 index 000000000..b66ada474 --- /dev/null +++ b/04_this_cohort/live_code/module_2/DISTINCT.sql @@ -0,0 +1,14 @@ +--DISTINCT + +--make a selection of columns distinct +--what does each vendor have on inventory and their price +SELECT DISTINCT vendor_id, product_id, original_price +FROM vendor_inventory; + +--what prices does the vendor sell things at +SELECT DISTINCT vendor_id, original_price +FROM vendor_inventory; + +-- what products does each vendor have on inventory and their price every day they were at the farmers market +SELECT DISTINCT vendor_id, product_id, original_price, market_date +FROM vendor_inventory \ No newline at end of file diff --git a/04_this_cohort/live_code/module_2/INNER_JOIN.sql b/04_this_cohort/live_code/module_2/INNER_JOIN.sql new file mode 100644 index 000000000..dbbcbb0fd --- /dev/null +++ b/04_this_cohort/live_code/module_2/INNER_JOIN.sql @@ -0,0 +1,14 @@ +--INNER JOIN + +SELECT * +FROM product -- on the left +INNER JOIN product_category + ON product.product_category_id = product_category.product_category_id; + + +SELECT +product_category_name, p.* -- only product_category_name and then everything from p product +FROM product_category AS pc -- on the left +INNER JOIN product AS p -- on the right + ON pc.product_category_id = p.product_category_id -- same result set (but we lost row for "product_category_id = 4" because not in the product table +ORDER BY pc.product_category_id ASC; \ No newline at end of file diff --git a/04_this_cohort/live_code/module_2/LEFT_JOIN.sql b/04_this_cohort/live_code/module_2/LEFT_JOIN.sql new file mode 100644 index 000000000..4cb50cb4a --- /dev/null +++ b/04_this_cohort/live_code/module_2/LEFT_JOIN.sql @@ -0,0 +1,14 @@ +--LEFT JOIN + +SELECT * +FROM product_category AS pc +LEFT JOIN product AS p + ON pc.product_category_id = p.product_category_id +ORDER BY pc.product_category_id ; + +-- different output! because product_category_id 4 is not in the product table! +SELECT * +FROM product AS pc +LEFT JOIN product_category AS p + ON pc.product_category_id = p.product_category_id +ORDER BY pc.product_category_id ; diff --git a/04_this_cohort/live_code/module_2/SELECT_FROM.sql b/04_this_cohort/live_code/module_2/SELECT_FROM.sql new file mode 100644 index 000000000..6ef045f45 --- /dev/null +++ b/04_this_cohort/live_code/module_2/SELECT_FROM.sql @@ -0,0 +1,23 @@ +--SELECT / FROM + +--SELECT * FROM customer + +/* -- comment out several lines of code +SELECT product_name, +product_category_id, +product_qty_type +FROM product +*/ + +SELECT +'2025' as this_year, +customer_first_name, +10*5 as calculation, +12 / 3, +1+1 + +FROM customer; + +SELECT +customer_first_name as first_name +FROM customer diff --git a/04_this_cohort/live_code/module_2/WHERE.sql b/04_this_cohort/live_code/module_2/WHERE.sql new file mode 100644 index 000000000..2884c490b --- /dev/null +++ b/04_this_cohort/live_code/module_2/WHERE.sql @@ -0,0 +1,33 @@ +--WHERE + +SELECT * +FROM customer +--WHERE customer_postal_code = 'M1L' -- only M1L +WHERE customer_postal_code IN ('M1L', 'M8Y'); -- M1L or M8Y + + -- dates farmers market was open in JUNE +SELECT * +FROM market_date_info +WHERE market_date BETWEEN '2022-06-01' AND '2022-06-30'; + +-- products with _tat_ or _ato_ in their names +SELECT * +FROM product +WHERE product_name LIKE '%tat%' +OR product_name LIKE '%ato%'; + + +SELECT * +FROM market_date_info +WHERE market_year <> 2022; -- not 2022, but could be 2021, 2023, etc + +-- what product_qty_types are missing in the product table +SELECT * +FROM product +WHERE product_qty_type IS NULL; + +-- what product_qty_type is blank +SELECT * +FROM product +WHERE product_size = '' -- must be two SINGLE QUOTES + diff --git a/04_this_cohort/live_code/module_2/multiple_table_joins.sql b/04_this_cohort/live_code/module_2/multiple_table_joins.sql new file mode 100644 index 000000000..2eda7b650 --- /dev/null +++ b/04_this_cohort/live_code/module_2/multiple_table_joins.sql @@ -0,0 +1,15 @@ +-- Multiple Table Joins + +--making the customer_purchases table have a lot more meaning and information +SELECT market_date, quantity, cost_to_customer_per_qty, +product_name, vendor_name, customer_first_name, customer_last_name, city, district + +FROM customer_purchases cp +INNER JOIN product p + ON p.product_id = cp.product_id +INNER JOIN vendor v + ON v.vendor_id = cp.vendor_id +INNER JOIN customer c + ON c.customer_id = cp.customer_id +INNER JOIN postal_data pd + ON c.customer_postal_code = pd.postal_code_3 \ No newline at end of file diff --git a/04_this_cohort/live_code/module_3/COUNT.sql b/04_this_cohort/live_code/module_3/COUNT.sql new file mode 100644 index 000000000..638acd764 --- /dev/null +++ b/04_this_cohort/live_code/module_3/COUNT.sql @@ -0,0 +1,28 @@ +-- count + +--count the number of products +SELECT +count(product_id) AS num_of_prod + +FROM product; + +-- how many products per product_qty_type +SELECT +product_qty_type, +count(product_id) +FROM product +WHERE product_qty_type IS NOT NULL -- remove the NULL rows +GROUP BY product_qty_type; + +-- how many UNIQUE products were bought +SELECT count(DISTINCT product_id) +FROM customer_purchases; + +-- how many products per product_qty_type AND per their product_size +SELECT +product_size, +product_qty_type, +count(product_id) + +FROM product +GROUP BY product_size, product_qty_type diff --git a/04_this_cohort/live_code/module_3/CTEs.sql b/04_this_cohort/live_code/module_3/CTEs.sql new file mode 100644 index 000000000..f7d6d9aef --- /dev/null +++ b/04_this_cohort/live_code/module_3/CTEs.sql @@ -0,0 +1,36 @@ +--CTEs + +--calculate sales per vendor per day +WITH vendor_daily_sales AS ( +SELECT +md.market_date +,market_day +,market_week +,market_year +,vendor_name +,sum(quantity*cost_to_customer_per_qty) as sales + +FROM customer_purchases cp +INNER JOIN vendor v + ON cp.vendor_id = v.vendor_id +INNER JOIN market_date_info md + ON cp.market_date = md.market_date + +GROUP BY md.market_date, v.vendor_id +), -- this comma for the next cte + +-- not doing anything, but still makes the 2nd cte, don't use with again +another_cte AS +( +select * from product +) + +--re-aggregate the daily sales within each market_week for each vendor +SELECT +market_year +,market_week +,vendor_name +,sum(sales) + +FROM vendor_daily_sales +GROUP BY market_year, market_week, vendor_name \ No newline at end of file diff --git a/04_this_cohort/live_code/module_3/HAVING.sql b/04_this_cohort/live_code/module_3/HAVING.sql new file mode 100644 index 000000000..1161fc1c8 --- /dev/null +++ b/04_this_cohort/live_code/module_3/HAVING.sql @@ -0,0 +1,11 @@ +--HAVING + +--how many products were bought +SELECT +count(product_id) as number_products +,product_id + +FROM customer_purchases +WHERE product_id <= 8 +GROUP BY product_id +HAVING count(product_id) BETWEEN 300 AND 500 diff --git a/04_this_cohort/live_code/module_3/MIN_MAX.sql b/04_this_cohort/live_code/module_3/MIN_MAX.sql new file mode 100644 index 000000000..b240cad6e --- /dev/null +++ b/04_this_cohort/live_code/module_3/MIN_MAX.sql @@ -0,0 +1,34 @@ +--MIN & MAX + +-- what is the most expensive product +SELECT +--product_id +product_name +,MAX(original_price) -- doesnt handle ties very well + +FROM vendor_inventory vi +INNER JOIN product p + ON p.product_id = vi.product_id; + +-- what is the least expensive per each type +SELECT +product_name +,product_qty_type +,MIN(original_price) -- doesnt handle ties very well + +FROM vendor_inventory vi +INNER JOIN product p + ON p.product_id = vi.product_id + +GROUP BY product_qty_type; + +--PROVE THE ABOVE QUERY +SELECT DISTINCT +product_name +,product_qty_type +,original_price + +FROM vendor_inventory vi +INNER JOIN product p + ON p.product_id = vi.product_id +ORDER BY product_qty_type, original_price \ No newline at end of file diff --git a/04_this_cohort/live_code/module_3/SUM_AVG.sql b/04_this_cohort/live_code/module_3/SUM_AVG.sql new file mode 100644 index 000000000..302d126ad --- /dev/null +++ b/04_this_cohort/live_code/module_3/SUM_AVG.sql @@ -0,0 +1,21 @@ +--SUM & AVG + +--how much did a customer spend on each day +SELECT +market_date +,customer_id +,SUM(CAST(quantity AS decimal(5,2))*cost_to_customer_per_qty) as total_cost + +FROM customer_purchases +GROUP BY market_date, customer_id; + +--how much does each customer spend on average +SELECT +customer_first_name +,customer_last_name +,ROUND(AVG(quantity*cost_to_customer_per_qty),2) as avg_spend + +FROM customer_purchases cp +INNER JOIN customer c + ON cp.customer_id = c.customer_id +GROUP BY c.customer_id diff --git a/04_this_cohort/live_code/module_3/TEMP tables.sql b/04_this_cohort/live_code/module_3/TEMP tables.sql new file mode 100644 index 000000000..b84d0ac2e --- /dev/null +++ b/04_this_cohort/live_code/module_3/TEMP tables.sql @@ -0,0 +1,23 @@ +--temp tables + +-- if a table named "new_vendor_inventory exist, delete it, otherwise do NOTHING +DROP TABLE IF EXISTS new_vendor_inventory; + +--make the TABLE +CREATE TEMP TABLE new_vendor_inventory AS +--definition of the TABLE +SELECT * +, original_price * 5 as inflation +FROM vendor_inventory; + +SELECT * FROM new_vendor_inventory; + +-- put a temp table into another one +DROP TABLE IF EXISTS new_new_vendor_inventory; + +CREATE TEMP TABLE new_new_vendor_inventory AS +SELECT * +,inflation*2 as super_inflation +FROM new_vendor_inventory; + +SELECT * FROM new_new_vendor_inventory \ No newline at end of file diff --git a/04_this_cohort/live_code/module_3/arithmitic.sql b/04_this_cohort/live_code/module_3/arithmitic.sql new file mode 100644 index 000000000..482c31c23 --- /dev/null +++ b/04_this_cohort/live_code/module_3/arithmitic.sql @@ -0,0 +1,13 @@ +--arithmitic + +SELECT 10.0/3.0 as division +,CAST (10.0 as INT) / CAST(3.0 as INT) as integer_division +,sin(10) +,pi(); + +SELECT DISTINCT +cost_to_customer_per_qty +,cost_to_customer_per_qty / 2 as half +,CAST(cost_to_customer_per_qty as INT) / 2 -- equiv //2 + +FROM customer_purchases \ No newline at end of file diff --git a/04_this_cohort/live_code/module_3/dates.sql b/04_this_cohort/live_code/module_3/dates.sql new file mode 100644 index 000000000..236dea541 --- /dev/null +++ b/04_this_cohort/live_code/module_3/dates.sql @@ -0,0 +1,28 @@ +--dates + +SELECT +DATE('now') +,DATETIME('now') + +--strftime +,strftime('%Y-%m','now') +,strftime('%Y-%m-%d','now','+30 days') as the_future +,market_date +,strftime('%Y-%m-%d',market_date,'-1 year','+30 days') -- 11 months before the market_date +,strftime('%Y-%m-%d','now','-1 year') + +--dateadd +--last day of the month +,strftime('%Y-%m-%d',market_date,'start of month','-1 day') + +from market_date_info; + +select distinct market_date +--datediff +, julianday('now') +, julianday(market_date) +, julianday('now') - julianday(market_date) -- number of days between now and eaach market_date +, (julianday('now') - julianday(market_date)) / 365.25 -- number of years between now and each market_date +, (julianday('now') - julianday(market_date)) * 24 -- number of hours between now and each market_date + +from market_date_info diff --git a/04_this_cohort/live_code/module_3/subqueries_join.sql b/04_this_cohort/live_code/module_3/subqueries_join.sql new file mode 100644 index 000000000..646900375 --- /dev/null +++ b/04_this_cohort/live_code/module_3/subqueries_join.sql @@ -0,0 +1,21 @@ +--SUBQUERIES: join + +-- "what is the single item that has been bought in the greatest quantity + +--outer query +SELECT +product_name +,max(quantity) + +FROM product p +INNER JOIN ( + +--inner query + SELECT + product_id + ,count(quantity) as quantity + + FROM customer_purchases + GROUP BY product_id + +) x ON p.product_id = x.product_id \ No newline at end of file diff --git a/04_this_cohort/live_code/module_3/subqueries_where.sql b/04_this_cohort/live_code/module_3/subqueries_where.sql new file mode 100644 index 000000000..559809a8d --- /dev/null +++ b/04_this_cohort/live_code/module_3/subqueries_where.sql @@ -0,0 +1,14 @@ +--SUBQUERIES: WHERE + +--"what is the name of the vendor who sells pie" + +SELECT DISTINCT vendor_name--, product_id +FROM vendor_inventory vi +INNER JOIN vendor v + ON v.vendor_id = vi.vendor_id +WHERE product_id IN +( + SELECT product_id + FROM product + WHERE product_name LIKE '%pie%' +) \ No newline at end of file