Welcome to my data science logbook! This repository is the home for my hands-on journey into the world of data. I'm a firm believer that the best way to truly master data is to get your hands dirty, which is exactly what this collection of projects is all about.
My goal here is to move beyond pure theory and build a strong, practical foundation in the core pillars of data science:
- Statistical Rigor: It's not enough to just see a difference in numbers; I want to know if that difference is significant. This part of my journey is about using tools like hypothesis testing and confidence intervals to validate assumptions and make claims with statistical backing.
- Predictive Power: This is where the magic happens—building models that can forecast the future. My focus is on understanding not just how to build a model, but what drives its predictions, and how to iterate on it to improve its accuracy. I'm starting with the fundamentals like linear regression and plan to explore more complex algorithms as I go.
- Data Engineering: A model is only as good as the data it's fed. While my initial projects are analysis-focused, a key part of my learning path will be understanding how to build robust data pipelines, manage data storage, and ensure data quality from start to finish.
This repository will grow as I do. Each folder represents a new skill learned, a new concept mastered, and another step forward on my path to becoming a data scientist.
Here's a look at the projects I've tackled on my journey. Each folder is a self-contained analysis with its own dataset and README.
###📂 01-loan-analysis - Foundations of EDA
- Goal: To conduct a classic Exploratory Data Analysis (EDA) on loan data. This was about getting comfortable with the fundamentals: loading, cleaning, and visualizing a dataset to uncover initial insights into risk factors.
- Key Skills:
pandasfor data manipulation,matplotlibfor foundational plotting.
- Goal: To go beyond simple observation and use statistics to validate claims. I used hypothesis testing (t-tests) to determine if differences in diamond prices between various cuts were statistically significant.
- Key Skills:
scipy.statsfor hypothesis testing,seabornfor comparative visualizations.
- Goal: To build my first predictive model from scratch. I used simple and multiple linear regression to predict diamond prices, learning how to interpret model coefficients and evaluate performance with R-squared.
- Key Skills:
statsmodelsfor regression analysis, feature engineering (one-hot encoding).
- Goal: To tell a clear and compelling story using more advanced visualization techniques. This project on flight delays involved creating stacked bar charts, scatter plots with correlation, and grids of subplots to compare distributions.
- Key Skills: Advanced
matplotlibandseabornplotting techniques.
- Goal: To dive deeper into predictive modeling by analyzing a more complex loan dataset. This project focused on feature engineering (like creating a
DebtToIncomeratio) and building a multiple linear regression model to predict loan interest rates. - Key Skills: Advanced feature creation, in-depth model interpretation, identifying key predictive factors from model coefficients.
- Goal: To analyze historical stock market data to understand performance, volatility, and seasonality. The project looked at the impact of the COVID-19 pandemic on tech stocks and uncovered seasonal trading volume patterns for Netflix.
- Key Skills: Time-series analysis with
pandas, handling datetime data, creating rolling averages, usingpivotto reshape data for comparative plotting.
- Goal: To build a time-series forecasting model to predict weekly temperatures for agricultural planning. This project highlighted the importance of seasonal features in predictive modeling.
- Key Skills: Resampling time-series data, building and evaluating a regression model with seasonal components, calculating and interpreting residuals and Mean Absolute Error (MAE).
Here are the tools I've been using on this journey so far:
- Python
- Pandas
- NumPy
- Matplotlib
- Seaborn
- SciPy
- Statsmodels
The journey is just getting started! I plan to continue adding new projects as I explore more advanced machine learning models, tackle larger and more complex datasets, and begin to dive into the world of data engineering. Stay tuned!