diff --git a/README.md b/README.md index 2270a80..1d5d2e9 100644 --- a/README.md +++ b/README.md @@ -1,10 +1,10 @@
-CRMS - College Result Management System +CRMS - College Result Management System -# πŸŽ“ CRMS - College Result Management System +# CRMS - College Result Management System -### A Production-Grade DevOps & DevSecOps Platform Built From Zero +### A Production-Grade DevOps & DevSecOps Platform [![CI Pipeline](https://github.com/crms-devops/crms/actions/workflows/ci.yml/badge.svg)](https://github.com/crms-devops/crms/actions) [![DevSecOps Pre-Flight](https://github.com/crms-devops/crms/actions/workflows/devsecops-preflight.yml/badge.svg)](https://github.com/crms-devops/crms/actions) @@ -13,7 +13,7 @@ **Built over 22 weeks | 200+ commits | 24+ technologies | 12 automated security gates | 0% error rate under load** -[πŸ“‹ View All Releases](https://github.com/crms-devops/crms/releases) | [πŸ” Security Dashboard](https://github.com/crms-devops/crms/security) | [βš™οΈ CI/CD Runs](https://github.com/crms-devops/crms/actions) | [πŸ“– Documentation](docs/) +[View All Releases](https://github.com/crms-devops/crms/releases) | [Security Dashboard](https://github.com/crms-devops/crms/security) | [CI/CD Runs](https://github.com/crms-devops/crms/actions) | [Documentation](docs/)
@@ -23,30 +23,30 @@ Every semester, thousands of college students try to check their exam results at the same time - and the website crashes. We built a system that never crashes, no matter how many students use it simultaneously. -A production-grade, cloud-native microservices platform deployed on AWS EKS with full CI/CD automation, GitOps, observability, event-driven architecture, and a 12-gate DevSecOps security pipeline - built end-to-end by two students as a real-world DevOps learning project. +A cloud-native microservices platform deployed on AWS EKS with full CI/CD automation, GitOps, observability, event-driven architecture, and a 12-gate DevSecOps security pipeline - built end-to-end by two students as a real-world DevOps learning project. **This project demonstrates every skill an MNC DevOps/DevSecOps/SRE engineer uses daily.** --- -## πŸ† Achievements At A Glance +## Achievements | Metric | Result | |--------|--------| -| πŸš€ Load test (concurrent users) | **100 VUs - 0.00% error rate** | -| ⚑ Response time (p99) | **58.94ms** | -| πŸ“¦ Requests handled | **24,841 in 3 minutes** | -| πŸ”’ Security CVEs caught | **CVE-2024-33663 (CRITICAL)** | -| πŸ›‘οΈ Security gates in pipeline | **12 automated gates** | -| πŸ“Š Kubernetes pods autoscaled | **2 β†’ 10 pods under load** | -| ☁️ AWS infrastructure | **VPC + EKS + S3 - all via Terraform** | -| πŸ”„ Deployment method | **GitOps via ArgoCD** | -| πŸ“ˆ Observability | **Prometheus + Grafana dashboards** | -| πŸ“… Build duration | **22 weeks** | +| Load test (concurrent users) | **100 VUs - 0.00% error rate** | +| Response time (p99) | **58.94ms** | +| Requests handled | **24,841 in 3 minutes** | +| Security CVEs caught | **CVE-2024-33663 (CRITICAL)** | +| Security gates in pipeline | **12 automated gates** | +| Kubernetes pods autoscaled | **2 β†’ 10 pods under load** | +| AWS infrastructure | **VPC + EKS + S3 - all via Terraform** | +| Deployment method | **GitOps via ArgoCD** | +| Observability | **Prometheus + Grafana dashboards** | +| Build duration | **22 weeks** | --- -## πŸ€” The Problem We Solved +## The Problem We Solved collages and Universities publishes semester results online. Every result day, **5000+ students rush to check their results simultaneously.** The current portal: @@ -60,20 +60,20 @@ collages and Universities publishes semester results online. Every result day, * --- -## πŸ—οΈ System Architecture +## System Architecture ```mermaid graph TB - subgraph "πŸ‘¨β€πŸŽ“ Students" + subgraph " Students" S[5000+ Students
Result Day] end - subgraph "☁️ AWS Cloud - ap-south-1 Mumbai" - subgraph "🌐 Network Layer" + subgraph " AWS Cloud - ap-south-1 Mumbai" + subgraph " Network Layer" LB[AWS Load Balancer
Auto-distributes traffic] end - subgraph "βš™οΈ Kubernetes - AWS EKS" + subgraph " Kubernetes - AWS EKS" FE[React Frontend
2-5 pods] BE[FastAPI Backend
2-10 pods ← HPA] DB[PostgreSQL
7 tables] @@ -81,22 +81,22 @@ graph TB RD[Redis
Result cache] end - subgraph "πŸ“Š Observability" + subgraph " Observability" PR[Prometheus
Metrics collection] GR[Grafana
Live dashboards] end - subgraph "πŸ”„ GitOps" + subgraph " GitOps" AR[ArgoCD
Auto-deploys from Git] end - subgraph "πŸ—οΈ Infrastructure" + subgraph " Infrastructure" TF[Terraform
IaC - VPC + EKS] S3[S3
Terraform state] end end - subgraph "πŸ”§ Developer Workflow" + subgraph " Developer Workflow" GH[GitHub
Source code] CI[GitHub Actions
12-gate pipeline] CR[GHCR
Docker images] @@ -120,7 +120,7 @@ graph TB --- -## πŸ”„ How A Student Gets Their Result +## How A Student Gets Their Result ```mermaid sequenceDiagram @@ -154,7 +154,7 @@ sequenceDiagram --- -## πŸš€ CI/CD Pipeline - Every Push Is Tested +## CI/CD Pipeline - Every Push Is Tested ```mermaid flowchart LR @@ -197,32 +197,32 @@ flowchart LR --- -## πŸ›‘οΈ DevSecOps - 12 Security Gates +## DevSecOps - 12 Security Gates ```mermaid graph TD subgraph "Stage 1: Pre-Flight" - G1["πŸ” Gate 1: TruffleHog
Scans for leaked secrets
AWS keys, passwords, tokens"] - G2["πŸ“‹ Gate 2: Hadolint
Dockerfile best practices
Security misconfigurations"] - G3["πŸ—οΈ Gate 3: Checkov
Terraform + K8s security
IaC misconfigurations"] - G4["πŸ€– Gate 4: TerraSecure
ML-powered IaC scan
Our own tool"] + G1[" Gate 1: TruffleHog
Scans for leaked secrets
AWS keys, passwords, tokens"] + G2[" Gate 2: Hadolint
Dockerfile best practices
Security misconfigurations"] + G3[" Gate 3: Checkov
Terraform + K8s security
IaC misconfigurations"] + G4[" Gate 4: TerraSecure
ML-powered IaC scan
Our own tool"] end subgraph "Stage 2: Code Analysis" - G5["🐍 Gate 5: Bandit
Python SAST
SQL injection, weak crypto"] - G6["πŸ“˜ Gate 6: ESLint Security
TypeScript SAST
XSS, prototype pollution"] - G7["πŸ“Š Gate 7: SonarQube
Code quality gate
Coverage + tech debt"] + G5[" Gate 5: Bandit
Python SAST
SQL injection, weak crypto"] + G6[" Gate 6: ESLint Security
TypeScript SAST
XSS, prototype pollution"] + G7[" Gate 7: SonarQube
Code quality gate
Coverage + tech debt"] end subgraph "Stage 3: Supply Chain" - G8["πŸ”— Gate 8: Snyk
Dependency CVE scan
Python + Node packages"] - G9["πŸ“¦ Gate 9: Syft SBOM
Software Bill of Materials
Every package catalogued"] + G8[" Gate 8: Snyk
Dependency CVE scan
Python + Node packages"] + G9[" Gate 9: Syft SBOM
Software Bill of Materials
Every package catalogued"] end subgraph "Stage 4: Runtime" - G10["πŸ•·οΈ Gate 10: OWASP ZAP
DAST - attacks live app
XSS, SQLi, auth bypass"] - G11["πŸ“œ Gate 11: OPA
Policy as Code
K8s security policies"] - G12["πŸ” Gate 12: Vault Check
Secrets management
No hardcoded credentials"] + G10[" Gate 10: OWASP ZAP
DAST - attacks live app
XSS, SQLi, auth bypass"] + G11[" Gate 11: OPA
Policy as Code
K8s security policies"] + G12[" Gate 12: Vault Check
Secrets management
No hardcoded credentials"] end G1 --> G2 --> G3 --> G4 @@ -235,7 +235,7 @@ graph TD --- -## πŸ“ˆ Autoscaling - How We Handle 5000 Students +## Autoscaling - How We Handle 5000 Students ```mermaid graph LR @@ -261,7 +261,7 @@ graph LR --- -## πŸ›οΈ Database Schema +## Database Schema ```mermaid erDiagram @@ -319,7 +319,7 @@ erDiagram --- -## πŸ—“οΈ Project Timeline - 22 Weeks +## Project Timeline - 22 Weeks ```mermaid timeline @@ -353,7 +353,7 @@ timeline --- -## πŸ› οΈ Complete Technology Stack +## Complete Technology Stack ### Application | Layer | Technology | Why | @@ -410,7 +410,7 @@ timeline --- -## ⚑ Load Test Results +## Load Test Results ``` k6 run k6/load-test.js @@ -438,7 +438,7 @@ k6 run k6/load-test.js --- -## πŸš€ Quick Start +## Quick Start ### Run locally (5 minutes) @@ -492,11 +492,11 @@ Date of Birth: 2005-05-08 --- -## πŸ“ Repository Structure +## Repository Structure ``` crms/ -β”œβ”€β”€ πŸ“± backend/ FastAPI application +β”œβ”€β”€ backend/ FastAPI application β”‚ β”œβ”€β”€ app/ β”‚ β”‚ β”œβ”€β”€ api/ Route handlers (auth, results) β”‚ β”‚ β”œβ”€β”€ core/ Config, database, Kafka, security @@ -509,12 +509,12 @@ crms/ β”‚ β”œβ”€β”€ requirements.txt Pinned dependencies β”‚ └── start.sh alembic upgrade β†’ uvicorn β”‚ -β”œβ”€β”€ 🌐 frontend/ React application +β”œβ”€β”€ frontend/ React application β”‚ β”œβ”€β”€ src/pages/ LoginPage.tsx, ResultsPage.tsx β”‚ β”œβ”€β”€ public/ SIET logos, campus photos β”‚ └── Dockerfile Multi-stage Node + nginx build β”‚ -β”œβ”€β”€ πŸ—οΈ infra/terraform/ AWS Infrastructure as Code +β”œβ”€β”€ infra/terraform/ AWS Infrastructure as Code β”‚ β”œβ”€β”€ main.tf Provider + S3 backend config β”‚ β”œβ”€β”€ vpc.tf VPC, subnets, IGW, route tables β”‚ β”œβ”€β”€ eks.tf EKS cluster + node group @@ -522,7 +522,7 @@ crms/ β”‚ β”œβ”€β”€ variables.tf Input variables β”‚ └── outputs.tf VPC ID, subnet IDs, cluster endpoint β”‚ -β”œβ”€β”€ ☸️ k8s/base/ Kubernetes manifests +β”œβ”€β”€ k8s/base/ Kubernetes manifests β”‚ β”œβ”€β”€ namespace.yaml crms namespace β”‚ β”œβ”€β”€ backend-deployment.yaml FastAPI + HPA config β”‚ β”œβ”€β”€ frontend-deployment.yaml React/nginx + LoadBalancer @@ -533,21 +533,21 @@ crms/ β”‚ β”œβ”€β”€ configmap.yaml Non-secret app config β”‚ └── hpa.yaml Horizontal Pod Autoscaler β”‚ -β”œβ”€β”€ πŸ”„ k8s/argocd/ GitOps configuration +β”œβ”€β”€ k8s/argocd/ GitOps configuration β”‚ └── crms-application.yaml ArgoCD Application CRD β”‚ -β”œβ”€β”€ πŸ“Š observability/ Monitoring configuration +β”œβ”€β”€ observability/ Monitoring configuration β”‚ β”œβ”€β”€ prometheus-values.yaml kube-prometheus-stack Helm values β”‚ β”œβ”€β”€ prometheus-rules.yaml Alert rules (error rate, latency) β”‚ └── grafana-dashboards/ Custom CRMS dashboard JSON β”‚ -β”œβ”€β”€ πŸ§ͺ k6/ Load testing +β”œβ”€β”€ k6/ Load testing β”‚ └── load-test.js 100VU ramp test script β”‚ -β”œβ”€β”€ πŸ“‹ policy/ Security policies +β”œβ”€β”€ policy/ Security policies β”‚ └── k8s-security.rego OPA Rego policy - 4 rules β”‚ -β”œβ”€β”€ πŸ”§ .github/workflows/ CI/CD + Security pipelines +β”œβ”€β”€ .github/workflows/ CI/CD + Security pipelines β”‚ β”œβ”€β”€ ci.yml pytest + ESLint + Trivy β”‚ β”œβ”€β”€ cd.yml Push to GHCR on main merge β”‚ β”œβ”€β”€ devsecops-preflight.yml Gates 1-4 @@ -556,20 +556,20 @@ crms/ β”‚ β”œβ”€β”€ devsecops-dast.yml Gate 10 β”‚ └── devsecops-policy-vault.yml Gates 11-12 β”‚ -β”œβ”€β”€ πŸ“– docs/ Project documentation +β”œβ”€β”€ docs/ Project documentation β”‚ β”œβ”€β”€ week-01-notes.md through week-22-notes.md β”‚ β”œβ”€β”€ architecture.md Full architecture decisions β”‚ β”œβ”€β”€ college-proposal.md Formal proposal to SIET β”‚ └── decisions/ ADRs and migration plans β”‚ -└── πŸ–ΌοΈ assets/ Architecture diagrams +└── assets/ Architecture diagrams β”œβ”€β”€ crms_system_architecture.png └── crms_request_flow.png ``` --- -## πŸ“Š Milestones +## Milestones | Tag | Week | Milestone | |-----|------|-----------| @@ -597,7 +597,7 @@ crms/ --- -## πŸ’° AWS Cost Breakdown +## AWS Cost Breakdown | Resource | Cost | Notes | |----------|------|-------| @@ -611,7 +611,7 @@ crms/ --- -## πŸŽ“ Built By +## Built By @@ -628,7 +628,7 @@ crms/ --- -## πŸ“š Learning Resources +## Learning Resources Each week of this project has detailed notes in `docs/`: @@ -640,7 +640,7 @@ Each week of this project has detailed notes in `docs/`: --- -## πŸ“„ License +## License MIT License - see [LICENSE](LICENSE) for details. @@ -648,7 +648,7 @@ MIT License - see [LICENSE](LICENSE) for details.
-**If this project helped you learn something - leave a ⭐** +**If this project helped you learn something, leave a star!!**
\ No newline at end of file