Cloud-Native Data Engineering Portfolio & AWS Architecture
Full-stack containerized web platform built with Django 6 and Python 3.13, managed with uv, and architected for zero-cost high-availability deployment on AWS Cloud.
Technologies & Infrastructure Used
📐 System Architecture & Data Pipeline
Technical Diagram
Click on the diagram to expand in full resolution.
### Overview
This project represents the cloud-native infrastructure, software architecture, and deployment pipeline for my personal Data Engineering & Analytics Engineering portfolio. Built from scratch to showcase modern web practices and high-availability deployment, the platform serves as both an interactive showcase of technical case studies and a live demonstration of production-grade container orchestration on public cloud infrastructure.
---
### Key Architectural Highlights
* **Modern Python Toolchain:** Built with Python 3.13 and Django 6, managed using `uv` for ultra-fast dependency resolution and deterministic virtual environment builds.
* **Containerized Deployment:** Encapsulated in lightweight Docker containers orchestrated via Docker Compose, isolating application dependencies, database persistence, and reverse proxy networking.
* **Automatic HTTPS & Reverse Proxy:** Integrated with Caddy Server as a front-end reverse proxy, providing automatic TLS certificate provisioning (Let's Encrypt / ZeroSSL), HTTP/2-3 protocols, and secure WebSocket/HTTP proxying to the Django WSGI backend.
* **Cloud Infrastructure & Security:** Hosted on an AWS EC2 instance running Fedora Linux. Configured with strict CSRF trusted origins, cross-origin resource sharing, secure session handling, and environment-isolated secret management.
* **Production Storage & Static Assets:** Structured media and static file pipelines utilizing Django's `collectstatic` with dedicated persistent Docker volume mounts for media dynamic uploads and SQLite database integrity.
---
### Technical Specifications
| Component | Technology | Purpose |
| :--- | :--- | :--- |
| **Backend Framework** | Django 6.0+ / Python 3.13 | Core MVC architecture, Object-Relational Mapping (ORM), and Admin CMS |
| **Package Manager** | `uv` (Astral) | Lightning-fast virtualenv management and lockfile enforcement |
| **Container Runtime** | Docker & Docker Compose | Multi-container application management and service isolation |
| **Reverse Proxy** | Caddy Server | SSL termination, automatic HTTPS renewal, and request forwarding |
| **Infrastructure** | AWS EC2 (Fedora Linux) | Low-cost, highly reliable cloud host execution environment |
| **Frontend UI** | HTML5, Tailwind CSS, JavaScript | Responsive, dark-themed UI components for case study presentations |
---
### What I Learned & Implemented
1. **Production Hardening:** Fine-tuned Django settings for proxy header evaluation (`SECURE_PROXY_SSL_HEADER`, `USE_X_FORWARDED_HOST`, `CSRF_TRUSTED_ORIGINS`) to handle secure HTTPS requests seamless through proxying layers.
2. **Zero-Downtime Pipeline Setup:** Developed clean container rebuild strategies using `--no-cache` and strict `uv` lock files to ensure byte-for-byte reproducibility across local dev environments and cloud instances.
3. **Database & Asset Persistence:** Configured volume mounts for SQLite and media storage to guarantee data retention across container lifecycle events and deployments.