Steam Recommendation Engine & MLOps Data Pipeline
End-to-end data pipeline transforming raw nested JSON telemetry datasets into optimized Parquet storage, powering a real-time FastAPI recommendation engine.
Technologies & Infrastructure Used
### 🎯 Business Problem & Project Goal
Recommendation systems are core drivers of user engagement and catalog discovery in digital storefronts. The objective of this project was to design an end-to-end data and machine learning pipeline that ingests raw, nested gaming telemetry data from Steam, optimizes storage and retrieval, and serves real-time game recommendations via a production-ready API.
---
### ⚙️ Data Engineering & Pipeline Architecture
* **ETL & Storage Optimization:** Ingested complex, deeply nested JSON datasets. Cleaned and restructured the schemas using Pandas, converting flat storage into column-oriented **Parquet** files to drastically minimize memory footprint and speed up query response times.
* **Feature Engineering & NLP:** Applied Sentiment Analysis algorithms to process user text reviews, creating quantitative user satisfaction scores used directly within the recommendation algorithms.
* **API Development & Serving Layer:** Engineered a high-performance RESTful API using **FastAPI**, exposing endpoint routes for real-time item-item similarity lookups and automated sentiment evaluation.
---
### 📈 Key Engineering Outcomes
* **Storage & Performance Gain:** Reduced data storage requirements using Parquet compression, eliminating RAM bottlenecks during API cold-starts.
* **Low-Latency Inference:** Deployed functional model endpoints capable of delivering instant recommendation payloads to end-user clients.