MLOps Learning

Roadmap

Phase1 Cloud Iac

IAM Basics
IAM Roles & Service Accounts
S3 Fundamentals
VPC Fundamentals
EC2 Fundamentals
Introduction to Terraform
Terraform State Management
Terraform Modules

Phase2 Containers K8s

Docker Deep Dive
Kubernetes Core Concepts
Kubernetes Hands-On with Kind
Kubernetes YAML Manifests
EKS Cluster Setup with Terraform
GitHub Actions Fundamentals
Complete CI/CD Pipeline

Phase3 Ml Lifecycle

ML Fundamentals for MLOps
MLflow Experiment Tracking
MLflow Model Registry
Model Serving with FastAPI
Airflow Fundamentals
ML Pipeline with Airflow

Phase4 Capstone

Capstone Architecture Design
Capstone Infrastructure Setup
Deploy MLflow to Kubernetes
Capstone Training Pipeline
Capstone Model Serving
Monitoring and Observability
Data and Model Drift Detection
Capstone CI/CD Integration
Documentation and Portfolio

Phase4 Capstone

Capstone Training Pipeline

Introduction

Now you build the training pipeline that runs in your infrastructure. It will: - Ingest data from S3 - Preprocess and validate - Train a model - Log to MLflow - Register in the model registry - Optionally promote to production

Why This Matters

This is the core ML pipeline that will run repeatedly — on schedule or triggered by new data. Everything you've learned comes together: - Airflow for orchestration - S3 for data storage - MLflow for tracking - Kubernetes for compute The training pipeline is often the most complex part of an MLOps platform. Getting it right means reliable, reproducible model training.

Tasks

Deploy Airflow to your EKS cluster (or use managed MWAA)

Hint: Helm chart is the easiest way

Create the training DAG with these stages:

- data_validation: check data quality, schema, freshness

- preprocessing: clean data, feature engineering, save to S3

- training: train model in a Kubernetes pod, log to MLflow

- evaluation: evaluate model, log metrics

- registration: register model in MLflow registry

- promotion: conditionally promote to staging/production

Use KubernetesPodOperator to run training in isolated containers

Configure the DAG to run on a schedule

Add alerting for failures

Test the full pipeline end-to-end

Document the pipeline: what runs, when, dependencies

Check Understanding

AI Assistant

Ask questions about this topic. The assistant has context about what you're learning.