S3 Fundamentals
Introduction
S3 (Simple Storage Service) is AWS's object storage. Think of it as a giant, infinitely
scalable filesystem that you access via HTTP.
In MLOps, S3 is everywhere:
- Training data lives in S3
- Model artifacts get saved to S3
- Logs, metrics, and experiment tracking data often land in S3
- Terraform state files should be stored in S3
Why This Matters
As a test automation engineer, you've probably worked with local files and maybe network shares.
S3 is different — it's an API-first storage system designed for scale and durability.
Understanding S3 patterns matters because:
- You'll configure training jobs to read from and write to S3
- You'll set up model registries backed by S3
- You'll store Terraform state in S3 (critical for team collaboration)
- You'll need to understand permissions (bucket policies) for secure data access
S3 is also cheap and nearly infinitely scalable, which is why it's the default choice
for ML data and artifacts.
Tasks
Hint: Use your-name-mlops-learning or similar
Hint: aws s3 cp localfile.txt s3://your-bucket-name/
Hint: S3 doesn't really have folders — they're just key prefixes