Totally agree! Schema drift was a great lesson today. dlt handles schema versioning out of the box, storing it as YAML in the destination. It definitely gives me peace of mind knowing the evolution is tracked even if the API changes. Thanks for the insight!
25.02.2026 17:48 — 👍 0 🔁 0 💬 0 📌 0
#DEZoomcamp W6: dlt & DuckDB! 🏁
Schema drift: API changed tip_amount to tip_amt. ⚠️
DuckDB's error logs pointed out the fix instantly.
Running #dlt on an AWS t3.micro (1GB RAM).
0.2666 Credit prop | 6063.41 Tips.
Repo: github.com/CodingJhames/de-zoomcamp-james
#DataEngineering #AWS #BuildInPublic
Thanks, Jeremy! Performance took a hit due to disk I/O, but the goal was to prove that logic beats a tight budget. I used the zone data for a join, and you're right—it’s the perfect case for broadcast joins to avoid exploding that 1GB RAM limit!.
23.02.2026 14:45 — 👍 0 🔁 0 💬 0 📌 0
Data Engineering Week 5: Done! 🏁
Pivoted to #AWS from GCP. ☁️
Ran #PySpark on a t3.micro (1GB RAM) using a 4GB Swapfile. Processed NYC Taxi data smoothly without crashes. 🧠
Adaptability > Tools. 🦾
Code: github.com/CodingJhames...
#DataEngineering #Spark #AWS #OpenSource
Week 4 Analytics Engineering: Complete! 👨💻
Mastered dbt with NYC Taxi data in AWS Athena. Overcoming those Parquet type mismatches was the final boss! 🚀
Project link:
github.com/CodingJhames...
#DataEngineering #dbt #BuildInPublic
Week 3 #DataZoomcamp done! 🚀
Migrated the DW logic to AWS Athena & S3 ☁️
🔹 20M+ records with #Kestra 🔹 Optimized queries: 310MB → 26MB scan using Partitioning & Clustering 🔹 Mastered cloud-agnostic DW concepts
Check my repo: 🔗 github.com/CodingJhames...
#DataEngineering #AWS #Athena
