Hasan Geren

Data ingestion with dlt and Dagster: An end-to-end pipeline tutorial Ingest Data from Bluesky API to AWS S3 Using dlt and deploy it on Dagster in Just 15 Minutes.

Data ingestion with dlt and Dagster: An end-to-end pipeline tutorial:

Curious like us to see what people are sharing with #dataBS and #datasky? Check out this post to learn how to do it using dlt!"
@matthausk.bsky.social
@datateam.bsky.social
@hgeren.bsky.social
@hopefanhe.bsky.social
#dlt

Week 0/32 - A Comprehensive Data Engineering Interview Preparation Guide Join us every Saturday on This New Journey

We are starting a 32-week Data Engineering Interview Guide program, covering everything from fundamentals to advanced topics, with sessions every Saturday.
Do you think we're missing any critical topics? We're curious about your opinions😊
#dataBS
#datasky

As a Data Engineer, understanding the data storage lifecycle and data retention policies is critical for designing efficient, cost-effective, and compliant data systems.
@joereis.bsky.social
#dataBS #datasky

substack.com/@pipeline2in...

10 Pipeline Design Patterns for Data Engineers How to leverage Design Patterns for scalable and efficient data pipelines

In our new post, we've covered 10 of the most popular data pipeline design patterns.

We’d love to hear your thoughts. For more details, please check out the full post created by (@hgeren.bsky.social and @hopefanhe.bsky.social ): open.substack.com/pub/pipeline...

#dataBS #datasky

As a Data Engineer and Monster Hunter fan, love this metaphor!

Introduction to data load tool (dlt): A Python Library for Simple Data Ingestion Discover the basics of dlt and its role in modern data engineering workflows

Discover how dlt simplifies data ingestion.
Learn its origins and real-world use cases. Follow a step-by-step guide to build your first pipeline and join the growing dlt community!
@matthausk.bsky.social
@datateam.bsky.social
@hgeren.bsky.social
@hopefanhe.bsky.social

#dataBS #datasky

Hi, wishing everyone a great Thanksgiving!

Recently we wrote about how SQL queries are executed behind the scenes.

If you are interested, check out our post: open.substack.com/pub/pipeline...

#dataBS #datasky

Storage Fundamentals For Data Engineers Why organised and durable storage is the cornerstone of Data Engineering?

Storage is at the heart of Data Engineering.
In this post, we explore the hierarchy of data storage from the ground up, drawing inspiration from Fundamentals of Data Engineering by
@joereis.bsky.social
and Matt Housley, as well as insights from the DE Professionals on Coursera.
#dataBS #datasky

Thank you so much! I am also planning to study cost estimation step in detail soon, so I will definitely write about it when I deepen my knowledge 🙏🏻

SQL Behind the Curtain: How Are Queries Executed? Explore the journey of your SQL query guided by execution plans

Hey #dataBS and #datasky folks,

Our new post about "how understanding Big O Notation & Execution Plans can optimize SQL queries" has just been posted.

Check it out if you're interested, and we'd love to hear your thoughts! @hopefanhe.bsky.social
open.substack.com/pub/pipeline...

yeah you are right, it was posted about 10 days ago 😊

Yeah, maybe Data Science can also be the navigation system with its predictions capabilities and Data Analytics can be driving assistants. While Data Engineering ensuring the whole coordination.

Hey #dataBS, I've been thinking of an analogy for Data Teams' roles.

Imagine a company as a vehicle. How would you map Data Engineering, Analytics, and Science to vehicle parts? Teams could have multiple parts or overlap with other Teams.

Curious about your thoughts!

YouTube video by Rill Data Data Talks on the Rocks 5 - Hannes Mühleisen, DuckDB

Looking for a distraction? Try this great interview between @hannes.muehleisen.org and @medriscoll.bsky.social covering all things @duckdb.org. I especially enjoyed the philosophy around improving SQL usability. www.youtube.com/watch?v=a-Rm... #databs

#dstaBS can you repost?

Filled up the first 150 and so am creating a second starter pack! Let’s all keep finding each other and make this place the best for all things data

Week #1: 100 Days of SQL Optimisation How Small Tweaks Transformed Our Queries, Saving Time and Resources

Week 1 of "100 Days of SQL Optimisation" covered key techniques like column selection, multicolumn indexes, filtering, window functions, Rank, CTE and composite indexes with IMDb data.

Check out the full post for more!
@hgeren.bsky.social
#dataBS #datasky

I made an infra engineer starter pack. Folks posting about databases, stream processing, durable execution, orchestrators, service meshes, and more.

go.bsky.app/SCZe42X

Hello everyone! I’m Hasan.

I transitioned from Industrial Engineering to Data Science, then found my passion in Data Engineering. Currently, doing a PhD in distributed stream processing while working as a Data Engineer.

Looking forward to connecting with fellow data enthusiasts to learn and share.

I’d say SQL

Just joined and heard #dataBS and #datasky are where the cool kids hang.

Wanted to introduce our blog where we regularly write about Data Engineering concepts, news, and tools.

pipeline2insights.substack.com