Gabriel, @thedatasitter's Avatar

Gabriel, @thedatasitter

@thedatasitter.com.bsky.social

Taming Data Tantrums. Senior Data Engineer @ Caylent (2024 AWS Partner of the Year) - subscribe to https://thedatasitter.substack.com/welcome 🇧🇷 - opinions are my own

22 Followers  |  55 Following  |  19 Posts  |  Joined: 27.02.2025  |  1.4849

Latest posts by thedatasitter.com on Bluesky

Preview
My story of job hopping in Data Engineering Off Topic 6

Last friday I published my most personal Substack post so far. I tell the tale of my job hopping in Data Engineering early in my career.

Check it out:
thedatasitter.substack.com/p/my-history...

15.04.2025 15:57 — 👍 0    🔁 0    💬 0    📌 0
Post image Post image

Let's talk about Partitioning and Bucketing as a performance strategy for Spark!

open.substack.com/pub/thedatas...

17.03.2025 15:16 — 👍 0    🔁 0    💬 0    📌 0
Post image

Friday's post will be about the Architect role. Hope you guys like it!

12.03.2025 11:58 — 👍 1    🔁 0    💬 0    📌 0

After some good years of job hopping, today is the first time I have a one-year anniversary at a company. Gotta say, I love my job at Caylent

12.03.2025 11:58 — 👍 0    🔁 0    💬 0    📌 0

How can AI kill software engineering jobs if it can't even kill SQL? lol

11.03.2025 16:45 — 👍 3    🔁 0    💬 0    📌 0
Preview
Netflix's WAP: guarantee data quality [Under The Hood #2] Write, Audit and Publish your data to minimize those tantrums.

How do you guarantee you won't have any bad rows maculating your source of truth?

Your answer is here:
thedatasitter.substack.com/p/netflixs-w...

11.03.2025 13:56 — 👍 1    🔁 0    💬 0    📌 0

Nevertheless, the spirit of the division is the same. You'll have your go-to production data at the end - the gold or analytics layer.

These are supposed to be the most trusted, up-to-date, clean, and good looking tables.

11.03.2025 13:56 — 👍 0    🔁 0    💬 1    📌 0

AWS names them “raw, staging, and analytics.”

Databricks likes the medallion convention of “bronze, silver, and gold.”

In some scenarios, you would add a layer for events. Some call them “transient,” others “landing.”

11.03.2025 13:56 — 👍 0    🔁 0    💬 1    📌 0

If you hold 10 data engineers in a room and ask them the correct naming convention for the layers of a data lake, odds are you'll receive 11 different answers.

11.03.2025 13:56 — 👍 0    🔁 0    💬 1    📌 0
@thedatasitter | Linktree Data Engineering educational tech content.

Also, if you missed, this week I wrote about parquet files and vectors. You can take a look at thedatasitter.com

06.03.2025 12:09 — 👍 3    🔁 0    💬 0    📌 0
Post image

Just finished preparing The Data Sitter's Friday post. I'll discuss memoir books. Hope it doesn't feel off, as it is a tech blog

06.03.2025 12:09 — 👍 0    🔁 0    💬 0    📌 0
Post image

“My kid never got vaccinated and he’s fine”

The kid:

28.02.2025 22:30 — 👍 1    🔁 0    💬 0    📌 0
Post image

The most “going to gym in a small brazilian town” pic you’re going to see today:

28.02.2025 17:00 — 👍 1    🔁 0    💬 0    📌 0

preach

28.02.2025 14:16 — 👍 0    🔁 0    💬 0    📌 0
Post image

So far, this has been the book of the year for me. I had no idea of how messy is Elon’s management of Twitter/X.

There was a time when he ended contracts with the companies that cleaned the Twitter offices lol

Employees had to bring toilet paper from home or go to bathrooms in nearby coffee shops

27.02.2025 23:12 — 👍 0    🔁 0    💬 0    📌 0

The only thing that can pacify the world nowadays

27.02.2025 16:55 — 👍 0    🔁 0    💬 0    📌 0

I tried and failed to study math for ML for 3 years now, and finally now, after treatment started, I can concentrate on these complex topics and actually learn lol

27.02.2025 16:37 — 👍 0    🔁 0    💬 0    📌 0

Earlier this year I've been diagnosed with ADHD. I started medication on Monday w/ Concerta and it's been life changing.

27.02.2025 16:37 — 👍 0    🔁 0    💬 1    📌 0

What's the best `hello_world` for Bluesky?

27.02.2025 16:33 — 👍 0    🔁 0    💬 0    📌 0

@thedatasitter.com is following 19 prominent accounts