We may be college educated but we’re always self taught

2. Old boring tech will teach you the fundamentals which you will never need re-learn.

3. Once you get a job, you can explore new technologies for fun

Learn things that don’t change!

1. 99.5% of companies are still using old, boring technologies like MSSQL and SSIS. This is not just in foreign countries, this is today here in the US.

Recently was talking with someone looking to get a job as a data engineer. Unfortunately he was learning about LLMs and other ML stuff so I had to set him straight 😁

The second one is not as easily solved. If they distrust data in favor of their intuition they've probably been burned in the past or they fundamentally believe their intuition is superior.

The first one is easily solved. Upon learning a solid methodology, they apply it regularly, get smarter and improve their area of the business.

There are two reasons most managers don't use data to improve the business:
1. They lack a solid methodology
2. They have a fundamental distrust of data

Gas grill?

Not to burst your bubble but that’s probably either a bot or intern 😀

Apparently I’m not the only one who thinks this way. AE needs to be expanded

Everyone should do the same. Why build an expensive API when you can just download a single parquet file.

Just like a map is the representation of the territory, data is the representation of business as it currently stands.

And just like a map data works best when it provides guidance and options on how to proceed.

For example the number of visits to a product page in the last week.
This is basically feature engineering and it’s one of the most powerful ways to finding high leverage insights.

Segmenting by customer attributes is even more powerful. These could be behavioral, temporal or demographic attributes.
Behavioral and temporal attributes don’t exist a-priori which means they’d have to be created.

Now imagine you segment this month’s sales by product:
- Product A sold $5000
- Product B sold $2000
- Products C and D sold $1500 each

Right away we have some incredible insights! They’re not actionable yet but we could either try to figure out why the other products didn’t sell or why A sold.

Suppose you run an ecommerce store that sells 10 products. Last month you sold $8,000 and this month $10,000. Say the growth is within your range of goals.

All is good right?

The most powerful analytics pattern, capable of producing incredible insights, is segmentation.
There are two forms of segmentation:
1. By product attributes
2. By customer attributes

JSON

I mean my gripe is mainly with dbt making interesting things hard

I hate jinja and yaml. That shit can go to hell

Typical data storytelling:
- Look at some data and make up a story
- Choose the data that fits the story

Proper data storytelling:
- Make up a theory and validate it with data
- Update theory until it fits data

When I finally grasped statistics…

I think executives are intimidated by data.

They’re expected to know how to use data but nobody showed them how to do it properly.

Using data properly starts by first determining what the goals or objectives are.

Next you need to map the goals to a metric that facilitates fast feedback loops.

It’s the same thing as writing code. The skill moves from output generation to output judgment. Those who know how to write (prose or code) will be able to judge good output from bad output.

Is there a platform you’re NOT on?? 😆

How The Los Angeles Dodgers Used Analytics To Become MLB's Most Feared Team The Dodgers went from being sold out of bankruptcy court to the league's second-most valuable team — and it's all because of a decade-long investment in analytics.

How the Dodgers used analytics to win the World Series (albeit there not a lot about analytics)
open.substack.com/pub/huddleup...

Ah yes I have read this one

That’s a given actually! Cedric is a good friend.

No, have link?

I’m starting to get curious about high level questions like “what do highly effective data driven organizations look like?” Any good resources (books, articles, papers) I can dive into?