omg literally my thoughts on paper. can we please do this?? how can I help
07.04.2025 17:16 โ ๐ 5 ๐ 0 ๐ฌ 0 ๐ 0@iamrami.bsky.social
Open source developer and content creator โจ Simplifying data analysis and visualization ๐ Try it out: http://dataline.app ๐
omg literally my thoughts on paper. can we please do this?? how can I help
07.04.2025 17:16 โ ๐ 5 ๐ 0 ๐ฌ 0 ๐ 0Have to say, I'm really happy for them with how much their Github contributors have grown.
Now that Airflow v3 is language agnostic, they can also get contributions for different SDKs which will get SO MANY more contributions. This wasn't just an architectural unlock!
โจ DAG Versioning
Straightforward: Now possible to view historical versions of DAGs along with their run information. Much better visibility, prevents loss of information for debugging/auditing
โจ Modernized UI
Not just a facelift, now uses React, Typescript, Chakra UI, Neutrino, React Query.
โฌ๏ธ
โจ Run workers "in any language":
- Airflow 3 is language agnostic
- Current language SDKs available: Python
- Upcoming support: Probably Golang
- This is now possible thanks to the new Task Execution Interface!
โฌ๏ธ
โจ Run workers "anytime" means:
- Event based triggers were already kinda there, but now can also trigger automatically based on external resource changes, no user setup needed
- Ad-hoc executions
- Process data in partitions, independently (ex. time or segment partitions) โฌ๏ธ
โจ Run workers "anywhere" means:
- Workers can be upgraded independently! (this is such a pain point in all task exec frameworks)
- Flexible deployment (public, hybric, private cloud, on-prem, ...)
- Better task isolation == security (Task exec interface architecture) โฌ๏ธ
๐ ๐ฅ Just heard about Airflow 3.0 coming end of month! I decided to take some time to write this and help by sharing.
To keep this short, picking the top features/improvements:
- Run workers anywhere, anytime, any lang
- DAG versioning
- Modernized UI
- Stronger security
โฌ๏ธ
If youโre into AI, I can highly recommend checking out AI Builders Amsterdam.
I've had the pleasure of speaking about Dataline there last week, and it was very motivating seeing such an active AI community. Not just saying that!
youtu.be/fV7AgrtfvGo?...
I've resisted the uv hype as long as I could, only to realize it was actually warranted...
Idk how docs.astral.sh/uv actually works but I'm stunned @crmarsh.com
7 MINUTES off my average build times? 1 min would have been impressive, 7 is just crazy.
Thanks for the PR @anthonymalkoun.com ๐
GIF of DataLine, a tool that allows exploring databases and plotting charts purely using plain text with a focus on data privacy.
983 stars ๐ซฃ
So close to getting my first thousand-starred Github repo ๐ฅน
What is it?
-> Plain text database exploration and chart generation with a focus on data privacy and security.
github.com/RamiAwar/dat...
Only works if your standing desk is low enough and then your monitor arm is high enough.
Then your elbows are above the wrist, and the arm can go up and down while the wrists stay stable.
It's a pretty delicate balance I'd say.
GIF of DataLine, a tool that allows exploring databases and plotting charts purely using plain text with a focus on data privacy.
983 stars ๐ซฃ
So close to getting my first thousand-starred Github repo ๐ฅน
What is it?
-> Plain text database exploration and chart generation with a focus on data privacy and security.
github.com/RamiAwar/dat...
We came back from vacation to a great feature article that breaks down DataLine even better than we could have!
If you're thinking about using AI to answer data questions securely in your business, give this a quick read:
neuralnotebook.substack.com/p/dataline-a...
Anyone whoโs worked with Amnesty International knows theyโre extremely cautious about using the word genocide. Many criticize them for it, but it greatly magnifies the significance of statements like this. amnesty.ca/gazagenocide
05.12.2024 01:57 โ ๐ 18 ๐ 5 ๐ฌ 1 ๐ 0Although it's adopted, I love my other open source project as much as I do my own.
I think most devs should use a CLI snippet manager, especially more mature ones with greater responsibilities and 99 commands to keep track of!
v1 out, give it a go ๐
https://buff.ly/3CXNZam
Cause it was never marketed๐
but I intend to change that!
I was lucky to have found it a long time ago randomly tbh, never would've thought I'd be the maintainer one day ๐
Although it's adopted, I love my other open source project as much as I do my own.
I think most devs should use a CLI snippet manager, especially more mature ones with greater responsibilities and 99 commands to keep track of!
v1 out, give it a go ๐
https://buff.ly/3CXNZam
Good month, even though I've been totally AFK.
SEO is important kids.
Kinda late but we finally did it.
Chart export โ
Table export as CSV โ
Works for massive results too! This builds the CSV on the fly from a stream (no in-mem sadness). Code is open source, link in bio.
Now you can just make your DB queries here and send that CSV to the team ๐
What a difference a model makes.
Even among #openai models, 3.5-turbo still the best performing weirdly enough.
Trying to use DataLine (dataline.app) with open source language models (qwen2.5 7b) be like:
Very interesting infra too. I want to dive into this whole 'data ownership' / 1 SQLite file per person design.
Imagine every software out there gave people control of their data like this.
I want any software I build in the future to offer that.
Every time I rant about this to friends they'd be like "aaah but bluetooth!"
Still not over it... My old QC headphones still far exceed any of the expensive wireless crap out there