DAG code
Airflow 3.0 is is finally going to allow us to run backfills from the UI.
At my team, we're currently using a DAG with a `BashOperator` that allows us to "run backfills from the UI".
How do the rest of you handle this?
cwiki.apache.org/confluence/d...
#airflow #dataengineering #databs
10.12.2024 09:44 — 👍 1 🔁 0 💬 0 📌 0
Reproducible bugs are candies 🍭🍬
tinkerer interested in distributed systems, compilers, and sometimes philosophy. he/him.
Researcher, advisor, writer, formal verification eng @ Confluent.
Everything data (dist sys, databases, messaging, data eng/analytics).
https://jack-vanlightly.com, https://www.hotds.dev
Credit: ESO/B. Tafresh
Editor Data Engineering Weekly; subscribe www.dataengineeringweekly.com. In Prgress, LakeByte
Co-Founder of @responsive.dev. Views are my own.
Former U.S. Chief Data Scientist
GP Great Point Ventures
Devoted Health, RelateIQ, LinkedIn
I build things
Co-founder @restate.dev: Building resilient applications made easy! Apache Flink Committer and PMC member. Previously @ververica.com
Systems engineer @turbopuffer.bsky.social. Former CTO @materialize.com.
Messing around with boats and databases.
Sometimes data systems, sometimes security, sometimes ai/ml, sometimes a blend of it all.
Building https://typedef.ai | host @ https://techontherocks.show | Done some cool stuff with trinodb | ex-RudderStack | previously CEO @ Blendo
Engineering Manager @clickhouse.com | Databases | Kubernetes | http://parallelthoughts.xyz | https://berlinsystems.xyz/ | https://cal.com/manishgill | AI Research @TomTom
Delhi raised, Berlin based.
Co-founder/CTO @fennel.ai / Databases #DataBS / Distributed Systems / Infrastructure. @bothra90 on Twitter.
Working at Datadog on Go Profiling, Performance, OpenTelemetry and more.
Before:
- Manufacturing Observability at Apple
- Co-founder at Transloadit
- Node.js Core Dev
Staff Engineer at Confluent, Committer and PMC Member at Apache Flink.
Co-Founded data Artisans. Member of the Apache Software Foundation.
Looker Founder, makes things with bits.
Trying to make a better SQL with Malloy
Senior Software Engineer at Ceph, IBM
You’ll find me talking about computers, databases, fiction books, movies, music, photography, building Lego and puzzles!
🌐 shraddhaag.dev
Product Manager AWS. Golang. Distributed Systems.
Views and opinions are my own.
