Apache Airflow 3 Fundamentals: the concepts every data engineer needs
Airflow is the orchestrator I use the most. Astronomer's Fundamentals certification reviews the basics and the big changes in version 3.
by Gabriel Henrique · · 2 min read · Ler em português
I'm Gabriel Henrique, a Data Engineer, and I work with Apache Airflow on the governance automations and pipelines I maintain. In October 2025 I earned Apache Airflow 3 Fundamentals from Astronomer.
I took the exam on version 3 to review the basics and understand what changed.
The core concepts
- DAG: the workflow, a graph of tasks with no cycles.
- Task and operator: a task is a step, and the operator defines what it does.
- Scheduler: decides when each task should run.
- Executor and workers: where tasks actually run.
- Metadata database: stores the state of every run.
Scheduling without surprises
Most beginner questions come from scheduling. These are the points the exam covers and that I see cause trouble in production:
- start_date and schedule: together they define when the first run happens.
- catchup: when enabled, Airflow runs every past interval since the start. Forgetting it on a new DAG can trigger hundreds of runs.
- Logical date: each run represents a data interval, and tasks should use that date, not the current time.
What changed in Airflow 3
Airflow 3 was the biggest update in years. The points that affect my work the most:
- A new UI, rewritten from scratch.
- DAG versioning, showing which version of the code ran in each execution.
- Assets replacing Datasets, for scheduling on data updates.
- Scheduler-managed backfills, triggered from the UI or the API.
- Task SDK and remote task execution, separating task code from the Airflow core.
Connections and variables
Credentials and settings live in connections and variables, not in the DAG code. In a corporate environment, the right move is to back them with a secrets manager, so no password ever shows up in the repository.
How to study
Astronomer offers the exam material for free at Astronomer Academy. Running Airflow locally and breaking things on purpose, like enabling catchup or changing the start_date, teaches more than rereading the theory.
Next step
The natural follow-up is Airflow 3 DAG Authoring, which I took the following month and which covers how to write production-ready DAGs.
See my other projects on the portfolio of Gabriel Henrique, Data Engineer. I'm Gabriel Henrique, and I write about orchestration and data engineering.