Local Executor in Airflow

Table of Contents

1. Setup for local executor

2. Creating a python file

3. Importing the modules

4. Creating a DAG Object

5. Creating Tasks

6. Creating callable functions

7. Setting Dependencies

8. Voila, It’s a DAG File

9. Code Reference

10. Running the DAG

11. What’s Next?

Setup for Local Executor

Creating a python file

Importing the modules

from airflow import DAG
from airflow.operators.python import PythonOperator
from datetime import datetime, timedelta
import time

Creating a DAG object

with DAG(
dag_id="local_executor_demo",
start_date=datetime(2021,1,1),
schedule_interval="
@hourly",
catchup=False) as dag:

Creating Tasks

task1=PythonOperator(
task_id="hello_function",
python_callable=hello_function
)
task2_1=PythonOperator(
task_id="sleepy_1",
python_callable=sleeping_function
)
task2_2=PythonOperator(
task_id="sleepy_2",
python_callable=sleeping_function
)
task3=PythonOperator(
task_id="bye_function",
python_callable=last_function
)

Creating callable functions

def hello_function():
print('Hello, this is the first task of the DAG')
time.sleep(5)
def last_function():
print('DAG run is done.')
def sleeping_function():
print("Sleeping for 5 seconds")
time.sleep(5)

Setting dependencies

task1>>[task2_1,task2_2]>>task3

Voila, it’s a DAG file

DAG File

Code Reference

Running the DAG

Webserver Home Screen
DAG Run
Gantt Chart

What’s next?

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store