How to configure the Airflow dag to execute at specified time on daily basis no matter what happens, something exactly like crons.
I know that similar behaviour could be obtained using TimeSensor, but in this case it depends upon the sensor tasks and which might conflict with the dag execution time.
Example: With sensor approach if I have sensor to run at 0 hour 15th minutes but if dag is executed at later then my task is delayed, so even for the sensor approach I need to make sure that the Dag is executed on right time.
So how to make sure that Dag is executed at specified time?
3 Answers
To start a DAG for example everyday on 2:30 AM in the morning you can do the following:
DAG( dag_id='dag_id', # start date:28-03-2017 start_date= datetime(year=2017, month=3, day=28), # run this dag at 2 hours 30 min interval from 00:00 28-03-2017 schedule_interval='30 2 * * *') Before configuring the schedule the interpretation of the cron interval can verified and tested here:
5@ruhong I see in a comment you are wondering how to do every other day. The Month is the third parameter and if you do a 2 30 */2 * * it will run every other day (at 2:30am). It calculates it a little weird sometimes depending on the month. You can force it to run even or odd days by specifying the range:
# Will only run on odd days: 2 30 1-31/2 * * command # Will only run on even days: 2 30 2-30/2 * * command You can set the schedule_interval to a string cron expression when you instantiate a DAG:
schedule_interval='0 * * * *' 
