Scheduler

Scheduler {#scheduler configuration}

To run DAGs automatically, you need to run the dagu scheduler process on your system.

Cron Expression

You can specify the schedule with cron expression in the schedule field in the config file as follows.

schedule: "5 4 * * *" # Run at 04:05.
steps:
  - name: scheduled job
    command: job.sh

Or you can set multiple schedules.

schedule:
  - "30 7 * * *" # Run at 7:30
  - "0 20 * * *" # Also run at 20:00
steps:
  - name: scheduled job
    command: job.sh

Stop Schedule

If you want to start and stop a long-running process on a fixed schedule, you can define start and stop times as follows. At the stop time, each step's process receives a stop signal.

schedule:
  start: "0 8 * * *" # starts at 8:00
  stop: "0 13 * * *" # stops at 13:00
steps:
  - name: scheduled job
    command: job.sh

You can also set multiple start/stop schedules. In the following example, the process will run from 0:00-5:00 and 12:00-17:00.

schedule:
start:
  - "0 0 * * *"
  - "12 0 * * *"
stop:
  - "5 0 * * *"
  - "17 0 * * *"
steps:
  - name: some long-process
    command: main.sh

Restart Schedule

If you want to restart a DAG process on a fixed schedule, the restart field is also available. At the restart time, the DAG execution will be stopped and restarted again.

schedule:
  start: "0 8 * * *"    # starts at 8:00
  restart: "0 12 * * *" # restarts at 12:00
  stop: "0 13 * * *"    # stops at 13:00
steps:
  - name: scheduled job
    command: job.sh

The wait time after the job is stopped before restart can be configured in the DAG definition as follows. The default value is 0 (zero).

restartWaitSec: 60 # Wait 60s after the process is stopped, then restart the DAG.
steps:
  - name: step1
    command: python some_app.py

Run Scheduler as a Daemon

The easiest way to make sure the process is always running on your system is to create the script below and execute it every minute using cron (you don't need root account in this way).

#!/bin/bash
process="dagu scheduler"
command="/usr/bin/dagu scheduler"

if ps ax | grep -v grep | grep "$process" > /dev/null
then
    exit
else
    $command &
fi

exit

Configuration

If you need to place DAGs in a different location, set the DAGU_DAGS environment variable to specify the directory of the DAGs.