Taskgraph¶
Taskgraph is a Python library to generate DAGs of tasks for Taskcluster, Mozilla’s CI service. The nodes of the DAG represent tasks, while the edges represent the dependencies between them.
Taskgraph is designed to scale to any level of complexity. From a handful of tasks, to the over 30,000 tasks and counting that make up Firefox’s CI.
Installation¶
Taskgraph is on Pypi and can be installed via:
pip install taskcluster-taskgraph
This provides the taskgraph binary, see taskgraph --help for available
commands. To integrate Taskgraph in your project, see
Creating a Simple Task Graph.
Alternatively you can install it by cloning the repo. This is useful if you need to test against a specific revision:
git clone https://github.com/taskcluster/taskgraph
cd taskgraph
python setup.py develop
How It Works¶
Taskcluster tasks are defined in the tasks section of a .taskcluster.yml
file at the repository root. This is adequate for simple CI systems which only
have a few tasks. But as the complexity of a CI system grows, trying to fit
all task definitions into a single file becomes difficult to maintain.
Instead of defining everything in the .taskcluster.yml, Taskgraph consumers
define only a single task called the Decision Task. This task runs the
command taskgraph decision which at a very high level:
Reads a set of input YAML files defining tasks.
Runs each task definition through a set of transforms.
Submits the resulting DAG of tasks (with Decision task as the root) to Taskcluster via its REST API.
Taskgraph’s combination of static configuration with logic layered on top, allows a project’s CI to grow to arbitrary complexity.