Run Taskgraph Locally#
When first starting out with Taskgraph, it’s tempting to test changes by pushing to a pull request (or try server) and checking whether your modifications have the desired effect on the impacted task(s). This isn’t ideal because the turn around time is slow, you may hit easily preventable errors in the Decision Task, and it wastes money running tasks that are irrelevant to your changes.
So before you even push your changes, it’s best practice to verify whether graph generation succeeds as well as to sanity check that your changes are having the desired effect.
Generating Graphs#
Note
If you haven’t done so already, make sure Taskgraph is installed.
Graphs can be generated via the taskgraph
binary. This binary provides
subcommands that correspond to the phases of graph generation. For instance:
Running
taskgraph full
produces thefull_task_graph
.Running
taskgraph target
produces thetarget_task_graph
.etc.
For a list of available sub-commands, see:
taskgraph --help
Useful Arguments#
Here are some useful arguments accepted by most taskgraph
subcommands. For
a full reference, see Command Line.
-J/--json
#
By default only the task labels are displayed as output, but when -J/--json
is used, the full JSON representation of all task definitions are displayed.
Note
Using -J/--json
can often result in a massive amount of output. Consider
using the --tasks
and/or --target-kind
flags in conjunction to
filter the result down to a manageable level.
--tasks/--tasks-regex
#
A regular expression that matches against task labels. Useful for filtering down the output to only display desired tasks.
-k/--target-kind
#
Only generate tasks of the given kind
or any kinds listed in that kind’s
kind-dependencies
key. Can be passed in multiple times.
-p/--parameters
#
Generate the graph with the specified parameter set. The following formats are accepted:
Path to a
parameters.yml
file.A value of
task-id=<decision task id>
. Theparameters.yml
artifact from the decision task specified by<decision task id>
will be downloaded and used.A value of
project=<project>
. Theparameters.yml
artifact from the latest decision task on<project>
will be downloaded and used.Path to a directory containing multiple parameter files. Any
.yml
file in the directory will be considered a parameter set.
The -p/--parameters
flag can be passed in multiple times. If passed in
multiple times, or if a directory containing more than one parameter set is
specified, then one graph generation per parameter set will occur in parallel.
Log generation will be disabled and output will be captured and emitted at the
end under parameter specific headings. This feature is primarily useful for
Diffing Graphs.
Note
If you do not specify parameters, the default values for each parameter will be used instead. This may result in a different graph than what is generated in CI.
--fast
#
Skip schema validation and other extraneous checks. This results in a faster generation at the expense of correctness.
Note
When using --fast
you may miss errors that will cause the decision task
to fail in CI.
Validating Your Changes#
Most changes to your Taskgraph configuration will likely fall under one of two buckets:
Modifications to the task definitions. This involves changes to the
kind.yml
files or any transform files that it references.Modifications to where the task runs. This is a subset of the above, but occurs when you modify values that affect the
target_task
phase, such as therun-on-projects
orrun-on-tasks-for
keys.
Different testing approaches are needed to validate each type.
Validating Changes to Task Definitions#
If you’re only modifying the definition of tasks, then you want to generate the
full_task_graph
. This is because task definitions are frozen (with minor
exceptions) after this phase. You’ll also want to use the -J/--json
flag and
likely also the --tasks
flag to filter down the result.
For example, let’s say you modify a task called build-android
. Then you
would run the following command:
taskgraph full -J --tasks "build-android"
Then you can inspect the resulting task definition and validate that everything is configured as you expect.
Validating Changes to Where Tasks Run#
If you’re modifying where a task runs, e.g by changing a key that impacts the
target_tasks_method
parameter (such as run-on-projects
or
run-on-tasks-for
), you’ll want to generate up until the
target_task_graph
phase.
Unlike when modifying the definition, we don’t care about the contents of the
task so passing the -J/--json
flag is unnecessary. Instead, we can simply
inspect whether the label exists or not. However it is important to make sure
we’re generating under the appropriate context(s) via the -p/--parameters
flag.
For example, let’s say you want to modify the test-sensitive
task so it
runs on pushes to the main
branch, but does not run on pull requests
(because it needs sensitive secrets you don’t want to expose to PRs). First you
would go to the main branch, find a decision task and copy it’s taskId
.
Then you would run:
taskgraph target -p task-id=<decision taskId from main push>
Now you would verify that the test-sensitive
label shows up in the
resulting output.
Next you would go to a pull request, find a decision task, and again
copy its taskId
. Then you’d again run:
taskgraph target -p task-id=<decision taskId from PR>
This time, you’d verify that the label does not show up.
Note
If there are certain parameter sets you find yourself needing over and over,
consider checking them into your repo under taskcluster/test/params
,
like the Fenix repository does. This way you can pass a path to the
appropriate parameters file rather than searching for a decision task.
Diffing Graphs#
Another strategy for testing your changes is to generate a graph with and
without your changes, and then diffing the output of the two. Taskgraph has a
built-in --diff
flag that makes this process simple. Both Mercurial and Git
are supported.
Because the --diff
flag will actually update your VCS’s current directory,
make sure you don’t have any uncommitted changes (the taskgraph
binary will
error out if you do). Then run:
taskgraph full -p <params> --diff
Taskgraph will automatically determine which revision to diff against (defaulting to your entire local stack). But you may optionally pass in a revision specifier, e.g:
# git
taskgraph full -p <params> --diff HEAD~1
# hg
taskgraph full -p <params> --diff .~1
Instead of the normal output (either labels or json), a diff will be displayed.
Note
The --diff
flag composes with every other flag on the taskgraph
binary. Meaning you can still filter using --tasks
or --target-kind
.
It can also diff any output format (labels or json).
Excluding Keys from the Diff#
Sometimes you might be making changes that impact many tasks (in the case of
Firefox’s CI, this is often thousands). You might have some expected changes
you know you made, but you want to check that there aren’t any additional
changes beyond that. You can pass in the --exclude-key
flag to filter out
certain properties of the task definition.
For example, let’s say you added an environment variable called “FOO” to every task. You now want to make sure that you didn’t make any changes beyond this, but the diff is so large this is difficult. You can run:
taskgraph full -p <params> --diff --exclude-key "task.payload.env.FOO"
This will first remove the task.payload.env.FOO
key from every task before
performing the diff. Ensuring that the only differences left over are the ones
you didn’t expect.