Migration Guide#

This page can help when migrating Taskgraph across major versions.

22.x -> 23.x#

taskgraph.parameters.extend_parameters_schema() now requires a msgspec Schema subclass instead of a voluptuous schema dict. Convert any custom parameter schemas to msgspec.

21.x -> 22.x#

If you have tasks using docker-worker’s relengapi-proxy feature, it is no longer supported. Remove all references to it in your task definitions.
Remove references to <REPO>_EXTRA_REFS in all run-task based tasks. These refs must now be fetched as part of the task payload.

20.x -> 21.x#

If you have tasks using docker-worker’s dind, privileged or loopback-audio features, they are no longer supported. Either avoid using these features or migrate to a generic-worker pool with D2G payloads enabled.
Ensure any task’s using the run-task or decision images work with the new Debian 13 base.

19.x -> 20.x#

Binaries belonging to packages that get installed by run-task as part of the PIP_REQUIREMENTS variable, are now installed to ~/.local/bin instead of ~/.local/lib/<python>/site-packages/bin. Update any task commands that hardcode the binary path accordingly.

18.x -> 19.x#

The Schema class in taskgraph.util.schema is now based on msgspec instead of voluptuous. The old voluptuous-based class has been renamed to LegacySchema. Both will continue to be supported.

If you have custom schemas using Schema, you have two options:

Option 1: Switch to LegacySchema to keep using voluptuous:
```
from taskgraph.util.schema import LegacySchema
```
Option 2: Migrate to the new msgspec-based Schema:
```
from taskgraph.util.schema import Schema

class MySchema(Schema):
    foo: str
    bar: int = 10
```
The new Schema class uses kebab-case renaming and forbids unknown fields by default. See the msgspec documentation for more details.
validate_schema now supports both voluptuous and msgspec schemas, and optionally_keyed_by accepts a use_msgspec=True flag for msgspec output.

17.x -> 18.x#

Stop setting the run.sparse-profile key in all tasks which perform a Mercurial clone. If sparse profiles are still required, the task must perform its own clone and not rely on the run-task script to do it.
run-task no longer cleans up the fetches directory. If using generic-worker’s multiuser engine, this should have no impact. If you are using a worker that does not create separate task workspaces and you require the fetches dir to be cleaned up, adjust your tasks’ commands to remove $MOZ_FETCHES_DIR explicitly.

16.x -> 17.x#

Upgrade to Python 3.9 or above.
The base_ref parameter could now be None if it wasn’t explicitly passed into the Decision task. If this was relied upon, do the following instead:
```
from taskgraph.util.vcs import get_repository

repo = get_repository()
base_ref = repo.default_branch
```
The base_rev parameter is no longer being reset to the merge-base. This means in some cases, base_rev might not actually be an ancestor of head_rev. If you need the merge-base, you can use:
```
from taskgraph.util.vcs import get_repository

repo = get_repository()
base_rev = repo.find_latest_common_revision(parameters["base_rev"], parameters["head_rev"])
```
The base_ref is no longer being fetched by run-task. If you were relying on it, you can run git fetch origin <base_ref> in a subprocess.
The run-task scripts no longer fetches all heads when head_ref isn’t specified. If this behaviour was relied upon, you can run git fetch +refs/heads/*:refs/remotes/work/* in a subprocess.

15.x -> 16.x#

Stop using the --tag flag with taskgraph build-image. Images must now be tagged manually via Docker post build.
Running taskgraph load-task will no longer pause task execution and instead immediately execute the task’s command after loading. Now you must use the -i/--interactive flag to pause execution.
Utility functions in taskgraph.util.taskcluster no longer accept the use_proxy argument. Instead the presence of TASKCLUSTER_PROXY_URL in the environment will be used to determine whether or not to use the Taskcluster proxy. Remove use_proxy from all function calls, and then either set TASKCLUSTER_PROXY_URL or unset it depending on whether using the proxy is desired or not.
Utility functions in taskgraph.util.taskcluster now raise taskcluster.TaskclusterRestFailure instead of requests.HTTPError.

14.x -> 15.x#

get_primary_dependency now requires a primary-dependency-label to be set on the task it is passed instead of primary-kind-dependency. Update any tasks you were calling this for to set this attribute. (If your only usage of this is indirectly through from_deps, no change is needed: from-deps generated tasks will set this for you).

13.x -> 14.x#

The {task_workdir} string in environment variables no longer gets interpolated by run-task. This is backing out a feature introduced in version 13.x, so this release introduces no new backwards incompatibilities with version 12.x or earlier.

12.x -> 13.x#

Remove all run.cache-dotcache keys. If it was set to true, replace it with:
```
run:
  use-caches: [checkout, <caches>]
```
Where caches can be any of cargo, pip, uv or npm. If the task was setting up .cache for another tool, a mount will need to be created for it manually. If use-caches was previously set to false, omit checkout in the example above. If use-caches was previously set to true, replace true with the value above (including checkout).

In Dockerfiles, replace VOLUME /builds/worker/.cache by VOLUME /builds/worker/.task-cache/{uv, cargo, pip, npm} as necessary.
Invert any usage of the dict keys and values returned by get_ancestors:

For example, if you were using:
```
for label, taskid in get_ancestors(...):
  ...
```
Change it to:
```
for taskid, label in get_ancestors(...):
  ...
```
Note that due to this change get_ancestors may return multiple tasks with the same label now, which your code may need to deal with.

11.x -> 12.x#

Add target_tasks_method to the front of the filters parameter wherever you are also using a custom filter.

For example, if you are passing in:
```
filters: ["my_custom_filter"]
```
Change it to:
```
filters: ["target_tasks_method", "my_custom_filter"]
```
No action is necessary if the filters parameter was empty.
Change all references from build-docker-image to docker-image.

10.x -> 11.x#

A hardcoded path to a Python installation was removed for MacOS generic-workers. If Mac tasks start failing after upgrade and you are able to change the worker environment, ensure the python3 binary is available on the $PATH. If you cannot change the worker environment, add the following to the definitions of the failing tasks:
```
run:
  run-task-command: ["/tools/python36/bin/python3", "run-task"]
```
When defining custom actions with taskgraph.actions.registry.register_callback_action, make the following changes:
- If you are passing generic=True to the function, remove this argument.
- If the argument isn’t present, or you are passing generic=False, then add a new argument called permission=<cb_name>, where <cb_name> is the value of whatever you are passing to the cb_name argument.

9.x -> 10.x#

Directories listed as VOLUME in Dockerfiles are created before any other instructions, so those instructions may need to be updated (e.g. RUN mkdir)
fetch-content no longer relies on file extension to detect archives, so you. may need to explicitly disable extract for some fetches.

8.x -> 9.x#

Replace references to taskgraph.util.memoize.memoize with functools.cache. E.g, change @memoize to @cache. If using Python 3.8, use @functools.lru_cache(maxsize=None) instead.
Pay close attention to tasks that use task-defaults to merge configuration containing by-<attribute> keys. The taskgraph.util.templates.merge() function will no longer attempt to merge keys containing these attributes, which may result in changes to your graph. You can use the diff feature to help detect possible changes.

7.x -> 8.x#

Replace all references to taskgraph.files_changed. Instead, use one of:
- The files_changed parameter
- The get_files_changed method on an instance of taskgraph.util.vcs.Repository
- Mercurial repositories relying on hgmo’s json-automationrelevance endpoint will need to in-line this logic into their own custom Taskgraph logic
In tasks using the from_deps transforms, remove from-deps.set-name if it is set to true
Update any references to pull request cached task indexes from {cache_prefix}.cache.head.{head_ref}... to {cache_prefix}.cache.pr... (i.e, add pr and remove the head.{head_ref})

6.x -> 7.x#

Upgrade to Python 3.8 or higher
Ensure root_dir now points to taskcluster instead of taskcluster/ci. Typically this value is not passed in explicitly by consumers, but updates are likely required if you have custom code that uses any of the following objects:
- taskgraph.config.GraphConfig
- taskgraph.config.load_graph_config
- taskgraph.generator.TaskGraphGenerator
- taskgraph.generator.load_tasks_for_kinds
- The -r/--root flag on the taskgraph binary
Rename the run_job_using decorator to run_task_using
Move config.yml from taskcluster/ci to taskcluster
Rename the taskcluster/ci directory to taskcluster/kinds
Replace references to taskgraph.transforms.job with taskgraph.transforms.run
Replace references to taskgraph.transforms.release_notifications with taskgraph.transforms.notify
Replace references to taskgraph.target_tasks._target_task with taskgraph.target_tasks.register_target_task
Stop using or inline taskgraph.util.decision.make_decision_task
Stop using the decision-mobile docker image
Ensure MacOS workers that need Mercurial have hg on their PATH

5.x -> 6.x#

Replace all uses of command-context with the more generalized task-context

4.x -> 5.x#

Upgrade to Python 3.7 or higher

3.x -> 4.x#

Remove all uses of the disable-seccomp key in the worker section of task definitions.

2.x -> 3.x#

Use a decision image at least as recent as this one.
Rename config.kind_dependencies_tasks to config.kind_dependencies_tasks.values().
Rename vcs.head_ref to vcs.head_rev. vcs.head_ref still exists but points to the actual reference instead of the revision.
Rename vcs.base_ref to vcs.base_rev. Same rationale as above.

1.x -> 2.x#

For all kinds using the transform loader, rename the following keys in both the kind.yml file and any files referenced in jobs-from:
```
jobs -> tasks
jobs-from -> tasks-from
job-defaults -> task-defaults
```
Rename taskgraph.util.schema.WHITELISTED_SCHEMA_IDENTIFIERS to taskgraph.util.schema.EXCEPTED_SCHEMA_IDENTIFIERS.
Rename any instances of taskgraph.optimize.Either to taskgraph.optimize.Any.
Add a deadline parameter as the third argument to any custom optimization strategies’ should_replace_task() function. For migration purposes it doesn’t need to be used.
Replace taskgraph.util.taskcluster.status_task with taskgraph.util.taskcluster.state_task.

Migration Guide

Contents