taskgraph.transforms.run package

Submodules

taskgraph.transforms.run.common module

Common support for various task types. These functions are all named after the worker implementation they operate on, and take the same three parameters, for consistency.

taskgraph.transforms.run.common.add_artifacts(config, task, taskdesc, path)

taskgraph.transforms.run.common.add_cache(task, taskdesc, name, mount_point, skip_untrusted=False)

Adds a cache based on the worker’s implementation.

Parameters:

• task (dict) – Tasks object.

• taskdesc (dict) – Target task description to modify.

• name (str) – Name of the cache.

• mount_point (str) – Path on the host to mount the cache.

• skip_untrusted (bool) – Whether cache is used in untrusted environments (default: False). Only applies to docker-worker.

taskgraph.transforms.run.common.docker_worker_add_artifacts(config, task, taskdesc)

Adds an artifact directory to the task

taskgraph.transforms.run.common.generic_worker_add_artifacts(config, task, taskdesc)

Adds an artifact directory to the task

taskgraph.transforms.run.common.get_vcsdir_name(os)

taskgraph.transforms.run.common.should_use_cache(name: str, use_caches: bool | list[str], has_checkout: bool) → bool

taskgraph.transforms.run.common.support_caches(config: TransformConfig, task: dict[str, Any], taskdesc: dict[str, Any])

Add caches for common tools.

taskgraph.transforms.run.common.support_vcs_checkout(config, task, taskdesc, repo_configs)

Update a task with parameters to enable a VCS checkout.

This can only be used with run-task tasks, as the cache name is reserved for run-task tasks.

taskgraph.transforms.run.index_search module

This transform allows including indexed tasks from other projects in the current taskgraph. The transform takes a list of indexes, and the optimization phase will replace the task with the task from the other graph.

class taskgraph.transforms.run.index_search.IndexSearchRunSchema(using: Literal['index-search'], index_search: list[str])

Bases: Schema

index_search: list[str]

using: Literal['index-search']

taskgraph.transforms.run.index_search.fill_template(config, task, taskdesc)

taskgraph.transforms.run.index_search.run_task_schema

alias of IndexSearchRunSchema

taskgraph.transforms.run.run_task module

Support for running tasks that are invoked via the run-task script.

class taskgraph.transforms.run.run_task.RunTaskRunSchema(*, using: Literal['run-task'], command: list[str | TaskRefTypeSchema] | str | TaskRefTypeSchema, checkout: bool | dict[str, dict], workdir: str, use_caches: bool | list[Literal['cargo', 'checkout', 'npm', 'pip', 'uv']] | None = None, cwd: str | None = None, exec_with: Literal['bash', 'powershell'] | None = None, run_task_command: list | None = None, run_as_root: bool | None = None)

Bases: Schema

checkout: bool | dict[str, dict]

command: list[str | TaskRefTypeSchema] | str | TaskRefTypeSchema

cwd: str | None

exec_with: Literal['bash', 'powershell'] | None

run_as_root: bool | None

run_task_command: list | None

use_caches: bool | list[Literal['cargo', 'checkout', 'npm', 'pip', 'uv']] | None

using: Literal['run-task']

workdir: str

taskgraph.transforms.run.run_task.common_setup(config, task, taskdesc, command)

taskgraph.transforms.run.run_task.docker_worker_run_task(config, task, taskdesc)

taskgraph.transforms.run.run_task.generic_worker_run_task(config, task, taskdesc)

taskgraph.transforms.run.run_task.run_task_schema

alias of RunTaskRunSchema

taskgraph.transforms.run.run_task.script_url(config, script)

taskgraph.transforms.run.toolchain module

Support for running toolchain-building tasks via dedicated scripts

class taskgraph.transforms.run.toolchain.ToolchainRunSchema(*, using: Literal['toolchain-script'], script: str, toolchain_artifact: str, workdir: str, arguments: list[str] | None = None, resources: list[str] | None = None, toolchain_alias: str | list[str] | None = None, toolchain_env: dict[str, object] | None = None)

Bases: Schema

arguments: list[str] | None

resources: list[str] | None

script: str

toolchain_alias: str | list[str] | None

toolchain_artifact: str

toolchain_env: dict[str, object] | None

using: Literal['toolchain-script']

workdir: str

taskgraph.transforms.run.toolchain.common_toolchain(config, task, taskdesc, is_docker)

taskgraph.transforms.run.toolchain.docker_worker_toolchain(config, task, taskdesc)

taskgraph.transforms.run.toolchain.generic_worker_toolchain(config, task, taskdesc)

taskgraph.transforms.run.toolchain.get_digest_data(config, run, taskdesc)

taskgraph.transforms.run.toolchain.toolchain_run_schema

alias of ToolchainRunSchema

Module contents

Convert a run description into a task description.

Run descriptions are similar to task descriptions, but they specify how to run the task at a higher level, using a “run” field that can be interpreted by run-using handlers in taskcluster/taskgraph/transforms/run.

class taskgraph.transforms.run.AlwaysOptimizedRunSchema(using: Literal['always-optimized'])

Bases: Schema

using: Literal['always-optimized']

class taskgraph.transforms.run.FetchesEntrySchema(artifact: str, dest: str | None = None, extract: bool | None = None, verify_hash: bool | None = None)

Bases: Schema

artifact: str

dest: str | None

extract: bool | None

verify_hash: bool | None

class taskgraph.transforms.run.RunConfig(*, using: str, workdir: str | None = None)

Bases: Schema

using: str

workdir: str | None

class taskgraph.transforms.run.RunDescriptionSchema(*, run: RunConfig, name: str | None = None, label: str | None = None, description: str, worker_type: str, priority: Literal['highest', 'very-high', 'high', 'medium', 'low', 'very-low', 'lowest'] | None=None, attributes: dict[str, object] | None=None, task_from: str | None = None, dependencies: dict[str, object] | None=None, soft_dependencies: list[str] | None = None, if_dependencies: list[str] | None = None, requires: Literal['all-completed', 'all-resolved'] | None=None, deadline_after: str | None = None, expires_after: str | None = None, routes: list[str] | None = None, scopes: list[str] | None = None, tags: dict[str, str] | None=None, extra: dict[str, object] | None=None, treeherder: bool | TreeherderConfig | None = None, index: IndexSchema | None = None, run_on_projects: Any, <taskgraph.util.schema.OptionallyKeyedBy object at 0x77422b5ce660>] | None=None, run_on_tasks_for: list[str] | None = None, run_on_git_branches: list[str] | None = None, shipping_phase: Literal['build', 'promote', 'push', 'ship'] | None=None, always_target: bool | None = None, optimization: None | OptimizationTypeSchema = None, needs_sccache: bool | None = None, when: WhenConfig | None = None, fetches: dict[str, list[str | ~taskgraph.transforms.run.FetchesEntrySchema]] | None=None, worker: dict | None = None)

Bases: Schema

always_target: bool | None

attributes: dict[str, object] | None

deadline_after: str | None

dependencies: dict[str, object] | None

description: str

expires_after: str | None

extra: dict[str, object] | None

fetches: dict[str, list[str | FetchesEntrySchema]] | None

if_dependencies: list[str] | None

index: IndexSchema | None

label: str | None

name: str | None

needs_sccache: bool | None

optimization: None | OptimizationTypeSchema

priority: Literal['highest', 'very-high', 'high', 'medium', 'low', 'very-low', 'lowest'] | None

requires: Literal['all-completed', 'all-resolved'] | None

routes: list[str] | None

run: RunConfig

run_on_git_branches: list[str] | None

run_on_projects: OptionallyKeyedBy object at 0x77422b5ce660>] | None

run_on_tasks_for: list[str] | None

scopes: list[str] | None

shipping_phase: Literal['build', 'promote', 'push', 'ship'] | None

soft_dependencies: list[str] | None

tags: dict[str, str] | None

task_from: str | None

treeherder: bool | TreeherderConfig | None

when: WhenConfig | None

worker: dict | None

worker_type: str

class taskgraph.transforms.run.WhenConfig(files_changed: list[str] | None = None)

Bases: Schema

files_changed: list[str] | None

taskgraph.transforms.run.always_optimized(config, task, taskdesc)

taskgraph.transforms.run.configure_taskdesc_for_run(config, task, taskdesc, worker_implementation)

Run the appropriate function for this task against the given task description.

This will raise an appropriate error if no function exists, or if the task’s run is not valid according to the schema.

taskgraph.transforms.run.get_attribute(dict, key, attributes, attribute_name)

Get attribute_name from the given attributes dict, and if there is a corresponding value, set key in dict to that value.

taskgraph.transforms.run.make_task_description(config, tasks)

Given a build description, create a task description

taskgraph.transforms.run.rewrite_when_to_optimization(config, tasks)

taskgraph.transforms.run.run_description_schema

alias of RunDescriptionSchema

taskgraph.transforms.run.run_task_using(worker_implementation, run_using, schema=None, defaults={})

Register the decorated function as able to set up a task description for tasks with the given worker implementation and run.using property. If schema is given, the task’s run field will be verified to match it.

The decorated function should have the signature using_foo(config, task, taskdesc) and should modify the task description in-place. The skeleton of the task description is already set up, but without a payload.

taskgraph.transforms.run.set_implementation(config, tasks)

taskgraph.transforms.run.set_label(config, tasks)

taskgraph.transforms.run.use_fetches(config, tasks)