Contributing to Taskgraph

Thanks for your interest in Taskgraph! To participate in this community, please review our code of conduct.

Clone the Repo

To contribute to Taskgraph or use the debugging tools, you’ll need to clone the repository, activate a virtualenv and install dependencies:

# first fork taskgraph
git clone https://github.com/<user>/taskgraph
cd taskgraph
git remote add upstream https://github.com/taskcluster/taskgraph
python -m venv taskgraph && source taskgraph/bin/activate
pip install -r requirements/dev.txt
python setup.py develop

Running Tests

Tests are run with the pytest framework:

pytest src

We use tox to run tests across multiple versions of Python.

Running Checks

Linters and formatters are run via pre-commit. To install the hooks, run:

$ pre-commit install -t pre-commit -t commit-msg

Now checks will automatically run on every commit. If you prefer to run checks manually, you can use:

$ pre-commit run

Some of the checks we enforce include black, flake8 and yamllint. See pre-commit-config.yaml for a full list.

Working with Documentation

The Taskgraph repo uses Sphinx to generate the documentation. To work on the docs, run:

make livehtml

This will start a live server that automatically re-generates when you edit a documentation file. Alternatively you can generate static docs under the docs/_build directory:

make html

Taskgraph also uses the autodoc extension to generate an API reference. When new modules are created, be sure to add an autodoc directive for them in the source reference.

Managing Dependencies

Warning

Ensure you always update packages using the minimum supported Python. Otherwise you may break the workflow of people trying to develop Taskgraph with an older version of Python. The pyenv tool can help with managing multiple Python versions at once.

To help lock dependencies, Taskgraph uses a tool called pip-compile-multi. To add or update a dependency first edit the relevant .in file under the requirements directory. If the dependency is needed by the actual Taskgraph library, edit requirements/base.in. If it’s required by the CI system, edit requirements/test.in. And if it’s only needed for developing Taskgraph, edit requirements/dev.in.

Next run the following command from the repository root:

pip-compile-multi -g base -g test -g dev --allow-unsafe

If you’d like to add a new package without upgrading any of the existing ones, you can run:

pip-compile-multi -g base -g test -g dev --allow-unsafe --no-upgrade

Releasing

In order to release a new version of Taskgraph, you will need permission to the taskcluster-taskgraph project in PyPI. The following are required steps:

  1. Update CHANGELOG.md

  2. Update version in setup.py

  3. Commit, and land the above changes

  4. Make sure your git status is clean

  5. Checkout the latest public revision git checkout main

  6. Pull latest revision git pull upstream main

  7. Verify git show outputs the desired revision

  8. Remove previously packaged releases rm -rf ./dist/*

  9. Package the app python setup.py sdist bdist_wheel

  10. Upload to PyPI using twine twine upload dist/* providing your username and API token