General | |
Pipeline | |
Tools | |
VC | |
Github | |
No longer maintained, because I migrated to Github Actions.
First configure a github service connection
It is suggested to use a generic name, such as github so forks can also configure the same.
You can find this in Project Settings => Service connections in the Azure Devops dashboard for your project. Project settings is located in the bottom left corner of the UI as of 2020-09-02. Below I'm using the endpoint name github.
Now add this to the beginning of the `azure-pipelines.yml`:
resources:
repositories:
- repository: cielquan
type: github
endpoint: github
name: cielquan/azure-pipelines-template
ref: refs/master
This will make the templates in this repository available in the cielquan
namespace. Note the ref allows you to change the template version you want,
you can use refs/master
if you want latest unstable version or
refs/tags/<TAG>
to pin the version to an specific tag
.
Current supported python version: - 3.6 - 3.7 - 3.8 - 3.9 (all os) - changed 0.4.0 - 3.10 (only linux) - added 0.4.0
The job templates can be found in the jobs
directory.
Added in version 0.1.0.
Supported python version
Logic
This job template will run tox
for a given set of tox environments on given
os.
Features and functionality:
- each specified toxenv target maps to a single Azure Pipelines job, but split over multiple architectures via the image matrix (each matrix will set the image_name variable to macOs, linux or windows depending on the image used)
- make tox available in the job: provision a python (3.8) and install a specified tox into that
- provision a python needed for the target tox environment
- provision the target tox environment (create environment, install dependencies)
- invoke the tox target
- if a junit file is found under .toxjunit.{toxenv}.xml upload it as test report
- if coverage is requested, run a tox target that should generate the .toxcoverage.xml or .tox.coverage and upload those as a build artifact (also enqueue a job after all these job succeed to merge the generated coverage reports)
- if coverage was requested queue a job that post all toxenv runs will merge all the coverages via a tox target
- if set also sends coverage data to
code climate
Steps
This template will:
- build a test matrix for the given os and architectures (win only)
- install a python interpreter for tox only
- install tox
- install the test target python version
- run global custom setup scripts
- run job custom setup scripts
- print PATH
- print python interpreter info
- print tox environments
- generate tox environments
- run tox environments
- upload junit file from
.tox/junit.{toxenv}.xml
(if present) - upload coverage artifacts from
.tox/*
(if present) - run job custom tear down scripts
- run global custom tear down scripts
After the steps above the template will also add a job to merge the uploaded
coverage artifacts and publish them with Cobertura
.
Parameters
The following parameters can be set at root level:
dependsOn
: List of jobs everytox_env
should depend on. Can be overwritten for singletox_env
.default_python
: Version of python to use when non is detectable or set (default: 3.8).additional_variables
: Mapping of additional variables to set for alltox_env
. Can be overwritten for singletox_env
.checkout_submodules
: If submodules should also be checked out when the repository gets checked out (default: true).tox_version
: Pin thetox
version to use (default: newest available).tox_python
: Python version to use for tox (default: 3.8).pre_test
: List of steps to run before invoking everytox_env
.post_test
: List of steps to run after running everytox_env
.tox_envs
: Mapping oftox
environments (key) and their parameters (values) to run:display_name
: Name to use for the job instead of thetox_env
name. Must be set fortox_env
with a '-' in their name, because dashes are not allowed in job names.dependsOn
: List of jobs thistox_env
should depend on. Overwrites the globaldependsOn
.os
: List of os to run thistox_env
on:linux
/lin
- ubuntu-latestwindows
/win
- windows-latestmacos
/osx
- macOS-latest- not set - fallback to ubuntu-latest.
architectures
: List of architectures to run thistox_env
on:x64
- defaultx86
- only available forwindows
.
py_version
: determines python version to use for thistox_env
, if not set will be derived from the key or fallback todefault_python
:py36
or starts withpy36-
- Python 3.6py37
or starts withpy37-
- Python 3.7py38
or starts withpy38-
- Python 3.8py39
or starts withpy39-
- Python 3.9py310
or starts withpy310-
- Python 3.10 latest pre-release (only available on linux -- it is installed from deadsnakespypy3
or starts withpypy3-
- PyPy 3
additional_variables
: Mapping of additional variables to set for thistox_env
. Overwrites the globaladditional_variables
.pre_test
: List of steps to run before thistox_env
. Runs after the globalpre_test
.post_test
: List of steps to run after thistox_env
. Runs before the globalpost_test
.
coverage
: List of settings used for coverage processing if set:with_toxenv
: Name of thetox_env
to do coverage collecting and normalizing with. Runs after everytox_env
infor_envs
and as a final jobreport_coverage
(with the merge-coverage.yml template) after alltox_env
runs finished to merge the coverage data.for_envs
: List oftox_env
to collect coverage data from. Referredtox_env
must generate.tox/.coverage
and.tox/coverage.xml
files
send_cov_to_codeclimate
: Boolean if covearge data should be send tocode climate
. NeedsCC_TEST_REPORTER_ID
variable to work: see notify-codeclimate.yml (default: false) Added in version 0.3.0.
Example
The following example will run the following jobs with tox
version 3.15.0
called via python 3.7:
pre_commit
on linux with python 3.7py38
on all three os and on windows also on x86py39
on linuxpypy3
on linux and macosdocs_test_html
on linux withdefault_python
version 3.6docs_test_linkcheck
on linux withdefault_python
version 3.6report_coverage
on linux withdefault_python
version 3.6 to merge the coverage data generated bypy38
,py39
andpypy3
.
use python 3.7 to call tox
in version* 3.15.0* for
jobs:
- template: jobs/run-tox.yml@cielquan
parameters:
tox_version: '3.15.0'
tox_python: '3.7'
default_python: '3.6'
tox_envs:
pre-commit:
display_name: pre_commit
py_version: '3.7'
py38:
os: [linux, windows, macOs]
architectures: [x86, x64]
py39: null
pypy3:
os: [linux, macOs]
docs-test-html:
display_name: docs_test_html
docs-test-linkcheck:
display_name: docs_test_linkcheck
coverage:
with_toxenv: 'coverage'
for_envs: [py38, py39, pypy3]
Added in version 0.2.0.
Logic
This job template will use poetry to build and publish the Python package (both sdist and wheel) to PyPI or a custom repository.
Parameters
The following parameters can be set at root level:
python_version
: Python version to use (default: 3.8).dependsOn
: List of jobs this job should depend on.custom_repository
: Boolean for using a custom repository over PyPI (default: false)
Pipeline variables
For this job to work credentials for the target repository are needed. They are served via Pipeline Variables, which you have to set in the pipelines Web-UI settings (see here for help).
If you want to publish to PyPI (which is the default) you have to set either:
POETRY_PYPI_TOKEN_PYPI
as a secret variable
or
POETRY_HTTP_BASIC_PYPI_USERNAME
as a non-secret variable andPOETRY_HTTP_BASIC_PYPI_PASSWORD
as a secret variable
If you want to publish to a custom repository you have to set:
POETRY_REPOSITORIES_CUSTOM_URL
as a non-secret variable
and for the credentials you have to set (similar to PyPI) either:
POETRY_PYPI_TOKEN_CUSTOM
as a secret variable
or
POETRY_HTTP_BASIC_CUSTOM_USERNAME
as a non-secret variable andPOETRY_HTTP_BASIC_CUSTOM_PASSWORD
as a secret variable
NOTE:
Currently there are issues with the token variables not being recognized by
poetry as is should. As a workaround for PyPI and
TestPyPI you can set the username to __token__
and the password to the token including the pypi-
at the beginning.
Example
This example builds and publishes the package to PyPI.org after the jobs
report_coverage
, pre_commit
and docs
ran successfully.
- ${{ if startsWith(variables['Build.SourceBranch'], 'refs/tags/') }}:
- template: jobs/publish-pypi-poetry.yml@cielquan
parameters:
dependsOn: [report_coverage, pre_commit, docs]
Added in version 0.3.0.
Logic
This job template will install the code climate
reporter tool and notify
code climate
about a new build.
Parameters
The following parameters can be set at root level:
send_cov_to_codeclimate
: Boolean ifcode climate
should be notified. (default: false)
Pipeline variables
For this job to work a test reporter id from code climate
is needed
(see here for help).
It is served via Pipeline Variables, which you have to set in the pipelines Web-UI settings
(see here for help).
Set a variable called CC_TEST_REPORTER_ID
with the id from code climate
.
Inspired by:
- https://github.com/tox-dev/azure-pipelines-template
- https://github.com/asottile/azure-pipeline-templates
No active maintenance is intended for this project. You may leave an issue if you have a questions, bug report or feature request, but I cannot promise a quick response time.