@@ -50,13 +50,54 @@ a specific test case, which is useful when developing functions.
5050$ pytest array_api_tests/test_creation_functions.py::test_zeros
5151```
5252
53+ ## Releases
54+
55+ The test suite has tagged releases on
56+ [ GitHub] ( https://github.com/data-apis/array-api-tests/releases ) . If you run
57+ the test suite in your CI system, we recommend pinning against a release tag.
58+
59+ We use [ calender versioning] ( https://calver.org/ ) for the releases. You should
60+ expect that any version may be "breaking" compared to the previous one, in the
61+ sense that there may have been additional tests added which cause a previously
62+ passing library to fail.
63+
64+ For now, the test suite is
65+ not installable as a Python package. You can use it by cloning the repo and
66+ running ` pytest ` as described above. If it would help you to be able to
67+ install it as a package, [ please let us
68+ know] ( https://github.com/data-apis/array-api-tests/issues/85 ) .
69+
70+ * Test suite maintainer note:* to make a release of the test suite, make an
71+ annotated tag with the version:
72+
73+ ```
74+ git tag -a 2022.1
75+ ```
76+
77+ (for the message, just write something like "array-api-tests version 2022.1").
78+ Be sure to use the calver version number for the tag name. Versioneer will
79+ automatically set the version number of the ` array_api_tests ` package based on
80+ the git tag.
81+
82+ Then push the tag to GitHub
83+
84+ ```
85+ git push --tags origin 2022.1
86+ ```
87+
88+ Finally go to the [ tags page on
89+ GitHub] ( https://github.com/data-apis/array-api-tests/tags ) and convert the tag
90+ into a release. If you want, you can add release notes to the release page on
91+ GitHub.
92+
93+
5394## What the test suite covers
5495
5596We are interested in array libraries conforming to the
5697[ spec] ( https://data-apis.org/array-api/latest/API_specification/index.html ) .
5798Ideally this means that if a library has fully adopted the Array API, the test
58- suite passes. We take great care to _ not_ test things which are out-of-scope, so
59- as to not unexpectedly fail the suite.
99+ suite passes. We take great care to _ not_ test things which are out-of-scope,
100+ so as to not unexpectedly fail the suite.
60101
61102### Primary tests
62103
0 commit comments