The CI/CD has very sensitive dependencies on the versions of the tools being used for checking the code

Question

The CI/CD has very sensitive dependencies on the versions of the tools being used for checking the code

sjswerdloff opened this issue a year ago · comments

Describe the bug

When I create a pull request, and I have dutifully used black, pytest, mypy, built the docs, etc, I still get errors in the CI/CD.
There are no explicit requirements (in requirements.txt or pyproject.toml) for versions of these tools, and it's a bit cumbersome to expect that those dev-tools will all be completely up to date at the moment of creation of the pull request.
Further, if that is the expectation, then the dev tools should be in pyproject.toml (which is then not necessarily valid depending on what version of poetry one has installed) or in the requirements.txt (or maybe a requirements-dev.txt).

It also appears that the build is broken on it's own, because of changes to these tools (Sphinx for example).

Expected behavior
I expect that either any version of the various dev tools used for checking the suitability of a PR will work, or the specific version range required will be contained in part of the definition of the project.
Given that the build is broken on its own because of changes to particular tools (e.g. Sphinx), I think pinning the dev tools to a particular major version would be best, and perhaps have separate tests (that don't shut down a PR) that test what happens if one unpins the version of each tool. @SimonBiggs has done some great work on this in pymedphys (I don't claim to grok it all).

Steps To Reproduce
Install versions of black, mypy, pytest, Sphinx that are one major version less than whatever is de-jure.
Your environment
Linux-5.15.0-73-generic-aarch64-with-glibc2.35
Python 3.9.14 (main, Sep 18 2022, 13:30:32)
[GCC 11.2.0]
pydicom 2.3.1
pynetdicom 2.1.0.dev0

And as of this moment:
my_pynetdicom_piplist.txt

mrbean-bremen · Answer 1 · Sat Jun 17 2023 14:40:42 GMT+0800 (China Standard Time)

Thanks - agreed. We had a similar problem in pydicom with mypy which kept finding new issues with every new version, and pinned the version in the end.

Having pinned versions for the PR tests, and separate tests with the newest versions that don't break the build sounds like a good compromise. Unfortunately, the maintainer of pynetdicom has not been available for quite some time now, so the project is a bit stuck. The rest of the pydicom team doesn't know that code very well (yet), so we are grateful for any help here (like your other PR).

As a fix first fix, pinning the versions would be ok in my opinion, and afterwards we can setup something like @SimonBiggs did in pymedphys (or he can do it himself, though it looks like his workload is already quite high given the projects he is working on).

Do you want to make a PR with the fixed versions that you know to work?

Simon Biggs · Answer 2 · Sat Jun 17 2023 15:53:31 GMT+0800 (China Standard Time)

As a fix first fix, pinning the versions would be ok in my opinion, and afterwards we can setup something like @SimonBiggs did in pymedphys (or he can do it himself, though it looks like his workload is already quite high given the projects he is working on).

Hi @mrbean-bremen, it shouldn't take me too long to replicate the approach I set up in PyMedPhys, given I have done battle with it before :).

I'd be more than happy to help 😊.

Maybe let me know once the tests are passing on main with a set of pinned dependencies, and then I can make a PR from there?

mrbean-bremen · Answer 3 · Sat Jun 17 2023 16:21:32 GMT+0800 (China Standard Time)

@SimonBiggs - thanks a lot! I'll wait for a response from @sjswerdloff, and either he'll make the PR, or I will.

sjswerdloff · Answer 4 · Sat Jun 17 2023 17:50:12 GMT+0800 (China Standard Time)

@SimonBiggs - thanks a lot! I'll wait for a response from @sjswerdloff, and either he'll make the PR, or I will.

I'll work with @SimonBiggs , we are used to teaming up on things.
I will work on construction of a reasonably recent set of versions that doesn't break on the current main branch

sjswerdloff · Answer 5 · Sun Jun 18 2023 05:02:15 GMT+0800 (China Standard Time)

Sphinx ext links is badly broken in 6.x and later.
They may call it a feature, but when I follow their exact example and it doesn't work, I'd call it borked.
The proximal code is in docs/conf.py where ext links are defined, e.g. :dcm:, :gh: and most importantly
:issue: and :pr:
Those are most important because rather than having None as the second part of the tuple they have something other than None (in particular they have the # symbol).
ext links is attempting to do some kind of rather clever interpolation and... not succeeding.
Plain text will fail, the escaped %% will fail, %%s will fail, an empty string will fail.
pynetdicom apparently just wanted the link to a PR or Issue to show up as
#839
Which works well in GitHub (I guess they have different programmers).
But it will not in Sphinx 6.x and later.

And now the "rock and a hard place": CircleCI does not honour the version of Sphinx specified in the pyproject.toml
So... the only approach I see working until someone figures out how to fix Sphinx ext links is for us to replace the # symbol with None in conf.py and rather than have
#839
we'll have https://github.com/pydicom/pynetdicom/issues/839

No commit or PR will pass CircleCI until some specific action is taken.
I'm sort of wondering why CircleCI is in use, given that GitHub Actions are performing essentially the same function (although I can't really be sure, because nothing passes Circle CI so... it's not clear what it does other than spend 60 seconds failing every commit)

So I'm going to make that change to conf.py, and until someone comes up with a better solution, I encourage everyone to just live with it...

There is no available historical evidence that anything from pynetdicom has ever passed CircleCI.

mrbean-bremen · Answer 6 · Sun Jun 18 2023 15:05:17 GMT+0800 (China Standard Time)

I'm sort of wondering why CircleCI is in use, given that GitHub Actions are performing essentially the same function

I think that is just the same as used in pydicom. pydicom has been using CircleCI before for CI, and switched to GH actions after they became usable and fast. As far as I remember, the documentation generation has not been switched because the documentation preview feature was not available in GH Actions, and it worked fine in CircleCI.

I wonder why it has stopped working in pynetdicom (it certainly worked before, as the documentation was generated), but not in pydicom, which essentially uses the same workflow.

There is no available historical evidence that anything from pynetdicom has ever passed CircleCI.

I disgree. The fact that the documentation exists and the links in the release notes work as expected shows that it did work, at least until the latest release (which admittedly was more than a year ago)

Given that almost the same CircleCI workflow seemed to work fine with the latest pydicom release a few days ago lets me wonder what is different here... I checked and it used sphinx 7.0, apparently without problems.

sjswerdloff · Answer 7 · Mon Jun 19 2023 04:30:35 GMT+0800 (China Standard Time)

There is no available historical evidence that anything from pynetdicom has ever passed CircleCI.

I disgree. The fact that the documentation exists and the links in the release notes work as expected shows that it did work, at least until the latest release (which admittedly was more than a year ago)

I stand corrected.
Allow me to clarify: CircleCI only shows jobs for the last few months. None of those shown showed it passing.
So there wasn't a straight forward way for me to see when things stopped working.

Given that almost the same CircleCI workflow seemed to work fine with the latest pydicom release a few days ago lets me wonder what is different here... I checked and it used sphinx 7.0, apparently without problems.

This will depend on the contents of the conf.py that a given project provides for Sphinx to make use of.

pydicom uses:

'dcm': (
        'http://dicom.nema.org/medical/dicom/current/output/chtml/%s',
        None
    ),
    'gh': (
        'https://github.com/pydicom/%s',
        None
    ),
    "issue": ("https://github.com/pydicom/pydicom/issues/%s", "#%s"),
    "pr": ("https://github.com/pydicom/pydicom/pull/%s", "#%s"),
}

while pynetdicom had been using

 "dcm": ("http://dicom.nema.org/medical/dicom/current/output/chtml/%s", None),
    "gh": ("https://github.com/pydicom/%s", None),
    "issue": ("https://github.com/pydicom/pynetdicom/issues/%s", "#"),
    "pr": ("https://github.com/pydicom/pynetdicom/pull/%s", "#"),

So... that's the key! I will change that and see how it goes. (yup, built fine with sphinx 7 in my local environment! woo-hoo!)
And I apologise for my (exasperated) tone, I was beating my head against the proverbial wall over a very long flight.

mrbean-bremen · Answer 8 · Mon Jun 19 2023 13:43:05 GMT+0800 (China Standard Time)

Great! I was to have a closer look myself at the weekend, but didn't get to it due to a combination of heat and broken internet at home...

mrbean-bremen · Answer 9 · Mon Jun 19 2023 15:27:02 GMT+0800 (China Standard Time)

And I should have known this, given that I committed the relevant change in pydicom myself... Though it was initially fixed by @darcymason.