Giters

pylhc / omc3

Python 3 codes for beam optics measurements and corrections in circular particle accelerators

Home Page:https://pylhc.github.io/omc3/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Implement full matrix test builds (Travis / Github Actions)

fsoubelet opened this issue · comments

commented

It is time to test our package(s) on all supported python versions across major operating systems.

From personal experience:

  • Travis builds are extremely slow under a big matrix as only 2 or 3 will run concurrently
  • Travis' macOS VMs take a very long time to spin up (~10-15 min)
  • Travis support for Windows is still experimental.
  • Github Actions don't have these problems
  • Github Actions "only" offers macos-latest and windows-latest for these systems.

I started setting up github actions in the feature/github_actions branch.
This was branched off from bugs/219/madx_macros as it should soon be merged into master, but did reset the tfs-pandas dependency to >=1.0 in order for the install step to not fail

We're expecting to find bugs in the new builds, will gather findings in this issue.

Commits that were pushed previously without referencing this issue: 92bed47, 73a3cf3, 6474dba and 10cf7b0

commented

First finding:

3.8 builds seem to fail when using pandas>=1.0. Reverting to pandas==0.25.* solved it in the next builds (10cf7b0, see also the github builds and travis builds)

commented

(Only basic tests ran for these pushes btw)

commented

Second finding:

The following plotting tests fail on macOS machines (see github build):

  • tests/unit/test_amplitude_detuning.py::test_ampdet_plot
  • tests/unit/test_plot_optics_measurements.py::test_phase
  • tests/unit/test_plot_optics_measurements.py::test_phase_delta
  • tests/unit/test_plot_optics_measurements.py::test_rdt_0040
  • tests/unit/test_plot_spectrum.py::test_basic_functionality
  • tests/unit/test_plot_spectrum.py::test_combined_bpms_stem_plot
  • tests/unit/test_plot_spectrum.py::test_no_tunes_in_files_plot

I can confirm the same tests fail locally on my machine, on macOS 10.15.6.
These errors are not present in windows-latest builds.

These all fail with AttributeError: 'NoneType' object has no attribute 'points_to_pixels'. As @mihofer suggested, this is probably a backend thing. Maybe we should set a specific backend in the test files?

commented

Worth noting the documentation workflow is currently not good as it reproduces the travis-sphinx command and still needs to be modified

commented

The plotting tests failing on macOS machines is solved in commit 2e7cb3b (tests failed because of a typo that was fixed in commit c46c029).

The problem was indeed a backend / canvas issue, due to the use of matplotlib.figure.Figure instead of matplotlib.pyplot.figure. As Figure does not nicely set up a canvas for us (while figure does), the backend is also not properly setup and it's our job to do so.

I elected to simply command matplotlib to use a specific backend (chosen as the non-interactive Agg) in test files rather than changing code in the repo, as this issue only focuses on CI.

A SO post talking about the same phenomenon: https://stackoverflow.com/questions/17538235/unable-to-save-matplotlib-figure-figure-canvas-is-none

commented

As of commit d7ae2ef the Basic Tests workflow actually runs all tests so I can see everything works fine (it does).
Will now be going back to allowing pandas 1.x and try to understand what happens.

EDIT: the good (?) news is the only test failing when combining Python 3.8 and pandas 1.x is test_amplitude_detuning.py::test_bbq_plot which runs in the Basic Tests workflow so I will revert it to the normal config of python -m pytest -m "not extended" to save time on builds.

commented

As suspected from previous builds' DeprecationWarnings, the issue on Python 3.8 + pandas 1.x was that pandas stopped registering matplotlib converters automatically for datetime plotting.

The fix is to remember to manually register_matplotlib_converters() in plot_bbq, but also all plotting modules using time data in the future. I'm still confused as to why this issue only happens with Python 3.8 considering the same version of pandas is installed on 3.7 and 3.8. If someone knows, let us know.

As of commit 48cc331, this is fixed and the full matrix tests pass.

commented

one small other thing, github action is limited to 2000 minutes per month for the free account .. do we have any estimate how many PR/ basic tests we can run before we run out of those?
bc since it kinda complements travis, i'd think we could only be using it every X days and for special PRs to check that macos and windows builds also work

or .. somebody sacrifices their office computer and we use it as self hosted runner ..

commented

one small other thing, github action is limited to 2000 minutes per month for the free account .. do we have any estimate how many PR/ basic tests we can run before we run out of those?
bc since it kinda complements travis, i'd think we could only be using it every X days and for special PRs to check that macos and windows builds also work

or .. somebody sacrifices their office computer and we use it as self hosted runner ..

nvm, 2000 minutes only applies to private repos

commented

Failed ededb0d as I merged and resolved conflicts by accepting the state of test files from master, which removed the forcing of a proper backend for tests.

Fixed in 1530061

commented

Also worth writing here that through in-person discussions we have elected to get rid of the Travis CI.