pangeo-forge / pangeo-forge-recipes

Python library for building Pangeo Forge recipes.

Home Page:https://pangeo-forge.readthedocs.io/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Speed up unit test runs

cisaacstern opened this issue · comments

Working on #603 just now, I noted that our unit tests take a long time to run. It would be good to speed this up, as the slow running tests do impede development iteration cycles to some degree.

Some areas that could potentially speed things up:

  • Slim down test envs (though the current caching does provide some performance here). This could also entail simply removing the ci/py3*.yaml files in favor of various (smaller) sets of pip-installable optional dependencies, which would have a benefit for the realism with which our test environment replicates user environments. Note reverting #603 may be part of this work if the issue linked there is resolved (or if we don't use conda).
    • Edit: Done in #605
  • Reduce fixture redundancy. I'm to blame for this one. A while ago, a fixtures refactor I did introduced a fair amount of grouping/lazy-fixturing in conftest such that a lot of tests that only need to run once, actually end up running in multiple parametrizations.
  • (Re)consider role http server plays in tests. We start a real (local) http server in the tests to serve files back to ourselves. Not sure how much performance we'd gain by using this more sparingly, but might be worth considering.

Yeah I was super surprised by the complex conda setups here - IMO they add a lot of complexity and slow everything down. We can and should actually go pip only - once pangeo-forge/pangeo-forge-runner#90 lands, there's no conda in the default image. I also don't use conda locally for anything, and haven't had problems with running the tests. There's also well developed pip caching for github actions supported upstream.