package tries to install all tf-nightly versions for each and everyday for this year!

[Cagri Demir]

I want to do some experiments on the project so I'm trying to install the package into a virtual environment.

It tries to install tf-nightly builds for each and every day this year (like 100 versions of the same package). Given that each nightly build for tf is around 450mb this quickly eats up my limited storage and only after it fills that 50gb, it stops downloading tf nightly builds and finishes up the installation that github package.

[Jordy Van Landeghem]

I have the same issue! The command I used:

pip install "git+https://github.com/google/uncertainty-baselines.git#egg=uncertainty_baselines[models]"

Collecting tf-nightly
Downloading tf_nightly-2.6.0.dev20210505-cp38-cp38-manylinux2010_x86_64.whl (453.0 MB)
|████████████████████████████████| 453.0 MB 7.6 kB/s
Downloading tf_nightly-2.6.0.dev20210504-cp38-cp38-manylinux2010_x86_64.whl (453.0 MB)
|████████████████████████████████| 453.0 MB 42 kB/s
Downloading tf_nightly-2.6.0.dev20210503-cp38-cp38-manylinux2010_x86_64.whl (456.9 MB)
|████████████████████████████████| 456.9 MB 40 kB/s
Downloading tf_nightly-2.6.0.dev20210502-cp38-cp38-manylinux2010_x86_64.whl (456.9 MB)
|████████████████████████████████| 456.9 MB 45.0 MB/s
Downloading tf_nightly-2.6.0.dev20210501-cp38-cp38-manylinux2010_x86_64.whl (456.9 MB)
|████████████████████████████████| 456.9 MB 49 bytes/s
Downloading tf_nightly-2.6.0.dev20210430-cp38-cp38-manylinux2010_x86_64.whl (456.9 MB)
|████████████████████████████████| 456.9 MB 30.2 MB/s
Downloading tf_nightly-2.6.0.dev20210429-cp38-cp38-manylinux2010_x86_64.whl (456.8 MB)
|████████████████████▏ | 287.8 MB 29.4 MB/s eta 0:00:06^C
ERROR: Operation cancelled by user

[Cagri Demir]

I tried to use conda and never ending installation tf nightly builds stopped. But a new one emerged. The exact steps I followed are:

1. create a new environment with conda
2. activate it and install pip and git using conda
3. install uncertainty-baselines using pip (after activating new conda environment)
4. installation of uncertainty_baselines didn't handled required packages well, so I installed uncertainty-metrics, edward2, and tensorflow-addons explicitly using pip and conda
5. After that the error I got is

Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0

6. After a quick internet search, found that some people overcame the issue by installing tf-gpu, so I installed it with:

conda install tensorflow-gpu cudatoolkit=10.1

7. After this importing uncertainty-baselines outputted this issue (the end of a long referencing error):

ImportError: cannot import name 'LayerNormalization' from 'tensorflow.python.keras.layers.normalization'

I tried different things along the way (like installing keras explicitly and re-installing uncertainty-baselines again and again) but, hey, no help. I think google needs a better package maintenance if the team really wants to make this package public and their research re-producible!

[Jordy Van Landeghem]

@cagdemir I tried similar things!

What worked for me to solve the latter issue is to install tensorflow 2.4.1, followed by tf-nightly, since there LayerNormalization is properly referenced.

I still cannot run sngp.py properly, I am afraid there is too little documentation on which exact FLAGS to run...

[Dustin Tran]

Thanks for raising this! That is a really strange behavior..

Does this issue also pop up if you install tf-nightly by itself? What about tf-models-nightly?

@Jordy-VL: I still cannot run sngp.py properly, I am afraid there is too little documentation on which exact FLAGS to run...

The default flags work to exactly reproduce the table results. To help with this, I added more guidance to the README.md (see the "Launching with TPUs." section).

We're also working with the Google Research Platforms team to open-source the launcher to start jobs with the associated GPU/TPU. This is a WIP, but the aim is that we can provide the exact snippets to launch every job to reproduce results---let alone the flags, TensorBoards, and checkpoints.

[Zachary Nado]

IIUC this was due to some bizarre pip dependency resolution, I believe it should be fixed at head now (I believe removing the gast dep from setup.py fixed it).

I tested on a fresh local virtualenv and it seemed to work, let me know if this is still the issue for you after pulling and reinstalling and I'll reopen and investigate.

[Jordy Van Landeghem]

Hi @znado,

Thanks, that did the trick. I used the command as above with [models] in a fresh environment.
However, when I run any of the baselines, I get an ImportError from robustness_metrics:

  File "sngp.py", line 41, in <module>
    import robustness_metrics as rm
  File "/home/jordy/.virtualenvs/test/lib/python3.6/site-packages/robustness_metrics/__init__.py", line 20, in <module>
    from robustness_metrics import metrics
  File "/home/jordy/.virtualenvs/test/lib/python3.6/site-packages/robustness_metrics/metrics/__init__.py", line 40, in <module>
    from robustness_metrics.metrics import serialization
ImportError: cannot import name 'serialization'

Probably something to check to ensure a stable build.

[Zachary Nado]

Coordinated with the robustness metrics devs, they did another push to fix it, hopefully it should work now (theyre also going to add tfp-nightly as a dep in their setup.py but as long as you have tensorflow_probability installed, which our setup.py does, you should be set, at least my local tests worked).