Failing to work on top of ATLAS StatAnalysis

Question

Failing to work on top of ATLAS StatAnalysis

matthewfeickert opened this issue a year ago · comments

Will Buttinger has pointed out that even though cvmfs-venv works for LCG views and AnalysisBase environments it is failing to work as expected for StatAnalysis. So something about the way StatAnalysis sets up its environment is different compared to LCG views and AnalysisBase. This would be good to guard against and support.

Minimal Failing Example

Using 759cbd0

#!/bin/bash

mkdir -p ~/.local/bin
export PATH=~/.local/bin:"${PATH}"  # If ~/.local/bin not on PATH already
curl -sL https://raw.githubusercontent.com/matthewfeickert/cvmfs-venv/759cbd066026f2209b6f541baf3f41085bfc1620/cvmfs-venv.sh -o ~/.local/bin/cvmfs-venv
chmod +x ~/.local/bin/cvmfs-venv

# Guard against this being run in a subshell
export ATLAS_LOCAL_ROOT_BASE=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase
. "${ATLAS_LOCAL_ROOT_BASE}/user/atlasLocalSetup.sh" -3  # setuptATLAS

echo "# asetup StatAnalysis,latest,master"
asetup StatAnalysis,latest,master
echo "PATH=$PATH"
echo "PYTHONPATH=$PYTHONPATH"
echo "$ python3: $(command -v python3)"
echo "# python3 --version --version"
python3 --version --version
echo "# python3 -m pip list | wc -l"
python3 -m pip list | wc -l
echo "# python3 -c 'import numpy; print(numpy)'"
python3 -c 'import numpy; print(numpy)'

echo ""
python3 -m venv failing-example
. cvmfs-venv working-example
echo "PATH=$PATH"
echo "PYTHONPATH=$PYTHONPATH"
echo "# python -m pip list"
python -m pip list
echo "# python -c 'import numpy; print(numpy)'"
python -c 'import numpy; print(numpy)'

Relevant output:

# asetup StatAnalysis,latest,master
Using StatAnalysis/0.2.0 [cmake] with platform x86_64-centos7-gcc11-opt
	at /cvmfs/atlas-nightlies.cern.ch/repo/sw/master_StatAnalysis_x86_64-centos7-gcc11-opt/2023-03-20T2101
StatAnalysis Software Versions (Please ask sw developers to submit updates):

ROOT                : v6-28-00
Python              : 3.10.6
RooFitExtensions    : v1.0.4 2023-03-14 16:11:29 +0100
xRooFit             : b8c8a621 2023-03-14 10:33:22 +0000
TRExFitter          : f3a290a7 2023-02-14 18:41:44 +0100
quickFit            :  
RooUnfold           : 924c8b60 2022-07-18 14:52:20 +0200
xmlAnaWSBuilder     : a89c1178 2023-03-15 03:36:35 +0100
workspaceCombiner   : 8c0a7050 2022-03-27 08:29:01 +0200
HistFitter          :  
BootstrapGenerator  : 71e6ebe2 2021-12-10 20:25:28 +0100
Significance        : d59b9a2 2023-03-14 12:30:24 +0000
pyhf                : 0.7.0
Unchanged: COOL_ORA_ENABLE_ADAPTIVE_OPT=Y
PATH=/tmp/feickert/.asetup-sysbin-TokIPE_4156917:/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_StatAnalysis_x86_64-centos7-gcc11-opt/2023-03-20T2101/StatAnalysis/0.2.0/InstallArea/x86_64-centos7-gcc11-opt/share:/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_StatAnalysis_x86_64-centos7-gcc11-opt/2023-03-20T2101/StatAnalysis/0.2.0/InstallArea/x86_64-centos7-gcc11-opt/bin:/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/Cmake/3.24.3/Linux-x86_64/bin:/cvmfs/sft.cern.ch/lcg/releases/gcc/11.2.0-8a51a/x86_64-centos7/bin:/cvmfs/sft.cern.ch/lcg/releases/binutils/2.37-355ed/x86_64-centos7/bin:/home/feickert/.local/bin:/home/feickert/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin:/opt/dell/srvadmin/bin
PYTHONPATH=/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_StatAnalysis_x86_64-centos7-gcc11-opt/2023-03-20T2101/StatAnalysis/0.2.0/InstallArea/x86_64-centos7-gcc11-opt/python:/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_StatAnalysis_x86_64-centos7-gcc11-opt/2023-03-20T2101/StatAnalysis/0.2.0/InstallArea/x86_64-centos7-gcc11-opt/lib
$ python3: /cvmfs/atlas-nightlies.cern.ch/repo/sw/master_StatAnalysis_x86_64-centos7-gcc11-opt/2023-03-20T2101/StatAnalysis/0.2.0/InstallArea/x86_64-centos7-gcc11-opt/bin/python3
# python3 --version --version
Python 3.10.6 (main, Mar 16 2023, 21:14:05) [GCC 11.2.0]
# python3 -m pip list | wc -l
55
# python3 -c 'import numpy; print(numpy)'
<module 'numpy' from '/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_StatAnalysis_x86_64-centos7-gcc11-opt/2023-03-20T2101/StatAnalysis/0.2.0/InstallArea/x86_64-centos7-gcc11-opt/lib/python3.10/site-packages/numpy-1.22.3-py3.10-linux-x86_64.egg/numpy/__init__.py'>

# Creating new Python virtual environment 'working-example'
PATH=/tmp/feickert/working-example/bin:/tmp/feickert/.asetup-sysbin-TokIPE_4156917:/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_StatAnalysis_x86_64-centos7-gcc11-opt/2023-03-20T2101/StatAnalysis/0.2.0/InstallArea/x86_64-centos7-gcc11-opt/share:/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_StatAnalysis_x86_64-centos7-gcc11-opt/2023-03-20T2101/StatAnalysis/0.2.0/InstallArea/x86_64-centos7-gcc11-opt/bin:/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/Cmake/3.24.3/Linux-x86_64/bin:/cvmfs/sft.cern.ch/lcg/releases/gcc/11.2.0-8a51a/x86_64-centos7/bin:/cvmfs/sft.cern.ch/lcg/releases/binutils/2.37-355ed/x86_64-centos7/bin:/home/feickert/.local/bin:/home/feickert/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin:/opt/dell/srvadmin/bin
PYTHONPATH=/tmp/feickert/working-example/lib/python3.10/site-packages:/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_StatAnalysis_x86_64-centos7-gcc11-opt/2023-03-20T2101/StatAnalysis/0.2.0/InstallArea/x86_64-centos7-gcc11-opt/python:/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_StatAnalysis_x86_64-centos7-gcc11-opt/2023-03-20T2101/StatAnalysis/0.2.0/InstallArea/x86_64-centos7-gcc11-opt/lib
# python -m pip list
Package    Version
---------- -------
pip        23.0.1
setuptools 67.6.0
wheel      0.40.0
# python -c 'import numpy; print(numpy)'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'numpy'

Minimal Working Example with AnalysisBase

Using 759cbd0

#!/bin/bash

mkdir -p ~/.local/bin
export PATH=~/.local/bin:"${PATH}"  # If ~/.local/bin not on PATH already
curl -sL https://raw.githubusercontent.com/matthewfeickert/cvmfs-venv/759cbd066026f2209b6f541baf3f41085bfc1620/cvmfs-venv.sh -o ~/.local/bin/cvmfs-venv
chmod +x ~/.local/bin/cvmfs-venv

# Guard against this being run in a subshell
export ATLAS_LOCAL_ROOT_BASE=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase
. "${ATLAS_LOCAL_ROOT_BASE}/user/atlasLocalSetup.sh" -3  # setuptATLAS

echo "# asetup AnalysisBase,22.2.113"
asetup AnalysisBase,22.2.113
echo "PATH=$PATH"
echo "PYTHONPATH=$PYTHONPATH"
echo "$ python3: $(command -v python3)"
echo "# python3 --version --version"
python3 --version --version
echo "# python3 -m pip list | wc -l"
python3 -m pip list | wc -l
echo "# python3 -c 'import numpy; print(numpy)'"
python3 -c 'import numpy; print(numpy)'

echo ""
. cvmfs-venv ab-working-example
echo "PATH=$PATH"
echo "PYTHONPATH=$PYTHONPATH"
echo "# python -m pip list"
python -m pip list
echo "# python -c 'import numpy; print(numpy)'"
python -c 'import numpy; print(numpy)'

Relevant output:

# asetup AnalysisBase,22.2.113
Using AnalysisBase/22.2.113 [cmake] with platform x86_64-centos7-gcc11-opt
	at /cvmfs/atlas.cern.ch/repo/sw/software/22.2
Unchanged: COOL_ORA_ENABLE_ADAPTIVE_OPT=Y
PATH=/tmp/feickert/.asetup-sysbin-lkkd3E_2481758:/cvmfs/atlas.cern.ch/repo/sw/software/22.2/AnalysisBase/22.2.113/InstallArea/x86_64-centos7-gcc11-opt/share:/cvmfs/atlas.cern.ch/repo/sw/software/22.2/AnalysisBase/22.2.113/InstallArea/x86_64-centos7-gcc11-opt/bin:/cvmfs/atlas.cern.ch/repo/sw/software/22.2/AnalysisBaseExternals/22.2.113/InstallArea/x86_64-centos7-gcc11-opt/bin:/cvmfs/atlas.cern.ch/repo/sw/software/22.2/AnalysisBaseExternals/22.2.113/InstallArea/x86_64-centos7-gcc11-opt/share:/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/Cmake/3.24.3/Linux-x86_64/bin:/cvmfs/sft.cern.ch/lcg/releases/gcc/11.2.0-8a51a/x86_64-centos7/bin:/cvmfs/sft.cern.ch/lcg/releases/binutils/2.37-355ed/x86_64-centos7/bin:/home/feickert/.local/bin:/home/feickert/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin:/opt/dell/srvadmin/bin
PYTHONPATH=/cvmfs/atlas.cern.ch/repo/sw/software/22.2/AnalysisBase/22.2.113/InstallArea/x86_64-centos7-gcc11-opt/python:/cvmfs/atlas.cern.ch/repo/sw/software/22.2/AnalysisBase/22.2.113/InstallArea/x86_64-centos7-gcc11-opt/lib:/cvmfs/atlas.cern.ch/repo/sw/software/22.2/AnalysisBaseExternals/22.2.113/InstallArea/x86_64-centos7-gcc11-opt/lib:/cvmfs/atlas.cern.ch/repo/sw/software/22.2/AnalysisBaseExternals/22.2.113/InstallArea/x86_64-centos7-gcc11-opt/python:/cvmfs/atlas.cern.ch/repo/sw/software/22.2/AnalysisBaseExternals/22.2.113/InstallArea/x86_64-centos7-gcc11-opt/lib/python3.9/site-packages
$ python3: /cvmfs/atlas.cern.ch/repo/sw/software/22.2/AnalysisBaseExternals/22.2.113/InstallArea/x86_64-centos7-gcc11-opt/bin/python3
# python3 --version --version
Python 3.9.12 (main, Mar 16 2023, 02:25:00) 
[GCC 11.2.0]
# python3 -m pip list | wc -l
9
# python3 -c 'import numpy; print(numpy)'
<module 'numpy' from '/cvmfs/atlas.cern.ch/repo/sw/software/22.2/AnalysisBaseExternals/22.2.113/InstallArea/x86_64-centos7-gcc11-opt/lib/python3.9/site-packages/numpy-1.22.3-py3.9-linux-x86_64.egg/numpy/__init__.py'>

# Creating new Python virtual environment 'ab-working-example'
PATH=/tmp/feickert/ab-working-example/bin:/tmp/feickert/.asetup-sysbin-lkkd3E_2481758:/cvmfs/atlas.cern.ch/repo/sw/software/22.2/AnalysisBase/22.2.113/InstallArea/x86_64-centos7-gcc11-opt/share:/cvmfs/atlas.cern.ch/repo/sw/software/22.2/AnalysisBase/22.2.113/InstallArea/x86_64-centos7-gcc11-opt/bin:/cvmfs/atlas.cern.ch/repo/sw/software/22.2/AnalysisBaseExternals/22.2.113/InstallArea/x86_64-centos7-gcc11-opt/bin:/cvmfs/atlas.cern.ch/repo/sw/software/22.2/AnalysisBaseExternals/22.2.113/InstallArea/x86_64-centos7-gcc11-opt/share:/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/Cmake/3.24.3/Linux-x86_64/bin:/cvmfs/sft.cern.ch/lcg/releases/gcc/11.2.0-8a51a/x86_64-centos7/bin:/cvmfs/sft.cern.ch/lcg/releases/binutils/2.37-355ed/x86_64-centos7/bin:/home/feickert/.local/bin:/home/feickert/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin:/opt/dell/srvadmin/bin
PYTHONPATH=/tmp/feickert/ab-working-example/lib/python3.9/site-packages:/cvmfs/atlas.cern.ch/repo/sw/software/22.2/AnalysisBase/22.2.113/InstallArea/x86_64-centos7-gcc11-opt/python:/cvmfs/atlas.cern.ch/repo/sw/software/22.2/AnalysisBase/22.2.113/InstallArea/x86_64-centos7-gcc11-opt/lib:/cvmfs/atlas.cern.ch/repo/sw/software/22.2/AnalysisBaseExternals/22.2.113/InstallArea/x86_64-centos7-gcc11-opt/lib:/cvmfs/atlas.cern.ch/repo/sw/software/22.2/AnalysisBaseExternals/22.2.113/InstallArea/x86_64-centos7-gcc11-opt/python:/cvmfs/atlas.cern.ch/repo/sw/software/22.2/AnalysisBaseExternals/22.2.113/InstallArea/x86_64-centos7-gcc11-opt/lib/python3.9/site-packages
# python -m pip list
Package    Version
---------- -------
Cython     0.29.28
LHAPDF     6.5.1
numpy      1.22.3
pip        23.0.1
setuptools 67.6.0
wheel      0.40.0
xrootd     5.4.3
# python -c 'import numpy; print(numpy)'
<module 'numpy' from '/cvmfs/atlas.cern.ch/repo/sw/software/22.2/AnalysisBaseExternals/22.2.113/InstallArea/x86_64-centos7-gcc11-opt/lib/python3.9/site-packages/numpy-1.22.3-py3.9-linux-x86_64.egg/numpy/__init__.py'>

Matthew Feickert · Answer 1 · Wed Mar 22 2023 04:22:38 GMT+0800 (China Standard Time)

Oops, I'm tired as I had pasted from another example incorrectly

python3 -m venv failing-example
. cvmfs-venv failing-example

instead of

. cvmfs-venv failing-example

😬

This failing example still demonstrates the bug

#!/bin/bash

mkdir -p ~/.local/bin
export PATH=~/.local/bin:"${PATH}"  # If ~/.local/bin not on PATH already
curl -sL https://raw.githubusercontent.com/matthewfeickert/cvmfs-venv/759cbd066026f2209b6f541baf3f41085bfc1620/cvmfs-venv.sh -o ~/.local/bin/cvmfs-venv
chmod +x ~/.local/bin/cvmfs-venv

# Guard against this being run in a subshell
export ATLAS_LOCAL_ROOT_BASE=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase
. "${ATLAS_LOCAL_ROOT_BASE}/user/atlasLocalSetup.sh" -3  # setuptATLAS

echo "# asetup StatAnalysis,latest,master"
asetup StatAnalysis,latest,master
echo "PATH=$PATH"
echo "PYTHONPATH=$PYTHONPATH"
echo "$ python3: $(command -v python3)"
echo "# python3 --version --version"
python3 --version --version
echo "# python3 -m pip list | wc -l"
python3 -m pip list | wc -l
echo "# python3 -c 'import numpy; print(numpy)'"
python3 -c 'import numpy; print(numpy)'

echo ""
. cvmfs-venv failing-example
echo "PATH=$PATH"
echo "PYTHONPATH=$PYTHONPATH"
echo "# python -m pip list"
python -m pip list
echo "# python -c 'import numpy; print(numpy)'"
python -c 'import numpy; print(numpy)'

Relevant output:

...
# asetup StatAnalysis,latest,master
Using StatAnalysis/0.2.0 [cmake] with platform x86_64-centos7-gcc11-opt
	at /cvmfs/atlas-nightlies.cern.ch/repo/sw/master_StatAnalysis_x86_64-centos7-gcc11-opt/2023-03-21T2101
StatAnalysis Software Versions (Please ask sw developers to submit updates):

ROOT                : v6-28-02
Python              : 3.10.6
RooFitExtensions    : v1.0.4 2023-03-14 16:11:29 +0100
xRooFit             : 5cc1c234 2023-03-20 20:48:57 +0100
TRExFitter          : f3a290a7 2023-02-14 18:41:44 +0100
quickFit            : e56ede5b 2023-03-16 12:28:03 +0100
RooUnfold           : 924c8b60 2022-07-18 14:52:20 +0200
xmlAnaWSBuilder     : a89c1178 2023-03-15 03:36:35 +0100
workspaceCombiner   : d41b333b 2023-03-15 03:54:49 +0100
HistFitter          :  
BootstrapGenerator  : 71e6ebe2 2021-12-10 20:25:28 +0100
Significance        : d59b9a2 2023-03-14 12:30:24 +0000
pyhf                : 0.7.0
Unchanged: COOL_ORA_ENABLE_ADAPTIVE_OPT=Y
PATH=/tmp/feickert/.asetup-sysbin-DLJMa8_2951995:/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_StatAnalysis_x86_64-centos7-gcc11-opt/2023-03-21T2101/StatAnalysis/0.2.0/InstallArea/x86_64-centos7-gcc11-opt/share:/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_StatAnalysis_x86_64-centos7-gcc11-opt/2023-03-21T2101/StatAnalysis/0.2.0/InstallArea/x86_64-centos7-gcc11-opt/bin:/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/Cmake/3.24.3/Linux-x86_64/bin:/cvmfs/sft.cern.ch/lcg/releases/gcc/11.2.0-8a51a/x86_64-centos7/bin:/cvmfs/sft.cern.ch/lcg/releases/binutils/2.37-355ed/x86_64-centos7/bin:/home/feickert/.local/bin:/home/feickert/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin:/opt/dell/srvadmin/bin
PYTHONPATH=/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_StatAnalysis_x86_64-centos7-gcc11-opt/2023-03-21T2101/StatAnalysis/0.2.0/InstallArea/x86_64-centos7-gcc11-opt/python:/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_StatAnalysis_x86_64-centos7-gcc11-opt/2023-03-21T2101/StatAnalysis/0.2.0/InstallArea/x86_64-centos7-gcc11-opt/lib
$ python3: /cvmfs/atlas-nightlies.cern.ch/repo/sw/master_StatAnalysis_x86_64-centos7-gcc11-opt/2023-03-21T2101/StatAnalysis/0.2.0/InstallArea/x86_64-centos7-gcc11-opt/bin/python3
# python3 --version --version
Python 3.10.6 (main, Mar 21 2023, 19:52:59) [GCC 11.2.0]
# python3 -m pip list | wc -l
55
# python3 -c 'import numpy; print(numpy)'
<module 'numpy' from '/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_StatAnalysis_x86_64-centos7-gcc11-opt/2023-03-21T2101/StatAnalysis/0.2.0/InstallArea/x86_64-centos7-gcc11-opt/lib/python3.10/site-packages/numpy-1.22.3-py3.10-linux-x86_64.egg/numpy/__init__.py'>

# Creating new Python virtual environment 'failing-example'
PATH=/tmp/feickert/failing-example/bin:/tmp/feickert/.asetup-sysbin-DLJMa8_2951995:/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_StatAnalysis_x86_64-centos7-gcc11-opt/2023-03-21T2101/StatAnalysis/0.2.0/InstallArea/x86_64-centos7-gcc11-opt/share:/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_StatAnalysis_x86_64-centos7-gcc11-opt/2023-03-21T2101/StatAnalysis/0.2.0/InstallArea/x86_64-centos7-gcc11-opt/bin:/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/Cmake/3.24.3/Linux-x86_64/bin:/cvmfs/sft.cern.ch/lcg/releases/gcc/11.2.0-8a51a/x86_64-centos7/bin:/cvmfs/sft.cern.ch/lcg/releases/binutils/2.37-355ed/x86_64-centos7/bin:/home/feickert/.local/bin:/home/feickert/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin:/opt/dell/srvadmin/bin
PYTHONPATH=/tmp/feickert/failing-example/lib/python3.10/site-packages:/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_StatAnalysis_x86_64-centos7-gcc11-opt/2023-03-21T2101/StatAnalysis/0.2.0/InstallArea/x86_64-centos7-gcc11-opt/python:/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_StatAnalysis_x86_64-centos7-gcc11-opt/2023-03-21T2101/StatAnalysis/0.2.0/InstallArea/x86_64-centos7-gcc11-opt/lib
# python -m pip list
Package    Version
---------- -------
pip        23.0.1
setuptools 67.6.0
wheel      0.40.0
# python -c 'import numpy; print(numpy)'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'numpy'