New version and Nick version of Rucio client setup
ericvaandering opened this issue · comments
@nsmith- asked via e-mail for a different version of python to be used in Rucio so that the GFAL tools could be used. I believe that is fine for SL7, but it has some issues on SL6 because of a dependency which is python 2.7. However, the old way for SL6 works fine except for the GFAL part.
Also, there is a new version of the Rucio client 1.22.7 with some things important to CMS in it.
Pending confirmation from Nick, can we get the following:
- The old setup script explicitly labeled as OLD or SL6 or whatever so people on that aging OS can get some functionality.
- The test on the IB made as the default for SL7
- Both versions using the newest versions of the Rucio code?
Thanks in advance @smuzaffar and/or @bockjoo
A new Issue was created by @ericvaandering Eric Vaandering.
@Dr15Jones, @silviodonato, @dpiparo, @smuzaffar, @makortel can you please review it and eventually sign/assign? Thanks.
cms-bot commands are listed here
@ericvaandering , I do not think that old setup works for SLC6 too , for example on slc6 I get [a]
[a]
> source /cvmfs/cms.cern.ch/rucio/setup.sh
/cvmfs/cms.cern.ch/slc7_amd64_gcc820/external/python/2.7.15-bcolbf/bin/python2: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /cvmfs/cms.cern.ch/slc7_amd64_gcc820/external/python/2.7.15-bcolbf/lib/libpython2.7.so.1.0)
assign externals
New categories assigned: externals
@smuzaffar,@mrodozov,@tulamor you have been requested to review this Pull request/Issue and eventually sign? Thanks
Personally I am OK with saying SL6 is not supported. As Nick and I discussed yesterday, I don't think there is really a need to do anything in Rucio and an old CMS release together. And that should be the only reason someone is hanging back on SL6.
Yeah as far as I know the previous setup did not work for SL6, I had made dmwm/CMSRucio#123 at some point even.
I am OK with the new version in IB being the default, but I let Eric comment about updating the client.
I have installed rucio client 1.22.7 under /cvmfs/cms-ib.cern.ch/rucio . Please try to source /cvmfs/cms-ib.cern.ch/rucio/setup.sh and test it. gfal-* tools should work under this env as it uses system python. Note that it only works for slc7. On slc6 it will give error
source /cvmfs/cms-ib.cern.ch/rucio/setup.sh
ERROR: Unsupported architecture x86_64/slc6. Supported architectures are:
x86_64/slc7
let me know if this looks good so that I can deploy it under /cvmfs/cms.cern.ch/rucio
I tried few rucio and gfal-* commands under new rucio env and they work but gfal-copy fails (may be I do not have rights to copy).
[a]
> source /cvmfs/cms-ib.cern.ch/rucio/setup.sh
rucio 1.22.7
> export RUCIO_ACCOUNT=muzaffar
> rucio ping
1.22.6.post1
> rucio whoami
status : ACTIVE
account : muzaffar
account_type : USER
created_at : 2020-04-28T23:20:02
updated_at : 2020-04-28T23:20:02
suspended_at : None
deleted_at : None
email : my.email@domain.com
> gfal-ls -l 'srm://dpmhead-rc.cern.ch:8446/srm/managerv2?SFN=/dpm/cern.ch/home/dteam/2.txt'
-rw-rw-r-- 1 48 47 2 Sep 24 2019 srm://dpmhead-rc.cern.ch:8446/srm/managerv2?SFN=/dpm/cern.ch/home/dteam/2.txt
> gfal-stat 'srm://dpmhead-rc.cern.ch:8446/srm/managerv2?SFN=/dpm/cern.ch/home/dteam/2.txt'
File: 'srm://dpmhead-rc.cern.ch:8446/srm/managerv2?SFN=/dpm/cern.ch/home/dteam/2.txt'
Size: 2 regular file
Access: (0664/-rw-rw-r--) Uid: 47 Gid: 48
Access: 1970-01-01 01:00:00.000000
Modify: 2019-09-24 10:40:05.000000
Change: 2020-05-18 21:03:28.000000
> gfal-copy 'srm://dpmhead-rc.cern.ch:8446/srm/managerv2?SFN=/dpm/cern.ch/home/dteam/2.txt' /tmp/muzaffar/2.txt
Copying srm://dpmhead-rc.cern.ch:8446/srm/managerv2?SFN=/dpm/cern.ch/home/dteam/2.txt [FAILED] after 0s
gfal-copy error: 13 (Permission denied) - Could not open source: globus_ftp_client: the server responded with an error 530 530-globus_xio: Authentication Error 530-globus_gsi_callback_module: Could not verify credential 530-globus_gsi_callback_module: Could not verify credential 530-globus_gsi_callback_module: Invalid CRL: The available CRL has expired 530 End.
out of curiosity - how does this setup evolve to support things like cc8 and python3?
@davidlange6 , The setup install rucio under arch/os (e.g. x86_64/slc7 or x86_64/cc8). So for cc8 it will just use a different directory.
As it uses system python and gfal, so we have to stick to the python version used by gfal. If system gfal links against py3 then we just install rucio for python3.
ha - so indeed, having python scripts in /usr/bin that start with #/usr/bin/python2 is not very promising:(
It is not only scripts but there is a shared library ( /usr/lib64/python2.7/site-packages/gfal2.so ) which comes with it
anyway, all the more reason the system python is the thing to use here
@ericvaandering , new rucio version 1.22.7 is now available in default location /cvmfs/cms.cern.ch/rucio . New version uses system python.
In case this does not work then you can still source /cvmfs/cms.cern.ch/rucio/setup-old.sh to get the old rucio setup (version 1.22.2)
new version using system python is now available