ModuleNotFoundError: No module named 'azure.storage.cloudstorageaccount'
skertz opened this issue · comments
SUMMARY
ISSUE TYPE
- Bug Report
COMPONENT NAME
ANSIBLE VERSION
ansible [core 2.14.6]
config file = None
configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /root/.local/lib/python3.11/site-packages/ansible
ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
executable location = /usr/local/bin/ansible
python version = 3.11.5 (main, Sep 22 2023, 15:34:29) [GCC 8.5.0 20210514 (Red Hat 8.5.0-20)] (/bin/python3)
jinja version = 3.1.3
libyaml = Tru
COLLECTION VERSION
Collection Version
------------------ -------
azure.azcollection 2.2.0
CONFIGURATION
OS / ENVIRONMENT
Red Hat Enterprise Linux 8.9 (Ootpa)
STEPS TO REPRODUCE
yum install python311
sudo update-alternatives --config python3
python3 -m ensurepip
python3 -m pip install ansible
ansible-galaxy collection install azure.azcollection
pip3 install -r ~/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt
Playbook:
---
- hosts: azurevm
become: true
collections:
- azure.azcollection
vars:
ansible_python_interpreter: /bin/python3
tasks:
- name: Print message
ansible.builtin.debug:
msg: Hello from Ansible
- name: Query all the resources in the resource group
azure_rm_resource_info:
resource_group: "rg-myRG"
resource_type: resources
EXPECTED RESULTS
ACTUAL RESULTS
TASK [Query all the resources in the resource group] ********************************************************************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ModuleNotFoundError: No module named 'azure.storage.cloudstorageaccount'
fatal: [dv-ansible-4]: FAILED! => changed=false
msg: Failed to import the required Python library (ansible[azure] (azure >= 2.0.0)) on pd-ansible-4's Python /bin/python3. Please read the module documentation and install it in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter
@skertz The Azure-storage already abandoned (v0.37.0), but the error "azure. storage. cloudstorageaccount 'should no longer appear in the abandoned after package. Suspicion is due to incompatible reasons. Could you please provide some information about your working environment? (pip list, python version, System....) ? Thank you!
Red Hat Enterprise Linux 8.9 (Ootpa) deployed to Azure - clean build. I've deployed RH7 and RH8 and rebuilt different times.
There are 3 programs which provide 'python3'.
Selection Command
-----------------------------------------------
* 1 /usr/bin/python3.6
2 /usr/bin/python3.8
+ 3 /usr/bin/python3.11
pip list
Package Version
------------------------------------ -----------
adal 1.2.7
aiohttp 3.9.3
aiosignal 1.3.1
ansible 9.3.0
ansible-core 2.16.4
anyio 4.3.0
applicationinsights 0.11.10
argcomplete 1.12.3
attrs 23.2.0
azure-cli-core 2.34.0
azure-cli-telemetry 1.0.6
azure-common 1.1.11
azure-containerregistry 1.1.0
azure-core 1.28.0
azure-identity 1.14.0
azure-iot-hub 2.6.1
azure-keyvault 4.2.0
azure-keyvault-certificates 4.7.0
azure-keyvault-keys 4.8.0
azure-keyvault-secrets 4.7.0
azure-mgmt-apimanagement 3.0.0
azure-mgmt-authorization 2.0.0
azure-mgmt-automation 1.0.0
azure-mgmt-batch 16.2.0
azure-mgmt-cdn 11.0.0
azure-mgmt-compute 26.1.0
azure-mgmt-containerinstance 9.0.0
azure-mgmt-containerregistry 9.1.0
azure-mgmt-containerservice 20.0.0
azure-mgmt-core 1.3.0
azure-mgmt-cosmosdb 6.4.0
azure-mgmt-datafactory 2.0.0
azure-mgmt-datalake-store 1.0.0
azure-mgmt-devtestlabs 9.0.0
azure-mgmt-dns 8.0.0
azure-mgmt-eventhub 10.1.0
azure-mgmt-hdinsight 9.0.0
azure-mgmt-iothub 2.2.0
azure-mgmt-keyvault 10.0.0
azure-mgmt-loganalytics 12.0.0
azure-mgmt-managedservices 6.0.0
azure-mgmt-managementgroups 1.0.0
azure-mgmt-marketplaceordering 1.1.0
azure-mgmt-monitor 3.0.0
azure-mgmt-network 19.1.0
azure-mgmt-notificationhubs 7.0.0
azure-mgmt-nspkg 2.0.0
azure-mgmt-privatedns 1.0.0
azure-mgmt-rdbms 10.0.0
azure-mgmt-recoveryservices 2.0.0
azure-mgmt-recoveryservicesbackup 3.0.0
azure-mgmt-redis 13.0.0
azure-mgmt-resource 21.1.0
azure-mgmt-search 8.0.0
azure-mgmt-servicebus 7.1.0
azure-mgmt-sql 3.0.1
azure-mgmt-storage 19.0.0
azure-mgmt-trafficmanager 1.0.0b1
azure-mgmt-web 6.1.0
azure-nspkg 2.0.0
azure-storage-blob 12.11.0
bcrypt 4.1.2
certifi 2024.2.2
cffi 1.16.0
charset-normalizer 3.3.2
cryptography 42.0.5
Deprecated 1.2.14
frozenlist 1.4.1
h11 0.14.0
h2 4.1.0
hpack 4.0.0
httpcore 1.0.4
httpx 0.27.0
humanfriendly 10.0
hyperframe 6.0.1
idna 3.6
importlib-metadata 6.11.0
isodate 0.6.1
Jinja2 3.1.3
jmespath 1.0.1
knack 0.9.0
MarkupSafe 2.1.5
microsoft-kiota-abstractions 1.3.1
microsoft-kiota-authentication-azure 1.0.0
microsoft-kiota-http 1.3.1
microsoft-kiota-serialization-json 1.1.0
microsoft-kiota-serialization-text 1.0.0
msal 1.27.0
msal-extensions 0.3.1
msgraph-core 1.0.0
msgraph-sdk 1.0.0
msrest 0.7.1
msrestazure 0.6.4
multidict 6.0.5
oauthlib 3.2.2
opentelemetry-api 1.23.0
opentelemetry-sdk 1.23.0
opentelemetry-semantic-conventions 0.44b0
packaging 21.3
paramiko 2.12.0
pendulum 3.0.0
pip 24.0
pkginfo 1.10.0
portalocker 1.7.1
psutil 5.9.8
pycparser 2.21
Pygments 2.17.2
PyJWT 2.8.0
PyNaCl 1.5.0
pyOpenSSL 24.1.0
pyparsing 3.1.2
PySocks 1.7.1
python-dateutil 2.9.0.post0
PyYAML 6.0.1
requests 2.31.0
requests-oauthlib 1.4.0
resolvelib 0.8.1
setuptools 65.5.1
six 1.16.0
sniffio 1.3.1
std-uritemplate 0.0.55
tabulate 0.9.0
time-machine 2.14.0
typing_extensions 4.10.0
tzdata 2024.1
uamqp 1.6.8
urllib3 2.2.1
wrapt 1.16.0
xmltodict 0.13.0
yarl 1.9.4
zipp 3.18.1
@skertz See you have three python interpretation versions, is it not in the execution, the python interpreter path is wrong, thank you!
Typically the path is Python /usr/libexec/platform-python
However, I am setting
vars:
ansible_python_interpreter: /usr/bin/python3
lrwxrwxrwx. 1 root root 25 Feb 18 16:54 /bin/python3 -> /etc/alternatives/python3
[root@pd-ansible-4 ~]# ll /etc/alternatives/python3
lrwxrwxrwx. 1 root root 19 Mar 18 17:07 /etc/alternatives/python3 -> /usr/bin/python3.11
When setting to Python 3.6 - I see I'm missing the packaging module as expected because I did not install with 3.6
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ModuleNotFoundError: No module named 'packaging'
fatal:
It is when trying to use Python 3.8 and 3.11 I see the same error for azure.storage.cloudstorageaccount
If trying to use the default /usr/libexec/platform-python3.6 I can not even install the azure collection requirements.
Could not find a version that satisfies the requirement msgraph-sdk==1.0.0 (from -r /root/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt (line 4)) (from versions: 1.0.0a0, 1.0.0a1, 1.0.0a2, 1.0.0a3, 1.0.0a4, 1.0.0a7, 1.0.0a8, 1.0.0a9, 1.0.0a10, 1.0.0a11, 1.0.0a12, 1.0.0a13)
No matching distribution found for msgraph-sdk==1.0.0 (from -r /root/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt (line 4))
@skertz I see you made a soft link to '/usr/bin/python3'. What version of python3 did you have before you built the link (the minimum python version 3.6 required by msgraph-sdk)? The module cannot be found. Is it because the interpreter of the soft link you specified does not match with the installed directory that it cannot work? Could you please help confirm these two points? Thank you!
The version of python originally is 3.6. This is a fresh azure vm deployment. However when using 3.6 - msgraphsdk cannot be installed from the azcollection requirements file due to missing version available.
[root@pd-ansible-4 ~]# python3 --version
Python 3.6.8
[root@pd-ansible-4 ~]# python3 -m pip install -r ~/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt
WARNING: Running pip install with root privileges is generally not a good idea. Try `__main__.py install --user` instead.
Collecting packaging (from -r /root/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt (line 1))
Using cached https://files.pythonhosted.org/packages/05/8e/8de486cbd03baba4deef4142bd643a3e7bbe954a784dc1bb17142572d127/packaging-21.3-py3-none-any.whl
Requirement already satisfied: requests[security] in /usr/lib/python3.6/site-packages (from -r /root/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt (line 2))
Collecting xmltodict (from -r /root/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt (line 3))
Using cached https://files.pythonhosted.org/packages/94/db/fd0326e331726f07ff7f40675cd86aa804bfd2e5016c727fa761c934990e/xmltodict-0.13.0-py2.py3-none-any.whl
Collecting msgraph-sdk==1.0.0 (from -r /root/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt (line 4))
Could not find a version that satisfies the requirement msgraph-sdk==1.0.0 (from -r /root/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt (line 4)) (from versions: 1.0.0a0, 1.0.0a1, 1.0.0a2, 1.0.0a3, 1.0.0a4, 1.0.0a7, 1.0.0a8, 1.0.0a9, 1.0.0a10, 1.0.0a11, 1.0.0a12, 1.0.0a13)
No matching distribution found for msgraph-sdk==1.0.0 (from -r /root/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt (line 4))
Which is why I've tried Python 3.8, 3.9 , 3.11 , and different versions of the collections. All comes back to the same cloudstorageaccount error. My setup is simple, deploy a vm in azure. And then run these commands:
yum install python311
sudo update-alternatives --config python3
python3 -m ensurepip
python3 -m pip install ansible
ansible-galaxy collection install azure.azcollection
pip3 install -r ~/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt
@skertz Yes, msgraph-sdk relies on a version of python higher than python 3.8. Could you please try it according to the following instructions, retry your playbook! thanks!
yum install python3.9
sudo update-alternatives --config python3
pip3 install ansible[azure]
ansible-galaxy collection install azure.azcollection
pip3 install -r ~/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt
@Fred-sun - yes the msgraph -sdk is needed by Python >= 3.9 -
But as noted prior to this suggestion. I have tried Python 3.9 and 3.11. Everything all comes back to the error for cloudstorageaccount
The error was: ModuleNotFoundError: No module named 'azure.storage.cloudstorageaccount'
@skertz Yes, I rebuilt a VM and ran the following command without any problems. thank you.
sudo update-alternatives --config python3
pip3 install ansible[azure]
ansible-galaxy collection install azure.azcollection
pip3 install -r ~/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt
I have done this on RedHat 8, RedHat9, CentOS 8.5 deployed fresh into Azure
I have done this with Python3.9 and 3.10 and 3.11
All similiar results with the same error for cloudstorageaccount
Here is my history file on brand new vm in Azure
8 yum install python3.9
9 sudo update-alternatives --config python3
10 python3 -m ensurepip
11 pip3 install ansible[azure]
12 ansible-galaxy collection install azure.azcollection
13 /usr/local/bin/ansible-galaxy collection install azure.azcollection
14 /usr/local/bin/ansible-galaxy collection install azure.azcollection --force
15 pip3 install -r ~/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt
16 historyI run a playbook.
become: true
collections:
- azure.azcollection
tasks:
- name: Print message
ansible.builtin.debug:
msg: Hello from Ansible
# - name: Get Azure subscription
- name: Query all the resources in the resource group
azure_rm_resource_info:
resource_group: "rg-myResourceGroup"
resource_type: resources
Then this error
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ModuleNotFoundError: No module named 'azure.storage.cloudstorageaccount'
@skertz Re-test on redhat 8 to obtain resource information. The following is the installation process. Thank you.
1 python3
2 cat /etc/ssh/sshd_config
3 sudo cat /etc/ssh/sshd_config
4 sudo yum update
5 sudo yum install python3.9
6 sudo update-alternatives --config python3
7 python3 -m ensurepip
8 sudo pip3 install ansible[azure]
9 ansible --version
10 ansible-galaxy collection install azure.azcollection
11 ansible-galaxy collection install azure.azcollection --force
12 sudo pip3 install -r ~/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt
13 pip3 list
Playbook:
---
- name: For test
hosts: localhost
become: true
collections:
- azure.azcollection
tasks:
- name: Print message
ansible.builtin.debug:
msg: Hello from Ansible
# - name: Get Azure subscription
- name: Query all the resources in the resource group
azure_rm_resource_info:
resource_group: v-xisuRG02
resource_type: resources
register: output
- debug:
var: output
Return info:
TASK [Query all the resources in the resource group] ************************************************************
ok: [localhost]
TASK [debug] ****************************************************************************************************
ok: [localhost] => {
"output": {
"changed": false,
"failed": false,
"response": [
{
"id": "/subscriptions/xxxxxxxxxxxxx/resourceGroups/v-xisuRG02/providers/Microsoft.Network/virtualNetworks/sqlmi003",
"location": "eastus",
"name": "sqlmi003",
"type": "Microsoft.Network/virtualNetworks"
},
{
..........
@skertz In addition, I see that the python version you specified is 3.11, if you directly specify the python version is 3.9, what is the result?
@skertz I still need more information to clarify this issue, can you help to provide '-vvvv 'information to execute the script, as well as your entire operation process, thank you!
This is issue can be marked resolved. Working with colleague, found that the ansible automation controller was missing the collections, even though the server we were executing on had all the installation pieces. If using "delegate" it would work on the server where items installed. Strange that the error was for missing azure cloudstorageaccount module.