Giters

Azure / WALinuxAgent

Microsoft Azure Linux Guest Agent

Home Page:http://azure.microsoft.com/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Provisioning any Azure VM Extension runs Enable Function of all other extensions previously provisioned

simratsatia opened this issue · comments

commented

Describe the bug: A clear and concise description of what the bug is.

Issue Description:
When I execute RunShellCommand on the VM, I notice that 'enable' function specified in HandlerManifest.json of ALL the previously existing extensions are executed again. I'm not sure if this is an intended behaviour but not sure why it would be intended to begin with.

Repro Steps:

  1. Provision the VM with any extension (eg: AzureHybridBenefit.AHBForRHEL)
  2. Provision the VM with another VM Extension (eg: CustomScriptExtension For Linux)
  3. The enable function of all existing extensions are run again

This can be confirmed using the /var/log/waagent.log file:

  • Distro and Version: [e.g. Red Hat Enterprise Linux 7.8]
  • WALinuxAgent version:
    [root@RHEL-7 log]# waagent --version
    WALinuxAgent-2.2.46 running on redhat 7.8
    Python: 2.7.5
    Goal state agent: 2.7.1.0

Additional context
Add any other context about the problem here.

Log file attached

/var/log/waagent.log:

2022-04-19T15:38:15.551349Z INFO ExtHandler ExtHandler Fetched new vmSettings [correlation ID: 5e948538-45ce-4ee4-a6e5-82cd6fbb9ae6 New eTag: 10419572670385096668]
2022-04-19T15:38:15.552741Z INFO ExtHandler ExtHandler Fetching goal state [incarnation 169]
2022-04-19T15:38:15.660185Z INFO ExtHandler ExtHandler Found private key matching thumbprint EE0D8FCF76284FF8BB43E99FDC0AE9E08DF578C3
2022-04-19T15:38:15.660402Z INFO ExtHandler ExtHandler Certificate with thumbprint 3C4B7460E6E4DB6B5E0C2FACEDC39FD34FF7ECC6 has no matching private key.
2022-04-19T15:38:15.677250Z INFO ExtHandler ExtHandler Fetch goal state completed
2022-04-19T15:38:15.681286Z INFO ExtHandler ExtHandler ProcessExtensionsInGoalState started [Incarnation: 169; Activity Id: 419de99a-1c6a-43e1-9284-c59013a87406; Correlation Id: 374bf7f6-dce4-4295-994a-5b0a8087e663; GS Creation Time: 2022-04-19T15:38:11.043457Z]
2022-04-19T15:38:15.721132Z INFO ExtHandler [Microsoft.Azure.AzureHybridBenefit.AHBForRHEL-1.0.2] Target handler state: enabled [incarnation 169]
2022-04-19T15:38:15.722032Z INFO ExtHandler [Microsoft.Azure.AzureHybridBenefit.AHBForRHEL-1.0.2] [Enable] current handler state is: enabled
2022-04-19T15:38:15.722273Z INFO ExtHandler [Microsoft.Azure.AzureHybridBenefit.AHBForRHEL-1.0.2] Update settings file: 0.settings
2022-04-19T15:38:15.722717Z INFO ExtHandler [Microsoft.Azure.AzureHybridBenefit.AHBForRHEL-1.0.2] Requested extension state: enabled
2022-04-19T15:38:15.723139Z INFO ExtHandler [Microsoft.Azure.AzureHybridBenefit.AHBForRHEL-1.0.2] Enable extension: [bin/ahb-extension-rhel enable]
2022-04-19T15:38:15.723851Z INFO ExtHandler [Microsoft.Azure.AzureHybridBenefit.AHBForRHEL-1.0.2] Executing command: /var/lib/waagent/Microsoft.Azure.AzureHybridBenefit.AHBForRHEL-1.0.2/bin/ahb-extension-rhel enable with environment variables: {"AZURE_GUEST_AGENT_UNINSTALL_CMD_EXIT_CODE": "NOT_RUN", "AZURE_GUEST_AGENT_EXTENSION_VERSION": "1.0.2", "AZURE_GUEST_AGENT_EXTENSION_SUPPORTED_FEATURES": "[{"Value": "1.0", "Key": "ExtensionTelemetryPipeline"}]", "AZURE_GUEST_AGENT_EXTENSION_PATH": "/var/lib/waagent/Microsoft.Azure.AzureHybridBenefit.AHBForRHEL-1.0.2", "ConfigSequenceNumber": "0", "AZURE_GUEST_AGENT_WIRE_PROTOCOL_ADDRESS": "168.63.129.16"}
2022-04-19T15:38:17.731976Z INFO ExtHandler [Microsoft.Azure.AzureHybridBenefit.AHBForRHEL-1.0.2] Command: bin/ahb-extension-rhel enable
[stdout]
[stderr]
2022-04-19T15:38:17.789842Z INFO ExtHandler [Microsoft.Azure.Extensions.CustomScript-2.1.6] Target handler state: enabled [incarnation 169]
2022-04-19T15:38:17.790213Z INFO ExtHandler [Microsoft.Azure.Extensions.CustomScript-2.1.6] [Enable] current handler state is: notinstalled
2022-04-19T15:38:17.791070Z INFO ExtHandler [Microsoft.Azure.Extensions.CustomScript-2.1.6] Downloading extension package: https://umsanxck4gvg3rq3n3k5.blob.core.windows.net/5237dd14-0aad-f051-0fad-1e33e1b63091/5237dd14-0aad-f051-0fad-1e33e1b63091_2.1.6.zip
2022-04-19T15:38:17.930492Z INFO ExtHandler [Microsoft.Azure.Extensions.CustomScript-2.1.6] Unzipping extension package: /var/lib/waagent/Microsoft.Azure.Extensions.CustomScript__2.1.6.zip
2022-04-19T15:38:18.068702Z INFO ExtHandler [Microsoft.Azure.Extensions.CustomScript-2.1.6] Initializing extension Microsoft.Azure.Extensions.CustomScript-2.1.6
2022-04-19T15:38:18.070030Z INFO ExtHandler [Microsoft.Azure.Extensions.CustomScript-2.1.6] Update settings file: 0.settings
2022-04-19T15:38:18.070514Z INFO ExtHandler [Microsoft.Azure.Extensions.CustomScript-2.1.6] Install extension [bin/custom-script-shim install]
2022-04-19T15:38:18.071070Z INFO ExtHandler [Microsoft.Azure.Extensions.CustomScript-2.1.6] Executing command: /var/lib/waagent/Microsoft.Azure.Extensions.CustomScript-2.1.6/bin/custom-script-shim install with environment variables: {"AZURE_GUEST_AGENT_UNINSTALL_CMD_EXIT_CODE": "NOT_RUN", "AZURE_GUEST_AGENT_EXTENSION_VERSION": "2.1.6", "AZURE_GUEST_AGENT_EXTENSION_SUPPORTED_FEATURES": "[{"Value": "1.0", "Key": "ExtensionTelemetryPipeline"}]", "AZURE_GUEST_AGENT_EXTENSION_PATH": "/var/lib/waagent/Microsoft.Azure.Extensions.CustomScript-2.1.6", "ConfigSequenceNumber": "0", "AZURE_GUEST_AGENT_WIRE_PROTOCOL_ADDRESS": "168.63.129.16"}
2022-04-19T15:38:20.084423Z INFO ExtHandler [Microsoft.Azure.Extensions.CustomScript-2.1.6] Command: bin/custom-script-shim install
[stdout]

  • /var/lib/waagent/Microsoft.Azure.Extensions.CustomScript-2.1.6/bin/custom-script-extension install
    time=2022-04-19T15:38:18Z version=v2.1.6/git@fc181d8-dirty operation=install seq=0 event=start
    time=2022-04-19T15:38:18Z version=v2.1.6/git@fc181d8-dirty operation=install seq=0 status="not reported for operation (by design)"
    time=2022-04-19T15:38:18Z version=v2.1.6/git@fc181d8-dirty operation=install seq=0 event="migrate to mrseq" error="Can't find out seqnum from /var/lib/waagent/Microsoft.Azure.Extensions.CustomScript-2.1.6/status, not enough files"
    time=2022-04-19T15:38:18Z version=v2.1.6/git@fc181d8-dirty operation=install seq=0 event="created data dir" path=/var/lib/waagent/custom-script
    time=2022-04-19T15:38:18Z version=v2.1.6/git@fc181d8-dirty operation=install seq=0 event=installed
    time=2022-04-19T15:38:18Z version=v2.1.6/git@fc181d8-dirty operation=install seq=0 status="not reported for operation (by design)"
    time=2022-04-19T15:38:18Z version=v2.1.6/git@fc181d8-dirty operation=install seq=0 event=end
    [stderr]
    2022-04-19T15:38:20.086226Z INFO ExtHandler [Microsoft.Azure.Extensions.CustomScript-2.1.6] Requested extension state: enabled
    2022-04-19T15:38:20.086650Z INFO ExtHandler [Microsoft.Azure.Extensions.CustomScript-2.1.6] Enable extension: [bin/custom-script-shim enable]
    2022-04-19T15:38:20.087345Z INFO ExtHandler [Microsoft.Azure.Extensions.CustomScript-2.1.6] Executing command: /var/lib/waagent/Microsoft.Azure.Extensions.CustomScript-2.1.6/bin/custom-script-shim enable with environment variables: {"AZURE_GUEST_AGENT_UNINSTALL_CMD_EXIT_CODE": "NOT_RUN", "AZURE_GUEST_AGENT_EXTENSION_VERSION": "2.1.6", "AZURE_GUEST_AGENT_EXTENSION_SUPPORTED_FEATURES": "[{"Value": "1.0", "Key": "ExtensionTelemetryPipeline"}]", "AZURE_GUEST_AGENT_EXTENSION_PATH": "/var/lib/waagent/Microsoft.Azure.Extensions.CustomScript-2.1.6", "ConfigSequenceNumber": "0", "AZURE_GUEST_AGENT_WIRE_PROTOCOL_ADDRESS": "168.63.129.16"}
    2022-04-19T15:38:22.098333Z INFO ExtHandler [Microsoft.Azure.Extensions.CustomScript-2.1.6] Command: bin/custom-script-shim enable
    [stdout]
    Writing a placeholder status file indicating progress before forking: /var/lib/waagent/Microsoft.Azure.Extensions.CustomScript-2.1.6/status/0.status
  • nohup /var/lib/waagent/Microsoft.Azure.Extensions.CustomScript-2.1.6/bin/custom-script-extension enable
    time=2022-04-19T15:38:20Z version=v2.1.6/git@fc181d8-dirty operation=enable seq=0 event=start
    time=2022-04-19T15:38:20Z version=v2.1.6/git@fc181d8-dirty operation=enable seq=0 event=pre-check
    time=2022-04-19T15:38:20Z version=v2.1.6/git@fc181d8-dirty operation=enable seq=0 event="comparing seqnum" path=mrseq
    time=2022-04-19T15:38:20Z version=v2.1.6/git@fc181d8-dirty operation=enable seq=0 event="seqnum saved" path=mrseq
    time=2022-04-19T15:38:20Z version=v2.1.6/git@fc181d8-dirty operation=enable seq=0 event="reading configuration"
    time=2022-04-19T15:38:20Z version=v2.1.6/git@fc181d8-dirty operation=enable seq=0 event="read configuration"
    time=2022-04-19T15:38:20Z version=v2.1.6/git@fc181d8-dirty operation=enable seq=0 event="validating json schema"
    time=2022-04-19T15:38:20Z version=v2.1.6/git@fc181d8-dirty operation=enable seq=0 event="json schema valid"
    time=2022-04-19T15:38:20Z version=v2.1.6/git@fc181d8-dirty operation=enable seq=0 event="parsing configuration json"
    time=2022-04-19T15:38:20Z version=v2.1.6/git@fc181d8-dirty operation=enable seq=0 event="parsed configuration json"
    time=2022-04-19T15:38:20Z version=v2.1.6/git@fc181d8-dirty operation=enable seq=0 event="validating configuration logically"
    time=2022-04-19T15:38:20Z version=v2.1.6/git@fc181d8-dirty operation=enable seq=0 event="validated configuration"
    time=2022-04-19T15:38:20Z version=v2.1.6/git@fc181d8-dirty operation=enable seq=0 event="creating output directory" path=/var/lib/waagent/custom-script/download/0
    time=2022-04-19T15:38:20Z version=v2.1.6/git@fc181d8-dirty operation=enable seq=0 event="created output directory"
    time=2022-04-19T15:38:20Z version=v2.1.6/git@fc181d8-dirty operation=enable seq=0 files=0
    time=2022-04-19T15:38:20Z version=v2.1.6/git@fc181d8-dirty operation=enable seq=0 event="executing command" output=/var/lib/waagent/custom-script/download/0
    time=2022-04-19T15:38:20Z version=v2.1.6/git@fc181d8-dirty operation=enable seq=0 event="executing protected commandToExecute" output=/var/lib/waagent/custom-script/download/0
    time=2022-04-19T15:38:20Z version=v2.1.6/git@fc181d8-dirty operation=enable seq=0 event="executed command" output=/var/lib/waagent/custom-script/download/0
    time=2022-04-19T15:38:20Z version=v2.1.6/git@fc181d8-dirty operation=enable seq=0 event=enabled
    time=2022-04-19T15:38:20Z version=v2.1.6/git@fc181d8-dirty operation=enable seq=0 event=end
    [stderr]
    2022-04-19T15:38:22.160421Z INFO ExtHandler [Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.17.5] Target handler state: enabled [incarnation 169]
    2022-04-19T15:38:22.161438Z INFO ExtHandler [Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.17.5] [Enable] current handler state is: enabled
    2022-04-19T15:38:22.161601Z INFO ExtHandler [Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.17.5] Update settings file: 0.settings
    2022-04-19T15:38:22.162036Z INFO ExtHandler [Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.17.5] Requested extension state: enabled
    2022-04-19T15:38:22.162433Z INFO ExtHandler [Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.17.5] Enable extension: [./shim.sh -enable]
    2022-04-19T15:38:22.162984Z INFO ExtHandler [Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.17.5] Executing command: /var/lib/waagent/Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.17.5/./shim.sh -enable with environment variables: {"AZURE_GUEST_AGENT_UNINSTALL_CMD_EXIT_CODE": "NOT_RUN", "AZURE_GUEST_AGENT_EXTENSION_VERSION": "1.17.5", "AZURE_GUEST_AGENT_EXTENSION_SUPPORTED_FEATURES": "[{"Value": "1.0", "Key": "ExtensionTelemetryPipeline"}]", "AZURE_GUEST_AGENT_EXTENSION_PATH": "/var/lib/waagent/Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.17.5", "ConfigSequenceNumber": "0", "AZURE_GUEST_AGENT_WIRE_PROTOCOL_ADDRESS": "168.63.129.16"}
    2022/04/19 15:38:22 Azure Monitoring Agent for Linux started to handle.
    2022/04/19 15:38:22 [Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.17.5] cwd is /var/lib/waagent/Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.17.5
    2022/04/19 15:38:22 [Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.17.5] Change log file to /var/log/azure/Microsoft.Azure.Monitor.AzureMonitorLinuxAgent/extension.log
    2022-04-19T15:38:24.176194Z INFO ExtHandler [Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.17.5] Command: ./shim.sh -enable
    [stdout]
    2022/04/19 15:38:23 [Microsoft.Azure.Monitor.AzureMonitorLinuxAgent-1.17.5] Enable,success,0,Enable succeeded
    [stderr]
    2022-04-19T15:38:24.202969Z INFO ExtHandler [Microsoft.Azure.Security.Monitoring.AzureSecurityLinuxAgent-2.19.250] Target handler state: enabled [incarnation 169]
    2022-04-19T15:38:24.203892Z INFO ExtHandler [Microsoft.Azure.Security.Monitoring.AzureSecurityLinuxAgent-2.19.250] [Enable] current handler state is: enabled
    2022-04-19T15:38:24.204051Z INFO ExtHandler [Microsoft.Azure.Security.Monitoring.AzureSecurityLinuxAgent-2.19.250] Update settings file: 0.settings
    2022-04-19T15:38:24.204439Z INFO ExtHandler [Microsoft.Azure.Security.Monitoring.AzureSecurityLinuxAgent-2.19.250] Requested extension state: enabled
    2022-04-19T15:38:24.204826Z INFO ExtHandler [Microsoft.Azure.Security.Monitoring.AzureSecurityLinuxAgent-2.19.250] Enable extension: [asm-extension enable]
    2022-04-19T15:38:24.205379Z INFO ExtHandler [Microsoft.Azure.Security.Monitoring.AzureSecurityLinuxAgent-2.19.250] Executing command: /var/lib/waagent/Microsoft.Azure.Security.Monitoring.AzureSecurityLinuxAgent-2.19.250/asm-extension enable with environment variables: {"AZURE_GUEST_AGENT_UNINSTALL_CMD_EXIT_CODE": "NOT_RUN", "AZURE_GUEST_AGENT_EXTENSION_VERSION": "2.19.250", "AZURE_GUEST_AGENT_EXTENSION_SUPPORTED_FEATURES": "[{"Value": "1.0", "Key": "ExtensionTelemetryPipeline"}]", "AZURE_GUEST_AGENT_EXTENSION_PATH": "/var/lib/waagent/Microsoft.Azure.Security.Monitoring.AzureSecurityLinuxAgent-2.19.250", "ConfigSequenceNumber": "0", "AZURE_GUEST_AGENT_WIRE_PROTOCOL_ADDRESS": "168.63.129.16"}

2022-04-19T15:38:26.215183Z INFO ExtHandler [Microsoft.Azure.Security.Monitoring.AzureSecurityLinuxAgent-2.19.250] Command: asm-extension enable
[stdout]
[stderr]
[AzureSecurityLinuxAgent] 2022/04/19 15:38:24 INFO: seqnum: 0
[AzureSecurityLinuxAgent] 2022/04/19 15:38:24 INFO: distro info: redhat 7
[AzureSecurityLinuxAgent] 2022/04/19 15:38:24 Performing Enable...
[AzureSecurityLinuxAgent] 2022/04/19 15:38:24 azsecd enable [OK]
[AzureSecurityLinuxAgent] 2022/04/19 15:38:24 azsecmond enable [OK]
[AzureSecurityLinuxAgent] 2022/04/19 15:38:24 VSA enable [OK]
[AzureSecurityLinuxAgent] 2022/04/19 15:38:24 Enable [OK]
2022-04-19T15:38:26.249284Z INFO ExtHandler [Microsoft.CPlat.Core.RunCommandLinux-1.0.3] Target handler state: enabled [incarnation 169]
2022-04-19T15:38:26.250214Z INFO ExtHandler [Microsoft.CPlat.Core.RunCommandLinux-1.0.3] [Enable] current handler state is: enabled
2022-04-19T15:38:26.250510Z INFO ExtHandler [Microsoft.CPlat.Core.RunCommandLinux-1.0.3] Update settings file: 152.settings
2022-04-19T15:38:26.251109Z INFO ExtHandler [Microsoft.CPlat.Core.RunCommandLinux-1.0.3] Requested extension state: enabled
2022-04-19T15:38:26.251429Z INFO ExtHandler [Microsoft.CPlat.Core.RunCommandLinux-1.0.3] Enable extension: [bin/run-command-shim enable]
2022-04-19T15:38:26.252182Z INFO ExtHandler [Microsoft.CPlat.Core.RunCommandLinux-1.0.3] Executing command: /var/lib/waagent/Microsoft.CPlat.Core.RunCommandLinux-1.0.3/bin/run-command-shim enable with environment variables: {"AZURE_GUEST_AGENT_UNINSTALL_CMD_EXIT_CODE": "NOT_RUN", "AZURE_GUEST_AGENT_EXTENSION_VERSION": "1.0.3", "AZURE_GUEST_AGENT_EXTENSION_SUPPORTED_FEATURES": "[{"Value": "1.0", "Key": "ExtensionTelemetryPipeline"}]", "AZURE_GUEST_AGENT_EXTENSION_PATH": "/var/lib/waagent/Microsoft.CPlat.Core.RunCommandLinux-1.0.3", "ConfigSequenceNumber": "152", "AZURE_GUEST_AGENT_WIRE_PROTOCOL_ADDRESS": "168.63.129.16"}

2022-04-19T15:38:28.267335Z INFO ExtHandler [Microsoft.CPlat.Core.RunCommandLinux-1.0.3] Command: bin/run-command-shim enable
[stdout]
Not writing a placeholder status file, already exists: /var/lib/waagent/Microsoft.CPlat.Core.RunCommandLinux-1.0.3/status/152.status

  • nohup /var/lib/waagent/Microsoft.CPlat.Core.RunCommandLinux-1.0.3/bin/run-command-extension enable
    time=2022-04-19T15:38:26Z version=v1.0.3/git@3b6c175-dirty operation=enable seq=152 event=start
    time=2022-04-19T15:38:26Z version=v1.0.3/git@3b6c175-dirty operation=enable seq=152 event=pre-check
    time=2022-04-19T15:38:26Z version=v1.0.3/git@3b6c175-dirty operation=enable seq=152 event="comparing seqnum" path=mrseq
    time=2022-04-19T15:38:26Z version=v1.0.3/git@3b6c175-dirty operation=enable seq=152 event=exit message="this script configuration is already processed, will not run again"
    [stderr]
    2022-04-19T15:38:28.271066Z INFO ExtHandler ExtHandler ProcessExtensionsInGoalState completed [Incarnation: 169; 12589 ms; Activity Id: 419de99a-1c6a-43e1-9284-c59013a87406; Correlation Id: 374bf7f6-dce4-4295-994a-5b0a8087e663; GS Creation Time: 2022-04-19T15:38:11.043457Z]
    2022-04-19T15:38:28.303333Z INFO ExtHandler ExtHandler Extension status: [(u'Microsoft.Azure.AzureHybridBenefit.AHBForRHEL', u'success'), (u'Microsoft.Azure.Extensions.CustomScript', u'success'), (u'Microsoft.Azure.Monitor.AzureMonitorLinuxAgent', u'success'), (u'Microsoft.Azure.Security.Monitoring.AzureSecurityLinuxAgent', u'success'), (u'Microsoft.CPlat.Core.RunCommandLinux', u'success')]
    2022-04-19T15:38:28.304061Z INFO ExtHandler ExtHandler All extensions in the goal state have reached a terminal state: [(u'Microsoft.Azure.AzureHybridBenefit.AHBForRHEL', u'success'), (u'Microsoft.Azure.Extensions.CustomScript', u'success'), (u'Microsoft.Azure.Monitor.AzureMonitorLinuxAgent', u'success'), (u'Microsoft.Azure.Security.Monitoring.AzureSecurityLinuxAgent', u'success'), (u'Microsoft.CPlat.Core.RunCommandLinux', u'success')]

As we can see, the 'enable' function of each extension is provisioned. Is this intended behaviour?

commented

@simratsatia Yes, this is the intended behavior. All the extensions in the Model of the VM need to be executed when anything changes in the VM.