krkn-chaos / krkn

Chaos and resiliency testing tool for Kubernetes with a focus on improving performance under failure conditions. A CNCF sandbox project.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Hog scenarios not tracking the status of the chaos runner correctly

chaitanyaenr opened this issue · comments

Running the litmus io hog scenario, the status is reported as failed even when the runner is in running state:

2022-08-29 19:03:39,609 [INFO] Waiting for node-io-stress to be completed                                                                                                                                                                                                                  
2022-08-29 19:03:49,629 [INFO] Waiting for node-io-stress to be completed                                                                                                                                                                                                                  
192.241.212.238 - - [29/Aug/2022 19:03:57] code 400, message Bad request version ('À\x14À')                                                                                                                                                                                                
192.241.212.238 - - [29/Aug/2022 19:03:57] "��PÑL�-JÕÕê[Ø*+´<R#>¢C3zÿé�dÍÀ/À+ÀÀÀÀ       ÀÀ" 400 -                                                                                                                                                                                          
2022-08-29 19:03:59,646 [INFO] Waiting for node-io-stress to be completed                                                                                                                                                                                                                  
2022-08-29 19:04:09,654 [INFO] Waiting for node-io-stress to be completed                                                                                                                                                                                                                  
2022-08-29 19:04:09,661 [ERROR] Chaos engine node-io-stress took longer than 5 minutes to be completed                                                                                                                                                                                     
2022-08-29 19:04:09,672 [INFO] Chaos scenario:nginx-chaos failed with error: N/A                                                                                                                                                                                                           
2022-08-29 19:04:09,672 [INFO] See 'kubectl get chaosresult nginx-chaos-node-io-stress -n litmus -o yaml' for full results                                                                                                                                                                 
2022-08-29 19:04:09,672 [INFO] Scenario: scenarios/openshift/node_io_engine.yaml was not successfully injected, please check

Pod status from the run:

time="2022-08-29T19:18:53Z" level=info msg="helper pod status: Running"
time="2022-08-29T19:18:54Z" level=info msg="helper pod status: Running"
time="2022-08-29T19:18:55Z" level=info msg="helper pod status: Running"
time="2022-08-29T19:18:56Z" level=info msg="helper pod status: Running"
time="2022-08-29T19:18:56Z" level=info msg="[Status]: The running status of Pods are as follows" Pod=node-io-stress-helper-uojjdr Status=Running
time="2022-08-29T19:18:57Z" level=info msg="[Status]: Getting the status of application node"
time="2022-08-29T19:18:57Z" level=info msg="The Node status are as follows" Node=ip-10-0-196-56.us-west-2.compute.internal Ready=true
time="2022-08-29T19:18:57Z" level=info msg="[Cleanup]: Deleting the helper pod"
time="2022-08-29T19:19:01Z" level=info msg="[Confirmation]: node-io-stress chaos has been injected successfully"
time="2022-08-29T19:19:01Z" level=info msg="[Status]: Verify that the AUT (Application Under Test) is running (post-chaos)"
time="2022-08-29T19:19:01Z" level=info msg="[Status]: No appLabels provided, skipping the application status checks"
time="2022-08-29T19:19:01Z" level=info msg="[The End]: Updating the chaos result of node-io-stress experiment (EOT)"