k8gb-io / k8gb

A cloud native Kubernetes Global Balancer

Home Page:https://www.k8gb.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

`TestFailoverPlayground ` flaky terratest

ytsarev opened this issue Β· comments

Even after #1340 there TestFailoverPlayground if failing from time to time

Example run: https://github.com/k8gb-io/k8gb/actions/runs/7188544157

Terratests Results πŸš€
--- ❌ FAIL: TestFailoverPlayground (143.83s)
--- βœ“ PASS: TestFailoverPlayground/failover_on_two_concurrent_clusters_with_TestApp_running (0.75s)
--- βœ“ PASS: TestFailoverPlayground/stop_podinfo_on_eu_cluster (45.94s)
--- ❌ FAIL: TestFailoverPlayground/start_podinfo_again_on_eu_cluster (6.52s)

Detailed related logs from the dump:

estFailoverPlayground 2023-12-12T23:36:13Z logger.go:66: Running command kubectl with args [--context k3d-test-gslb1 -n k8gb-test-korpdu run -i --rm busybox --restart Never --image busybox --overrides {"spec":{"dnsConfig":{"nameservers":["10.43.52.104"]},"dnsPolicy": "None"}} -- sh -c wget -qO - playground-failover.cloud.example.com]
TestFailoverPlayground 2023-12-12T23:36:16Z logger.go:66: wget: server returned error: HTTP/1.1 503 Service Temporarily Unavailable
TestFailoverPlayground 2023-12-12T23:36:16Z logger.go:66: pod "busybox" deleted
TestFailoverPlayground 2023-12-12T23:36:16Z logger.go:66: pod k8gb-test-korpdu/busybox terminated (Error)
    extensions.go:437: 
        	Error Trace:	/home/runner/work/k8gb/k8gb/terratest/test/extensions.go:437
        	            				/home/runner/work/k8gb/k8gb/terratest/test/k8gb_failover_playground_test.go:60
        	            				/home/runner/work/k8gb/k8gb/terratest/test/k8gb_failover_playground_test.go:85
        	Error:      	Received unexpected error:
        	            	error while running command: exit status 1; pod k8gb-test-korpdu/busybox terminated (Error)
        	Test:       	TestFailoverPlayground
        	Messages:   	busybox%!(EXTRA []string=[sh -c wget -qO - playground-failover.cloud.example.com], string=wget: server returned error: HTTP/1.1 503 Service Temporarily Unavailable
        	            	pod "busybox" deleted
        	            	pod k8gb-test-korpdu/busybox terminated (Error))
=== CONT  TestFailoverPlayground
    extensions.go:245: killing Instance: k3d-test-gslb2:k8gb-test-109htg
TestFailoverPlayground 2023-12-12T23:36:16Z client.go:42: Configuring Kubernetes client using config file /home/runner/.kube/config with context k3d-test-gslb2
    extensions.go:245: killing Instance: k3d-test-gslb1:k8gb-test-korpdu
TestFailoverPlayground 2023-12-12T23:36:16Z client.go:42: Configuring Kubernetes client using config file /home/runner/.kube/config with context k3d-test-gslb1
--- FAIL: TestFailoverPlayground (143.83s)
    --- PASS: TestFailoverPlayground/failover_on_two_concurrent_clusters_with_TestApp_running (0.75s)
    --- PASS: TestFailoverPlayground/stop_podinfo_on_eu_cluster (45.94s)
    --- FAIL: TestFailoverPlayground/start_podinfo_again_on_eu_cluster (6.52s)

and

=== RUN   TestFailoverPlayground/start_podinfo_again_on_eu_cluster
=== CONT  TestFailoverPlayground/start_podinfo_again_on_eu_cluster
    testing.go:1343: test executed panic(nil) or runtime.Goexit: subtest may have called FailNow on a parent test
    --- FAIL: TestFailoverPlayground/start_podinfo_again_on_eu_cluster (6.52s)