Night CI 2024-02-16: Failed -- timeout when run test case R00007
weizhoublue opened this issue · comments
• [FAILED] [135.389 seconds]
Reliability [Reliability]
/home/runner/work/egressgateway/egressgateway/test/e2e/reliability/reliability_test.go:30
Test the drift of the EIP
/home/runner/work/egressgateway/egressgateway/test/e2e/reliability/reliability_test.go:31
restart components [R00007]
/home/runner/work/egressgateway/egressgateway/test/e2e/reliability/reliability_test.go:225
[It] restart calico-node
/home/runner/work/egressgateway/egressgateway/test/e2e/reliability/reliability_test.go:298
Timeline >>
> Enter [BeforeEach] Test the drift of the EIP - /home/runner/work/egressgateway/egressgateway/test/e2e/reliability/reliability_test.go:43 @ 02/16/24 20:47:51.991
succeeded to create the gateway: egw-a6c21cf9-9a5c-469d-a0c0-7171f2319ba0
v4DefaultEip: 172.18.6.3, v6DefaultEip: fc00:f853:ccd:e793::603
succeeded to create DaemonSet: ds-reliability-08af0855-5392-44c5-b03f-c4f1ed6b36bf
succeeded to create policy: policy-2f1a83b9-83e1-42a6-a073-9b6cb3f48a71lzbhl
succeeded to check the export IP of the daemonSet: ds-reliability-08af0855-5392-44c5-b03f-c4f1ed6b36bf
< Exit [BeforeEach] Test the drift of the EIP - /home/runner/work/egressgateway/egressgateway/test/e2e/reliability/reliability_test.go:43 @ 02/16/24 20:48:01.645 (9.654s)
> Enter [It] restart calico-node - /home/runner/work/egressgateway/egressgateway/test/e2e/reliability/reliability_test.go:298 @ 02/16/24 20:48:01.645
succeeded to check the export IP of the daemonSet: ds-reliability-08af0855-5392-44c5-b03f-c4f1ed6b36bf
succeeded to check the export IP of the daemonSet: ds-reliability-08af0855-5392-44c5-b03f-c4f1ed6b36bf
Automatically polling progress:
Reliability Test the drift of the EIP restart components restart calico-node (Spec Runtime: 29.656s)
/home/runner/work/egressgateway/egressgateway/test/e2e/reliability/reliability_test.go:298
In [It] (Node Runtime: 20.001s)
/home/runner/work/egressgateway/egressgateway/test/e2e/reliability/reliability_test.go:298
Spec Goroutine
goroutine 546 [syscall]
syscall.Syscall6(0xc000263550?, 0x6315e5?, 0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/opt/hostedtoolcache/go/1.20.5/x64/src/syscall/syscall_linux.go:91
os.(*Process).blockUntilWaitable(0xc000555a40)
/opt/hostedtoolcache/go/1.20.5/x64/src/os/wait_waitid.go:32
os.(*Process).wait(0xc000555a40)
/opt/hostedtoolcache/go/1.20.5/x64/src/os/exec_unix.go:22
os.(*Process).Wait(...)
/opt/hostedtoolcache/go/1.20.5/x64/src/os/exec.go:132
os/exec.(*Cmd).Wait(0xc0007991e0)
/opt/hostedtoolcache/go/1.20.5/x64/src/os/exec/exec.go:890
os/exec.(*Cmd).Run(0xc0004e1f00?)
/opt/hostedtoolcache/go/1.20.5/x64/src/os/exec/exec.go:590
os/exec.(*Cmd).CombinedOutput(0xc0007991e0)
/opt/hostedtoolcache/go/1.20.5/x64/src/os/exec/exec.go:1005
github.com/spidernet-io/egressgateway/test/e2e/common.CheckPodEgressIP({_, _}, _, {{{0x0, 0x0}, {0x0, 0x0}}, {{0xc0005e1880, 0x39}, {0xc0005e18c0, ...}, ...}, ...}, ...)
/home/runner/work/egressgateway/egressgateway/test/e2e/common/check_eip.go:93
github.com/spidernet-io/egressgateway/test/e2e/common.CheckDaemonSetEgressIP({_, _}, {_, _}, _, {0x1, 0x1, {{0xc0000e649c, 0x4}, 0x0, ...}, ...}, ...)
/home/runner/work/egressgateway/egressgateway/test/e2e/common/check_eip.go:43
> github.com/spidernet-io/egressgateway/test/e2e/reliability_test.glob..func3.1.5(0xc00024bcb0, 0x1bf08eb000)
/home/runner/work/egressgateway/egressgateway/test/e2e/reliability/reliability_test.go:259
|
| for !isRerun.Load() {
> err := common.CheckDaemonSetEgressIP(ctx, cli, config, egressConfig, daemonSet,
| policy.Status.Eip.Ipv4, policy.Status.Eip.Ipv6, true)
| Expect(err).NotTo(HaveOccurred())
reflect.Value.call({0x2373d00?, 0xc0000000c0?, 0x0?}, {0x25adea1, 0x4}, {0xc0000ae360, 0x2, 0xc0007d7d78?})
/opt/hostedtoolcache/go/1.20.5/x64/src/reflect/value.go:586
reflect.Value.Call({0x2373d00?, 0xc0000000c0?, 0x29842a0?}, {0xc0000ae360, 0x2, 0x2})
/opt/hostedtoolcache/go/1.20.5/x64/src/reflect/value.go:370
> github.com/onsi/ginkgo/v2.invokeFunction({0x2373d00?, 0xc0000000c0}, {0xc0001d8100, 0x2, 0x0?})
/home/runner/work/egressgateway/egressgateway/vendor/github.com/onsi/ginkgo/v2/table_dsl.go:335
| }
|
> return reflect.ValueOf(function).Call(inValues)
| }
|
> github.com/onsi/ginkgo/v2.generateTable.func2.2()
/home/runner/work/egressgateway/egressgateway/vendor/github.com/onsi/ginkgo/v2/table_dsl.go:299
| panic(err)
| }
> invokeFunction(internalBody, entry.parameters)
| })
| }
> github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0xc00078c660, 0xc0007d7f80})
/home/runner/work/egressgateway/egressgateway/vendor/github.com/onsi/ginkgo/v2/internal/node.go:463
|
| body := arg.(func())
> return func(SpecContext) { body() }, false
| }
|
> github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
/home/runner/work/egressgateway/egressgateway/vendor/github.com/onsi/ginkgo/v2/internal/suite.go:889
| }()
|
> node.Body(sc)
| finished = true
| }()
> github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
/home/runner/work/egressgateway/egressgateway/vendor/github.com/onsi/ginkgo/v2/internal/suite.go:876
| failureC := make(chan types.Failure)
|
> go func() {
| finished := false
| defer func() {
Goroutines of Interest
goroutine 39 [select]
> github.com/onsi/ginkgo/v2/internal/interrupt_handler.(*InterruptHandler).registerForInterrupts.func1()
/home/runner/work/egressgateway/egressgateway/vendor/github.com/onsi/ginkgo/v2/internal/interrupt_handler/interrupt_handler.go:107
| pollTicker := time.NewTicker(ABORT_POLLING_INTERVAL)
| for {
> select {
| case <-pollTicker.C:
| if handler.client.ShouldAbort() {
> github.com/onsi/ginkgo/v2/internal/interrupt_handler.(*InterruptHandler).registerForInterrupts
/home/runner/work/egressgateway/egressgateway/vendor/github.com/onsi/ginkgo/v2/internal/interrupt_handler/interrupt_handler.go:104
| if handler.client != nil {
| abortChannel = make(chan interface{})
> go func() {
| pollTicker := time.NewTicker(ABORT_POLLING_INTERVAL)
| for {
goroutine 40 [select, 3 minutes]
> github.com/onsi/ginkgo/v2/internal/interrupt_handler.(*InterruptHandler).registerForInterrupts.func2(0xc0005ded20)
/home/runner/work/egressgateway/egressgateway/vendor/github.com/onsi/ginkgo/v2/internal/interrupt_handler/interrupt_handler.go:131
| var interruptCause InterruptCause
| for {
> select {
| case <-signalChannel:
| interruptCause = InterruptCauseSignal
> github.com/onsi/ginkgo/v2/internal/interrupt_handler.(*InterruptHandler).registerForInterrupts
/home/runner/work/egressgateway/egressgateway/vendor/github.com/onsi/ginkgo/v2/internal/interrupt_handler/interrupt_handler.go:128
| }
|
> go func(abortChannel chan interface{}) {
| var interruptCause InterruptCause
| for {
goroutine 41 [select, 3 minutes]
> github.com/onsi/ginkgo/v2/internal.RegisterForProgressSignal.func1()
/home/runner/work/egressgateway/egressgateway/vendor/github.com/onsi/ginkgo/v2/internal/progress_report.go:32
| go func() {
| for {
> select {
| case <-signalChannel:
| handler()
> github.com/onsi/ginkgo/v2/internal.RegisterForProgressSignal
/home/runner/work/egressgateway/egressgateway/vendor/github.com/onsi/ginkgo/v2/internal/progress_report.go:30
| }
| ctx, cancel := context.WithCancel(context.Background())
> go func() {
| for {
| select {
goroutine 549 [sleep]
time.Sleep(0x3b9aca00)
/opt/hostedtoolcache/go/1.20.5/x64/src/runtime/time.go:195
github.com/spidernet-io/egressgateway/test/e2e/common.DeletePodsUntilReady({0x[2973](https://github.com/spidernet-io/egressgateway/actions/runs/7935604457/job/21670155659#step:10:2974)030, 0xc0001ba008}, {0x297ac00, 0xc0007001b0}, 0x43e000?, 0x0?)
/home/runner/work/egressgateway/egressgateway/test/e2e/common/pod.go:269
> github.com/spidernet-io/egressgateway/test/e2e/reliability_test.glob..func3.1.5.1()
/home/runner/work/egressgateway/egressgateway/test/e2e/reliability/reliability_test.go:254
| defer GinkgoRecover()
|
> err := common.DeletePodsUntilReady(ctx, cli, labels, timeout)
| Expect(err).NotTo(HaveOccurred())
| }()
> github.com/spidernet-io/egressgateway/test/e2e/reliability_test.glob..func3.1.5
/home/runner/work/egressgateway/egressgateway/test/e2e/reliability/reliability_test.go:249
| wg.Add(1)
|
> go func() {
| defer wg.Done()
| defer isRerun.Store(true)
succeeded to check the export IP of the daemonSet: ds-reliability-08af0855-5392-44c5-b03f-c4f1ed6b36bf
succeeded to check the export IP of the daemonSet: ds-reliability-08af0855-5392-44c5-b03f-c4f1ed6b36bf
succeeded to check the export IP of the daemonSet: ds-reliability-08af0855-5392-44c5-b03f-c4f1ed6b36bf
succeeded to check the export IP of the daemonSet: ds-reliability-08af0855-5392-44c5-b03f-c4f1ed6b36bf
succeeded to check the export IP of the daemonSet: ds-reliability-08af0855-5392-44c5-b03f-c4f1ed6b36bf
succeeded to check the export IP of the daemonSet: ds-reliability-08af0855-5392-44c5-b03f-c4f1ed6b36bf
succeeded to check the export IP of the daemonSet: ds-reliability-08af0855-5392-44c5-b03f-c4f1ed6b36bf
succeeded to check the export IP of the daemonSet: ds-reliability-08af0855-5392-44c5-b03f-c4f1ed6b36bf
succeeded to check the export IP of the daemonSet: ds-reliability-08af0855-5392-44c5-b03f-c4f1ed6b36bf
succeeded to check the export IP of the daemonSet: ds-reliability-08af0855-5392-44c5-b03f-c4f1ed6b36bf
succeeded to check the export IP of the daemonSet: ds-reliability-08af0855-5392-44c5-b03f-c4f1ed6b36bf
succeeded to check the export IP of the daemonSet: ds-reliability-08af0855-5392-44c5-b03f-c4f1ed6b36bf
succeeded to check the export IP of the daemonSet: ds-reliability-08af0855-5392-44c5-b03f-c4f1ed6b36bf
succeeded to check the export IP of the daemonSet: ds-reliability-08af0855-5392-44c5-b03f-c4f1ed6b36bf
succeeded to check the export IP of the daemonSet: ds-reliability-08af0855-5392-44c5-b03f-c4f1ed6b36bf
check egress gateway status
check egress policy status
[FAILED] Unexpected error:
<*errors.errorString | 0xc0005b75c0>:
error timeout
{s: "error timeout"}
occurred
In [It] at: /home/runner/work/egressgateway/egressgateway/test/e2e/reliability/reliability_test.go:255 @ 02/16/24 20:50:06.638
< Exit [It] restart calico-node - /home/runner/work/egressgateway/egressgateway/test/e2e/reliability/reliability_test.go:298 @ 02/16/24 20:50:06.638 (2m4.992s)