panic: runtime error: index out of range [0] with length 0 expiring zero versions in ILM

[Viktor3434]

Hello!

Expected Behavior

I am using helm chart's operator and tennant.
I decided to upgrade the chart version from 5.0.9 to 5.0.14
I was expecting the minio to be stable and the pods to not restart.

Current Behavior

Cotntainers in pods crash with an error after a while and restart.

Possible Solution

-

Steps to Reproduce (for bugs)

I think you can try the following

1. install version 5.0.9 from the chart
2. upgrade to version 5.0.14

Context

version update

Regression

-

Your Environment

• Version used (minio-operator): 5.0.14
• Environment name and version (e.g. kubernetes v1.17.2): k8s v1.26.5
• Server type and version: CentOS Stream 8
• Operating System and version (uname -a):

Linux k8s-master 4.18.0-500.el8.x86_64 minio/operator#1 SMP Wed Jun 28 00:07:07 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Linux node01        4.18.0-500.el8.x86_64 minio/operator#1 SMP Wed Jun 28 00:07:07 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

• Link to your deployment file:
  (my values for the chart)
  minio.yaml.txt
  minio-operator.yaml.txt

Error

API: SYSTEM()
Time: 10:11:31 UTC 05/02/2024
DeploymentID: 77bf4ae8-eca7-4d07-82aa-3dfd7cd304e
Error: Drive: http://minio-pool-0-0.minio-hl.data-store.svc.cluster.local:9000/export/data returned drive not found (*fmt.wrapError)э
       endpoint="http://minio-pool-0-0.minio-hl.data-store.svc.cluster.local:9000/export/data"
       3: internal/logger/logger.go:250:logger.LogAlwaysIf()
       2: cmd/prepare-storage.go:51:cmd.glob..func24.1()
       1: cmd/erasure-sets.go:235:cmd.(*erasureSets).connectDisks.func2()
panic: runtime error: index out of range [0] with length 0

goroutine 3307 [running]:github.com/minio/minio/cmd.newerNoncurrentTask.OpHash(...)	github.com/minio/minio/cmd/bucket-lifecycle.go:149
github.com/minio/minio/cmd.(*expiryState).enqueueByNewerNoncurrent(0xc03f2ed380?, {0xc03f889ec0?, 0xc03f8e6810?}, {0xc03f252540?, 0x0?, 0x0?}, {0x2, {0xc0010314e8, 0x14}, {0x0, ...}, ...})
	github.com/minio/minio/cmd/bucket-lifecycle.go:237 +0x26cgithub.com/minio/minio/cmd.(*scannerItem).applyNewerNoncurrentVersionLimit(0xc03ff75248, {0x4d4c7d8, 0xc03f829cc0}, {0x92d?, 0x5?}, {0xc03f8f8000, 0x6, 0x6}, 0x202020313f2ed201?)
github.com/minio/minio/cmd/data-scanner.go:1054 +0xf90github.com/minio/minio/cmd.(*scannerItem).applyVersionActions(0xc03ff75248, {0x4d4c7d8?, 0xc03f829cc0?}, {0x4d7e948?, 0xc001f50000?}, {0xc03f8f8000?, 0x0?, 0xc03f8f8000?}, 0x6?)	
	github.com/minio/minio/cmd/data-scanner.go:1061 +0x52github.com/minio/minio/cmd.(*xlStorage).NSScanner.func2({{0xc03f239960, 0x6c}, {0xc03f889ec0, 0x7}, {0xc03f2e1620, 0x9}, {0xc03f889ed2, 0x45}, {0x0, 0x0}, ...})	
github.com/minio/minio/cmd/xl-storage.go:597 +0xaedgithub.com/minio/minio/cmd.(*folderScanner).scanFolder.func2({0xc03f2e1604?, 0xc00208002b?}, 0x0)	
	github.com/minio/minio/cmd/data-scanner.go:490 +0xbdagithub.com/minio/minio/cmd.readDirFn({0xc03f239880, 0x64}, 0xc03ff76348)	
github.com/minio/minio/cmd/os_unix.go:249 +0x72fgithub.com/minio/minio/cmd.(*folderScanner).scanFolder(0xc03fa32000, {0x4d4c7d8, 0xc03f829cc0}, {{0xc03f7f18c0?, 0xc03ff76bc8?}, 0xc03f840660?, 0x4170fa?}, 0xc03f8f45b0)	
	github.com/minio/minio/cmd/data-scanner.go:417 +0x794github.com/minio/minio/cmd.(*folderScanner).scanFolder.func3({{0xc03f7f18c0?, 0x57?}, 0xc03f840660?, 0x4?})	
github.com/minio/minio/cmd/data-scanner.go:571 +0xd6github.com/minio/minio/cmd.(*folderScanner).scanFolder(0xc03fa32000, {0x4d4c7d8, 0xc03f829cc0}, {{0xc03f5ad158?, 0x0?}, 0xc000d3e2e0?, 0x410c25?}, 0xc03f825d40)	
	github.com/minio/minio/cmd/data-scanner.go:618 +0x1127github.com/minio/minio/cmd.(*folderScanner).scanFolder.func3({{0xc03f5ad158?, 0x11?}, 0xc000d3e2e0?, 0x14?})	
github.com/minio/minio/cmd/data-scanner.go:571 +0xd6github.com/minio/minio/cmd.(*folderScanner).scanFolder(0xc03fa32000, {0x4d4c7d8, 0xc03f829cc0}, {{0xc03fa04187?, 0x7?}, 0x0?, 0x3ccb25a0?}, 0xc03ff79188)	
	github.com/minio/minio/cmd/data-scanner.go:647 +0x154dgithub.com/minio/minio/cmd.scanDataFolder({0x4d4c7d8, 0xc03f829cc0}, {0xc03fa02380, 0x4d2ba40?, 0x4}, {_, _}, {{{0xc03fa04187, 0x7}, 0x4cc7e, ...}, ...}, ...)	
github.com/minio/minio/cmd/data-scanner.go:345 +0x3f2github.com/minio/minio/cmd.(*xlStorage).NSScanner(0xc0000ee5a0, {0x4d4c7d8?, 0xc03f829cc0}, {{{0xc03fa04187, 0x7}, 0x4cc7e, {0x355a2288, 0xeddc55d94, 0x64e5f80}, 0x0, ...}, ...}, ...)	
	github.com/minio/minio/cmd/xl-storage.go:552 +0x710github.com/minio/minio/cmd.(*xlStorageDiskIDCheck).NSScanner(0xc0010c0900, {0x4d4c7d8, 0xc03f829cc0}, {{{0xc03fa04187, 0x7}, 0x4cc7e, {0x355a2288, 0xeddc55d94, 0x64e5f80}, 0x0, ...}, ...}, ...)	
github.com/minio/minio/cmd/xl-storage-disk-id-check.go:245 +0x1ffgithub.com/minio/minio/cmd.erasureObjects.nsScanner.func2(0x2?)	
	github.com/minio/minio/cmd/erasure.go:530 +0x624
created by github.com/minio/minio/cmd.erasureObjects.nsScanner in goroutine 3008	
	github.com/minio/minio/cmd/erasure.go:483 +0x8ff

[Viktor3434]

I saw a similar problem here URL

[Harshavardhana]

Please upgrade to latest release of MinIO @Viktor3434

commit 383489d5d96e673a6b5ce43facdad70c0b0b1d20
Author: Krishnan Parthasarathi <[e-mail]>
Date:   Tue Mar 19 17:10:58 2024 -1000

    Handle zero versions qualified for expiration (#19301)
    
    When objects have more versions than their ILM policy expects to retain
    via NewerNoncurrentVersions, but they don't qualify for expiry due to
    NoncurrentDays are configured in that rule.
    
    In this case, applyNewerNoncurrentVersionsLimit method was enqueuing empty
    tasks, which lead to a panic (panic: runtime error: index out of range [0] with
    length 0) in newerNoncurrentTask.OpHash method, which assumes the task
    to contain at least one version to expire.

Fixed since RELEASE.2024-03-21T23-13-43Z your yaml image says

  image:
    repository: quay.io/minio/minio
    tag: RELEASE.2024-03-15T01-07-19Z
    pullPolicy: IfNotPresent

[Klaus Post]

Nah. No relation. It seems you are running version before this: #18926

Please upgrade.

[Viktor3434]

thanks!
maybe fix the default values in the chart?