SIGSEGV

Question

SIGSEGV

VladyslavLukyanenko opened this issue 2 months ago · comments

Vladyslav Lukyanenko commented 2 months ago

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x95e28d]

goroutine 3244 [running]:
github.com/redis/rueidis.(*lru).purge(0xc0027e1200, {0xc0cb1c53c8, 0x13}, 0xc04b7fbba0)
	/home/runner/go/pkg/mod/github.com/redis/rueidis@v1.0.33/lru.go:299 +0xad
github.com/redis/rueidis.(*lru).Delete(0xc0027e1200, {0xc0dca2c340, 0x1, 0x0?})
	/home/runner/go/pkg/mod/github.com/redis/rueidis@v1.0.33/lru.go:318 +0x15f
github.com/redis/rueidis.(*pipe).handlePush(0xc000a27440, {0xc0afeaf300, 0x0?, 0xc0afeaf300?})
	/home/runner/go/pkg/mod/github.com/redis/rueidis@v1.0.33/pipe.go:640 +0x4fa
github.com/redis/rueidis.(*pipe)._backgroundRead(0xc000a27440)
	/home/runner/go/pkg/mod/github.com/redis/rueidis@v1.0.33/pipe.go:487 +0x32a
github.com/redis/rueidis.(*pipe)._background(0xc000a27440)
	/home/runner/go/pkg/mod/github.com/redis/rueidis@v1.0.33/pipe.go:342 +0x88
created by github.com/redis/rueidis.(*pipe).background.func1 in goroutine 3134
	/home/runner/go/pkg/mod/github.com/redis/rueidis@v1.0.33/pipe.go:325 +0x56

Rueian · Answer 1 · Wed Apr 10 2024 22:40:36 GMT+0800 (China Standard Time)

Hi @VladyslavLukyanenko,

Could you provide the go build command, the GOARCH, and the GOOS env variable you used for building the executable? I would like to locate where the +0xad points to.

Vladyslav Lukyanenko · Answer 2 · Wed Apr 10 2024 23:08:18 GMT+0800 (China Standard Time)

@rueian

go build command: go build -o ${{ env.BINARY_NAME }} cmd/productpage/main.go
GOARCH: amd64
GOOS: linux

It was built in github action, using golang version 1.22, on runs-on: ubuntu-latest

Rueian · Answer 3 · Wed Apr 10 2024 23:55:02 GMT+0800 (China Standard Time)

Thank you for the information. According to the amd64 object dump:

TEXT github.com/redis/rueidis.(*lru).purge(SB) /Users/ruian/Code/go/rueidis/lru.go
func (c *lru) purge(key string, kc *keyCache) {
  0x5f7be0		4c8d6424e8		LEAQ -0x18(SP), R12	
  0x5f7be5		4d3b6610		CMPQ R12, 0x10(R14)	
  0x5f7be9		0f86e4010000		JBE 0x5f7dd3		
  0x5f7bef		55			PUSHQ BP		
  0x5f7bf0		4889e5			MOVQ SP, BP		
  0x5f7bf3		4881ec90000000		SUBQ $0x90, SP		
  0x5f7bfa		48899c24a8000000	MOVQ BX, 0xa8(SP)	
	if kc != nil {
  0x5f7c02		4885ff			TESTQ DI, DI		
  0x5f7c05		7460			JE 0x5f7c67		
  0x5f7c07		48898424a0000000	MOVQ AX, 0xa0(SP)	
  0x5f7c0f		4889bc24b8000000	MOVQ DI, 0xb8(SP)	
  0x5f7c17		48898c24b0000000	MOVQ CX, 0xb0(SP)	
  0x5f7c1f		48899c24a8000000	MOVQ BX, 0xa8(SP)	
		for cmd, ele := range kc.cache {
  0x5f7c27		488b1f			MOVQ 0(DI), BX			
  0x5f7c2a		488d7c2430		LEAQ 0x30(SP), DI		
  0x5f7c2f		488d7fe0		LEAQ -0x20(DI), DI		
  0x5f7c33		660f1f840000000000	NOPW 0(AX)(AX*1)		
  0x5f7c3c		0f1f4000		NOPL 0(AX)			
  0x5f7c40		48896c24f0		MOVQ BP, -0x10(SP)		
  0x5f7c45		488d6c24f0		LEAQ -0x10(SP), BP		
  0x5f7c4a		e8fc46e7ff		CALL 0x46c34b			
  0x5f7c4f		488b6d00		MOVQ 0(BP), BP			
  0x5f7c53		488d4c2430		LEAQ 0x30(SP), CX		
  0x5f7c58		488d0501110400		LEAQ 0x41101(IP), AX		
  0x5f7c5f		90			NOPL				
  0x5f7c60		e81b89e1ff		CALL runtime.mapiterinit(SB)	
  0x5f7c65		eb13			JMP 0x5f7c7a			
}
  0x5f7c67		4881c490000000		ADDQ $0x90, SP		
  0x5f7c6e		5d			POPQ BP			
  0x5f7c6f		c3			RET			
		for cmd, ele := range kc.cache {
  0x5f7c70		488d442430		LEAQ 0x30(SP), AX		
  0x5f7c75		e8268be1ff		CALL runtime.mapiternext(SB)	
  0x5f7c7a		488b542430		MOVQ 0x30(SP), DX		
  0x5f7c7f		90			NOPL				
  0x5f7c80		4885d2			TESTQ DX, DX			
  0x5f7c83		74e2			JE 0x5f7c67			
  0x5f7c85		488b742438		MOVQ 0x38(SP), SI		
  0x5f7c8a		488b36			MOVQ 0(SI), SI			
			e := ele.Value.(*cacheEntry)
  0x5f7c8d		488b4618		MOVQ 0x18(SI), AX	
  0x5f7c91		488d1da8dc0300		LEAQ 0x3dca8(IP), BX	
  0x5f7c98		0f1f840000000000	NOPL 0(AX)(AX*1)	
  0x5f7ca0		4839d8			CMPQ AX, BX		
  0x5f7ca3		0f851d010000		JNE 0x5f7dc6		
  0x5f7ca9		4c8b4620		MOVQ 0x20(SI), R8	
			if e.val.typ != 0 { // do not delete pending entries
  0x5f7cad		4180786800		CMPB 0x68(R8), $0x0	
  0x5f7cb2		74bc			JE 0x5f7c70		
		for cmd, ele := range kc.cache {
  0x5f7cb4		4889742420		MOVQ SI, 0x20(SP)	
			e := ele.Value.(*cacheEntry)
  0x5f7cb9		4c89442428		MOVQ R8, 0x28(SP)	
		for cmd, ele := range kc.cache {
  0x5f7cbe		488b0a			MOVQ 0(DX), CX		
  0x5f7cc1		488b7a08		MOVQ 0x8(DX), DI	
				if delete(kc.cache, cmd); len(kc.cache) == 0 {
  0x5f7cc5		488b9424b8000000	MOVQ 0xb8(SP), DX			
  0x5f7ccd		488b1a			MOVQ 0(DX), BX				
  0x5f7cd0		488d0589100400		LEAQ 0x41089(IP), AX			
  0x5f7cd7		e8c4c2e1ff		CALL runtime.mapdelete_faststr(SB)

The offset 0xad(173) from the 0x5f7be0 should be the

0x5f7c8d		488b4618		MOVQ 0x18(SI), AX

which is

e := ele.Value.(*cacheEntry)

While I am still trying to figure out how could this happen, I have prepared a mitigation #529 and a pre-release tag https://github.com/redis/rueidis/releases/tag/v1.0.35-dev1

@VladyslavLukyanenko, would you mind using the v1.0.35-dev1 to verify if the issue has been solved?

Vladyslav Lukyanenko · Answer 4 · Thu Apr 11 2024 00:34:15 GMT+0800 (China Standard Time)

@VladyslavLukyanenko, would you mind using the v1.0.35-dev1 to verify if the issue has been solved?

I'm going to do so, but it's the first time i see this issue over couple months of using the library, will let you know whether i spot it again

Rueian · Answer 5 · Fri Apr 12 2024 23:29:18 GMT+0800 (China Standard Time)

@VladyslavLukyanenko, would you mind using the v1.0.35-dev1 to verify if the issue has been solved?

I'm going to do so, but it's the first time i see this issue over couple months of using the library, will let you know whether i spot it again

Thank you for your cooperation. I will merge the mitigation and close this issue first.