moleculer-go / moleculer

🚀 Progressive microservices framework for Go - based and compatible with https://github.com/moleculerjs/moleculer

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Broker hang if multiple goroutines make calls

casskir opened this issue · comments

Describe the bug
Sometimes when we make calls from multiple goroutines broker is hanged. Also, a timeout error is not fired.

To Reproduce
Make calls from multiple goroutines. Often reproduced on a slow machine.

broker := broker.New(config)

for t := 0; t < 10000; t++ {
		wg := sync.WaitGroup{}
		wg.Add(3)

		go func() {
			defer wg.Done()

			<-broker.Call(action, params)
			log.Info("first done")
		}()

		go func() {
			defer wg.Done()

			<-broker.Call(action, params)
			log.Info("second done")
		}()

		go func() {
			defer wg.Done()

			<-broker.Call(action, params)
			log.Info("third done")
		}()

		wg.Wait()
	}

Expected behavior
All requests successfully finished or execution timeout error returned.

Additional context
My investigation showed that multiple async calls generate requests with the same ID:
image

And the main problem in:

id := util.RandomString(12)

This function using random generator that unsafe for multiple goroutines:

// NewSource returns a new pseudo-random Source seeded with the given value.
// Unlike the default Source used by top-level functions, this source is not
// safe for concurrent use by multiple goroutines.

Another way how reproduce:

t := make([]string, 1000)

	wg := sync.WaitGroup{}
	wg.Add(1000)

	for i := 0; i < 1000; i++ {
		go func(idx int) {
			defer wg.Done()
			t[idx] = util.RandomString(12)
		}(i)
	}

	wg.Wait()

	v := map[string]interface{}{}
	for i := 0; i < len(t); i++ {
		v[t[i]] = nil
	}

	fmt.Printf("Uniq: %d", len(v))

generates less than 1000 unique strings.