How to retry on failed executions in singleflight.Do

[Idiakose O. Sunday]

Hello everyone, i'm looking to update some singleflight code to retry if the cached call errored out.
So far, i have this.

type Result struct {
	v int
	k string
}

var group singleflight.Group

// see https://encore.dev/blog/advanced-go-concurrency
func main() {

	if true {
		for k := 0; k <= 2; k++ {
			go doGroup(context.Background(), "sameKey")
		}

		<-time.Tick(5 * time.Second)

		for k := 0; k <= 3; k++ {
			go doGroup(context.Background(), "sameKey")
		}

		<-time.Tick(30 * time.Second)
	}

}
func doGroup(ctx context.Context, key string) (*Result, error) {

	log.Println("Inside normal call")

	results, err, shared := group.Do(key, func() (interface{}, error) {
		r, e := doExpensive(ctx, key)

		// Do this; so if it encountered an error;
		// subsequent calls will retry
		// didnt work
		// perhaps because of timing
		if e != nil {
			group.Forget(key)
		}

		return r, e
	})

	fmt.Printf("Call to multiple callers: %v\n", shared)
	// does not retry if error occured

	if err != nil {
		wrapped := fmt.Errorf("error bruh %s: %w", key, err)
		fmt.Printf("%s\n", wrapped.Error())
		return nil, wrapped
	}

	fmt.Printf("Results: %v\n", results)

	return results.(*Result), err
}

func doExpensive(ctx context.Context, key string) (*Result, error) {
	log.Printf("Inside Expensive function with key %s\n", key)
	<-time.Tick(time.Second * 10)

	dice := rand.Int31n(10)

	if true {
		// <-time.Tick(time.Millisecond * time.Duration(dice*100))
		return nil, errors.New("operation failed")
	}

	<-time.Tick(time.Second * time.Duration(dice))
	return &Result{
		v: int(dice),
		k: key,
	}, nil
}

I'm trying to force a retry by doing a group.Forget(key) to no avail. I've been unable to trigger a reexecution of the doExpensive function.

I've simulated waits between calls to doGroup so the key is actually forgotten by the second call.
But the doExpensive function only seems to ever be called once.

A reproduction is available with golang playground here https://go.dev/play/p/psGjFTypU6C

[Ian Lance Taylor]

I don't think that singleflight works well for that kind of case. It was designed for cases where an error applies to all concurrent calls.

[Idiakose O. Sunday]

What options do i have to build such a thing. I like what singleflight offers, I just dont want to always serve errors, in case of intermittent errors for instance.

[Ian Lance Taylor]

Perhaps, on an intermittent error, you could have have the function retry itself, rather than returning an error to the singleflight caller.

[Idiakose O. Sunday]

ok; thats one way around it.
I think at this point I may be fighting singleflight and going against what its meant for.
Thanks again Ian, I'd try that.