How to keep the order of result?

[Hoang Trinh]

Hi everyone,

I'm using conc to handle concurrency.
For example, for each user_id, I have 2 tasks:

• Fetch user data from the user service (returned data is called userData)
• Fetch loyalty points from the reward service (returned data is called loyaltyPoints)

I want to do these 2 tasks concurrently, and the result is saved into the results variable
I expected the userData = results[0] and loyaltyPoints = results[1] (the order that they were added to the pool.NewWithResults[any]().WithContext(ctx))

But I realized that the order of result is not guaranteed, sometimes loyaltyPoints = results[0] and vice versa

How can I keep the order of result?
Thanks

[Camden Cheek]

Hi @piavgh!

You're right that a result pool does not collect results in order. For something where you have just two tasks, I'd recommend doing something like the following:

func fetchUserDataAndLoyaltyPoints(ctx context.Context, userID int) (UserData, LoyaltyPoints, error) {
    p := pool.New().WithContext()
    
    var userData UserData
    p.Go(func(ctx context.Context) error {
        var err error
        userData, err = fetchUserData(ctx, userID)
        return err
    })
    
    var loyaltyPoints LoyaltyPoints
    p.Go(func(ctx context.Context) error {
        var err error
        loyaltyPoints, err = fetchLoyaltyPoints(ctx, userID)
        return err
    })
    
    if err := p.Wait(); err != nil {
        return nil, nil, err
    }
    
    return userData, loyaltyPoints, nil
}

In general, if you want to maintain the order of results submitted, you can use iter.Map() for a predetermined set of inputs, or you can use the stream.Stream, which will call its callbacks in the same order the tasks were submitted.

[Hoang Trinh]

Hi @piavgh!

You're right that a result pool does not collect results in order. For something where you have just two tasks, I'd recommend doing something like the following:

func fetchUserDataAndLoyaltyPoints(ctx context.Context, userID int) (UserData, LoyaltyPoints, error) {
    p := pool.New().WithContext()
    
    var userData UserData
    p.Go(func(ctx context.Context) error {
        var err error
        userData, err = fetchUserData(ctx, userID)
        return err
    })
    
    var loyaltyPoints LoyaltyPoints
    p.Go(func(ctx context.Context) error {
        var err error
        loyaltyPoints, err = fetchLoyaltyPoints(ctx, userID)
        return err
    })
    
    if err := p.Wait(); err != nil {
        return nil, nil, err
    }
    
    return userData, loyaltyPoints, nil
}

In general, if you want to maintain the order of results submitted, you can use iter.Map() for a predetermined set of inputs, or you can use the stream.Stream, which will call its callbacks in the same order the tasks were submitted.

@camdencheek : Thanks a lot, this looks much much more elegant

Yesterday, I tried to use Stream to handle this problem, but because Stream does not handle the error propagation like Pool does, the code looks really ugly (I have to define 2 other error variables outside)
And another plus point is that I don't have to type assert from any to UserData or LoyaltyPoints

If Stream supports error propagation like Pool does, I think this conc library will be much better. Do you guys have it in the roadmap?

[Camden Cheek]

If Stream supports error propagation like Pool does, I think this conc library will be much better. Do you guys have it in the roadmap?

Probably not before 1.0 (which should be soon). The API of the stream package is a little awkward to avoid having a massive soup of generics. I've been experimenting with some different API designs there, but am struggling to get something I'm happy with because streaming is somewhat difficult to fit into a "scoped" paradigm