Mutliple execution with different payload

[Piyush Nath]

In our application we can use only one execution for a payload. Now to do power tuning ,is there at ways in which i can execute with different payloads .

option one : execute with payload and weight , but system tried to execute it for different memory configuration with same set of payload. (ref Weighted Payloads)
option two : have a post process to reset value for payload data, not sure how can this be done in aws lambda power tuning state machine
option three: have a pre process to create different payload, not sure how we can do this, ref :Pre/Post-processing functions of https://serverlessrepo.aws.amazon.com/applications/arn:aws:serverlessrepo:us-east-1:451282441545:applications~aws-lambda-power-tuning

[Alex Casalboni]

Hi @zombaki 👋 thanks for asking :)

It really depends on what your function is doing. Could you share more details about the function to power-tune?

For example, if your function needs to delete a record from a database, I'd recommend setting up a pre-processor to create the record in the database. Because all power configurations will run in parallel (even with parallelInvocation: false), you might need to randomize the ID / primary key, so that a few invocations can work in parallel.

To achieve that, you need to create a Lambda function that will act as pre-processor and then provide it as preProcessorARN when you run the state machine. This pre-processor will receive the original invocation payload and will be able to return an altered payload for the actual power-tuning invocation.

It looks like this:

function Executor:
  iterate from 0 to num:
    [alteredPayload = execute Pre-processor (payload)]
    results = execute Main Function (alteredPayload)

Let me know if you need more help/info, or if your use case is different.

[Piyush Nath]

my limitation is that my lambda can execute once for a particular payload, so wanted to know if i can pass multiple payloads for execution, so that a payload is used only once!

[Alex Casalboni]

You can pass multiple payloads using weighted payloads.

For example, with num: 10 you could pass 10 different payloads (see example below).

This way, each payload would be used only once *for each power configuration*.

{
    "parallelInvocation": false,
    "num": 10,
    "payload": [
        { "payload": { "key": "A" }, "weight": 1 },
        { "payload": { "key": "B" }, "weight": 1 },
        { "payload": { "key": "C" }, "weight": 1 },
        { "payload": { "key": "D" }, "weight": 1 },
        { "payload": { "key": "E" }, "weight": 1 },
        { "payload": { "key": "F" }, "weight": 1 },
        { "payload": { "key": "G" }, "weight": 1 },
        { "payload": { "key": "H" }, "weight": 1 },
        { "payload": { "key": "I" }, "weight": 1 },
        { "payload": { "key": "J" }, "weight": 1 },
    ]
}

The problem is that each power configuration will use these payloads. For example, if you're power-tuning with powerValues: [128, 256, 512], you will have 3 concurrent Lambda invocations with payload "A", then 3 concurrent invocations with payload "B", and so on.

If your function is interacting with the same database (or service) and you need 100% unique payloads, this won't work.

That is why I recommended using pre-processors above.

[Alex Casalboni]

If you implement a pre-processor, you'll be able to do something like this:

{
    "parallelInvocation": true,
    "num": 50,
    "payload": { "key": "A" },
    "preProcessorARN": "ARNofYourPreProcessorFunction"
}

and your pre-processor logic might look like this:

exports.handler =  async function(event, context) {
  const randomString = /* generate random string here */;
  const key = event.key + randomString; // fetch event key from original payload and append random string
  const primaryKey = await db.createNewRecord(key); // create a new db object and fetch new primary key
  return {key: primaryKey}; // return new payload for power-tuning invocation
}

[Alex Casalboni]

Closing this for now, please feel free to reopen if the solution above didn't work for you :)