Improve weighted payload logging in case of invocation error
cp-k1nakayama opened this issue · comments
I prepared each function for CRUD operation of data.
Among them, I wanted to collect the statistics of the function for deletion, but the deletion target put in Payload is deleted by the first execution, and an error occurs from the second time.
In order to avoid this, I prepared data to be deleted more than the total number of executions and gave the payload a weight of all 1 and executed it, but I could not execute it because of an Invalid payload weight (num is too small) error was.
Please tell me a good way to handle such a function.
Hi @cp-k1nakayama, thanks for reporting this.
Based on the error message, I believe you've been using an incorrect weighted payload. Is it possible that one of the weights you configured is zero or negative?
Can you please share your state machine input payload?
Thank you for your reply.
The payload I entered into the state machine is:
{
"lambdaARN": "arn:aws:lambda:ap-northeast-1:************:function:manager-backend-DeploySta-deleteOrganizationFuncti-1SWSWWNFWCNBV",
"powerValues": [
128,
256,
512,
1024,
2048,
3008
],
"num": 5,
"payload": [
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"1024616\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"1043330\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"1043330\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"1263738\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"1509560\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"1578032\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"1641783\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"1720964\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"1725691\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"1906792\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"1922442\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"1990699\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"2006918\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"2141353\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"2456075\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"2477393\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"2672457\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"2722396\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"2853427\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"3062096\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"3072622\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"3400131\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"3452433\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"3830803\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"4212111\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"4375014\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"4521897\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"4596692\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"4751382\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"4828651\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"5045787\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"5245486\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"5378162\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"5418361\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"5460927\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"5523596\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"5608519\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"5885199\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"5947230\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"6068734\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"6122687\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"6138701\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"6146121\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"6158087\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"6485729\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"6526360\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"6884684\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"7039870\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"7188506\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"7383084\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"7451105\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"7466019\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"7484416\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"7509895\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"7681763\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"8001743\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"8445216\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"8544480\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"8859075\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"8895302\"}}",
"weight": 1
},
{
"payload": "{\"headers\":{},\"requestContext\":{\"authorizer\":{\"cognito:username\":\"0123456\"}},\"pathParameters\":{\"id\":\"8948757\"}}",
"weight": 1
}
],
"parallelInvocation": true,
"strategy": "balanced",
"balancedWeight": 0.3
}Lambda of manager-backend-DeploySta-deleteOrganizationFuncti-1SWSWWNFWCNBV, which is the target above, implements the process of receiving the organization ID and deleting the corresponding organization in the API that manages the organization.
I prepared 60 organizations in advance.
Then, all 1s were given to the payloads that specified those IDs so that all weights would be uniform.
Since the number of tests for each memory size was set to a minimum of 5 times, it is assumed that 30 times of execution will be performed as a whole, and a doubled number of payloads are prepared. It is assumed that processing will be performed by specifying.
However, when I ran this, I got an error, probably because I was given more than 60 payloads than num = 5.
Thanks for sharing.
Yes you are correct, it's because of "num": 5.
You'd need to set num to at least count(payloads) (in your case, 61). This way, the input function will be invoked exactly 61 times, each time with a different payload.
Apologies if this isn't well documented in the READ.me.
Let me know if it works!
After all, it is necessary to specify a value larger than the number of payloads in num.
In that case, like Lambda Function this time, what method can be used to collect statistics about the function that causes an error when the data is updated by the execution and the same payload is executed. do you have?
I think that if you can specify the processing to be executed before and after execution of each memory value, you can solve it, but I do not think there is a function to specify such processing.
If there are other solutions, please let me know.
That's a very useful feature request :)
Would you be ok with the state machine logging the current invocation's payload in case of error? That way, you'd be able to identify which payload is causing the error.
I didn't think of this problem initially because I didn't expect Lambda Power Tuning to be used with so many different payloads (60+). This sounds like a very useful feature for your case.
If you agree, I can work on it later this week!
Thank you very much.
Would you be ok with the state machine logging the current invocation's payload in case of error?
I think it is okay to log the error.
I am very much looking forward to solving this problem, because I can use AWS Lambda Power Tuning in more situations and incorporate it into all Lambda Function CI / CDs.
Ok great, so I'll rename this issue and work on it asap. Thanks again!