Support for headers like Content-Disposition or Content-Type with putObject
immavalls opened this issue · comments
As suggested in https://community.k6.io/t/s3serviceerror-using-s3client/5586/4, it would be nice to be able to add some headers like Content-Disposition or Content-Type when using AWS PutObject.
Looking at the code it seems we always pass empty headers, maybe we could add a parameter to the putObject() function to optionally indicate additional headers. I might be wrong and there is an option to do that already.
/**
* Adds an object to a bucket.
*
* You must have WRITE permissions on a bucket to add an object to it.
*
* @param {string} bucketName - The bucket name containing the object.
* @param {string} objectKey - Key of the object to put.
* @param {string | ArrayBuffer} data - the content of the S3 Object to upload.
* @throws {S3ServiceError}
* @throws {InvalidSignatureError}
*/
putObject(bucketName: string, objectKey: string, data: string | ArrayBuffer) {
// Prepare request
const method = 'PUT'
const host = `${bucketName}.${this.host}`
const signedRequest = this.signature.sign(
{
method: method,
protocol: 'https',
hostname: host,
path: `/${objectKey}`,
headers: {},
body: data,
},
{}
)
const res = http.request(method, signedRequest.url, signedRequest.body, {
headers: signedRequest.headers,
})
this._handle_error('PutObject', res)
}
This is a good point! 🤝
I looked into the documentation, and the canonical way to do that, in my view, would be to define a PutObjectRequestParameters interface (or class) containing all the various options, which we map to headers in the context of the function call: serializing what needs to be serialized, etc.
Then the putObject method would look like: putObject(bucketName: string, objectKey: string, data: string | ArrayBuffer, parameters: PutObjectRequestParameters = {})
Here's my proposal for the interface, based on what's described in the documentation. I believe most fields should be optional, and that some of them (such as server side encryption related ones) could be grouped under a dedicated interface themselves:
// TODO: all the field could/should be optional. Some fields depend on each other, like
// the ones related to server-side encryption, and should probably be grouped together into
// their own dedicated interface, embedded in this one?
export interface PutObjectRequestParameters {
/**
* The canned ACL to apply to the object.
*
* This action is not supported by Amazon S3 on Outposts.
*/
ACL: CannedACL
/**
* Can be used to specify caching behavior along the request/reply chain.
*
* For more information, see https://www.rfc-editor.org/rfc/rfc9111#name-cache-control.
*/
CacheControl: string
/**
* Specifies presentational information for the object.
*
* For more information, see https://www.rfc-editor.org/rfc/rfc6266#section-4.
*/
ContentDisposition: string
/**
* Specifies what content encodings have been applied to the object and thus
* what decoding mechanisms must be applied to obtain the media-type referenced
* by the ContentType option.
*
* For more information, see https://www.rfc-editor.org/rfc/rfc9110.html#field.content-encoding.
*/
ContentEncoding: string
/**
* The language the content is in.
*/
ContentLanguage: string
// TODO: we should make sure this is automatically computed, and override it with the
// one from the options if it's provided.
/**
* Size of the body in bytes. This parameter is useful when the size of the body cannot be
* determined automatically.
*
* For more information, see https://www.rfc-editor.org/rfc/rfc9110.html#name-content-length.
*/
ContentLength: string
/**
* The base64-encoded 128-bit MD5 digest of the message (without the headers) according to RFC 1864.
* This header can be used as a message integrity check to verify that the data is the same data that
* was originally sent.
*
* Although it is optional, we recommend using the Content-MD5 mechanism as an end-to-end integrity
* check.
*/
ContentMD5: string
/**
* A standard MIME type describing the format of the contents.
*
* For more information, see https://www.rfc-editor.org/rfc/rfc9110.html#name-content-type.
*/
ContentType: string
/**
* The date and time at which the object is no longer cacheable.
*
* For more information, see https://www.rfc-editor.org/rfc/rfc7234#section-5.3.
*/
ChecksumAlgorithm: string
/**
* Can be used as a data integrity check to verify that the data received is the same data that was originally sent.
* Specifies the base64-encoded, 32-bit CRC32 checksum of the object.
*
* For more information, see Checking object integrity in the Amazon S3 User Guide.
*/
ChecksumCRC32: string
/**
* Can be used as a data integrity check to verify that the data received is the same data that was originally sent.
* Specifies the base64-encoded, 32-bit CRC32C checksum of the object.
*
* For more information, see Checking object integrity in the Amazon S3 User Guide.
*/
ChecksumCRC32C: string
/**
* Can be used as a data integrity check to verify that the data received is the same data that was originally sent.
* Specifies the base64-encoded, 160-bit SHA-1 digest of the object.
*
* For more information, see Checking object integrity in the Amazon S3 User Guide.
*/
ChecksumSHA1: string
/**
* Can be used as a data integrity check to verify that the data received is the same data that was originally sent.
* Specifies the base64-encoded, 256-bit SHA-256 digest of the object.
*
* For more information, see Checking object integrity in the Amazon S3 User Guide.
*/
ChecksumSHA256: string
/**
* The date and time at which the object is no longer cacheable.
*
* For more information, see https://www.rfc-editor.org/rfc/rfc7234#section-5.3.
*/
Expires: string
/**
* Gives the grantee READ, READ_ACP, and WRITE_ACP permissions on the object.
*
* This action is not supported by Amazon S3 on Outposts.
*/
GrantFullControl: string
/**
* Allows grantee to read the object data and its metadata.
*
* This action is not supported by Amazon S3 on Outposts.
*/
GrantRead: string
/**
* Allows grantee to read the object ACL.
*
* This action is not supported by Amazon S3 on Outposts.
*/
GrantReadACP: string
/**
* Allows grantee to write the ACL for the applicable object.
* This action is not supported by Amazon S3 on Outposts.
*/
GrantWriteACP: string
/**
* The server-side encryption algorithm used when storing this object in Amazon S3.
*/
ServerSideEncryption: ServerSideEncryption
/**
* By default, Amazon S3 uses the STANDARD Storage Class to store newly created objects.
* The STANDARD storage class provides high durability and high availability.
* Depending on performance needs, you can specify a different Storage Class.
*
* Amazon S3 on Outposts only uses the OUTPOSTS Storage Class.
*
* For more information, see Storage Classes in the Amazon S3 User Guide.
*/
StorageClass: StorageClass
/**
* If the bucket is configured as a website, redirects requests for this object
* to another object in the same bucket or to an external URL.
*
* Amazon S3 stores the value of this option in the object metadata.
*/
WebsiteRedirectLocation: string
/**
* Specifies the algorithm to use to when encrypting the object
*/
SSECustomerAlgorithm: ServerSideEncryption
/**
* Specifies the customer-provided encryption key for Amazon S3 to use in encrypting data.
* This value is used to store the object and then it is discarded; Amazon S3 does not store the encryption key.
* The key must be appropriate for use with the algorithm specified in the SSECustomerAlgorithm option.
*/
SSECustomerKey: string
/**
* Specifies the 128-bit MD5 digest of the encryption key according to RFC 1321.
* Amazon S3 uses this option for a message integrity check to ensure that the encryption key was transmitted without error.
*/
SSECustomerKeyMD5: string
/**
* If the ServerSideEncryption option is present and has the value of aws:kms, this header specifies the ID of the AWS
* Key Management Service (AWS KMS) symmetrical encryption customer managed key that was used for the object.
*
* If you specify ServerSideEncryption=aws:kms, but do not provide SSEKMSKeyId, Amazon S3 uses the AWS managed key
* to protect the data. If the KMS key does not exist in the same account issuing the command, you must use the full
* ARN and not just the ID.
*/
SSEKMSKeyId: string
/**
* Specifies the AWS KMS Encryption Context to use for object encryption.
*
* The value of this header is a base64-encoded UTF-8 string holding JSON with the encryption context key-value pairs.
*/
SSEKMSEncryptionContext: string
/**
* Specifies whether Amazon S3 should use an S3 Bucket Key for object encryption with
* server-side encryption using AWS KMS (SSE-KMS). Setting this option to true causes Amazon S3
* to use an S3 Bucket Key for object encryption with SSE-KMS.
*
* Specifying this option with a PUT action doesn’t affect bucket-level settings for S3 Bucket Key.
*/
BucketKeyEnabled: boolean
/**
* Confirms that the requester knows that they will be charged for the request.
* Bucket owners need not specify this parameter in their requests.
*
* For information about downloading objects from Requester Pays buckets, see
* Downloading Objects in Requester Pays Buckets in the Amazon S3 User Guide.
*/
RequestPayer: RequestPayer
// TODO: we don't have a construct for this yet. Something along the lines of Record<string, string>
// should do. Also, the recent SQS PR added a form url encoding function that would prove useful here.
/**
* The tag-set for the object.
*
* The tag-set must be encoded as URL Query parameters.
*/
Tagging: string
/**
* The Object Lock mode that you want to apply to this object.
*/
ObjectLockMode: ObjectLockMode
/**
* The timestamp of the date and time when you want this object's Object Lock to expire.
*/
ObjectLockRetainUntilDate: number
/**
* Specifies whether a legal hold will be applied to this object.
*
* For more information about S3 Object Lock, see Object Lock.
*/
ObjectLockLegalHoldStatus: ObjectLockLegalHold
/**
* The account ID of the expected bucket owner.
*/
ExpectedBucketOwner: string
}
/**
* The S3CannedACL type is a string union of the possible S3 canned ACLs.
*/
export type CannedACL = "private" | "public-read" | "public-read-write" | "authenticated-read" | "aws-exec-read" | "bucket-owner-read" | "bucket-owner-full-control"
/**
* The S3ServerSideEncryption type is a string union of the possible S3 server-side encryption algorithms.
*/
export type ServerSideEncryption = "AES256" | "aws:kms"
/**
* The Request Payer type is a string union of the possible S3 request payer values.
*/
export type RequestPayer = "requester"
/**
* The Object Lock Mode type is a string union of the possible S3 object lock modes.
*/
export type ObjectLockMode = "GOVERNANCE" | "COMPLIANCE"
/**
* The Object Lock Legal Hold type is a string union of the possible S3 object lock legal hold values.
*/
export type ObjectLockLegalHold = "ON" | "OFF"PS: I don't know how familiar with Typescript you might be, so I'd like to clarify that interfaces here are used to give a hint to the typescript compiler, and javascript interpreter as to the shape of the object we pass in 👍🏻
Let me know what you think, this is just a proposal 🤝