collect IEnumerable to S3, Glue and DynamoDB
sam-goodwin opened this issue · comments
We want to be able to collect stream transformations to:
- S3
- Glue Tables
- DynamoDB Tables etc.
Some sort of Collector
interface could work, or individual toX
methods:
// attach a firehose delivery stream
stream.collect(scope, id, Collectors.toS3());
// or something like
stream.toS3(scope, id, {
encryption: BucketEncryption.KMS
});
const ddbTable = stream.collect(scope, id, Collectors.toDynamoDB());
const ddbTable = stream.toDynamoDB(scope, id);
const database = new Database(scope, 'database');
stream
.map(async event => ({
id: event.id,
timestamp: new Date()
}))
.collect(scope, id, Collectors.toGlue({
database,
schema: new Schema({
schemaName: 'table-name',
shape: {
id: string(),
timestamp,
}
timestampField: 'timestamp'
})));
// or ..
stream.toGlue(..);