Interoperability with CDK/Incremental Adoption?

[Max Dumas]

Hi there! First of all this is an incredible project, and I wish my team and I had discovered it sooner! The strong typing connection possible between GraphQL API resolvers and DynamoDB table specifications is incredibly powerful and exciting to us, as the loose typing here is a big source of bugs in our current work. Of course many other things about this paradigm seem amazing as well!

Unfortunately, our codebase is rather large and currently written entirely in CDK. I wonder if there is any possible story of incremental adoption here? Can punchcard constructs be used in the same stack as normal CDK constructs? More specifically to our usecase, is there are way we could leverage the strongly-typed VTL template generation in as minimal a way as possible? For example, is there a way that we could leverage this work to just generate VTL templates?

[sam]

Hey, thanks for reaching out! I'm glad you like the project.

This project is a playground full of many experiments where I try to push the TypeScript language as far as I can to build a virtual programming language. It's been a ton of work and a lot of fun, but it also means that some features have known edge cases with the compiler. I'm curious, have you played with it? What was your experience like dealing with the heavy use of type-level logic?

AppSync VTL abstraction

One of those edge cases is with the AppSync stuff. Sometimes, type inference on yield* breaks down. I'm proud of the implementation but I also think it deserves a version 2, maybe I can do that as part of a standalone Construct library compatible with the CDK? Would you be interested in helping design that DX?

RE: extracting the VTL abstraction from Punchcard - that is quite complicated. The interactions with services are encoded directly into the DynamoDB.Table, Lambda.Function, etc. constructs themselves. Those constructs are typed, so the data type is known and used to derive type-safe VTL behavior. How would you expect this to work with existing CDK constructs?

Incremental Adoption

It is possible but there is one gotcha you need to be aware of. Punchcard uses webpack to create the runtime bundle JS file uploaded to Lambda. In webpack, I do some hacky stuff to prune out all the CDK dependencies from the runtime bundle - it reduces the size down from >2MB to ~30-200KB and drastically improves the cold start.

This is a complication introduced by blending code that runs at CDK synthesize time and code that runs in the Lambda Function. If I just bundled the CDK app and ran it within lambda, then things blow up. For example, if a Construct references an asset, then it will run a bunch of fs calls in the lambda and likely break. It's slow and heavy - you don't want to run unnecessary code in your Lambda.

My solutiton works, but it has the unfortunate drawback that it diverges from the CDK. A better solution would be a closure serializer that is capable of extracting only the code required by a Function and serializing any state. Pulumi does some interesting things in this space, but alas, I went with a different approach.

In Punchcard, instead of importing a CDK module, like:

import * as cdk from '@aws-cdk/core'

Everything is wrapped in a Build<T> monad. Think of it like a Promise<T>. This lazily suspends all CDK code in a closure so that it can be safely erased from the runtime bundle.

const cdk = Build.lazy(() => require('@aws-cdk/core') as import('@aws-cdk/core'))

You could incrementally adopt your existing CDK application by creating a new Punchcard package that depends on your existing Constructs.

1. add some boilerplate to your existing package, you can model this off of https://github.com/punchcard/punchcard/tree/master/packages/%40punchcard/constructs

In your package's index.ts

import erasure = require('@punchcard/erasure');

// tell Punchcard to erase this module from the runtime bundle - it is only needed at build time.
erasure.erasePattern(/^my-construct-library$/);

2. create a new Punchcard package and lazily import your existing Construct library

const MyConstructLibrary = Build.lazy(() => require('my-construct-library') as import('my-construct-library'));

import { Core } from 'punchcard';

const app = new Core.App();

const existingStack = Build.concat(app.root, MyConstructLibrary).map(([app, myConstructLibrary]) => {
  // you can write normal CDK code inside this block, e.g. instantiating one of your existing stacks.
  return new myConstructLibrary.ExistingStack(app, 'MyExistingStack');
});

// now go and use Punchcard stuff
class MyShape extends Type(..)

const table = new DynamoDB.Table(..)

Now, you can author standard CDK constructs as you've already been doing and then instantiate them in your Punchcard app.

Note:
I think this inter-op is the thing most hurting Punchcard's DX. I'd love to do some closure serialization and drop this hack all together.