Feature request : generate only interface types ?

Question

Feature request : generate only interface types ?

alexisvisco opened this issue 3 years ago · comments

Alexis Viscogliosi commented 3 years ago

Hey, would it be possible to just generate only interface types without any implementation ?

If so it will permit for some people who rewrite urls to build their own custom client based on the types generated and removing protobuf and twirp from their dependencies.

Tate Thurston · Answer 1 · Tue Mar 08 2022 02:19:19 GMT+0800 (China Standard Time)

Hey @alexisvisco, that is certainly doable. Could you help me understand this use case a bit more?

Questions:

removing protobuf

So these would be JSON clients? Or would they leverage a different protobuf library?

and twirp

The client runtime implements error handling, so these clients would reimplement that?

Looking at the haberdasher example what would you expect to remain? What would need to be added? What would you expect the client and service types to look like?

Alexis Viscogliosi · Answer 2 · Tue Mar 08 2022 02:27:43 GMT+0800 (China Standard Time)

@tatethurston thanks for your reply ;)

So these would be JSON clients? Or would they leverage a different protobuf library?

Yes json clients

The client runtime implements error handling, so these clients would reimplement that?

Yes

I expect to remains only interfaces, no implementation.

Only that from the haberdasher example will remains :

export interface Haberdasher<Context = unknown> {
  /**
   * MakeHat produces a hat of mysterious, randomly-selected color!
   */
  MakeHat: (size: Size, context: Context) => Promise<Hat> | Hat;
}

/**
 * Size of a Hat, in inches.
 */
export interface Size {
  /**
   * must be > 0
   */
  inches: number;
}

/**
 * A Hat is a piece of headwear made by a Haberdasher.
 */
export interface Hat {
  inches: number;
  /**
   * anything but "invisible"
   */
  color: string;
  /**
   * i.e. "bowler"
   */
  name: string;
}

Our usecase

At the company were I work we are rewriting path because we decide to hide the relevant information that would lead someone to deduce that we are using protobuf and twirp.

The prefix is not enough as a configuration, so instead of having something like: https://api.domain.com/twirp/somepackage.dummy.v1.DummyService/Log we are rewriting the url to be like https://api.domain.com/v1/dummy/Log.

Another approach would be to use interceptors, maybe I could create a json dictionary with the key is the real path to the "public one".

To be honest if I was alone I would let the urls as their are.

Tate Thurston · Answer 3 · Tue Mar 08 2022 04:00:58 GMT+0800 (China Standard Time)

Got it, thanks @alexisvisco. If TwirpScript (or another solution) handled custom domain mapping for you, would that be preferable to generating only the types?

You're right that you could probably solve this using interceptors and a JSON dictionary.

I feel less strongly about url customization than the twirp core team, so I'm also open to considering URL customization. My inclination would be to use the http option, which I believe would be the idiomatic approach for the broader protobuf ecosystem. Defining it at the protobuf layer would facilitate out of the box interop with other tools.

Alexis Viscogliosi · Answer 4 · Tue Mar 08 2022 04:08:35 GMT+0800 (China Standard Time)

If TwirpScript (or another solution) handled custom domain mapping for you, would that be preferable to generating only the types?

We already made our own custom client because we were using grpc with grpc-gateway so either having a custom domain mapping or only the types would be awesome.

However removing the protobuf dependency would be great, so I am a bit ok with generating only types, we could also only generate grpc or json and combine this with mapping urls.

My inclination would be to use the http option, which I believe would be the idiomatic approach for the broader protobuf ecosystem. Defining it at the protobuf layer would facilitate out of the box interop with other tools.

It's possible but don't forget that twirp only use POST routes so I don't know how you will deal with someone who put a GET '/v1/dummy' for instance.

Another thing to considerate is do you keep the prefix or not ? ^^

Lot of questions with that option.

Tate Thurston · Answer 5 · Tue Mar 08 2022 04:45:28 GMT+0800 (China Standard Time)

We already made our own custom client because we were using grpc with grpc-gateway so either having a custom domain mapping or only the types would be awesome.

Got it. So you've switched to a twirp backend from gRPC? Have you looked into any of the TypeScript generators in the protobuf ecosystem to generate types for you? I'm curios where these projects haven't met your needs.

However removing the protobuf dependency would be great, so I am a bit ok with generating only types, we could also only generate grpc or json and combine this with mapping urls.

If you're using a modern client bundler like webpack (any bundler with tree shaking support will work) the runtime dependencies won't be present in client builds if you don't use those depdencies.

It's possible but don't forget that twirp only use POST routes so I don't know how you will deal with someone who put a GET '/v1/dummy' for instance.

Another thing to considerate is do you keep the prefix or not ? ^^

Yes those questions would need to be fleshed out. twirp-ts' gateway has done some interesting work in this space, so a solution like that is something to consider as well. I haven't looked into how the gateway definitions influence their generated client.

Tate Thurston · Answer 6 · Tue Mar 08 2022 04:53:16 GMT+0800 (China Standard Time)

I expect to remains only interfaces, no implementation.

Only that from the haberdasher example will remains :

export interface Haberdasher<Context = unknown> {
  /**
   * MakeHat produces a hat of mysterious, randomly-selected color!
   */
  MakeHat: (size: Size, context: Context) => Promise<Hat> | Hat;
}

/**
 * Size of a Hat, in inches.
 */
export interface Size {
  /**
   * must be > 0
   */
  inches: number;
}

/**
 * A Hat is a piece of headwear made by a Haberdasher.
 */
export interface Hat {
  inches: number;
  /**
   * anything but "invisible"
   */
  color: string;
  /**
   * i.e. "bowler"
   */
  name: string;
}

Circling back to this, would you want types for the client definitions as well? Eg something like:

/**
 * MakeHat produces a hat of mysterious, randomly-selected color!
 */
export type MakeHat = (size: Size): Promise<Hat>

Or would you use Haberdasher<T>["MakeHat"] as the type?

Alexis Viscogliosi · Answer 7 · Tue Mar 08 2022 06:20:41 GMT+0800 (China Standard Time)

Have you looked into any of the TypeScript generators in the protobuf ecosystem to generate types for you? I'm curios where these projects haven't met your needs.

I looked around for some ts generation repository but lot of them are not ready to be packaged as a library as easily as yours. Plus, if we decide to switch from json to protobuf in our web clients, it would only be possible with your code generation. (I think your typescript generation is the most usable on there for twirp)

Or would you use Haberdasher["MakeHat"] as the type?

I don't think so, ideally I would implement the interface on my clients. :

export interface Haberdasher<Context = unknown> {
  /**
   * MakeHat produces a hat of mysterious, randomly-selected color!
   */
  MakeHat: (size: Size, context: Context) => Promise<Hat> | Hat;
}

Apart of that, is there any reason why you have a _writeMessageJSON and _readMessageJSON ?
Why you aren't directly using JSON.stringify ? Is this a mechanism to only set authorized fields to the request ?

Alexis Viscogliosi · Answer 8 · Tue Mar 08 2022 06:24:36 GMT+0800 (China Standard Time)

Firstly what I can do is generate the code as twirpscript currently does and map the service urls to those that my proxy expects (therefore the public url) with an interceptor.

If you decide to implement one of the solutions we discussed let me know, if I can help too I would be delighted, I have a few hours left to spare.

Tate Thurston · Answer 9 · Tue Mar 08 2022 07:40:31 GMT+0800 (China Standard Time)

Firstly what I can do is generate the code as twirpscript currently does and map the service urls to those that my proxy expects (therefore the public url) with an interceptor.

If you decide to implement one of the solutions we discussed let me know, if I can help too I would be delighted, I have a few hours left to spare.

I'll plan to implement type only generation for now. I think the lift is going to be fairly low there -- maybe an hour or two? So I should be able to publish something tomorrow or Wednesday.

You're certainly welcome to take a swing at it if you're interested, I'm happy to review a PR. You'd trace the code path from twirpscript_opt=language in src/cli/index.ts.

I'm still thinking through whether this should be a top level parameter like language, or a something nested like the json options: twirpscript_opt=json=emitFieldsWithDefaultValues. In the nested case, it would fall under a typescript option, something like twirpscript_opt=typescript=emitDeclarationOnly . The rational for nesting would be to set the stage for any future TS configuration.

Tate Thurston · Answer 10 · Tue Mar 08 2022 07:48:19 GMT+0800 (China Standard Time)

Apart of that, is there any reason why you have a _writeMessageJSON and _readMessageJSON ? Why you aren't directly using JSON.stringify ? Is this a mechanism to only set authorized fields to the request ?

Protobuf specifies some complex rules around JSON serialization. TwirpScript isn't fully compliant -- there is some special handling of "well known types" that isn't (yet) implemented in TwirpScript.

In addition to field type specifics, by default protobuf JSON serialization is expected to omit any fields with default values. And when serializing, any missing fields are "hydrated" with the default value for the field.

Tate Thurston · Answer 11 · Fri Mar 18 2022 00:46:14 GMT+0800 (China Standard Time)

@alexisvisco this is available in v0.0.50

Alexis Viscogliosi · Answer 12 · Fri Mar 18 2022 04:28:33 GMT+0800 (China Standard Time)

@alexisvisco this is available in v0.0.50

Hoooo thanks you, you are awesome !!! 😍