`Schema.optional` struct prop restricts possible schema operations

[Giacomo Randazzo]

What version of Effect is running?

3.1.4

What steps can reproduce the bug?

const Doc = Schema.Struct({
  id: Schema.String,
  desc: Schema.optional(Schema.String, { as: "Option" }),
});

const DocId = Doc.pipe(Schema.pick("id"));
const DocIdAndVersion = DocId.pipe(Schema.extend(Schema.Struct({ version: Schema.Number })));

Crashes with unsupported schema or overlapping types.

Note that:

• const DocAndVersion = Doc.pipe(Schema.extend(Schema.Struct({ version: Schema.Number }))) works
• setting no optional options or { exact: true } in desc works
• setting optional options { nullable: true } or { default: ... } in desc crashes

What is the expected behavior?

No response

What do you see instead?

No response

Additional information

@effect/schema version 0.67.2

[Giulio Canti]

The issue arises from the fact that const DocId = Doc.pipe(Schema.pick("id")); returns a transformation (namely a ComposeTransformation) that isn't supported by extend. However, I think we can resolve it by not returning a transformation in the first place if none of the picked keys are related to a property signature transformation (as in this case).

p.s.
I'm not sure if the repro is intentionally contrived to showcase the problem, but if it's not, for your information, you can achieve the same result like this:

const DocIdAndVersion = Schema.Struct({
  id: Doc.fields.id,
  version: Schema.Number
})

[Giacomo Randazzo]

Thanks for the explanation and the fix.

I operate under the mental model that Schema closely maps to TypeScript. The model breaks in this case because, in TypeScript, I can Pick<Doc, "id"> & { version: number }.

In the past, I've encountered a similar issue when applying Schema.partial to a struct containing optional properties. I understand this might be harder to support since both Schema.partial and Schema.optional deal with optional properties.

[Giulio Canti]

Your mental model is correct and works well as long as we're talking about schemas representing a TypeScript type.

In TypeScript, I can use Pick<Doc, "id"> & { version: number }.

You actually can't, but the reason is because in TypeScript you can't represent Doc since it contains a transformation.

This definition

const Doc = Schema.Struct({
  id: Schema.String,
  desc: Schema.optional(Schema.String, { as: "Option" }),
});

is nothing more than a shortcut for this transformation:

const DocAsExplicitTransformation = Schema.transform(
  Schema.Struct({
    id: Schema.String,
    desc: Schema.optional(Schema.String)
  }),
  Schema.Struct({
    id: Schema.String,
    desc: Schema.OptionFromSelf(Schema.String)
  }),
  {
    decode: (input) => ({ id: input.id, desc: input.desc === undefined ? Option.none() : Option.some(input.desc) }),
    encode: (input) => Option.isSome(input.desc) ? { id: input.id, desc: input.desc.value } : { id: input.id }
  }
)

which cannot be represented in TypeScript's type system, and therefore you can't even use Pick on it.

You can consider Schema.pick as an expanded version of TypeScript's Pick. It functions effectively for schemas representing a TypeScript type and for certain types of transformations (though not all).

So optional properties themselves are not problematic, the issue arises when dealing with transformations. Schema.optional has some options (such as { as: "Option" }) that turn the corresponding field into a transformation.

[Giacomo Randazzo]

Gotcha, thanks for the clear explanation!