Caching for `ts-node` (deprecation of `typescript-cached-transpile` and usage of `swc`)
CMCDragonkai opened this issue · comments
Specification
Currently we use a third party package https://www.npmjs.com/package/typescript-cached-transpile to do caching for our ts-node
. This is important in PK tests because we test alot of program executions of polykey and without caching the compilation, the test times blow out.
However this third party package isn't very stable and it has become out of date with respect to the new ts-node release. This caused some timeout problems in MatrixAI/Polykey#379. For now we had to fix our ts-node version to exactly 10.7.0
.
Upstream ts-node has an official caching solution TypeStrong/ts-node#1364, that we should switch to when it is available as that will be more stable.
At the same time, we should explore the usage of swc
instead of tsc
. Note that swc
is only transpilation, not type checking. The tsc
is still needed when checking types. However our current usage of tsc
in ts-node
is to ignore type checking and only do transpilation anyway, so it should be fine for this usecase. The only issue would be automatic conversion from tsconfig
to swcrc
... It could also be used for jest tests itself, however I think jest testing is fine to keep using tsc.
Either way, it's not maintainable to keep using typescript-cached-transpile
.
Additional context
Tasks
- Try out
swc
and see how fast it is - See how
swc
can be integrated - See what happens if
swc
is only used byts-node
, we don't want to maintain another.swcrc
- Consider incremental typescript compilation build mode with project references
I've tested out swc. It is indeed quite quick. But the main reason is that it doesn't do any typechecking. So when it's only doing transpiling, it's able to do it in 20ms.
Now if we tell tsc to only do typechecking, and only emit type declaration files with emitDeclarationOnly: true
, this doesn't actually save any time. It takes 6 seconds to compile everything, it still takes 6 seconds after not requiring emitting any JS code.
So you don't really save any time when doing a proper build. You still want to check the types, and you still want to emit the type declaration files, which swc
cannot do.
The only place where it would save time, is 2 places:
- Where
ts-node
is used, it may be better to avoid any typechecking in the first place, and thusswc
makes sense here. - Where
jest
is used, it may be better to not require type checking when runningjest
. This is not such a gain due to jest caching, which is likely to cached compiled typescript code. At the same time, when running tests, it may actually be a good idea for the types to be checked.
The benefits of running ts-node
faster is great though... but it does require additional configuration and additional dependencies. Without a usable typescript-cached-transpile
, this can be quite problematic.
I'm not convinced bringing in swc
right now would be a great benefit. The usage of ts-node
is sufficient now with the caching. Furthermore, there's already an option applied to ts-node
to only transpile, and not type check, and even then it's still slow.
If ts-node
is an issue, and we have to upgrade it which makes typescript-cached-transpile
not work, then we may need to bring in swc
...
If we can use swc
but only because of ts-node
... and not require us to maintain .swcrc
or any swc-specific configuration, then we may be able to do this: TypeStrong/ts-node#908 (comment)
In another world, the tsc
incremental compilation could work without needing us to delete the dist
directory. It would maintain correctness and ensure any files deleted is also deleted in the target dist
directory. It looks like this is actually possible with a new mode of tsc
: https://www.typescriptlang.org/docs/handbook/project-references.html#tsc--b-commandline