Support multiple files
tmattio opened this issue · comments
It would be amazing if ts2ocaml could support generating bindings for multiple files at once (i.e supporting imports).
This would make creating bindings much more straightforward.
ts2ocaml can already read multiple files. See https://github.com/ocsigen/ts2ocaml/blob/bootstrap/generate_ts_bindings.sh
Adding an option to parse imports may be interesting, though.
Since bdd8cb3, we support non-relative imports.
Supporting relative imports is still to be discussed. I think it's a bit hard to bundle every relatively imported .d.ts files to one binding.
I think that's just the responsibility of the library authors on the TypeScript side in a sense. Namely I don't really think we need to address it at the very moment.
Just tried @ocsigen/ts2ocaml@1.0.0 on Cassandra driver and got a bunch of warning about imports:
warn: importing an identifier is not supported at line 27, col 1 of node_modules/cassandra-driver/index.d.ts
> import Long = types.Long;
warn: importing an identifier is not supported at line 28, col 1 of node_modules/cassandra-driver/index.d.ts
> import Uuid = types.Uuid;
warn: importing an identifier is not supported at line 29, col 1 of node_modules/cassandra-driver/index.d.ts
> import graph = datastax.graph;
warn: re-exporting an external module is not supported. at line 32, col 1 of node_modules/cassandra-driver/index.d.ts
> export { concurrent } from './lib/concurrent';
warn: re-exporting an external module is not supported. at line 33, col 1 of node_modules/cassandra-driver/index.d.ts
> export { mapping } from './lib/mapping';
warn: re-exporting an external module is not supported. at line 34, col 1 of node_modules/cassandra-driver/index.d.ts
> export { geometry } from './lib/geometry';
Are those relative or non-relative? Source code for index.d.ts: https://github.com/datastax/nodejs-driver/blob/master/index.d.ts#L20
@Lupus There are all relative references.
- Any imports which involve "./" are all relative references.
- It's difficult to handle relative references since
- JS/TS source codes have a different directory structure than OCaml source codes, so
- we're not very sure about the name and the location of the OCaml binding which the relative imports are referring to.
- Re-exporting is currently not supported since it is difficult to know what the imported thing is (module or type or value), which is needed to generate appropriate OCaml code for it.
Supporting these things is of low priority, because it should be fairly easy to fix it by hand. You can simply generate bindings for each files and add appropriate open statements.
I don't think it is a good idea to automatically detect the involving source codes, since the number of source codes can easily explode.
we're not very sure about the name and the location of the OCaml binding which the relative imports are referring to
Assuming that someone is binding a library, should't there be single .mli file encompassing all stuff being exported from that library? As someone willing to use cassandra-driver from OCaml, I don't care much about how internally .d.ts files are organized.
Re-exporting is currently not supported since it is difficult to know what the imported thing is (module or type or value), which is needed to generate appropriate OCaml code for it.
You could query TypeChecker API to get information about what that thing is, no?
Supporting these things is of low priority, because it should be fairly easy to fix it by hand. You can simply generate bindings for each files and add appropriate open statements.
Like creating a makefile that generates all included/reexported bindings and re-adding opens/includes on top of main .mli each time? Some knowledge on how type resolution works in ts2ocaml would be helpful to do that by hand, probably this should be a part of upcoming documentation site? May be even a guide on how to specifically handle imports/exports would be worth making, as typescript bindings are really rich on this pattern...
should't there be single .mli file encompassing all stuff being exported from that library?
I agree on that, but I haven't managed to implement it since it's difficult to erase relative imports/exports correctly.
You could query TypeChecker API to get information about what that thing is, no?
Yes but not that easy. Values are particularly problematic, since
- We have to emit type signatures correctly too. ts2ocaml currently works only on the input files to keep it sane, but emitting type signatures require processing all the referenced files at once.
- Overloaded functions are problematic. If there is an overloaded function
foo, the first overload will be named asfoo, the second onefoo', the third onefoo'', etc, which have nothing to do with the information obtained from TypeScript API. In order to emit overloaded functions correctly, we have to re-parse the generated.mlior to store some metadata to somewhere, both of which would add an extra complexity. - We actually turns off type checker in the current version of ts2ocaml.
- We are planning to release an webapp version of ts2ocaml, in which type checker is not available because it's not in the Node environment.
- If enabled, TypeScript API will start to parse every referenced files, which takes some time and sometimes fails with some TypeScript version/configuration differences.
Like creating a makefile that generates all included/reexported bindings and re-adding opens/includes on top of main .mli each time?
ts2ocaml shouldn't really be used in an automated context (@smorimoto did though 😅), because if the library author updates their type definition, the resulting .mli can have a completely different signature. For example, if an overload is added for the function foo, either the new overload is named foo' in OCaml or the existing one is renamed to foo' and the new one takes the name foo. Since OCaml doesn't support overloading, a human effort is always required when updating the binding.
I actually recommend to add the generated bindings directly to the project repository. See how I added the bindings generated by ts2fable directly to: https://github.com/ocsigen/ts2ocaml/tree/main/src/Bindings
#35 will add a better support for relative imports.
importof relative path will be converted to aopenstatement or a module alias.- The filename and the module name will be inferred by heuristics, so it might still require a human modification.
// node_modules/cassandra-driver/index.d.ts
import { auth } from './lib/auth';(* cassandra_driver.mli *)
module Auth = Cassandra_driver__auth.Export.Auth// node_modules/cassandra-driver/lib/mapping/index.d.ts
import { Client } from '../../';(* cassandra_driver__mapping.mli *)
module Client = Cassandra_driver.Export.Client- Indirect
importusing identifiers is not yet be supported by #35.- I think I can add some heuristics to here too. I'll try tomorrow or later.
import { types } from './lib/types';
import Uuid = types.Uuid; // we should be able to convert this to `module Uuid = Type.Uuid`- Direct
exportof an external module will not be supported.- Because of the reasons described above:
- We have to emit type signatures correctly too. ts2ocaml currently works only on the input files to keep it sane, but emitting type signatures require processing all the referenced files at once.
- Overloaded functions are problematic. If there is an overloaded function foo, the first overload will be named as foo, the second one foo', the third one foo'', etc, which have nothing to do with the information obtained from TypeScript API. In order to emit overloaded functions correctly, we have to re-parse the generated .mli or to store some metadata to somewhere, both of which would add an extra complexity.
- It should be better to leave a
(* FIXME .. *)to let users know a manual modification is required?
export { concurrent } from './lib/concurrent'; // this is actually hard to do in OCaml!I've left some notes here and there but I will later compose them to a doc in the PR.
Sounds awesome! @cannorin may be a command-line option could be added to make ts2ocaml follow relative imports and automatically call itself to produce dependent files with filenames matching those relative imports?