Alternative approach: Import on demand
martinheidegger opened this issue · comments
"Being able to know the dependencies after parsing" seems to be one big issue why dynamic imports are not allowed. This proposal goes straight against that. With this understanding I thought that the following approach might be a solution:
import on-demand helperFunction from './views/**/*.js'
helperFunction('./views/MyView.js')
.then(function (view) {
// Here the view can be used.
})With a syntax like:
import on-demand <variable-name> from <grep-path>
where variable-name will contain a reference to a helper method that allows to load all files that match <grep-path>, returning a Promise.
This way the parser would get a full variable path name that could be used for the dynamic lookup.
The loader would need to keep all the files of the given grep handy. Not quite sure how that would work in the browser environment (HEAD requests?) but it could be a compromise?!
Right, the point of this proposal is to allow the dynamic determination of dependencies at runtime, not after parsing. It's a non-goal to allow the parser to get a full variable path name. If you want to use static imports, use static imports; this proposal is for dynamic imports.
Closing, since this doesn't seem to be an issue with the current proposal but instead a completely separate proposal. Happy to use this closed thread to discuss your proposal if you want.
Sure, lets talk on a closed thread.
Introducing non-parsable dependencies to the language seems to be a no-go for the people I have had the chance to talk to. And I do get the sentiment: Having the ability to correctly parse files and find out about the dependencies is a necessity for frontend/web development that is hacked to CommonJS right now. That being said without the possibility to have dynamic dependency a whole different set of use-cases don't work.
In the proposal of this repo, the dynamic imports are implemented as a method, which looks a bit like an API to me, not like a language feature. With this proposal the "imports" seem oddly mixed and non-verbose between dynamic and non-dynamic.
The idea that I meant to express with this comment is a sectional divide between dynamic and static imports.
import on-demand helperFunction from './views/**/*.js'
here the './views/*/.js' would specify the "possible namespace" of the dynamic imports. Giving scope and restriction to the action (security side-benefit?) The resulting "helperFunction" would then behave like a regular function so it doesn't break syntax or metaphors much.
It seems to me like this approach would satisfy both the static and the dynamic camp somehow.
Introducing non-parsable dependencies to the language seems to be a no-go for the people I have had the chance to talk to.
How do these people react to the modern surge toward code-splitting and dynamic loading of parts of your app? Are they aware that you can already introduce dynamic dependencies by inserting <script> elements? As outlined in the readme, there are many well-motivated use cases for this; saying it's a "no-go" to determine dependencies only at runtime seems contrary to my data.
It's possible we're just talking to different sets of developers, which is fine. Your set can choose not to use this feature, and even create lint rules prohibiting its use, or constraining it.
In the proposal of this repo, the dynamic imports are implemented as a method, which looks a bit like an API to me, not like a language feature.
No, it's not a method, or even a function. It's a keyword-using syntactic form. And there's talk of making it a unary operator even; see #23.
With this proposal the "imports" seem oddly mixed and non-verbose between dynamic and non-dynamic.
I don't know what this means or how it relates to the discussion.
here the './views/*/.js' would specify the "possible namespace" of the dynamic imports.
The problem here is that per the language spec, the string passed is opaque. Certain host environments might give meaning to patterns like slashes or stars, but that's not something we can specify in the language spec.
(I am a bit tired to answer right now, family duty for the weekend, will come back to you)
Think about this a bit more over the last day or so gave me a better way to phrase what I'm trying to get at here.
Your idea, and the idea in generally that you want to be able to read out the potential imports from the source, is derived from the requirements of certain static analysis and bundling tools. But the problems you're trying to solve are not because of constraints imposed by the language itself. As this proposal proves, the language has no problems with fully-dynamic import. Rather, it's because of the constraints of these tools.
As such, a solution to the problems created by these constraints really belongs on the same level it is created: at the tooling level. That is, if a tool needs extra information on how to consume JavaScript source, then it should be supplied to the tool. It should not form restrictions on the JavaScript source itself. So, whatever tools it is that needs this information, such as "all my dynamic imports will be from files corresponding to the module specifiers ./views/**/*.js", should be given that information out of band.
There are many ways to do this. Probably the most currently-accepted in the community is via config files, e.g. the famous webpack config. That seems like a good solution. If there's a real desire to colocate the information in the source files for whatever reason, then we already have a mechanism in the language for embedding metadata that is not used by the JavaScript engine, but is used by source text analyzers: namely, comments. You can use a special comment syntax that your tool parses ahead of time to add this information inline. If multiple tools need this information, hopefully they'll be able to collaborate on a common config or comment format.
This way, the language stays simple and does not get extra restrictions or complexity imposed on it by tools, but those tools are still able to accomplish their purpose, in the case that people need those extra restrictions.
Does this make sense?
Thank you for taking the time to explain your point. Most of the things you are mentioning are clear and, indeed, I havn't really bothered about considering out-of-band notation. After doing some thinking of my own, I think there are still points I still havn't been able to convey yet. Let me try to add them to the conversation.
I agree that there is the case that knowing at parse time is good for tooling. But also it is good for the runtime. After the parsing step, runtimes (i.e. browsers) can immediately load & parse the dependencies. No need to wait for the files to be loaded and then process them until the import statement hits. Being able to do that after parsing will also result in code failing earlier and clearer in case of broken dependencies.
Putting that aside there are two other aspects:
Not every tool is a build tool. other tools: i.e. Editors, can inspect quicker with lower amount of resources (both memory and cpu) needed.
And... putting additional information in out-of-band information requires that the person who maintains the out-of-band information to know what files are needed (an information that might be held only by the writer of the module) which consequently increases the likeliness of mistakes to happen.
Summing up an answer:
Being able to statically analyze is not just good for build tools but also for the runtime and other tools like editors. Putting the information out-of-band information increases the complexity and effort needed to maintain; increases the amount of human error.
To expand on this, let me try to explain myself a point I havn't been able to convey before:
... mixed and non-verbose between dynamic and non-dynamic.
I don't know what this means or how it relates to the discussion.
import() (the method proposed here) and import ... from look the same for beginners. As it looks same and mostly behaves same the question arises: Why not use import() everywhere? Having the same syntax for the dynamic imports (as in mentioned in #23) doesn't mitigate that: a dynamic string in imports has unintended, complex consequences for the code and tools that are not easy to understand and if the language is not verbose about that distinction people will use whatever works without giving a second thought.
With the proposal in this comment I tried to mitigate language user-errors:
import on-demand helperFunction from './views/**/*.js'
would bring the developers to be verbose about the places that needs the dynamism. They couldn't simply use it where-ever they like.
The import statement here works analog to any other import statement. The helperFunction here would work just like any regular function/method. In other words: using this syntax would be mostly straight forward and easy to teach/explain.
(For the record: I thought briefly about using a simple: import helberFunction from './views/**/*.js' for they dynamism but i feared that would be too much magic)
Security feature
Maybe I didn't highlight this enough so I would like to expand on the security aspect of having that syntax:
import myVariable;
technically allows the variable to import everything from the file system in node. Which is generally a bad thing if you are worried about security since variables are generally prone to be tampered with (see SQL injection). Having the on-demand syntax would allow to make sure that only files that are "meant" to be dynamically loaded are indeed dynamically loaded.
To address the issue you have mentioned:
The problem here is that per the language spec, the string passed is opaque. Certain host
environments might give meaning to patterns like slashes or stars, but that's not something
we can specify in the language spec.
If that is the case then any import statement is at the mercy of the vm. As such also the on-demand behavior is at the mercy of the vm. Or, in order to explain things with an analogy: If the vm can decide what should be served at an import statement, the vm can also decide what to serve on an on-demand import statement.
Sidenote:
No, it's not a method, or even a function. It's a keyword-using syntactic form. And there's talk of
making it a unary operator even; see #23.
If it looks like a duck and quacks like a duck it would be awkward if it wasn't a duck: In other words: having more complex grammar is a bad idea imo. Predictability in the behavior of a language is a great plus. typeof is a horrible example of JavaScript being hard to learn and teach.
I agree that there is the case that knowing at parse time is good for tooling. But also it is good for the runtime. After the parsing step, runtimes (i.e. browsers) can immediately load & parse the dependencies. No need to wait for the files to be loaded and then process them until the import statement hits. Being able to do that after parsing will also result in code failing earlier and clearer in case of broken dependencies.
Sure! In that case, use static import syntax.
And... putting additional information in out-of-band information requires that the person who maintains the out-of-band information to know what files are needed (an information that might be held only by the writer of the module) which consequently increases the likeliness of mistakes to happen.
OK. It sounds like you'd prefer it in band, using comments then. That works too!
Why not use import() everywhere?
It's much less convenient to get a promise.
technically allows the variable to import everything from the file system in node.
So does require, especially via require("fs").readFile.
If that is the case then any import statement is at the mercy of the vm. As such also the on-demand behavior is at the mercy of the vm. Or, in order to explain things with an analogy: If the vm can decide what should be served at an import statement, the vm can also decide what to serve on an on-demand import statement.
We only create things that are useful for the language VM though.
technically allows the variable to import everything from the file system in node.
So does require, especially via require("fs").readFile.
Yes, never said that this is a good thing. Or desirable to be ported to other systems.
OK. It sounds like you'd prefer it in band, using comments then. That works too!
It is not about my "preference". Its about the benefits of putting the information "in-bands" vs "out-of-bands".
Sure! In that case, use static import syntax.
The developers can choose to restrict themselves but self restriction doesn't work for tool developers or vm builders. They always need to work on the "most loose" definition and expect everything.
Before this conversation gets completely out of hands though: My intent in raising this issue was to share my concerns and give light to problematic aspects of the problem you are trying to solve.
Was I able to convey the concerns in a way that you understand why it would be possible to disagree with your proposal?
Sure. But there are many people who disagree with any new language addition that adds new capabilities on the basis that it makes their existing workflows harder. That's not an argument against progress.