Giters

TypeStrong / ts-node

TypeScript execution and REPL for node.js

Home Page:https://typestrong.org/ts-node

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

ESM support: soliciting feedback

cspotcode opened this issue · comments

commented

Please use this ticket to provide feedback on our native ESM support. Your involvement is greatly appreciated to ensure the feature works on real-world projects.

Experimental warning

Node's loader hooks are EXPERIMENTAL and subject to change. ts-node's ESM support is as stable as it can be, but it relies on APIs which node can and will break in new versions of node.

When node breaks their APIs, it breaks loaders using their APIs. You have been warned!

Third-party docs: "Guide: ES Modules in NodeJS"

Someone has been maintaining a great reference document explaining how to use ts-node's ESM loader.

Guide: ES Modules in NodeJS

First-party docs

Our website explains the basics:

CommonJS vs native ECMAScript modules
Options: esm

Usage

Requirements

  • Set "module": "ESNext" or "ES2015" so that TypeScript emits import/export syntax.
  • Set "type": "module" in your package.json, which is required to tell node that .js files are ESM instead of CommonJS. To be compatible with editors, the compiler, and the TypeScript ecosystem, we cannot name our source files .mts nor .mjs.
  • Include file extensions in your import statements, or pass --experimental-specifier-resolution=node Idiomatic TypeScript should import foo.ts as import 'foo.js'; TypeScript understands this.
    • The language service accepts configuration to include the file extension in automatically-written imports. In VSCode:
      image

Invocation

ts-node-esm ./my-script.ts

ts-node --esm ./my-script.ts

# If you add "esm": true to your tsconfig, you can omit the CLI flag
ts-node ./my-script.ts

# If you must invoke node directly, pass --loader
node --loader ts-node/esm ./my-script.ts

# To force the use of a specific tsconfig.json, use the TS_NODE_PROJECT environment variable
TS_NODE_PROJECT="path/to/tsconfig.json" node --loader ts-node/esm ./my-script.ts

# To install the loader into a node-based CLI tool, use NODE_OPTIONS
NODE_OPTIONS='--loader ts-node/esm' greeter --config ./greeter.config.ts sayhello

ts-node-esm / --esm / "esm": true work by spawning a subprocess and passing it the --loader flag.

Configuration

When running ts-node --esm, ts-node-esm, or ts-node all CLI flags and configuration are parsed as normal. However, when passing --loader ts-node/esm, the following limitations apply:

  • tsconfig.json is parsed.
  • CLI flags are not parsed.
  • Environment variables are parsed.
  • ts-node must be installed locally, not globally. npm install ts-node or yarn add ts-node.

tsconfig will be resolved relative to process.cwd() or to TS_NODE_PROJECT. Specify ts-node options in your tsconfig file. For details, see our docs.

Use TS_NODE_PROJECT to tell ts-node to use a specific tsconfig, and put all ts-node options into this config file.

Versioning

As long as node's APIs are experimental, all changes to ESM support in ts-node, including breaking changes, will be released as minor or patch versions, NOT major versions. This conforms to semantic versioning's philosophy for version numbers lower than 1.0. Stable features will continue to be versioned as normal.

node's API change: v16.12.0, v17.0.0

Node made a breaking change in their ESM API in version 17, backported to 16.12.0. It may also be backported to 14 and 12.
This is the change: nodejs/node#37468

ts-node automatically supports both APIs, thanks to #1457. This relies on hard-coded version number checks. If/when this is backported to node 14 and 12, we will publish a new version of ts-node with the appropriate version number checks. Be sure you are always using the latest version of ts-node to avoid problems.

Note: things below this line may be out-of-date or inaccurate. These notes were used during initial implementation, but have not been updated since

Pending development work

  • Make resolution lookup use our fs caches
  • Create esm-script.mjs to do --script-mode?
    • Can read process.argv for config resolution?
  • Implement require('ts-node').esmImport(module, 'import-path')
  • Throw error when CJS attempts to require ESM, matching node's behavior for .js
    • See below: "Changes to existing functionality" > "require() hook"

The proposal

Below is the official proposal, explaining our implementation in detail.

I am asking node's modules team questions here: nodejs/modules#351

I was reading the threads about ESM support in ts-node, e.g. #935.

The @K-FOSS/TS-ESNode implementation is unfortunately incomplete; it does not attempt to typecheck. (it uses transpileModule)

So I did some research. Below is a proposal for ESM support in ts-node, describing the required behavior in detail.

This doesn't feel like an urgent feature to me, but I like having an official proposal we can work on.

Usage

node --loader ts-node/esm ./entrypoint.ts

Cannot be invoked as ts-node because it requires node flags; hooks cannot be enabled at runtime. This is unavoidable.

For simplicity, --require ts-node/register can be eliminated, because ts-node/esm automatically does that.

Alternatively, we publish an experimental ts-node-esm entry-point which invokes a node subprocess.

Don't forget allowJs! Affects the treatment of .js files. (Not .mjs nor .cjs because the TS language service won't look at them)

ESM hooks

Must implement ESM hooks to resolve extensionless imports to .ts files, resolve .js to .ts, classify .ts(x) and .jsx files as CJS or MJS, and compile .ts(x) and .jsx files.

resolve() hook:

Match additional file extensions: .ts, .tsx, .jsx.

Resolve .ts, .tsx, and .jsx if the import specifier says .js. Obey preferTsExts when doing this.

_

[Good idea?] Always ask default resolver first. If it finds something, we should not interfere.

--experimental-specifier-resolution=node does not obey require.extensions, unfortunately, so we can't use that.

getFormat hook:

If the resolved file is .ts, .tsx, or .jsx, behave as if extension was .js: use node's package.json discovery behavior to figure out if ESM or CJS.

This can be accomplished by appending .js to the URL path and delegating to built-in getFormat hook.

transformSource hook:

Same as today's code transformer. Relies on projects to be configured correctly for import/export emit.

Changes to existing functionality

require() hook

  • Use same getFormat logic to determine if node will treat file as CJS or ESM.
  • NOTE node already detects and throws some errors on its own. But if require.resolve points to a .ts file, we need to make the determination.
  • If ESM, throw the same error as NodeJS ("cannot load ESM via require()")

require() code transform

  • Must somehow allow import() calls.
  • Force consumers to use require('ts-node').esmImport(module, 'import-path')?

ts-node bin entry-point

ts-node CLI does NOT need to support import()ing ESM.

WHY? Because ESM hooks are an experimental feature which must be enabled via node CLI flag.

Thus we will be loaded via --require, and Node is responsible for loading the entry-point, either triggering our hook or our require.extensions.

Allow import() in CJS

If "module": "commonjs", compiler transforms import() into __importStar

No way to change this without a custom transformer, which IMO is too much complexity at this time.

Users should run their code as ESM.

If they can't do that, we can recommend the following workaround:

// This is in a CommonJS file:
const dynamicallyImportedEsmModule = await require('ts-node').importESM('./specifier-of-esm-module', module);

Emit considerations

NOTE we have not implemented the following, although initially I thought we might. Instead, we assume tsconfig is configured for either ESM or CJS as needed

We could intelligently emit both "module": "esnext" and "module": "commonjs" depending on the classification of a file.

In transpile-only mode this is simple. Call transpileModule with different options.

When typechecking, we can pull SourceFile ASTs from the language service / incremental compiler.

We'll need a second compiler, one for each emit format. Or we can hack it by using transpileModule for all ESM output. transpileModule is incompatible with certain kinds of TS code, (can't do const enums) but it might work for a first-pass implementation.

commented

TODO: turns out, users can tell the language service to include the .js file extension with automatically-written imports. So we do not need to automatically add them, though we do need to check if a .js import might point to a .ts or .tsx file.

The option is passed to the language service in a ts.UserPreferences object.
https://discordapp.com/channels/508357248330760243/640177429775777792/703301413337432114

commented

I was trying to figure out if ts-node needs to automatically switch the "module" option between CommonJS and ESNext depending if we need to emit CommonJS or ESM. I concluded we do not want to do this. Here's an explanation anyway, in case I am proven wrong.

Today, ts-node respects the tsconfig's module option. Users are required to set it appropriately. If the user incorrectly sets module to ESNext and then tries to require() a TS file, they get an error because the emitted code has import statements.

Alternatively, we can automatically override the module option to be CommonJS when emitting for require() and ESNext when emitting for ESM. This allows a single tsconfig to be used for both ESM and CommonJS.

After thinking about this, it doesn't make sense. Users will choose either ESM or CommonJS via their package.json file. They won't do a mix of both. Also, this would get pretty messy since we'd be doing something that doesn't match tsc's output.

Nevertheless, if we wanted to implement this:

If the module option is already correct, we can use the languageService's getEmitOutput() like we do today. If not, we can grab a reference to the SourceFile and transform it using the same technique as transpileModule's implementation. This allow custom emit while avoiding an expensive parse.

TypeScript has an internal sourceFileAffectingCompilerOptions array. If any of those options differ, a SourceFile cannot be reused. However, some are only relevant if you care about diagnostics. For swapping out the module flag, I think SourceFile can always be reused.

commented

We have released an experimental implementation of this in v8.10.1. Please test and share your feedback here.

commented

Thanks @cspotcode for the release! Everything seems be working minus one snafu. Importing named exports don't seem to be working, but this may be a Node module quirk. For example in index.ts:

import { graphql } from 'graphql';

will cause a syntax error of:

SyntaxError: The requested module 'graphql' does not provide an export named 'graphql'

but this can be solved by using destructuring:

import Graphql from 'graphql';
const { graphql } = Graphql;

Any way to support importing named exports in ts files?

commented

@chpeters I'd guess that would be because graphql is actually CommonJS and not an ES module. You can read more about it here: https://nodejs.org/api/esm.html#esm_interoperability_with_commonjs. Unfortunately it'll probably be messy for a while with TypeScript since the imports syntax is overloaded to represent both CommonJS and native ES modules.

commented

Using mocha and TypeScript with ES modules I am facing an issue and I don't quite understand it.

Running this cmd as my test cmd :

node --experimental-modules --loader ts-node/esm.mjs ./node_modules/mocha/bin/mocha --extension ts

I get this error :

import './unit/authentication.js';
^^^^^^

SyntaxError: Cannot use import statement outside a module

What did I do wrong ?

PS: I have my tsconfig.json module attribute set to "ES2015", my package.json type attribute to "module", ts-node installed locally

commented
commented

This is my project architecture :

src
  |_index.ts
test
  |_tests.ts
  |_unit
      |_authentication.ts
package.json
tsconfig.json

My package.json :

{
  "name": "my-project",
  "version": "1.0.0",
  "description": "My project",
  "main": "lib/index",
  "type": "module",
  "files": [
    "lib/**/*"
  ],
  "directories": {
    "test": "test"
  },
  "scripts": {
    "build": "tsc",
    "test": "node --experimental-modules --loader ts-node/esm.mjs ./node_modules/mocha/bin/mocha --extension ts"
  },
  "devDependencies": {
    "@types/chai": "^4.2.11",
    "@types/mocha": "^7.0.2",
    "@types/node": "^13.13.5",
    "chai": "^4.2.0",
    "mocha": "^7.1.2",
    "ts-node": "^8.10.1",
    "typescript": "^3.8.3"
  }
}

My tsconfig.json :

{
  "compilerOptions": {
    "target": "ES2015", 
    "module": "ES2015", 
    "lib": ["es6"], 
    "declaration": true,
    "outDir": "lib",
    "rootDir": "src",
    "strict": true, 
    "noImplicitAny": true,   
    "moduleResolution": "node",   
    "esModuleInterop": true,  
    "forceConsistentCasingInFileNames": true
  },
  "exclude": [
    "test/"
  ]
}

My test/tests.ts :

import './unit/authentication.js'

Typescript is building my files right.
The npm run test cmd returns throw the error I wrote before.

Do you need more context ?

commented

@NeilujD this is perfect, thanks.

It looks like, due to missing features in node's ESM support, mocha is using a hack to figure out whether a file should be loaded as ESM or CJS.
https://github.com/mochajs/mocha/blob/master/lib/esm-utils.js#L4-L23

ts-node's require() hook will need to be updated to match the error behavior of node's .js hook. When you try to require() a TS file that should be treated as ESM, we should throw an error.

At first I thought mocha could simply import() everything, since it automatically switches to CommonJS loading as needed. However, that would require our ESM hook to be installed in order to resolve and classify .ts files. They're forced to use require() to cater to legacy require() hooks.

commented

I think I can hack this by delegating to node's built-in .js extension, passing a fake filename.

require.extensions['.js']({_compile(){}}, filename + 'DOESNOTEXIST.js')

At the cost of a failed fs call, this will cause the built-in require hook to do its package.json lookup and see if the file should be treated as CommonJS or ESM.

> require.extensions['.js'].toString()
'function(module, filename) {\n' +
  "  if (filename.endsWith('.js')) {\n" +
  '    const pkg = readPackageScope(filename);\n' +
  "    // Function require shouldn't be used in ES modules.\n" +
  "    if (pkg && pkg.data && pkg.data.type === 'module') {\n" +
  '      const parentPath = module.parent && module.parent.filename;\n' +
  "      const packageJsonPath = path.resolve(pkg.path, 'package.json');\n" +
  '      throw new ERR_REQUIRE_ESM(filename, parentPath, packageJsonPath);\n' +
  '    }\n' +
  '  }\n' +
  "  const content = fs.readFileSync(filename, 'utf8');\n" +
  '  module._compile(content, filename);\n' +
  '}'

EDIT I've shared this hack with the node folks nodejs/modules#351 (comment) to see if they have any interest in exposing this API natively.

commented

Responding to #1007 (comment)

Cannot be invoked as ts-node because it requires node flags; hooks cannot be enabled at runtime. This is unavoidable.

Actually... it should be possible, by providing a very simple posix shell script wrapping the whole thing. That would be ideal when using shebangs, as it's not allowed to pass options to shebangs in Linux (although it's possible in Mac).

commented

@castarco the problem is cross-platform support that matches the npm ecosystem without adding complexity to ts-node. Typically this is handled by the package manager: npm, yarn, pnpm, etc.

We set up our package.json "bin" field, pointing to a file with a shebang, and it takes care of creating a symlink, .cmd shim, and .ps1 shim.

commented

@NeilujD The mocha issue you were seeing should be fixed by #1031 which has been merged to master.

If you're feeling adventurous, you can install ts-node directly from master.

npm i TypeStrong/ts-node#master --save

Or you can download and install the tarball artifact produced by CI.
image
image

npm install ts-node-packed.tgz --save
commented

FWIW I tried it out and got two issues one undocumented issue:

  1. Instructions say to include file extensions in import statements, so that's what I did. But TS complains about this (TS2691: An import path cannot end with a '.ts' extension. Consider importing '../../src/index' instead). I used a @ts-ignore here, and then was able to import that module, sortof.
  2. Wherever npm packages are imported I get the error TS7016: Could not find a declaration file for module 'tcp-port-used'. 'C:/Users/Matt/Documents/dev/http-server-group/node_modules/tcp-port-used/index.js' implicitly has an 'any' type. Try 'npm install @types/tcp-port-used' if it exists or add a new declaration (.d.ts) file containing 'declare module 'tcp-port-used';' even though I have those packages sitting in my node_modules, and I don't get this error when I build with tsc.

Anyways, just thought I would give some feedback since it was solicited. Keep up the great work @cspotcode!!

commented

@zenflow thanks, much appreciated.

Including file extensions is tricky; you need to include them in the way that typescript wants, which is to include the .js extension, not the .ts extension. This comment explains precisely why TypeScript does things this way. It has to do with maintaining compatibility with the pre-compiled code scenario. nodejs/modules#351 (comment)

commented

I've also been trying to get mocha to work with this and I've been following the rabbit hole of mochajs/mocha#4267

master works for me, but to have a usable setup I also needed esModuleInterop enabled and I had to run tests with

node --experimental-modules --loader ts-node/esm.mjs node_modules/mocha/lib/cli/cli.js src/**/*.spec.ts

Calling into the mocha internals is a bit ugly, and probably breaks some features but doesn't require patching node_modules at least.

commented

@thatsmydoing Thanks for sharing your experience; I'm sure it will help other people, too.

You should be able to omit --experimental-modules because it is implied by --loader.

We merged #1028 a few days ago, so you should be able to omit the .mjs extension if you want.

This simplifies things a bit:

node --loader ts-node/esm node_modules/mocha/lib/cli/cli 'src/**/*.spec.ts'

Unfortunately, node has a lot of work to do before this feels clean. There's no way for us to load our hooks at runtime, which would allow us to perform the equivalent of --loader on your behalf. There's also no good system for composing multiple loader hooks together. Right now, ts-node's hook is doing extra work that ideally should be handled by a robust hook composer.

commented

Thanks for your time on this! Works great, except I seem to be losing the source map support.

For a two-line test.ts:

type Foo = string;
throw new Error("Oh no!");

(without "type": "module" in package.json)

ts-node test.ts
...
Error: Oh no!(test.ts:2:7) 👍

(with "type": "module" in package.json)

node --loader ts-node/esm.mjs ./test.ts
...
Error: Oh no!(test.ts:1:7) 😢

Is there any easy fix?

commented

@akbr Good catch, thanks. It looks like files loaded as ESM have file:// URLs instead of paths.

We install source-map-support here:
https://github.com/TypeStrong/ts-node/blob/master/src/index.ts#L445-L451

It's a third-party library that handles sourcemaps automatically, rewriting stack traces. We give it access to a cache of TypeScript's compiler output so it can get the sourcemaps. But we're not handling file:// URLs correctly, so source-map-support is not able to get access to the sourcemaps.

Created #1060 to track this. If you feel like sending us a bugfix, that'd be awesome!

commented

thanks for the this initiative.
I experiment esm and cjs feature
heres my library package.json

  "type": "module",
  "main": "./cjs/my-lib.js",
  "module": "./my-lib.js",
  "typings": "./my-lib.d.ts",
  "exports": {
    "require": "./cjs/my-lib.js",
    "import": "./my-lib.js"
  },

if i combine esm and cjs in my code to call the "my-lib" it always use the cjs

import { addNumber } from 'my-lib'
const lib = require('my-lib')

console.log(lib.addNumber(1,2) + addNumber(1,2))

my tsconfig.json

{
  "compilerOptions": {
    "baseUrl": ".",
    "moduleResolution": "node",
    "module": "commonjs",
    "target": "es2018",
    "allowJs": true
  }
}

by this configuration and example it is possible to always call or use the esm module or
when there require it uses the cjs then when there import it use the esm?

commented

@aelbore Since your tsconfig.json file has module: "CommonJS", it will always require(…) the CJS files.

To import the ESM files, you need to set module: "ES2015", module: "ES2020" or module: "ESNext".

Also, you need to construct the require function in ESM modules yourself if you intend to use it:

import { createRequire } from 'module';
const require = createRequire(import.meta.url);
commented

Is there an easy answer to why __dirname is undefined? Easy to work around but feels unusual. Maybe it could be polyfilled as

import path from 'path';
const __dirname = path.dirname(new URL(import.meta.url).pathname);

As I saw here: https://techsparx.com/nodejs/esnext/dirname-es-modules.html

node --loader ts-node/esm.mjs ./ssr.ts
(node:117914) ExperimentalWarning: --experimental-loader is an experimental feature. This feature could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
ReferenceError: __dirname is not defined
    at new FileResourceLoader (file:///home/today/_/work/ssr.ts:11:43)
    at file:///home/today/_/work/ssr.ts:26:24
    at ModuleJob.run (internal/modules/esm/module_job.js:110:37)
    at Loader.import (internal/modules/esm/loader.js:179:24)
commented
commented

@cspotcode is there a way to use transpile-only mode? I'm trying to run this in a ts-node-dev way and closest I've got is running nodemon along with -x "node --loader ...", but it's still throwing in type errors instead of just ignoring them (like tsnd does)

commented
commented

@cspotcode It worked, thanks! trying this on a non-prod project and so far is has been good, excluding the annoyances of using destructured imports, but besides from that all is working (even TLA), will try to integrate jest as well.

Scripts if anyone is interested for faster testing:

"scripts": {
    "start": "node --loader ts-node/esm --experimental-specifier-resolution=node --experimental-top-level-await --no-warnings src/server.ts",
    "dev": "nodemon -q -e ts,js,json -x npm start"
  }
commented

@cspotcode , node-esm-resolve-implementation doesn't support '--experimental-specifier-resolution=node' flag specified trough NODE_OPTIONS env variable because it looks into process.execArgv. Issue can be fixed via '-r module-which-modifies-process-exec-argv.js", but it's really ugly hack.

commented

@VladimirGrenaderov Thanks, can you file this as an issue to help track implementation of a fix?

commented

@cspotcode looks a bit confusing:

code from my node_modules (latest version, 8.10.2) - line commented:
image

https://github.com/TypeStrong/ts-node/blame/master/raw/node-esm-resolve-implementation.js
image
image

Somebody change the code before publish?

Because https://registry.npmjs.org/ts-node/-/ts-node-8.10.2.tgz contains commented line too.

commented
commented

yarn info
image
image

npm view
image

Looks like tarball url is correct.

Tag v8.10.2 - https://github.com/TypeStrong/ts-node/blob/v8.10.2/raw/node-esm-resolve-implementation.js, line is not commented:
image

commented
commented

@cspotcode yep I catch it - dist-raw/node-esm-resolve-implementation.js & raw/node-esm-resolve-implementation.js are different. Thx, but what does it mean? Just experimental garbage?

As you asked, new issue - #1072.

commented

#1007 (comment)

The node --loader ts-node/esm node_modules/mocha/lib/cli/cli 'src/**/*.spec.ts' is not working in this case because the esm.mjs file has require for

const esm = require('./dist/esm')

but the folder dist doesn't exist when it is "installed" from TypeStrong/ts-node#master

A small hack is going to node_modules/ts-node and type npm install + npm run build-nopack

But if user doesn't specifiy --experimental-specifier-resolution=node the ts-node will start complaining about ERR_MODULE_NOT_FOUND. Because the normal resolution is not resolving missing .ts extension.

Eg.:

import {Something} from "../src/shared/TestFile";

Complains:

Error: ERR_MODULE_NOT_FOUND /src/shared/TestFile /tests/TestFile.spec.ts module
at finalizeResolution (/node_modules/ts-node/dist-raw/node-esm-resolve-implementation.js:360:9)

So the solution is having the specifier.

"test": "node --loader ts-node/esm --experimental-specifier-resolution=node node_modules/mocha/lib/cli/cli --config _config/.mocharc.cjs"

When do you plan next release of ts-node?

commented
commented

I have used yarn.

yarn add --dev TypeStrong/ts-node#master

seems it hasn't called the prepare event. yarnpkg/yarn#1671

Sorry for false fire!

yarn add --dev git+https://github.com/TypeStrong/ts-node
Works as is expected.

commented

@sionzeecz Thanks for sharing; I'm sure this will help others using yarn, too.

commented

for the time being, could we include a note about adding additional type-roots?

since ts-node --files isn't an options here, people (including me up until 5min ago) might not know how to do it otherwhise.

something like

note: to include typeroots specified in tsconfig, add env var TS_NODE_FILES=true

as a hashbang: #!/usr/bin/env TS_NODE_FILES=true node --loader ts-node/esm.mjs
note hasbang arguments arent available on some unix distros (including linux). it works on macOS for example.

besides that i't's generally been working very well and made coding ts+esm a much smoother experience

commented

EDIT: whoops, i missed the note for it:

Set "type": "module" in your package.json, which is required to tell node that .js files are ESM instead of CommonJS. To be compatible with editors, the compiler, and the TypeScript ecosystem, we cannot name our source files .mts nor .mjs.

have you considered adding support for .mts/.ctsextentions? i don't know how that would be handled by standard ts-compiler and wether that could be added as a compiler-plugin if not.

this isn't really nescessary of course, i just wanted to add a note about it

commented

I have. To elaborate on the explanation that you found:

The problem is it makes code incompatible with the TypeScript ecosystem. Try importing from a .mts or .cts file and see what the language service says.

The benefits of TypeScript largely come from the language service. Even if you may individually disagree with that statement, we can't be messing things up for people who do benefit greatly from the language service.

Additionally, runtime compiling in production doesn't make sense for obvious reasons. Again, some people may individually agree with this statement, which is fine, but we can't be encouraging code that's deliberately incompatible with precompilation.

EDIT: it'll be interesting to see if Deno causes any changes in the ecosystem, since they use the .ts file extension in their import specifiers, and they've had to write editor integrations to somehow modify behavior of the language service.

commented

The problem is it makes code incompatible with the TypeScript ecosystem. Try importing from a .mts or .cts file and see what the language service says.

never tried it, but i thougth so. that why i added the note about a compiler-plugin. there is a plugin system, but it's not used much and i have no idea how it works. was just a thought

The benefits of TypeScript largely come from the language service. Even if you may individually disagree with that statement, we can't be messing things up for people who do benefit greatly from the language service.

100% agree. my code style is basically vanilla-esm with type anotations and my default compile target is ESNEXT, so the language feature is almost the only reason why i use ts

The reason why i wondered is because of this vanilla-sentiment. it would allow for the emitted files to be even closer to the source. It's just something i like, to have that smooth transition and it makes compiler-side oddities and mistakes a breeze to track down. it's just satisfying to me how together with the typedeclarations that have the same vanilla-look, it merges together into prettymuch exactly what the source-code is on a visual level aswell.

EDIT: it'll be interesting to see if Deno causes any changes in the ecosystem, since they use the .ts file extension in their import specifiers, and they've had to write editor integrations to somehow modify behavior of the language service.

I haven't looked at deno in quite a while. i followed it's development early-on but i'm quite happy with node, so for now i don't see a personal need to use it. but it currently looks like it's here to stay, and some competition can cause benefitial changes on both sides.
I'm really looking forward to what developments will happen in nodes vm.module and policies. i think deno will have some influance on them.

commented

The TypeScript compiler intentionally enforces that extensions in import statements include the .js extension, because this makes the import statement in your source code exactly match the emitted import statement. import {foo} from './foo.js'; will be emitted. At compile-time it's looking at foo.ts. At runtime it's looking at foo.js. The import statement is identical both places.

commented

This doesn't appear to work with mixed module type dependencies. I have a dependency that has the following in its package.json:

"main": "./output-cjs/index.js",
"exports": {
	"import": "./output-esm/index.js",
	"require": "./output-cjs/index.js"
},

When I am in a project that references it and do

node --loader ts-node/esm.mjs --experimental-specifier-resolution=node tests/index.ts

I get an error:

The requested module '@zoltu/rlp-encoder' is expected to be of type CommonJS, which does not support named exports. CommonJS modules can be imported by importing the default export.

Why is it expecting the dependency package to be CJS? I am specifically trying to load everything as ESM modules. Both CJS and ESM are available, so why is failing to load whichever one it is trying to load?

commented
commented

Repro: https://github.com/MicahZoltu/eip-2718-tests/tree/ts-node-repro

This is the first project I've tried to use with ts-node & ESM, so I may be doing something wrong.

commented

@MicahZoltu thanks, having a complete example makes debugging straightforward.

I tried the following with node v14.2.0 on Linux:

# At the shell prompt
$ npm install @zoltu/rlp-encoder # EDIT: installed v2.0.3
$ node
# in the node REPL
# Ask node what it wants to load for CommonJS.  This part looks good.
> require.resolve('@zoltu-rlp-encoder')
..../repros/node_modules/@zoltu/rlp-encoder/output-cjs/index.js'
# Ask node to load it as ESM
> import('@zoltu/rlp-encoder')
Promise { <pending> }
> (node:5815) UnhandledPromiseRejectionWarning:..../repros/node_modules/@zoltu/rlp-encoder/output-esm/index.js:1
export function rlpEncode(item) {
^^^^^^

SyntaxError: Unexpected token 'export'
    at Object.compileFunction (vm.js:344:18)

Based on the above, and without using ts-node at all, it looks like node is trying to load the ESM files as CommonJS.

This makes sense because the file extension is .js, so node will use package.json to figure out how to load the file. This tells node to load it as CommonJS.

So it looks like the problem is unrelated to ts-node and is caused by a faulty build process for @zoltu/rlp-encoder.

commented

Hmm, thanks for looking into it, I'll dig a bit more. Looking at the script output you provided, it does appear to be loading the correct index file (it is using the exports property in package.json to figure out paths), but it then proceeds to complain that export is not a valid token in an ESM.... (which it definitely is) which is very bizarre.

commented

The node docs call out this behavior explicitly, so it's not a node bug. https://nodejs.org/api/esm.html#esm_conditional_exports

commented

The confusion from the docs was here: https://nodejs.org/api/esm.html#esm_dual_commonjs_es_module_packages

Node.js can now run ES module entry points, and a package can contain both CommonJS and ES module entry points (either via separate specifiers such as 'pkg' and 'pkg/es-module', or both at the same specifier via Conditional exports).

However, if I set type: 'module' it only works with import ... and if I don't set type: 'module' then it only works with require(...). I'll take this up with the NodeJS people though, since it sounds like the issue isn't with ts-node.

commented

For any future readers wanting to follow along on my journey toward functional dual module packages: nodejs/node#34515

commented

Yeah, under the "Conditional Exports" docs they say:

"import" - matched when the package is loaded via import or import(). Can reference either an ES module or CommonJS file, as both import and import() can load either ES module or CommonJS sources. Always matched when the "require" condition is not matched.

node intends you to use the .cjs and .mjs file extensions to specify the module's type, which overrides whatever package.json is saying. I'm not sure what would happen if you include a mostly-empty package.json file in the ./output-cjs directory which specifies a different "type".

commented

It's worth to mention that TypeScript compiler does not support Node package exports at the moment. There is bug report for that: microsoft/TypeScript#33079

commented

There are two distinct issues I would like to discuss with the current ESM loader implementation:

  • it does not play nice with deferring transformSource calls to the "next" (e.g. Node default) ESM loader, obstructing loader chaining.
  • .cjs and .mjs are not interoperable with TypeScript-transpiled sources, as the compiler does not support these file extensions.

The loader hooks resolve and getFormat eventually defer execution to the "next" (e.g. default) loader hook. However, `transformSource' swallows the entire call without deferring out-of-scope source transformations (e.g. non-TypeScript extensions) to the "next" loader hook, stifling composability for loader chaining.

I mention this issue of composability since ESM loader chaining will be available when the pull request nodejs/node#33812 has landed in Node.js. The currently proposed ESM loader chaining API is documented here. It would be preferable that the ts-node ESM loader does not have to be the last (i.e. rightmost) ESM loader, since it blindly swallows the call. If two distinct loaders never defer to the "next" loader, they are not easily composable in a chain; higher order code would be needed to compose them intelligently. There is an example ESM loader with good chaining composability available here - it only handles in-scope calls and defers out-of-scope calls.

In the present, only one ESM loader that can be registered with --experimental-loader/--loader, making loaders mutually exclusive (e.g. CoffeeScript and TypeScript). This can be resolved with a userland ESM loader chaining "polyfill" before the native chaining support is available (a shoddy example is available below). Once again, the same issue is prevalent - the ts-node ESM loader must be the "rightmost" loader in the chain, as it will not continue the chain.

I am interested in amending this behavior so that the loader chaining "just works", in almost any order.


A notable consequence of the current "catch-all" behavior: .cjs and .mjs files are not loadable and interoperable by TypeScript-transpiled ESMs (i.e. sources transpiled with "ESNEXT" target) unless they are either implicitly or explicitly ignored by the ts-node instance. It could be argued this is a pitfall of TypeScript's compiler itself and requires upstream support (see microsoft/Typescript#39840); regardless, an exception is thrown for seemingly out-of-scope compile calls (likely by file extension?). For example, non-valid TypeScript extensions generate the following error:

$ node --experimental-loader ts-node/esm.mjs index.mjs
(node:207696) ExperimentalWarning: --experimental-loader is an experimental feature. This feature could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)

C:\REDACTED\project\node_modules\typescript\lib\typescript.js:139175
                var error = new Error("Could not find source file: '" + fileName + "'.");
                            ^
Error: Could not find source file: 'C:/REDACTED/project/index.mjs'.
    at getValidSourceFile (C:\REDACTED\project\node_modules\typescript\lib\typescript.js:139175:29)
    at Object.getEmitOutput (C:\REDACTED\project\node_modules\typescript\lib\typescript.js:139560:30)
    at getOutput (C:\REDACTED\project\node_modules\ts-node\src\index.ts:562:32)
    at Object.compile (C:\REDACTED\project\node_modules\ts-node\src\index.ts:775:32)
    at C:\REDACTED\project\node_modules\ts-node\src\esm.ts:96:38
    at Generator.next (<anonymous>)
    at C:\REDACTED\project\node_modules\ts-node\dist\esm.js:8:71
    at new Promise (<anonymous>)
    at __awaiter (C:\REDACTED\project\node_modules\ts-node\dist\esm.js:4:12)
    at transformSource (C:\REDACTED\project\node_modules\ts-node\dist\esm.js:71:16)

(despite index.mjs existing and functional; it executes correctly without the --experimental-loader ts-node flag)

Notably, the same behavior is present with CommonJS's require.extensions when executing node -r ts-node/register index.cjs. ts-node seems to inject the TypeScript compiler in Module._extensions['.js'], but no native Module._extensions['.cjs'] is registered; thus a .cjs file defaults to the TypeScript compiler in Module._extensions['.js'], which throws the same error.

As a quick hack, installing a userland ESM loader.mjs can make them interoperable:

import Module from "module";
Module._extensions['.cjs'] = Module._extensions['.js'];

const tsNode = import("ts-node/esm.mjs");

let pendingResolve = false;
let tsEsmLoader;

export async function resolve(specifier, context, defaultResolve) {
    if (!pendingResolve) {
        pendingResolve = true;
        tsEsmLoader = await tsNode;
    }
    return Reflect.apply(tsEsmLoader ? tsEsmLoader.resolve : defaultResolve, this, arguments);
}

export async function getFormat(url, context, defaultGetFormat) {
    return Reflect.apply(tsEsmLoader ? tsEsmLoader.getFormat : defaultGetFormat, this, arguments);
}

export async function transformSource(source, context, defaultTransformSource) {
    return Reflect.apply(tsEsmLoader && !context.url.endsWith(".mjs") && !context.url.endsWith(".cjs") ? tsEsmLoader.transformSource : defaultTransformSource, this, arguments);
}

I strongly feel that the recommended .cjs and .mjs extensions should not be mangled when registering the ts-node loader. This can be fixed by either upstream support from microsoft/TypeScript or by the loaders in this project. I am unsure of what ramifications there are to this decision, though. Since the introduction of ES modules has also introduced the new convention of .cjs and .mjs, I feel this is also worth mentioning on this ticket.


I am not too familiar with the internals or API side of the TypeScript transpiler, so some of these issues might already have some solution, decision, or workaround that I am simply unaware of. Let me know what you think.

commented

@concision Thank you for taking an interest. Responding to both items:

it does not play nice with deferring transformSource calls to the "next" (e.g. Node default) ESM loader, obstructing loader chaining.

When I first implemented ts-node's ESM loader, node did not have any plans to support composing multiple loaders. I'm glad to see that they are now supporting this. We will certainly accept a pull request that allows ts-node to support composing with other ESM loaders. Are you able to send us one?

.cjs and .mjs are not interoperable with TypeScript-transpiled sources, as the compiler does not support these file extensions.

Is your usage of .cjs and .mjs incompatible with TypeScript's semantic analysis? If TypeScript's language service or CLI compiler are throwing a semantic diagnostic in these situations, then we will faithfully pass that semantic diagnostic to the user. Our --transpileOnly mode skips typechecking, so semantic diagnostics are not generated.

Are the errors you are seeing coming from TypeScript? If the same code is compiled by tsc, what are the errors? (if any)

This information will help us make a fix if appropriate.

commented

We will certainly accept a pull request that allows ts-node to support composing with other ESM loaders. Are you able to send us one?

I can definitely author a pull request for this. I would like to work out what we should consider the expected behavior first, though.

In transformSource, are the only sources we should transform be based on the file extensions .js, .jsx, .ts, .tsx , based on context.url? TypeScript does not seem to recognize .cjs and .mjs for the time being.

If so, should extensions be captured only be enabled based on tsconfig.json settings, similar to this? These enable rules seem reasonable to me to apply to the ESM loader as well.

I have also realized that no ESM loader exists that offers an equivalent of the following CommonJS loaders:

  • ts-node/register/transpile-only
  • ts-node/register/files

Perhaps it is also worth adding more ESM-equivalent loaders?

Are the errors you are seeing coming from TypeScript? If the same code is compiled by tsc, what are the errors? (if any)

Thanks for mentioning the --transpileOnly flag might be affecting it. It turns out it is related to TypeScript's compiler trying to type check the files and not liking the extension. TypeScript's error message could definitely be a bit more verbose.

As evidenced by running the following with an empty index.cjs file:

# throws "Error: Could not find source file", despite the file exists
$ node -r ts-node/register index.cjs

# executes successfully
$ node -r ts-node/register/transpile-only index.cjs

# compiling with tsc
$ tsc --allowJs index.cjs
error TS6054: File 'index.cjs' has an unsupported extension. The only supported extensions are '.ts', '.tsx', '.d.ts', '.js', '.jsx'.

Found 1 error.

I suppose this ideally requires upstream support from the pull request microsoft/Typescript#39840 being merged.

If we only transform files with the extensions [.js, .jsx, .ts, .tsx] in transformSource, then .mjs files will be loaded correctly (the current loader behavior does not allow loading .mjs files, only .js with "type": "module"). However, this is currently not the case in the CommonJS equivalent - .cjs files will not load correctly when allowJs filtering is on, as the TypeScript compiler will throw an error.

This could be resolved by defining if (config.allowJs) require.extensions['.cjs'] = require.extensions['.js'] before injecting require.extensions['.js']. However, this could prove to be an unintuitive and confusing behavior for an end user of ts-node, as some JavaScript sources will not be ran through the TypeScript compiler.

As it stands though, these discrepancies obstruct ts-node working with mixed codebases where .cjs and .mjs files are present.

What are your thoughts on the matter?

As a side note:
In addition to ESM loader chaining, they are considering moving ESM loaders to isolated worker threads with their own global context (see nodejs/node#31229). I am unsure of what ramifications this change would have on this project, or how to prepare for it. If anything, I am mentioning so that it is on this project's radar (as this is one of the few projects I have seen that even support --experimental-loader).

commented

@concision

After looking at the code again, our transformSource hook already defers via defer()
https://github.com/TypeStrong/ts-node/blob/master/src/esm.ts#L91
Are you seeing different behavior? I believe you're hitting the bug described in the next paragraph. Fixing that bug will not require any changes to esm.ts; only to our ignored() implementation.

The rules for which files we do/don't transform should match the rules implemented for our CommonJS loader. We will transform .ts and .tsx. We will additionally transform .js and .jsx if the allowJs option is turned on. We will not transform any other extensions, and we will not transform anything that's identified by our ignored function. Today, the ignored function has a bug when combined with our ESM loader. Instead of conditionally excluding .js and .jsx extensions, it should be conditionally including file extensions based on the rules I just described.

An ESM equivalent of ts-node/register/transpile-only might be useful. Feel free to file a separate issue for this. We already support loading all flags from a tsconfig file, so I tend to recommend --script-mode or ts-node-script and allow all other flags to be specified in tsconfig.json.

We're not going to support code that tsc and the language service don't support. ts-node is meant to be equivalent to tsc && node. If someone is using tsc, then they're forced by TS to write compatible code. They can use the package.json "type" field to specify if a subdirectory is loaded as CJS or MJS. If/when tsc adds additional features, we'll support them.

I've heard about their plans to move loaders to an isolated worker. If that happens, it'll mean the ESM loader does not auto-install the CJS loader, since it won't have access to the main thread's require.extensions. Otherwise I think it'll keep working as expected. Someone could file separate issues to set up some sort of communication channel between the isolated worker and the main thread, so that a single TS instance could handle compilation for both CJS and ESM. This might also be convenient to make sure both use the same tsconfig settings.

commented

@concision edited my post above, so pinging you in case you missed the edit.

commented

ok frinds here comes the sun you need to check the file befor your load it we do the same inside rollup for example we assume ES6+ import by default if a file has Top Level Import but has require in it we assume still ES6+ so if he uses createRequire it is still ES6 as soon as no TopLevel Import is there we handle it as it would be CommonJS as it is Common JS :)

commented

Oh by the way maybe mts and cts extensions would help? i saw here already

@cspotcode i am deno contributor i can tell you how it works inside deno it uses parcel internal to load and transpil the files. it bundels in memory on first run.

at present i am trying to prepare a language-server called rollup-lang-server that does the same but with rollups algo in the background as i am also rollup contributor i am familar with what it does and it works well as loader.

i also use rollup as webloader often

The Language Server will run rollup + typescript where rollup will do resolve and load and then hand over to typescript at this point rollup does know already what type of file that is via parsing it into a acron ast representation for analyze.

This will also open the gates for loader plugins for typescript

commented

By design, we match node's behavior for choosing ESM/CJS, which determines based on file extension and package.json, and does not read the source text.

commented

@cspotcode i know that typescript does not even handle .mjs or .cjs that is the core issue it will return Module not found for .mjs or .cjs

commented

@cspotcode
You are correct - I am hitting the bug you have described. I glossed over the ignored functionality thinking it was intended to handle something akin to tsconfig.json's excludes.

I will be moving this discussion over to #1098.

I have opened a separate issue for the transpile-only hook over at #1101.

commented

@cspotcode seems like the instructions in the OP are slightly off?

I needed to run

node --loader ts-node/esm.mjs ./my-script.ts

not

node --loader ts-node/esm ./my-script.ts
commented
commented

I'm trying to support ESM but the paths alias configured in tsconfig.json aren't being resolved in node-esm-resolve-implementation as relative or absolute path:

/home/guilima/www/financial-back/node_modules/ts-node/dist-raw/node-esm-resolve-implementation.js:680
  throw new ERR_MODULE_NOT_FOUND(packageName, fileURLToPath(base));
        ^
Error: ERR_MODULE_NOT_FOUND @middleware/errorHandler /home/guilima/www/financial-back/index.ts

tsconfig.json:

{
    "compilerOptions": {
      "baseUrl": ".",
      "paths": {
        "@root/*": ["*"],
        "@business/*": ["src/business/*"],
        "@data/*": ["src/data/*"],
        "@enums/*": ["src/enums/*"],
        "@middleware/*": ["src/middleware/*"],
        "@routes/*": ["src/routes/*"],
        "@schema/*": ["src/schema/*"],
        "@services/*": ["src/services/*"],
        "@utils/*": ["src/utils/*"],
        "*": ["node_modules/*"]
      },
      "outDir": "dist",
      "lib": ["ESNext"],
      "module": "ESNext",
      "target": "ESNext",
      "moduleResolution": "Node",
      "removeComments": true,
      "esModuleInterop": true
    },
    "include": [
      "index.ts"
    ]
}

Can it be a problem or something on my end?

commented
commented

Despite all suggestions offered in this thread, I am stuck in a loop with this and maybe I am missing something.
I have so far:

  • included the type: module in package.json
  • changed the tsconfig file to point to module 2017 and target esnext
  • changed both allowJs and transplileOnly to true
  • changed start to node --loader ts-node/esm.mjs ./src/cli.ts

and still hit the issue:

`(node:20072) ExperimentalWarning: The ESM module loader is experimental.
     internal/modules/run_main.js:54
     internalBinding('errors').triggerUncaughtException(
                          ^
    TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".ts" for...........
    at Loader.defaultGetFormat [as _getFormat] (internal/modules/esm/get_format.js:65:15)
    at Loader.getFormat (internal/modules/esm/loader.js:113:42)
    at Loader.getModuleJob (internal/modules/esm/loader.js:244:31)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at async Loader.import (internal/modules/esm/loader.js:178:17) {
    code: 'ERR_UNKNOWN_FILE_EXTENSION'
}`
commented

@rags2riches-prog have you tried --experimental-specifier-resolution=node flag?
node --experimental-specifier-resolution=node --loader ts-node/esm.mjs ./src/cli.ts

commented

Also double-check if TypeScript likes your code or not. You can use tsc for this. TypeScript requires you to import "foo.js" (js extension) even when you're importing a foo.ts file. I'm not sure if that's happening here, but it's worth double-checking.

commented

@evg656e @cspotcode
Thanks for coming back to me guys, much appreciated.
Same old story here. I have changed the script to node --experimental-specifier-resolution=node --loader ts-node/esm.mjs ./src/cli.ts to no avail.

 yarn run v1.22.4
 $ node --experimental-specifier-resolution=node --loader ts-node/esm.mjs ./src/cli.ts
 (node:18420) ExperimentalWarning: The ESM module loader is experimental.
 (node:18420) ExperimentalWarning: --experimental-loader is an experimental feature. This feature could change at any time
  internal/modules/cjs/loader.js:966
  const err = new Error(message);
          ^

 Error: Cannot find module './dist/esm'
 Require stack:
 C:\xxxxxxx\xxxxxx\xxxxxxxx\node_modules\ts-node\esm.mjs
 at Function.Module._resolveFilename (internal/modules/cjs/loader.js:966:15)
 at Function.Module._load (internal/modules/cjs/loader.js:842:27)
 at Module.require (internal/modules/cjs/loader.js:1026:19)
 at require (internal/modules/cjs/helpers.js:72:18)
 at file:///C:/xxxxxxxxxxxxxxx/node_modules/ts-node/esm.mjs:6:13
 at ModuleJob.run (internal/modules/esm/module_job.js:137:37)
 at async Loader.import (internal/modules/esm/loader.js:179:24)
 at async internal/process/esm_loader.js:69:9 {
 code: 'MODULE_NOT_FOUND',
 requireStack: [
    'C:\\xxxxxxx
  ]
 }
 error Command failed with exit code 1.
commented

I am not trying to pass the hot potato over to you guys, I just need to understand where I can take a look at this internal/cjs/loader function that triggers the error because this specific module in the stack comes up over and over again. This is a node thing isn't it?
Nor do I think that a shotgun approach would help here as in "try this, try that"

commented

Check your package.json if you are using latest version of ts-node. (the esm.mjs is not part of any released version).

package.json:

    "ts-node": "git+https://github.com/TypeStrong/ts-node",

Then I'm using as run script:

TS_NODE_PROJECT='src/_server/tsconfig.json' node --loader ts-node/esm.mjs --experimental-specifier-resolution=node --experimental-top-level-await --no-warnings src/_server/index.ts

(the important ones are --loader and --experimental-specifier-resolution=node)

In my case I have written custom loader (not using esm.mjs) but it was working fine for me.

// Edit:
@cspotcode corrected me, the last ts-node should contains the esm.mjs.

So please check node_modules/ts-node/dist if there is esm.js file and node_modules/ts-node should contains esm.mjs file.

If you are using #master version please check if you have proper linking in your package.json (yarn handles it differently than npm). You will find out the esm.mjs is missing in the mentioned directory above if this is true.

If you are not using #master version then please check if you have latest version in your package.json.

PS: Running on WSL 2.0 && Ubuntu

commented

@sionzeecz correction: esm.mjs is in the latest version.

https://unpkg.com/ts-node@8.10.2/esm.mjs

commented

@rags2riches-prog here is proof that it works, using the latest version of ts-node published to npm, with yarn, on Windows.

https://github.com/TypeStrong/ts-node-repros/tree/bfd23957-d44f-47c2-969b-7fe9b3ba8630

You can fork that branch and modify it so that it breaks in the way you're experiencing. Then we'll be able to verify on our end and propose a fix.

commented

@rags2riches-prog here is proof that it works, using the latest version of ts-node published to npm, with yarn, on Windows.

https://github.com/TypeStrong/ts-node-repros/tree/bfd23957-d44f-47c2-969b-7fe9b3ba8630

You can fork that branch and modify it so that it breaks in the way you're experiencing. Then we'll be able to verify on our end and propose a fix.

will take a look at it tomorrow and revert. I have already the latest version of your package and please, I am not trying to attack your package in any shape or form...I am trying to understand here despite having a quite busy schedule at work.

commented

Of course, I don't see it as an attack at all. I'm also busy, and I get questions from many other users, so this is the quickest way to diagnose your problem.

We get a lot of questions which are either very simple mistakes or problems completely unrelated to ts-node. Additionally, the questions often omit important information. Without knowing you personally, I have to assume the problem you're having is unrelated to ts-node. This is why I ask for people to send me code that I can actually run which shows the problem. Usually, in the process of attempting to send me this code, people end up figuring out their own problem. This is good, because it solves the problem for them and for me.

commented

@cspotcode works as expected, meaning I missed something about the configuration. Thank you for your help.

commented

v9.0.0 is out, including a few ESM improvements: https://github.com/TypeStrong/ts-node/releases/tag/v9.0.0

commented

ESM support works great for me, thank you! Any ETA on caching support?

commented

@sgtpep I'm glad to hear it's working well for you!

Please move caching discussion to the relevant tickets: #951, #908

commented

With v9.0.0, I notice a 1-second delay when starting the script with the loader. I guess that's expected?

echo "console.log('Hello world')" > hello.js
cp hello.js hello.ts
node --loader ts-node/esm hello.js  # instant
node --loader ts-node/esm hello.ts  # takes one second
commented
commented

I am trying to run this with mocha through npx. What I run is the following:
npx --node-arg "--loader ts-node/esm.mjs" mocha --config ".."
I run it this way since as I understand you can only register the ESM loader by argument to Node, and you cannot do it inside of a required file when mocha starts up. But I get this error:

TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension "" for node_modules/mocha/bin/mocha

Is it possible to have the default extension assumed 'js'?

commented

@archfz this is a mocha bug, caused by a nodejs limitation. You can check mocha's issue tracker to find the appropriate solution. I don't have a link on-hand.

I believe they've already made the necessary changes, but you may need to follow specific instructions that are written either in mocha's README, on its website, or in the relevant issue.

commented

I think this is of concern here as well. Suppose I have a binary that is infact js and executed with node (shebang) and I don't want to have an extension to it (that is the norm). Shouldn't in that case this loader work?

commented

@archfz node is saying it does not understand how to classify extensionless files, correct? We defer to node to classify files that we don't care about. Extensionless files are not handled by ts-node, so we defer to node.

Here is the relevant code:
https://github.com/TypeStrong/ts-node/blob/master/src/esm.ts#L58-L80

Additionally, it is having trouble classifying the file node_modules/mocha/bin/mocha, correct? This is tricky because mocha's package.json says that it should be loaded as CommonJS, not ESM. Should node be using the ESM loader in this case?

commented

Any word on node 12 support? Been trying to run with the --es-module-specifier-resolution=node --experimental-modules flags, instead of the --experimental-specifier-resolution=node that works with 13+. Here's the error I got:

TypeError [ERR_INVALID_RETURN_PROPERTY_VALUE]: Expected string to be returned for the "format" from the "loader resolve" function but got type undefined.
    at Loader.resolve (internal/modules/esm/loader.js:88:13)
    at Loader.getModuleJob (internal/modules/esm/loader.js:147:29)
    at Loader.import (internal/modules/esm/loader.js:131:17)
commented

@nguyensomniac The error you're getting doesn't seem related to the flags you're trying to use. It seems like a bug in node. Are you absolutely sure this error is being caused by ts-node's loader? Does the same error happen when you run ESM modules without ts-node on the same version of node 12?

Those node flags have to be parsed manually in our code. If you think ts-node is missing some logic for node 12, we will probably accept a pull request to fix it. My focus is on supporting node 13 and 14, since all of this is experimental anyway.

commented

@nguyensomniac Could you specify what version of node 12 you are receiving the error with? I was testing and there seems to be mixed results on various node 12 versions.

On Docker image node:12.18.3, I had no problems using just --loader ts-node/esm.
However, on Docker image node:12.0.0, different flags were required (i.e. --experimental-modules --es-module-specifier-resolution=node --loader ts-node/esm) and there was an error (different from the one you specified, though).

commented

I tried again with esm-usage-example in this repo using 12.18 (I was using 12.15 beforehand). I was able to make it work by adding an extension to the export at foo.ts and running the following command: node --loader ts-node/esm ./index

However, without the extension, I get an ERR_MODULE_NOT_FOUND with and without the --experimental-modules --es-module-specifier-resolution=node flags. Is that consistent with what you've found?

commented
commented

@cspotcode Thanks for the prompt replies—I was able to fix my issue by aliasing --es-module-specifier-resolution to --experimental-specifier-resolution. I'll open a PR with more info.

commented

Small bug: ts-node can't load a package if it defines the conditional imports directly at the root of the exports field.

Scratch that, i just didn't notice that it was imported twice, once missing the leading @
i thought it was odd since you are using a slight modification of the native esm loader.

commented

@KilianKilmister yeah, it's a good thing to keep an eye on. If node every makes bugfixes to their ESM loader, then we'll need to integrate those fixes into our copy. That's why the indentation in our copy is intentionally wrong: to hopefully make merging upstream changes easier if/when they happen.

commented

Pardon in advance if this isn't the right location for my comment/question 🙇 :

Is the following bullet point (from the pending development section) Implement require('ts-node').esmImport(module, 'import-path') related to tsconfig compilerOptions.paths? I think I'm encountering an expected bug, but wanted to confirm.

I was following the usage directions/examples and have my code working, with the exception of ES style imports using tsconfig paths.

More context on my situation:

// foo-bizzle.ts
import { getKnexInstance } from '@db/injector'; // @db is a ts path/alias

node --loader ts-node/esm scripts/foo-bizzle.ts

.../node_modules/ts-node/dist-raw/node-esm-resolve-implementation.js:650
  throw new ERR_MODULE_NOT_FOUND(packageName, fileURLToPath(base));
        ^
Error: ERR_MODULE_NOT_FOUND @db/injector.ts .../scripts/foo-bizzle.ts
    at packageResolve (.../node_modules/ts-node/dist-raw/node-esm-resolve-implementation.js:650:9)
commented

@chrisdchow

Is the following bullet point ... Implement require('ts-node').esmImport(module, 'import-path') related to tsconfig compilerOptions.paths?

Good question. No, it's not.

esmImport() is a way to preserve await import('foo') in a .ts file when you have "target": "CommonJS". TypeScript will normally transform await import('foo') into a require() call if you have "target": "CommonJS". This is normally what you want. But what if you want most imports to be converted to require(), but you need one dynamic await import() to be preserved? Node allows us to do await import() in a CommonJS file to load ESM modules, so this is occasionally necessary.

The hack is to write a helper function in another file, not being transformed by TypeScript, which does await import(). Then use this helper function in our CommonJS .ts file. esmImport() is this helper function.

compilerOptions.paths affects the way TS's typechecker resolves paths. You're only supposed to use it if the runtime environment does the same sort of path mapping. (webpack, rollup, etc) There are third-party libraries such as tsconfig-paths which can implement this non-standard path resolution for require(), but I don't think any of them support ESM.

commented

@boyangwang use

import type { SomeType } from './someType'

There's a compiler option --importsNotUsedAsValues--, which you can set to

  • preserve type only imports,
  • remove type only imports at compile time,
  • throw a compiler error on type only imports that aren't using import type { X } from 'y'

I believe your code should also throw the second error when running files pre-compiled with tsc -p <your-tsconfig-json>
I always set this option to error, as it significantly improves source-code readability, as everyone who sees it knows that something is just a virtual type/interface and not an actual object.

I would recommend setting the compiler option "importsNotUsedAsValues": "error" to anyone

commented 
type

Thank you! That's very helpful

I was in the mid of editing my question. For any future folks, here's my original question:

Does it support .d.ts file? I see 2 errors and couldn't make it work:

node-esm-resolve-implementation.js:319 throw new ERR_MODULE_NOT_FOUND

An import path cannot end with a '.d.ts' extension

commented

@boyangwang
This old comment is relevant. It explains why you generally need to include a .js extension when importing .ts files.
#1007 (comment)

Combined with @KilianKilmister's comment, it should give you everything you need to know. I agree with the compiler option recommendation.

commented

compilerOptions.paths affects the way TS's typechecker resolves paths. You're only supposed to use it if the runtime environment does the same sort of path mapping. (webpack, rollup, etc) There are third-party libraries such as tsconfig-paths which can implement this non-standard path resolution for require(), but I don't think any of them support ESM.

@cspotcode any chance the loader could support path aliasing?

Similar to this perhaps: ilearnio/module-alias#59 (comment)