sveltejs / svelte

Cybernetically enhanced web apps

Home Page:https://svelte.dev

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

TypeScript Roadmap via LSP

orta opened this issue · comments

Goal: TypeScript-style tooling in Svelte

Getting TS in Svelte might be a blocker for some, but I think the biggest underlaying advantage comes in that the "TS support" is just a stricter version of the JS tooling. Everyone using Svelte gets improvements in tooling from this, not just folks with TS.

This proposal makes a strong separation between the tooling and the svelte compiler. It explicitly minimizes the impact on core in favor of moving live-DX complexity to a svelte language server.

The language server provides editor support and CLI validation for working in svelte, not the core library svelte. This isn't a unique concept, it's based on the vue eco-system which has similar "extend HTML" constraints.

This pattern also allows different maintainers to work on JS/TS tooling independent of the svelte core team (which is how we do it in GraphQL), making it a good place for people who want to contribute to step up and own some of the svelte core ecosystem.

Three main steps to get there:

  1. Add support in core for transpiling TypeScript in tags script, but ignore type-checking. TS will always synchronously emit JS if it's reasonably legal JS. This is the only change required in core. When you encounter a TS script tag, you can look up whether they have TS/Babel in deps and throw if not. No need to force the dep.

  2. Create "svelte/svelte-language-tools" repo in main org:

    1. Add svelte-language-server
    2. Add svelte-vscode
    3. Create a new CLI tool which uses the LSP to validate - this makes LSP calls against all .svelte files to run TS/JS tooling for errors.
  3. Svelte Language Server improvements

Rationale

  1. This is to allow the editor support to make that assumption out of the box, and for a user to be able to safely write <script lang="typescript"> knowing that it will work with the core.

  2. Adding TypeScript support to Svelte can take a lot of tips from prior art done on Vue.
    Vetur acts as the higher level tooling framework for both providing the JS and TS tooling support in scripts.

    This pattern keeps most of the work in the LSP level, which means all editors can use it, the vscode extension would just be one client of many.

    The language server could turn into @svelte/language-server and the vscode-svelte can get some more maintainers added so that deploys can be done by others. If folks have other popular IDE integrations, they could live in this repo too.

    svelte check adds a way to validate on CI, which can handle folks without the extension not accidentally breaking things. Because TS will always emit JS even if it has type errors, on purpose.

  3. This is basically a "draw the rest of the owl"

Work on Svelte Lanaguge Server

Luckily, both vscode-svelte and svelte-langauge-server have already been built and worked on by @UnwrittenFun and he's interested in moving those to the org. They both are solid foundations to build upon, from looking through the code.

They're definitely good enough for JS today, and considerably better than nothing!

I think one of the first blockers on making it something recommended by default, is finding a way to make sure that money-labels (reactive designators) are handled in both JS and TS.

Because the $: syntax does more work than TypeScript can know about $: x = count + 1 would probably need transforming into something like const x = () => { count + 1 } as an in-memory transform, so that the variables exist in the "TS representation" of the LSP (but not on disk)

HTMLx

The quick route:

I think you can get autocompletion, but I'm not sure how feasible typechecking would be here in a cheap way.

For inside {} the LSP could make API requests to the known script tags above and request all the variables at the top level scope (basically the equivilient of adding a newline in that script and then checking what's available.)

In theory you could type check each one by making a unique TS document for all of them, with exposed globals from the main script - but that could be quite slow, and it's hard to say how well that could scale.

Longest term:

How TypeScript handles the LSP-ish server is by embedding it into the core codebase, perhaps over time it makes sense to give the LSP deeper access to the svelte compiler, so that the LSP can ask it questions about the document "what variables are available in this eval context?" as (I guess) you must have some of that information to do the compilation.

I'd see this as quite a long time away, and you can get a lot of value right now in the LSP without access to that.

The svelte language server could eventually split into a htmlx-language-server and the svelte version would wrap with svelte specific cases.

Things to think about

This opens up a lot of legitimate questions "like how does the tooling support x and y" - the answer to most of them is "that is something the language-server would have to handle" - the language server would need to keep track of files, their exports and more - it'll be a cool an interesting project for folks who care about the DX at editor level.

Anything in vetur today should be feasible for svelte.

/cc @octref who works on Vetur, and might have some insight.

My experience with the Typescript compiler API is limited to a few hours of poking around, but I would love to help out with this once the repositories are all set up.

I would love to help out with this. How can I get involved?

I'm going to give it at least the week to sit on the chance someone has a good blocker. I'd like to get an explicit "yes, this is OK" from @UnwrittenFun - because while it's legal to fork, and he's said it is a good idea before. I'd rather be sure.

In the meantime, I'll make a yarn workspace repo on my own user account which retains all the history of svelte-vscode and svelte-language-server and look into how their testing setup works. This can get transferred over as the base to work from.

As homework, I'd recommend reading the code for:

For folks who might be thinking this feels super complex types of code, don't worry, it really tends to be quite shallow code built on top of vscode-languageserver (because creating a LS is a reasonably common thing nowadays) - and a lot of the foundations are already set up.

I'd also love to lend a hand!

Sounds great! There is a project that has gone down this path that might be a great starting point:
https://github.com/simlrh/svelte-language-server/blob/feature/extend-ts-support/src/plugins/ts-svelte/service.ts as for handling the htmlx, https://github.com/halfnelson/svelte2tsx converts a svelte file to tsx for type checking. There is also a commandline tool that uses this infra

Thanks @halfnelson - I bet both of those tools have a lot of underlaying ideas we can get merged into the official LS tooling level!

/cc @simlrh

Exciting stuff, absolutely a "yes, this is OK" from me 👍

commented

Happy to share lessons learned or answer questions, but I wouldn't have time to commit any code as I don't even find enough time to develop Vetur/VLS.

Just a notice that transpiling gets expensive and complex soon. Consider auto completion. Transpilation need to happen in < 50ms in the worst case for completion to show up fast enough for each keystroke. Besides, your compiler needs to reasonably transpile all invalid states while user inputs svelte-specific JS such as $: x = count + 1 one symbol by another. I wish you success, but Vetur doesn't transpile <script> regions in Vue. Only interpolations are transpiled.

@orta I have pretty good svelte/typescript language server running locally using svelte2tsx, it does checking of both the scripts and the htmlx props, autocompletion, maybe some other stuff. I'll collect all the branches together and put them online somewhere so it's easy for others to try.

I think it's definitely the place to start, but does need some work - it's pretty incredible what @halfnelson achieved using string manipulation with https://github.com/rich-harris/magic-string, but I have some worries about correctness in edge cases, so I began a full rewrite of svelte2tsx using AST transformations (turning svelte htmlx ast into plausible/typecheckable babel jsx ast). I got about halfway through the necessary features, without source map support, before burning out on it (I don't use svelte for work so this is all hobby stuff). I'll put that online for reference.

Cool, so an update - doesn't feel like any blockers to the idea of merging and centralizing so far (other than, "there is more cool stuff we should integrate" - which is cool, we should try get that in)

I've started merging some of the tooling repos together: https://github.com/orta/language-tools and updating all the dependencies. If there's still no conceptual blockers I'll start poking folks to get this pushed into a new remote of sveltejs/langauge-tools on Friday.

@octref's point about $: x = count + 1 is legit, this might be a hard problem to solve performantly - however TypeScript can't infer that this creates a variable without some kind of transformation

@octref's point about $: x = count + 1 is legit, this might be a hard problem to solve performantly - however TypeScript can't infer that this creates a variable without some kind of transformation

I think there are 3 related issues here:

First just to clarify, this is valid JS and TS, the $: is a JS label and the only problem this causes for type checking is the 'unused label' warning.

Second is how to type check the x variable, as the typical way to write these statements omits the variable declaration. I've been handling this just by expecting the user to do:

let x: number;
$: x = count + 1;

Third is the issue of parsing the invalid file while the user is typing. acorn-loose does this, and babel has recently added error recovery which I believe does the same thing - worth looking in to.

For the $: blocks, why not allow some more concise interface like

let c = $(()=> count + 1);

Would be easier for TypeScript, as there's no magic syntax.
The generated code would be exactly the same, Svelte would see the code inside the lambda and do its transformations.
I would not mind loosing a bit of syntax sugar for a good tooling support

Second is how to type check the x variable, as the typical way to write these statements omits the variable declaration. I've been handling this just by expecting the user to do:

Shouldn't this work?

$: x: number = count + 1;

This could simply be translated to:

let x: number;
const updateXorWhatever = () => {
  x = count + 1;
}

Shouldn't this work?

$: x: number = count + 1;

Trying to parse this as TS gives an error, it seem to treat both $ and x as labels and number as a value. $: let x: number = count + 1 works ok for typescript but the svelte compiler throws an error "let is a reserved keyword".

In my humble opinion I'd be in favour of a not adding any new syntax, and doing the minimum to make existing svelte type checkable, hence simply adding let variable: type; before any reactive declarations. And that way you don't need transpilation at all.

@simlrh I thought the script content will be transformed before it will be checked by typescript. That's why I've written:

This could simply be translated to:

let x: number;
const updateXorWhatever = () => {
  x = count + 1;
}

Alright, doesn't sound like any blockers. Can an org admin please:

  • Create a new language services repo, and pull in orta/language-tools and I'll put some more polish in via PRs over the weekend

  • Make a #tooling (or #language-tools?) chat room in the Discord

I'm happy to handle incoming PRs for a while to help sveltejs/langauge-tools get bootstrapped (you can add me to the org, or make me a contributor to that repo), but I don't use svelte nor have constructive production experience with it. So ideally, like to hope that some of the folks in this issue end up taking over those responsibilities with some time - I'll be here to provide moral support :D

@orta you should have an org invite now giving you the access you need to transfer the repo(s) into the svelte org. 🚀

Thanks @antony!

Over the weekend I've built out https://github.com/sveltejs/language-tools and got it working as a solid set of foundations. Tests/CI/Debugging/etc.

Next up, this week, I'll try test a bunch of the tools with the community website and some open source examples and try give a sense of what a roadmap could look like for the tools in issues sveltejs/language-tools#6

You don't need to wait on me if you already have ideas - create issues! I'll be learning from scratch, so that comes with fresh perspective but does mean that I don't know what it's like to be doing something like this for a long time or for working on larger projects.

Just plugging this here in case it (or its concept) helps with the $: issue discussed above.

https://github.com/tree-sitter/tree-sitter

I would love to help out in this. How can I help?

Alright, I'm going to close this up - tooling work is improving at pretty constant rate thanks to @dummdidumm and @jasonlyu123 over at https://github.com/sveltejs/language-tools

What it looks like to declare the VS Code extension 1.0 and to ship a separate 'release build' to the extensions store is still undecided. We'll need help from folks who use svelte daily to make decisions around this because I don't have any svelte projects. There's also the possibility that the current "beta" could be renamed to "Svelte" and we ship that fast and often instead of occasionally and with stability.

Thanks @orta! I wonder if it'd be worth a separate ticket to discuss a possible CLI tool integration. I think you might be able to provide some help direction there. For example:

  • What are other popular framework with similar CLI support for TypeScript that we can look at for inspiration?
  • Any pointers on implementing it or what it needs to do? E.g. would it first have to invoke a preprocessor? Or do you imagine it would depend on #4797 instead?

@benmccann will this one work: #3103 ?

Perfect, @benmccann - I've extracted that out into an issue on the language tools repo. sveltejs/language-tools#68

Where the validate CLI command ends up in the long term depends on whether there is a svelte cli which is a bit in the air ( #3103 ) but work can be done independent of that choice and can be merged in later, it's not a big project

Thanks for creating the new issue @orta!

@antony yeah, I knew about that ticket, but wanted a place to discuss the LSP integration piece specifically without going too far off topic on the main thread :-)

Every issue about typescript that I can find is closed, meaning typescript implementation must be done!

Or if its not there should be some way to track / subscribe to a single issue to let people know if this is ready to use -_-

Every issue about typescript that I can find is closed, meaning typescript implementation must be done!

Or if its not there should be some way to track / subscribe to a single issue to let people know if this is ready to use -_-

How to use it today with sveltejs/template?

@frederikhors please keep support questions to discord. You can get the help you need in the language-tools channel.

@antony, I'm asking for two lines in the Readme. For everyone who comes here.

I do believe that TypeScript can be used with Svelte today and all you need to do is install svelte-preprocess. I haven't actually gotten a chance to try it yet, but spent a couple days this week getting lost navigating things just to figure out that much. Hopefully I'll get to give it a try this coming week

I created an issue with a handful of my own suggestions for improving the docs #4816. Please feel free to add more suggestions!

Could someone update the FAQ? It still points to the old issue and says that there is no TS support yet.

@frederikhors this is how to request some info in the readme / on the site about TypeScript support #4816 - thanks @benmccann

@l2dy would be great if you have time to open a PR to add those lines / add a section to the docs. I don't know the exact process.

Sadly, wikis on GitHub don't support PRs. It seems that only project members could edit this wiki.

https://github.com/sveltejs/svelte/wiki/FAQ#why-isnt-hot-module-reloading-working also needs to be updated on the wiki. Maybe someone could update both at once. Sadly you can't send a PR against the wiki

I've opened an issue to track both (#4819) and suggested moving the faq to svelte.dev (#4820)

I don't understand why this ticket is closed, since typescript is not well supported in svelte yet. I found this project in the meanwhile (https://github.com/mistlog/svelte-draft) . It's really promising!

@tezine it's not clear to me why you say typescript is not well-supported in svelte. The support seems pretty good overall to me. Please see the FAQ for instructions on setting up TypeScript support. If there's some particular issue you've hit, it'd be helpful to file a ticket with the respective subproject (svelte-preprocess for building or language-tools for IDE)

@benmccann maybe a new user doesn't know how to start with typescript and svelte using the default sveltejs/template project.

Yep, it's definitely not got to the "document stage" though from using the extension myself in an app, the support is feeling pretty production worthy

@orta it will be used mainstream as soon as we update/support/change/propose a default typescript template like the one now in "pure" js only.

commented

I agree that there is currently a lack of documentation (at least), as writing a Svelte app in TS now is pretty rough. I wouldn't say it is "working" for me, in the sense that both the Svelte compiler and vscode both work and agree together.

For instance, I have numerous errors (using Svelte Beta for vscode) such as:

  1. vscode can't find component imports without the '.svelte' extension (compilation with rollup is fine).
  2. vscode can't type event actions, e.g. on:edit={(ev) => dispatch('edit', { item: i, ...ev.detail })} gives Property 'detail' does not exist on type 'Event'.
  3. svelte-preprocess can't resolve imported ts files. For instance, in a Svelte component, the script tag is <script lang="ts" src="file.ts"></script> and file.ts itself imports another.ts (without the extension as the typescript plugin complains otherwise). The Svelte compiler gives Error: 'importedFunc' is not exported by another.svelte, imported by ....
  4. multiple weird parsing errors in vscode as Identifier expected in <script lang="ts" src="../app/file.ts"> (compilation is fine)
  5. vscode sometimes has errors such as "Type annotations can only be used in TypeScript files" (despite using <script lang="ts"> and compilation being fine)

Maybe something is wrong with my vscode or rollup configuration. I will try to isolate and submit independent issues, but just wanted to give feedback as a whole.

If you use lang="typescript" instead of lang="ts", errors 4 and 5 should go away. A fix for this is on the way, so should be working soon with lang="ts".

@hbbio if it isn't already tracked, you may open some issues on sveltejs/svelte-preprocess and sveltejs/language-tools?

commented

If you use lang="typescript" instead of lang="ts", errors 4 and 5 should go away. A fix for this is on the way, so should be working soon with lang="ts".

Thanks, it fixes some errors. Some remain, i.e.
Type '{ lang: string; typescript: true; }' is not assignable to type 'HTMLProps<HTMLElement>' Property 'typescript' does not exist on type 'HTMLProps<HTMLElement>' when there is also a <svelte:head> tag which loads an external script (compilation is fine).

@AnatoleLucet Yes, I will try to submit them to the right place but it's not always easy to identify the right component! There should really be one clean, official and documented way to setup TypeScript with Svelte :)

Edit: I started this repo https://github.com/hbbio/svelte-ts to isolate issues.

I was able to get this working with the following:

  1. Use Svelte Beta VSCode extension (uninstall all others)
  2. Install svelte and typescript into your package.json
  3. Create a svelte file, here's mine:
<script lang="typescript">
  import Post from './post.svelte'
  import { uid } from './lib'

  type Props = {
    posts: {
      title: string
      date: Date
    }[]
    author: {
      firstName: string
      lastName: string
      email: string
    }
  }

  export let posts: Props['posts'] = []
  export let author: Props['author']
</script>

<html lang="en">
  <body>
    {#each posts as post}
      <Post {post} {author}>{uid(9)}</Post>
    {/each}
  </body>
</html>

You'll find a few quirks here and there, but I'm very impressed with how usable it already is!

@matthewmueller Seems to be working well apart from a memory leak I'm getting with that Svelte Beta VSCode extension

commented

@matthewmueller I started this repo https://github.com/hbbio/svelte-ts to isolate issues (my current list is in the README, I did not yet create small examples for each). Simple demos are working (with typescript instead of ts), but there are a lot of small issues. It's just "fresh paint", I'm pretty sure Svelte+TS will rock soon!

@marcusnewton could you provide more info over at sveltejs/language-tools#151 ?

I don't know what is the etiquette here for discussions, but I'd like to say that reactive could be done simply like this:

import { reactive } from "svelte"; 
let foo = reactive(count + 1);

Then reactive could have this type:

declare function reactive<T>(t: T): T

Thus the TypeScript could also infer the type for the foo for us!

Because svelte is compiler, that function need not to exist except for IDE's and users sanity. The $: syntax is a bit eye-sore and in my opinion unnecessary syntax.

Hi guys,
if Svelte is compiler, so it should handle any syntax.
Now reactive prop uses $: , but why shouln't use something like

let $foo: string = 'reactive defined thing'

Just let with dollar in name, one simple rule.
When we import something like store and for reactivity, we just need use it with dollar, then we can use this power to define reactive variable in component. Syntax should be stay simple, shoudn't be like Vue or React (reactive, useState, etc...)

@daarxwalker It is based on JS Label:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/label

Theoretically, it is at least something similar to JS.

@lukaszpolowczyk
I get it, know labels, but if Svelte compiles code, why not check if variable start with $, no need of labels. That's my idea. I haven't read source code yet, but it will be great and simple, however if possible.

@daarxwalker Some of the creators must answer that.

This would be a breaking change given that variables prefixed with $ indicates syntax for store variables.

@GrygrFlzr So, unite them -> reactive prefix.

@daarxwalker $ is a valid symbol for starting identifier names in JavaScript: https://mathiasbynens.be/notes/javascript-identifiers#valid-identifier-names. Adding special semantics would clash with user-defined variable names.

@malj I know, what you mean, developers naming preferences, migrations to new syntax, etc... But Svelte isn't so standard thing, it's literally revolutionary, so if you add one simple special rule...don't see any problem. Honestly, who uses $ as starting symbol, maybe long time ago, with jQuery. Just trying find simple way :)

Plus, Svelte as compiler (framework) doing things with own way, has some strict rules and developer have to follow these rules.

@daarxwalker The suggestion is indeed valid, and could potentially resolve some issues with TypeScript, but the community sentiment is likely to keep the label syntax, even if an alternative was also added.

@marcusnewton
I understand, what you mean. As you say, I'm looking for a way to elegant solve TypeScript syntax.

It's pretty hard to gauge community sentiment if all the thumbs up here points towards replacing the $: syntax, maybe people here thumbing up are not representative.

However, I'd like to say in my opinion explicit syntax is better than implicit. After all you have to import all the other stuff from svelte too e.g. import { onMount } from "svelte"; adding a reactive there wouldn't make a much difference, it would just make it explicit.

I actually don't care too much how it's solved, it's just a bit verbose for a typescript right now, e.g.

  let urlUE: string;
  let titleUE: string;
  let facebookSharingUrl: string;
  let twitterSharingUrl: string;
  let linkedInSharingUrl: string;
  let whatsAppSharingUrl: string;
  let mailSharingUrl: string;
  let style: string;
  $: urlUE = ...;
  $: titleUE = ...;
  $: facebookSharingUrl = ...;
  $: twitterSharingUrl = ...;
  $: linkedInSharingUrl = ...;
  $: whatsAppSharingUrl = ...;
  $: mailSharingUrl = ...;

Looks a lot of syntax to me.

@Ciantic
It's and option, but it's starting looks like Vue3 ;)
My solution:

  let $urlUE: string = '...'

What's the problem to be solved regarding TypeScript again? Are we talking about this example from earlier in this thread:

$: x: number = count + 1;

These type annotations on the labels could be recognized and stripped out by the Babel parser. (See https://babeljs.io/docs/en/babel-parser#will-the-babel-parser-support-a-plugin-system for one possibility; another would be if it could become officially supported syntax enabled via an official plugin, like how it works with JSX).

The trickier question is how to actually do type checking on these. I think the labels would have to be transformed somehow to code that the TypeScript type-checker recognizes. This would complicate IDE integration as well.

Just some things to keep in mind while exploring options...

The way TypeScript support works is that svelte-preprocess processes the file and passes it to the TypeScript compiler and then to the Svelte compiler. We can probably support any syntax as long as svelte-preprocess handles it correctly. svelte-preprocess needs to be able to operate very quickly (for intellitype, autocomplete, etc. in the IDE) and today is just a regex, so it's best to keep that in mind while designing a syntax.

@benmccann how about we do just this then:

$: foo = count + 1

before passing to typescript turn it to this:

/* $: */ let foo = count + 1;

TypeScript should be able to even infer the type.

Then when it comes from Typescript, transpile back to $:?

Right now the biggest issue with $: syntax is that it can't infer the type for some reason.

OK, this thread has gone pretty far off topic and I'm going to lock it. You're welcome to create individual threads about their own topics.

Alright, popping back to give basically a final update. Today we declared the VS Code extension production worthy, and recently svelte-preprocessor became an official svelte project.

In my opinion, this is enough to call TypeScript support 1st class for Svelte.

  • The JavaScript and TypeScript tooling is solid, and does what you expect.
  • Use TypeScript pretty much everywhere in .svelte files
  • Use svelte-check to validate your builds

There is both official support for making TS files work in the compiler, and work in your editor.

There's a few docs + template issues to sort out but we're there and it took a bunch of work from a lot of folks to get there.

Specifically @dummdidumm, @jasonlyu123, @UnwrittenFun, @halfnelson and @octref who provided code, foundations and valuable advice. Y'all did great work. Thank you.

There we go, that's the right button. Issues with how bits work can go on either the IDE level in the language-tools repo, or at the svelte compiler level in svelte-preprocess.

Docs issues can be new issues on this repo like normal 👍

You'll probably get a better response if you open a new issue at https://github.com/sveltejs/language-tools/issues instead of adding a comment to a closed issue.

Hi all. Great work on this! Just getting into Svelte and TS is a huge bonus for me. Are there any issues that can be watched for when documentation will be added?

Hi everyone, contratulations con the great work

I followed this document and also tried starting directly with this template and I'm getting the following error when trying to import a .ts file from App.svelte:

App.svelte

<script lang="ts">
  import { todos } from './stores'
  export let name: string;

  console.log('todos', todos)
</script>

Error:

bundles src/main.ts → public/build/bundle.js...
[!] (plugin typescript) Error: Could not load [...]/svelte-typescript-app/src/stores.ts (imported by src/App.svelte): Debug Failure. False expression: Expected fileName to be present in command line
Error: Could not load [...]/svelte-typescript-app/src/stores.ts (imported by src/App.svelte): Debug Failure. False expression: Expected fileName to be present in command line

And this is my stores.ts

export const todos: number[] = [ 1, 2, 3]

Am I missing anything?

@opensas You should open a ticket in the language-tools repo

Time to re-lock this thread I guess 👋 - this is all prod ready, we've just not had a big announcement for it all, you can read the announcement bits here: #5101