@turf/helpers bundle size grew from 4.9 to 53kb minified between v6 and v7
VIKTORVAV99 opened this issue · comments
It seems like the bundle size between v6 and v7 increased by a substantial amount.
Here you can see that the bundle size for @turf/helpers@6 is 4.9kb: https://pkg-size.dev/@turf%2Fhelpers@v6
And here you can see that the latest version is at 53kb: https://pkg-size.dev/@turf%2Fhelpers
The amount of modules also went from 1 to 70 so my guess is that there was build system changes and there is an error in that?
The main culprit seems to be deep-equal and its dependencies.
Thanks for noting this @VIKTORVAV99. Is this causing a problem when you build or deploy? Or is it more of an FYI observation?
Thanks for noting this @VIKTORVAV99. Is this causing a problem when you build or deploy? Or is it more of an FYI observation?
It's not causing any direct issues but it's a lot of additional code to download since we use this package in a web app. This causes additional overhead affecting performance, especially on mobile devices which might not have the best network conditions.
And since it seems to import everything even if we only use 2 of the functions this is currently a dealbreaker for us to upgrade I'm afraid, especially since the current versions seems to be working just fine.
deep-equal was part of a precision fix issue, so it's not a build error that it now ships with turf-helpers.
Do hear what you're saying about the package size though. We would obviously love for everyone to jump on board the v7 train. Couple of options we could explore then:
- look for a smaller implementation of deep-equal, so we can get the package size back down while keeping the bug fixed
- explore options to enhance tree shake-ability
I'd prefer to steer away from 2. for the time being as it's a bit of a dark art, and isn't guaranteed to be supported by all environments anyway. So trimming back deep-equal seems our best bet. To help us find a potential compromise, what would you consider a palatable size increase?
Considering we only use multiLineString and multiPolygon which currently adds about 900 bytes minified (a little less actually) in v6 and 53 000 bytes on v7 that is almost a 60x increase, which is not acceptable.
I am not very familiar with the codebase (yet) but perhaps you can explain why this exist in the index file for the helpers package:
export * from "./lib/geojson.js";
export * from "./lib/geojson-equality.js";As far as I can tell at first glance this should not be needed at all for the helpers package and is also what is causing the tree-shaking issues that did not exist before.
If that is not needed I think the build sizes would be a lot more reasonable and we should have no problem upgrading.
Unfortunately I don't have the time to set this up and test locally today but I'll look into it myself as soon as I can if someone don't beat me to it.
Edit: The tree shaking might be solved by marking the package as side effects free, which should allow bundlers to strip exports that aren't used.
Thanks @VIKTORVAV99.
Those classes are exported for use by other turf packages. They used to be in a third party package, however that was causing multiple precision issues and was no longer being maintained. In lieu of having a turf-internals package for sharing code, we pulled that code into helpers so we could get v7 out the door with as many bug fixes as possible. That's lead to an unintended increase in the package size, which is why I'm here trying to help you.
So, per my last comment, do you have a size increase you believe would be acceptable? That will let us determine when any efforts to resolve this can be considered a success.
Fyi, turf-helpers is already marked as side effect free, so for whatever reason, tree shaking is not already working.
Thanks @VIKTORVAV99.
Those classes are exported for use by other turf packages. They used to be in a third party package, however that was causing multiple precision issues and was no longer being maintained. In lieu of having a turf-internals package for sharing code, we pulled that code into helpers so we could get v7 out the door with as many bug fixes as possible. That's lead to an unintended increase in the package size, which is why I'm here trying to help you.
So, per my last comment, do you have a size increase you believe would be acceptable? That will let us determine when any efforts to resolve this can be considered a success.
Fyi, turf-helpers is already marked as side effect free, so for whatever reason, tree shaking is not already working.
Honestly since the helpers themself don't even use this deep-equal code no increase would be the only acceptable increase in this case.
What I fail to understand is why it's part of the @turf/helpers package in this case. It's not a documented feature and it's just used in two other packages, @turf/boolean-equal and @turf/bolean-overlap. Therefore I don't think it should even be a part of the helpers package and should either be moved to those packages or moved to a stand-alone lib package (or @turf/meta).
Another acceptable option would be to split out all the helpers and publish them as separate packages. That way you should implicitly get almost the same benefits as if the tree-shaking worked (minus some additional overhead).
What I fail to understand is why it's part of the @turf/helpers package in this case
Because it was convenient, and as I mentioned fixing bugs ahead of the v7 release was the priority.
As the non-functional aspects of the package clearly aren't to your satisfaction, more than happy to take a look at refactoring. Can't guarantee that even if the code is removed entirely from helpers that the package won't change size though - there's been three years of Typescript and other toolchain developments since 6.5.0.
However, at that point you can decide if it meets your expectations, and upgrade should you choose. Will update this issue should a potential fix become available.
Just did some digging with a minimal vite repo in library mode with one import from @turf/helpers and no minification to try and see what the tree shaking issue might be:
If I remove this build setting from tsup.config.ts:
keepNames: true,
The build size changes from:
vite v5.2.13 building for production...
✓ 148 modules transformed.
dist/my-library.es.js 108.78 kB │ gzip: 23.77 kB
dist/my-library.umd.js 115.87 kB │ gzip: 24.08 kB
✓ built in 6.60s
to:
vite v5.2.13 building for production...
✓ 148 modules transformed.
dist/my-library.es.js 41.54 kB │ gzip: 9.67 kB
dist/my-library.umd.js 44.12 kB │ gzip: 9.79 kB
✓ built in 769ms
There is still a lot of overhead in there but it's a step I the right direction at least.
Would it be okay if I opened a PR for that?
EDIT:
For reference this is the v6 build size with the same config:
vite v5.2.13 building for production...
✓ 2 modules transformed.
dist/my-library.es.js 0.61 kB │ gzip: 0.26 kB
dist/my-library.umd.js 0.80 kB │ gzip: 0.33 kB
✓ built in 62ms
A PR under development produces the following results in a minimal Vite project in library mode:
vite v5.3.1 building for production...
Turf 6.5.0, turf-helpers isObject call
✓ 6 modules transformed.
dist/my-lib.js 2.50 kB │ gzip: 1.38 kB
dist/my-lib.umd.cjs 2.52 kB │ gzip: 1.40 k
Turf 7.0.0, turf-helpers isObject call
✓ 152 modules transformed.
dist/my-lib.js 72.95 kB │ gzip: 20.36 kB
dist/my-lib.umd.cjs 55.08 kB │ gzip: 18.26 kB
Turf 7.0.0 with PR, turf-helpers isObject call
✓ 6 modules transformed.
dist/my-lib.js 8.03 kB │ gzip: 3.01 kB
dist/my-lib.umd.cjs 6.89 kB │ gzip: 2.91 kB
The PR removes the heavyweight deep-equal import from turf-helpers, though for some reason tree shaking of other functions in turf-helpers isn't happening.
This is probably the best we can do right now. Will get that PR merged and it will be in a subsequent point release.
Potentially some more background to refer to for future refinements - https://stackoverflow.com/questions/74362685/tree-shaking-does-not-work-in-vite-library-mode
Ok, a bit more background. Long story short, when tree shaking rollup and esbuild take different cues as to what can be shook or not. sideEffects: false isn't enough for esbuild in many cases so individual top level exports need to be marked with /* @__PURE__ */ annotations.
Rerunning the above we get:
Turf 6.5.0, turf-helpers isObject call
✓ 6 modules transformed.
dist/my-lib.js 2.50 kB │ gzip: 1.38 kB
dist/my-lib.umd.cjs 2.52 kB │ gzip: 1.40 k
Turf 7.0.0 with PR and PURE annotations, turf-helpers isObject call
✓ 6 modules transformed.
dist/my-lib.js 2.63 kB │ gzip: 1.48 kB
dist/my-lib.umd.cjs 2.62 kB │ gzip: 1.47 kB
Which is much closer to parity. If I can convince myself it won't introduce unwanted side effects, will include an annotated helpers with this PR. Other modules we can do in a subsequent effort.
Ok, a bit more background. Long story short, when tree shaking rollup and esbuild take different cues as to what can be shook or not.
sideEffects: falseisn't enough for esbuild in many cases so individual top level exports need to be marked with/* @__PURE__ */annotations.Rerunning the above we get:
Turf 6.5.0, turf-helpers isObject call
✓ 6 modules transformed.
dist/my-lib.js 2.50 kB │ gzip: 1.38 kB
dist/my-lib.umd.cjs 2.52 kB │ gzip: 1.40 k
Turf 7.0.0 with PR and PURE annotations, turf-helpers isObject call
✓ 6 modules transformed.
dist/my-lib.js 2.63 kB │ gzip: 1.48 kB
dist/my-lib.umd.cjs 2.62 kB │ gzip: 1.47 kB
Which is much closer to parity. If I can convince myself it won't introduce unwanted side effects, will include an annotated helpers with this PR. Other modules we can do in a subsequent effort.
Thanks for your work on this!
Did you try to remove the keepNames setting from the build config? The output without it are what I would expect from a ESM build but with it includes a bunch of name declarations in the form of: __name(function, "function") which seems to prevent Vite from tree-shaking the code.
No worries. No, haven't changed the keepNames setting in Turf's build, and would be reluctant to it was absolutely necessary and the impact on all users was properly understood.
From what I can see here, vite (in both normal and library mode) is tree shaking ok, so long the annotations are present.
@VIKTORVAV99 a PR has been merged that removes deep-equal and disables keepNames. That's been publshed to npm as 7.1.0-alpha.7
Please give it a try and confirm that has resolved the issue 👍
@VIKTORVAV99 a PR has been merged that removes deep-equal and disables keepNames. That's been publshed to npm as 7.1.0-alpha.7
Please give it a try and confirm that has resolved the issue 👍
Just gave it a go and now the both the tree shaking works again and the bundle size is back to normal!
Thanks a lot for fixing it @smallsaucepan and @mfedderly!
That's great. Thanks for confirming.