Consider deprecating base+polyfill due to complexity and misuse
philipwalton opened this issue · comments
Since releasing the "base+polyfill" version of this library, we've received a number of reports that it's complex and confusing to use (e.g. #95).
The primary benefit you get from using the "base+polyfill" build is you can collect FID data from all browsers. You also potentially get more accurate visibility reporting, but we have no data on how often that accuracy affects the value of the metrics (i.e. it's more accurate in theory). In other words, if you don't want FID data in other browsers, it's likely not worth using the polyfill build.
Given that the vast, vast majority of sites across the web report good FID scores (more than 95%), given that Firefox now does support FID natively, and given that this library already exposes the INP metric (which was created as a replacement for FID and a more accurate measure of responsiveness), it's worth reconsidering whether or not the additional complexity of including the FID polyfill (both for end users as well as building this library) is worth it.
To answer that question, I wanted to look at how many people were using the polyfill, so I queried the HTTP Archive response_bodies table from June 1, 2022 to look for sites that were using web-vitals at all, and then specifically sites that were shipping code unique to the "base+polyfill" build.
Here were the results:
| Segment | Count |
|---|---|
| All versions | 706,302 |
| "base+polyfill" version | 22,905 |
So only about 3% of sites using web-vitals are using the "base+polyfill" version.
However, when I started looking at some of the individual sites that were using the polyfill, I noticed many, many cases where sites were using it incorrectly—including cases where sites were using a third-party service and that service was using the polyfill version incorrectly (raygun and catchpoint analytics stood out, if you work on these products please reach out and I can help with proper implementation).
Examples I found of misuse:
- Loading the base bundle without the polyfill script at all (meaning FID won't report).
- Loading the base bundle and the polyfill bundle async (the polyfill script needs to be inlined in the
<head>or it's not accurate). - Loading the polyfill script as a sync, blocking script in the head (which negatively affects load performance)
I was curious to see if I could query how often sites were using it correctly vs. incorrectly, and this is what I found.
| Segment | Count |
|---|---|
| Correct usage | 9,349 |
| Loading the polyfill via an external script (incorrect) | 13,556 |
| Loading the polyfill via an external, blocking script (incorrect) | 4,813 |
To summarize, based on my research, it looks like the majority of sites who use the polyfill use it incorrectly (so they get no benefit and likely report incorrect data), and a sizeable portion of sites use it in a way that negatively affects performance (blocking scripts). Only about 1% of sites that use the web-vitals library use the polyfill and use it correctly, and it's unclear how many of them actually look at the polyfill data they get for FID.
Based on all of these reasons, I think it makes sense to deprecate the polyfill in version 3 and then remove it in version 4, but I'd love to hear feedback from anyone who is currently using the polyfill and would not want to see it removed.
I don't think base+polyfill should be removed just warning.
Can you explain more why you feel that way @ShuaibZaker-Create ?