Need a callback for when children changed or parser finished parsing children
rniwa opened this issue · comments
There appears to be a strong desire for having some callback when the parser finishes parsing children or when a new child is inserted or an old child is removed.
Previously, children changed callback was deemed too expensive but I'm not certain that would necessarily be the case given all major browser engines (Blink, Gecko, and WebKit) have builtin mechanism to listen to children changes.
I think we should consider adding this callback once for all. As much as I'd like to make sure custom elements people write are good, developer ergonomics here is quite terrible, and people are resorting to very awkward workarounds like attaching mutation observers everywhere, which is arguably far slower.
Not sure if this is possible but would it be possible to simply capture the value of childrenChangedCallback immediately after the element is constructed? Then one could easily enable an optimization that doesn't bother watching the children at all if the callback isn't actually provided.
(This would be similar to the iterator.next changes in for-of loops that .next is only captured once at the start of the loop).
EDIT: It looks like the other methods already work like this by capturing the methods off the prototype (TIL).
All custom element callbacks are retrieved at the time of definition so such an optimization is already possible. See step 10 of https://html.spec.whatwg.org/multipage/custom-elements.html#dom-customelementregistry-define
I support adding a children changed callback as that is indeed a primitive the DOM Standard provides and specifications and implementations use. (Firefox also has internal synchronous mutation observers, but nothing in the platform relies on them, though Firefox does currently use them for the output element.)
Before we add this however I'd like whatwg/dom#732 (review) solved. In particular, the primitive I referenced above isn't defined that well and some refactoring is needed. It'd be good to settle on the exact shape in private first before exposing it publicly.
(Note that this is effectively #550 btw.)
As with #550, I do not support this if it is parser-specific. If it is just a CEReactions-time MutationObserver that only custom elements can install, then I'm open to investigating. In that case I think we should also seriously consider un-deprecating DOMNodeRemoved/DOMNodeInserted as an API shape, since that already exists in some browsers and we've needed to spec it properly for some time.
If it is just a CEReactions-time MutationObserver that only custom elements can install
I would love it to be a bit more than that, in that it observes projected children as well. I know that's covered slotchange, but currently you need to combine slotchange and a MO to implement "effective children changed".
I don't think observing the assigned slot should be conflated with this API.
A common ask that keeps coming up is the ability to know when a child node has been inserted & updated: e.g. #765, #619, #615
To do this, just knowing when a child is inserted or removed isn't quite enough due to async / out-of-order upgrading. Perhaps we need both childChangedCallback and childDefinedCallback the latter of which gets called when a child node is inserted & upgraded / defined.
Also see whatwg/dom#662
@rniwa do you foresee this as something that can be observed in a custom element or a more low level API that can be applied to any element? It is not clear from the title or description.
@rniwa do you foresee this as something that can be observed in a custom element or a more low level API that can be applied to any element? It is not clear from the title or description.
I think this needs to be a custom element reaction callback.
I think this needs to be a custom element reaction callback.
That's probably limiting. I feel that this is in the same boat as the connect/disconnect reaction that we were asking for last week during the F2F, something a lot more generic.
Now, if we were to add a new reaction callback, will that callback be invoked even when the CE is not connected to the DOM? e.g.:
const ce = document.createElement('x-foo');
ce.appendChild(document.createElement('p'));I think this needs to be a custom element reaction callback.
That's probably limiting. I feel that this is in the same boat as the connect/disconnect reaction that we were asking for last week during the F2F, something a lot more generic.
For general nodes, you can just use MutationObserver. If you wanted it sooner, then that's a discussion that has to happen in whatwg/dom#533
Now, if we were to add a new reaction callback, will that callback be invoked even when the CE is not connected to the DOM?
You mean when it's not connected to a document? If so, then yes. There is no reason to restrict this childChangedCallback to the connected nodes.
Remember the imo much more important part remains the childrenParsedCallback. Obviously that wouldn't make too much sense when the CE is not connected to the DOM. Currently CE only work in the upgrade case if they rely on children for setup.
Also, for a childChangedCallback we already have a working tool, MutationObserver. Why would we need an additional callback?
Let first state that the time at which the parser had finished running isn't a great way to do anything because DOM is inherently dynamic. Any node can be inserted or removed after the parser had finished parsing its contents, not to mention that HTML parser could later insert more children via adoption agency, etc...
The reason we want to add childChangedCallback is ergonomics for the same reason we have attributeChangedCallback even though attribute changes can be observed via MutationObserver as well.
Now, custom elements have unique characteristics that child elements may be dynamically upgraded later so just knowing when a child is inserted may not be enough for a parent element to start interacting with a child element. This is a rather fundamental problem with the current custom elements API. We need some way for a parent to discover a child custom element when it becomes well defined.
@rniwa how is whenDefined() not adequate for that?
Any node can be inserted or removed after the parser had finished parsing its contents
I'm not interested in that case. Imagine you want to have an element <table is="data-table"> and you need to set it up - how are you ever going to do this without having guaranteed access to all children? Many web projects still generate static HTML code on the serverside using classic template engines like Thymeleaf etc. This statically generated content inside a webcomponent needs to be reliably accessible for the webcomponent at some point - and this point should definitely be a lifecycle callback, not the funky stuff that frameworks like Google AMP do, which led to the creation of https://github.com/WebReflection/html-parsed-element
In the course of the discussion back in the days I created a gist that sums up the problem and suggests a solution that served as a basis for the development of the aforementioned package.
https://gist.github.com/franktopel/5d760330a936e32644660774ccba58a7
As already explained in multiple comments at the beginning of #551 that's not a pattern I want to encourage by adding API surface for it as it'll break down whenever someone dynamically uses such an element. I'm not sure why @rniwa mentioned the parser again. If we really want to reopen that discussion it should be in a separate issue as it's unlikely to get agreement, whereas children changed is an existing low-level primitive that would be vastly easier to get agreement on exposing.
There appears to be a strong desire for having some callback when the parser finishes parsing children
This is exactly what this issue addresses, unfortunately it mixes it up with
or when a new child is inserted or an old child is removed.
which is a completely different problem. I really think there is a strong necessity to solve the first problem first, because the second problem can already be tackled with using a MutationObserver. While that may not be as convenient as having a childrenChangedCallback, it definitely works and does so without going ten extra miles just to find the right time via deduction the type of which
if ([this, ...this.parentNodes].some(el=> el.nextSibling) || document.readyState !== 'loading') {
this.childrenAvailableCallback();
} else {
this.mutationObserver = new MutationObserver(() => {
if ([this, ...this.parentNodes].some(el=> el.nextSibling) || document.readyState !== 'loading') {
this.childrenAvailableCallback()
this.mutationObserver.disconnect()
}
}
this.mutationObserver.observe(this, {childList: true});
}does.
Please note that this is also explicitly mentioned in the entry post:
[...] and people are resorting to very awkward workarounds like attaching mutation observers everywhere
which is a completely different problem.
I don't think parser finishing and dynamic children are different problems at all. The parser adding nodes is just a sub-problem of dynamic child changes and a single API can handle both cases.
The fact that you can use MutationObservers after parsing is complete doesn't solve the current issue that developers have to piece together several APIs in order to know when their children change.
I'm not interested in that case. Imagine you want to have an element table is="data-table" and you need to set it up - how are you ever going to do this without having guaranteed access to all children?
Components that don't support updating its appearance upon dynamic DOM tree change is outside the scope of custom elements API. We don't intend to address such a use case.
To answer @annevk's comment about why I brought up the parser again: I think there is legitimate scenarios in which the most natural solution authors think of is finishedParsingChildrenCallback. As far as I dissected the problem space, I think the most common complain there is that out-of-order upgrades makes it impossible to know when a parent can start interacting with children. I think this is an unique requirement that built-in elements don't have. So while I tend to agree we don't want to add that exact callback, we may need something like childDefinedCallback as I suggested above to address the underlying use cases of issues which motivated folks to request finished parsing callback.
Components that don't support updating its appearance upon dynamic DOM tree change is outside the scope of custom elements API.
Can you please give reference on this? I assume such a heavy-weight decision is well-reasoned and -documented. What exactly is in scope of the custom elements API?
@rniwa it seems to me that the combination of childrenChangedCallback, customElements.whenDefined(), plus some bookkeeping is all you need for that. Though perhaps if the bookkeeping gets too complicated there is some kind of shortcut we could offer if libraries all end up with something similar. (Seems like something user land should figure out the pattern first for though.) Also, @domenic has a point though that whatwg/dom#305 perhaps should be flushed out first given that we're likely not able to get rid of mutation events. Nevertheless, iIt might still make sense to offer a childrenChangedCallback given its similar timing and more idiomatic custom element API.
@franktopel it was decided early on that we wanted to encourage custom elements that behave equivalently to built-in elements and would therefore not add hooks that built-in elements do not have. It might be a little tricky to find a definitive reference for that, but that's best discussed separately from this thread in a new issue.
@rniwa it seems to me that the combination of childrenChangedCallback, customElements.whenDefined(), plus some bookkeeping is all you need for that. Though perhaps if the bookkeeping gets too complicated there is some kind of shortcut we could offer if libraries all end up with something similar. (Seems like something user land should figure out the pattern first for though.)
To give some color about what I've been doing for this situation: I generally fire events from children when they're ready to interact with an ancestor, and have the child wait for the parent to be ready to receive the event with customElements.whenDefined(). This seems to be simpler code usually and avoids tightly coupling to the exact structure of the DOM (works with grandchildren as well).
The fact that you can use MutationObservers after parsing is complete
I think current work around is using MutationObserver on constructor, hence before the parsing is complete.
I generally fire events from children when they're ready to interact with an ancestor
That works only if children are Custom Elements.
A <sortable-list> might be a container for an UL and some LI, and knowing when the content is ready is crucial when it comes to setup.
Imagine the server producing the following based on some data:
<sortable-list>
<ul>
<li>a</li>
<li>b</li>
<ul>
</sortable-list>If you define upfront the sortable-list component, and you don't want to use ShadowDOM, which shouldn't be mandatory to setup Custom Elements, the only way to do that is to use a hack via html-parsed-element or similar, or to use MutationObserver within the constructor so that the component will inevitably flick due asynchronous nature of the MO.
I think @rniwa here nailed the issue me, and others, are complaining about.
I think there is legitimate scenarios in which the most natural solution authors think of is finishedParsingChildrenCallback.
Anything else would result in probably nice to have, but not an answer to the real issue.
The fact built-in elements don't have such lifecycle is misleading, 'cause a <select> internally setup likely once, instead of N times synchronously per each discovered <option> with it.
We, users, simply don't get to know when that setup should happen, which is independent of the usage of MO in case the component should be able to react on added/removed nodes, 'cause these are not mutually exclusive scenarios.
Did you test <select>? As far as I know that's not true. (Not even clear to me how that would work if you dynamically append to it.)
Did you test
<select>? As far as I know that's not true
OK then, let's talk about other builtins where some of those ship with Shadow DOM included and get rendered only once they are fully known (input, progress, etc)
I've never seen an input flicking while setting up its own SD or styles, and same happens if you use Shadow DOM in the constructor.
Although, does Shadow DOM need to be mandatory to have useful custom elements?
And how/when should we setup its internal events/behaviors?
In these cases a "parsed" callback would help. Maybe "parsed" is not the right term, but it's semantic with what we'd like to have.
P.S. I think GitHub broke its own comment parsing, which is why my previous message looks like that 😢
In those cases there's nothing to parse right? The element implementation has all the necessary bits already available? Your custom element input can also create a whole bunch of div elements with styles synchronously and inject them in a shadow tree at creation time.
so you are saying Shadow DOM should be mandatory to reliably setup elements without flicks, right?
otherwise, can you please explain to me how to setup this, generated from the server, and pre-registered as component on the client, without using Shadow DOM? Thanks
<sortable-list>
<ul>
<li>a</li>
<li>b</li>
<ul>
</sortable-list>I don't see how that case is equivalent to <input>, which has no (same node tree) children.
I'm a little worried we're dragging this too far off-topic, but could you elaborate on what kind of flickering you are encountering? Is there a demo somewhere? Mutation observer callbacks should in theory always happen before rendering. You might potentially see more elements get added on a per frame basis, but it's not clear to me how that is different from incremental rendering of built-in elements.
Mutation observer callbacks should in theory always happen before rendering.
I didn't know this, meaning the following pattern should grant the element has been parsed, right?
constructor() {
super();
new MutationObserver((, mo) => {
mo.disconnect();
this.finishedParsingChildrenCallback();
}).observe(this, {childList: true});
}if that's the case, I might update my html-parsed-element library then, yet @rniwa concern on abusing MO all over will still be valid, and I wish myself there was a finishedParsingChildrenCallback or similar method for CE instantiation.
edit also please note maybe a CE doesn't have children so that such method might never trigger, leaving the element partially initialized.
Another reason for desiring a reliable entry point that works no matter the circumstances, is the following one: a basic star rate component.
// <star-rate value=1-5>
customElements.define('star-rate', class extends HTMLElement {
attributeChangedCallback(name, old, value) { this[name] = value; }
get observedAttributes() { return ['value']; }
get value() {
return this.getAttribute('value');
}
set value(value) {
let input = this.querySelector(`input[value="${value}"]`);
let changed = false;
if (input && !input.checked)
input.checked = (changed = true);
else {
input = this.querySelector(`input:checked`);
if (input && input.checked)
input.checked = !(changed = true);
}
if (changed)
input.dispatchEvent(new CustomEvent('change', {bubbles: true}));
}
connectedCallback() {
this.addEventListener('change', this);
// ⚠ the following is unpredictable
// it might work or not
const {value} = this;
if (value)
this.value = value;
}
handleEvent(event) { this[`on${event.type}`](event); }
onchange(event) {
const {target} = event;
this.classList.toggle('checked', target.checked);
}
});This basic custom element would like to simply expose its own input value, and it can be represented on the HTML as such (so it's SSR compatible out of the box)
<star-rate value="3">
<input type="radio" name="star-rate" value="1">
<input type="radio" name="star-rate" value="2">
<input type="radio" name="star-rate" value="3">
<input type="radio" name="star-rate" value="4">
<input type="radio" name="star-rate" value="5">
</star-rate>Star rating is one of the most basic and common Web components out there, and while every framework/library would easily provide a way to do exactly what above custom element definition does, the standard way will likely fail because it's unpredictable.
For instance, the CodePen version will always work as expected, because the script is executed after the layout has been generated.
However, defining the custom element upfront will likely fail in Chrome, will probably fail in Firefox, and won't likely set it up in Safari.
Try this page to find out https://webreflection.github.io/test/star-rate/ and refresh on Firefox to see it works sometimes, and not some other times.
This is awkward (to put it gently)
The fact developers don't have an entry point to setup their own components in a reliable way across browsers is one of the reason people still don't go custom elements.
Every alternative for a basic star-rate component would be easier and more reliable, with Custom Elements we always have this chicken/egg issue if the definition is known upfront or not, and if the DOM is already live or not.
The strongest point of Custom Elements without Shadow DOM is SSR capability and graceful enhancement, and this is also the weakest feature in practice.
V0 used to have a createdCallback that was indicating the whole upgrade was done, and IIRC it was OK in that case to check for internal children state, but there's nothing in V1 that helps developers avoid shooting their own foot through the platform.
The ideal scenario
All it would take to actually solve every issue, is to be notified when the closing tag of a DOM node has been encountered (even if automatically closed, or if it's a void element)
I understand the platform never offered that, but it's also clear to me by now it should.
Until we have that information available, frameworks will always win in reliability too, because their declarations land at once, and whenever the component is initialized (mounted/connected/whatever) internal nodes would be already available to perform any sort of action.
I hope this extra example would clarify more why we need some mechanism to setup components.
Best Regards
I didn't know this, meaning the following pattern should grant the element has been parsed, right?
I'll test this later but I doubt it, mutation observer callbacks happen on the microtask queue so I'd imagine if there was flaky internet it could add a couple children, pause parsing to wait for more data and wind up running mutation observer callback microtasks before all children are parsed.
As far as I'm aware the only reliable way to detect end of parsing of a specific element is this thing from earlier in the thread.
@WebReflection your mutation observer hack… can't that still fire before the element is fully parsed? If 50% of the element had been parsed, and rendered, I'd expect a mutation observer callback before render.
Just wanted to add that I want keep wanting the "parsing done" callback, especially for progressive enhancement.
<my-custom-element>
…some acceptable pre-JS markup that's eventually enhanced…
</my-custom-element>In cases like this, I want to run JS when parsing is complete, then look at the contents, and use it to initialise the interactive element (perhaps move everything into shadow dom). Edits to element content are ignored after parsing.
I find myself having to do:
<my-custom-element>
…some acceptable pre-JS markup that's eventually enhanced…
</my-custom-element>
<script>
{
const el = document.currentScript.previousElementSibling;
customElements.whenDefined('my-custom-element')
.then(() => el.enhance());
}
</script>In that case I think we should also seriously consider un-deprecating
DOMNodeRemoved/DOMNodeInsertedas an API shape, since that already exists in some browsers and we've needed to spec it properly for some time.
Alternatively, we could have a { sync: true } option on mutation observers, which would fire immediately rather than on microtask. Then things like childList remain opt-in.
@jakearchibald I don't understand this snippet:
<my-custom-element>
…some acceptable pre-JS markup that's eventually enhanced…
</my-custom-element>
<script>
{
const el = document.currentScript.previousElementSibling;
customElements.whenDefined('my-custom-element')
.then(() => el.enhance());
}
</script>Why wouldn't .enhance() be called in my-custom-element's constructor?
Dur: sorry. I see, the parser yielding in the middle, the whole point of this thread. I'll leave this up anyway...
Edits to element content are ignored after parsing.
Much of this thread and previous ones is a discussion about how custom elements which have this sort of behavior are discouraged, and so adding an API to make them easier is not desirable for some of us.
I imagine you might be aware of that, but in case others are coming in via Twitter or similar, I thought it'd be good to re-establish that context.
Much of this thread and previous ones is a discussion about how custom elements which have this sort of behavior are discouraged, and so adding an API to make them easier is not desirable for some of us.
I imagine you might be aware of that, but in case others are coming in via Twitter or similar, I thought it'd be good to re-establish that context.
@domenic Thanks for thinking of people like me which landed on this thread without any context.
Can you please share the rationale behind this decision (to discourage such behaviour) or a link to it?
I was going to add my 2 cents, but perhaps knowing the history, will help answer some (of my) questions and avoid the conversation repeating itself. Thanks.
@dogoku the TL;DR of current state of the art is that given the following JS:
// your custom element
customElements.define('my-select', class extends HTMLElement {
finishedParsingChildrenCallback() {
console.log('ready to go');
}
});
// the currently needed utility
customElements.define('the-end', class extends HTMLElement {
connectedCallback() {
const {parentElement} = this;
parentElement.removeChild(this);
if (parentElement.finishedParsingChildrenCallback)
parentElement.finishedParsingChildrenCallback();
else
parentElement.dispatchEvent(new CustomEvent('finishedparsingchildren'));
}
});and the following layout:
<my-select>
<select>
<option>1</option>
<option>2</option>
<option>3</option>
</select>
<the-end/>
</my-select>you would know when your custom element, the one containing the select, is ready to setup itself also based on its content, as you can see in this code pen.
However, not only this awkward "hack" is needed because Custom Elements are missing a callback that tells the component it's OK to set itself up, there's always a chicken-egg issue about when a Custom Element is defined:
- is it already live?
- is it predefined and it could land at any time and be reliable in its setup?
The latter case is usually covered by the common situation where you land all custom elements and their content at once, at least via JS, but not necessarily via SSR.
The first point though is problematic, 'cause as soon as you have the definition order of the the-end component after a component that needs it, nothing works as expected.
As example, this other code pen simply defines the the-end component before others, and you won't see anything in the console, hence no setup granted.
The proposed feature here, is to inform a generic Custom Elements when such "end" of its content is reached, so that everyone can have a single entry point to setup components that depends on their content, same as native select, ul, table or other more complex HTML elements do.
There's a missing primitive that requires a lot of overhead per each component because of such missing information/callback/event, and every framework not based on custom elements has an easier life in setting itself up, 'casue the content is already accessible, which is also not always the case for Custom Elements.
As example, if you have a definition in your document head and the element is initialized while its opening tag is encountered, you need a lot of "dancing" to figure out when it's the right time to set anything up.
This is somehow a non-issue when your component uses Shadow DOM, as that's usually done in the constructor, but it becomes a foot gun prone approach when you'd like to have Server Side Render friendly components, and you don't control when these are defined in any hosting site.
I hope this summary helped a little to go back on track with the issue, and any possible solution.
@WebReflection your star element example from earlier in the thread, already gave me a clear understanding of your point and honestly it's something my team has been struggling with as well (ironically, a custom select element was one of the pain points), often resorting to ugly setTimeout hacks.
What I was trying to learn is the counter argument to this, which might not be immediately obvious if it comes down to performance or some other browser internals. By going through #550 as suggested by Domenic, I found this comment to be the most informative:
As I've numerously stated in various threads & issues, the right approach is for the child to report its insertion to the parent. Waiting for all children to be parsed in order to initialize is never a good idiom because the HTML parser may pause for seconds at a time when there is a network delay.
We used to have code like that in our builtin element implementations and we still do in some cases (due to historical reasons) but almost all such code end up causing bugs. It's simply not the right pattern by which to write an element.
Originally posted by @rniwa in #550 (comment)
(P.S: I also wanted to avoid adding noise, which seems I have failed in doing 😅)
@dogoku well, yes, if there was at least a contentChangedCallback that would automatically enable access to the Custom Element content (i.e. this.innerHTML or this.textContent, or this.querySelector(...)) we'll have an easier life, as that can help with unexpected flushes of the page and it will expose a precise point in time where you can at least start setting up your component.
After all, even <select> shows their content before their end tag is encountered:
<?php
header('Content-type: text/html; charset=utf-8');
echo '<select>';
for ($i = 0; $i < 10; $i++) {
echo '<option>'.$i.'</option>';
if ($i === 5) {
flush();
ob_flush();
sleep(5);
// explore the select up to 5 options
}
}
echo '</select>';
?>So while I personally dream about a contentParsedCallback that would happen independently of the content, as long as the closing tag is either explicitly encountered or automatically closed, anything else that could help setting up reliably a Custom Element would be better than current state.
edit the reason I believe the closing tag callback would be more efficient, is that a custom element could have visibility: hidden up to that point, so that a user prematurely interacting with an element that exposes some default down the stream won't fail when choosing an option, es example, before such stream ends.
<?php
header('Content-type: text/html; charset=utf-8');
echo '<select>';
for ($i = 0; $i < 10; $i++) {
echo '<option'.($i === 7 ? ' selected' : '').'>'.$i.'</option>';
if ($i === 5) {
flush();
ob_flush();
sleep(5);
// you have 5 seconds to change option
}
}
echo '</select>';
// see the user previous choice replaced by the 7th entry after 5 seconds
?>
Are there any active plans to fix this?
Currently, I have to define part of my elements inside DOMContentLoaded. That's quite incorrect and don't handles all possible scenarios, but works in my case.
window.addEventListener('DOMContentLoaded', () => {
window.customElements.define('my-elem', MyElem);
});
Someone suggested requestAnimationFrame to callback when parser finished parsing children.
connectedCallback() {
window.requestAnimationFrame(()=>{
this.querySelectorAll("input") // or whatever chlidren-specific operation
})
}Using this method, you will always have all child nodes available.
is this a good solution? seems straightforward and ultra-simple.
Source:
#551 (comment)
No. Browsers can render documents during parsing, and rendering includes calling requestAnimationFrame callbacks.
Here's an attempt at a concrete proposal:
Define an event called finishedparsingchildren, whose behavior is:
When a Node
nodeis popped off the stack of open elements of an HTML parser or XML parser, queue an element task on the DOM manipulation task source given thenodeto fire an event named finishedparsingchildren atnode.
I believe that should be relatively straightforward to implement, and should address the use cases discussed in this issue. A few comments/variations:
- The
finishedparsingchildrenevent should not bubble, nor should it be cancelable. - Instead of queueing the event, it could be run synchronously, or at a different timing.
- Perhaps the behavior could change depending on whether the parser was created as part of the HTML fragment parsing algorithm. Either don't fire at all for fragment parsing, or fire with a different timing?
Thoughts?
Why an event instead of a custom element reaction callback? What use cases does it serve besides custom elements?
We have bypassed this over the years in multiple ways.
I plea for Web Component gurus to work on real problems.
https://jsfiddle.net/WebComponents/ar7jengd/
<template id="MY-ELEMENT">
<img src="" onerror='this.getRootNode().host.parsed("img onerror")'>
<style onload='this.getRootNode().host.parsed("style onload")'></style>
</template>
<my-element>
<span></span>
<b></b>
</my-element>
<script>
customElements.define('my-element', class extends HTMLElement {
constructor() {
const templateById = (id) => document.getElementById(id);
const clone = (id) => templateById(id).content.cloneNode(true);
super().attachShadow({mode:"open"}).append(clone(this.nodeName));
}
connectedCallback() {
setTimeout(() => this.parsed("timeout"))
requestAnimationFrame(() => this.parsed("rAF"));
console.log("connectedCallback");
}
parsed(ref){
console.log("parsed() called by", ref, this.children)
}
});
</script>We have bypassed this over the years in multiple ways.
I plea for Web Component gurus to work on real problems.
If a not-so-uncommon use-case requires one of a multitude of timing-based workarounds, what could prove better there's a real issue here?
I am not denying the issue, but the last real contribution in this thread was september 2019
If this really was a Big issue, it would have been fixed.
There's a number of surprisingly-never-fixed issues. Take the example of asking an element whether its text is causing overflow. A simple thing that simply doesn't have an API to ask for, and has been a common cause of headache in just about any (application) framework.
Ok what about the second simple straightforward solution?
constructor() {
setTimeout(() => console.log(this.children.length), 0)
// prints correctly after parsing
}why not?
and if you find the name confusing then you just extract that into a named function for the sake of clarity
Why an event instead of a custom element reaction callback? What use cases does it serve besides custom elements?
Yeah, I'd be ok with a custom element reaction callback also. It'd be similarly easy to implement. I was just going for a general purpose solution, since it seemed easily possible to build one. But limiting to custom elements would seem to still meet the use case.
Ok what about the second simple straightforward solution?
For this and all of the similar questions, the answer is because they all fail in this case:
<my-element>
...Lots of content here...
{... the HTML parser yields here, renders the document, and spins the event loop.
All workarounds above fire here.}
...More content here, which are still children of <my-element>
</my-element>
@mfreed7 so the only correct solution is for the child nodes to report that they have connected to the parent element?
Possible solutions including:
- a Promise object set by the parent and resolve by one or more childs,
- Custom Events fired by childs and caught by the parent,
- a MutationObserver object set to observe changes in the parent's child list...
Source from a 2016 StackOverflow that seems to be more relevant than anything else mentioned in this thread and thus perhaps the most correct approach.
@Sleepful That's talking about custom elements spec version 0, though.
it still works tho, no? pragmatic and offers alternatives to the imo convoluted Mutation Observer.
It is also in the same vein of:
The parser adding nodes is just a sub-problem of dynamic child changes and a single API can handle both cases.
Ok what about the second simple straightforward solution?
For this and all of the similar questions, the answer is because they all fail in this case:
<my-element> ...Lots of content here... {... the HTML parser yields here, renders the document, and spins the event loop. All workarounds above fire here.} ...More content here, which are still children of <my-element> </my-element>
On the other hand. maybe whatever code which will run once all child nodes are parsed should be running whenever parser yields like that as well. Maybe we can make this callback be called in those cases with a boolean argument indicating the state:
class MyElement extends HTMLElement {
...
childrenParsed(bool didFinish) { // didFinish will be false when this is getting called due to parser yields.
}
...
}the one issue with having the child nodes "report" to the parent node is that this only works for child-nodes that are custom-elements and it does not work so well for child-nodes that are simple tags
After playing around a little bit, the mutation observer really seems like the best option for the component to be aware of its children.
<script>
window.customElements.define(
'elmt-container',
class extends HTMLElement {
observeMutations() {
const callback = (mutationList, observer) => {
for (const mutation of mutationList) {
if (mutation.type === 'childList') {
if (mutation.addedNodes) {
if (mutation.addedNodes[0] instanceof HTMLElement) {
console.log('A child node has been added:')
console.log(mutation.addedNodes[0])
}
}
}
}
}
const observer = new MutationObserver(callback)
this.observer = observer
const config = { childList: true, subtree: true }
observer.observe(this, config)
}
disconnectedCallback() {
const observer = this.observer
observer.disconnect()
}
constructor() {
super()
this.observeMutations()
}
}
)
window.customElements.define('elmt-test', class extends HTMLElement {})
</script>
<elmt-container>
<div>1</div>
<elmt-test>2</elmt-test>
<div><span>3</span></div>
</elmt-container>The funny thing is that this still doesn't signal when the parser "finished first pass" or whatever. So instead the CustomElement has to carefully count/wait/expect the child nodes that it depends on.
MutationObserver seems kind of awkward but to be honest it works for me 🤷
An advantage is that there is no leakage of browser-parsing details, the browser might not differentiate between first parser pass or dynamic node insertion, if it sees both as the same thing then the parser finished idea seems implausible. MutationObserver also doesn't care about the parser yielded behavior at all, it is a non-issue.
Funny thing with the observer though: sometimes the parser already loaded this.children so they are not new mutations that will go through the observer.
So basically what @matthewp already mention in this other thread #551 (comment)
An attempt at a concrete API for a Promise could be one in elementinternals:
class MyElement extends HTMLElement {
#internals = this.attachInternals()
async connectedCallback() {
await this.#internals.fullyParsed();
}
}So I wonder who is against this "parser notification" behavior and for what reasons, if anyone has any, that would be interesting.
So I wonder who is against this "parser notification" behavior and for what reasons, if anyone has any, that would be interesting.
This is covered earlier in the thread. #809 (comment) and #809 (comment) are good starting points.
Not to add too much noise here, but if the consensus is that this cannot be parser specific then I just want to point out hydration as a motivating use case (which I don't see in this thread?)
I think there is a lot of value in being able to write "when this component is fully parsed, hydrate it". That inherently needs to be a parser feature, since hydration would be unrelated to any mutations applied to the element mid-parse. This is especially important for streaming use cases and is technically necessary anytime you hydrate a component defined in an async script tag, since it may execute while the document is mid-parse of a hydratable element.
I just wanted to point out hydration as one use case which would really benefit from a parser notification here and I don't think any non-parser approach like MutationOberserver meets that need.
@dgp1130 I'm not sure that's a valid point. Say there was a multi-second network hiccup in the middle of a large custom element, why shouldn't it be rendered including all the children it has thus far, similar to built-in elements? I think "children changed", possibly rate-limited with "request animation frame" depending on what you need to be doing, is still the right approach there.
Btw, in case of shadow root, we have slotchange event that's fired once as soon as slot was rendered and inserted
https://dev.to/coryrylan/understanding-slot-updates-with-web-components-oah
I think there's multiple valid implementations of hydrating a component. One might be as @annevk suggests which allows individual elements inside a component to hydrate or be loaded independently.
However I think it's equally valid to consider an entire component as a "hydration unit" in which hydration happens for the whole thing at once. Take for example a simple counter (button and label which gets incremented each click). You could implement this via @annevk's approach and watch for DOM mutations, however this is 1) excessively complicated IMHO and 2) kinda pointless. It's completely unnecessary complexity given that neither the button nor the label are useful to hydrate independently. Taking the effort to observe them individually still requires waiting on both to be loaded anyways. Not only that, but a child could be half-parsed. A MutationObserver could trigger a callback part way through parsing the label for example and hydration might start prematurely, before it's full text is loaded (happy to include a more concrete example here, but on mobile ATM).
While some very large components might benefit from making individual elements independently hydratable, I think it is a very useful and common pattern for developer to treat the whole component as a hydration unit, and it is difficult to support that model without a parser-integrated hook to know when all it's input HTML has been fully applied to the DOM.
class MyElement extends HTMLElement { childrenParsed(bool didFinish) { // didFinish will be false when this is getting called due to parser yields. }
At first I was thinking this is a nice addition, but the case where childrenParser(false) is identically the same as just calling requestAnimationFrame(), right? So I'm not sure what you gain. With something like finishedParsingChildren(), it would be guaranteed to only be called once, and only after all children are done being parsed, which is the thing that isn't easily possible today.
An attempt at a concrete API for a Promise could be one in
elementinternals:class MyElement extends HTMLElement { #internals = this.attachInternals() async connectedCallback() { await this.#internals.fullyParsed(); } }
I like this approach also, perhaps better than the callback-that-gets-called-once approach! It achieves the same goal, and it's more clear that this promise just gets resolved once.
@dgp1130 I'm not sure that's a valid point. Say there was a multi-second network hiccup in the middle of a large custom element, why shouldn't it be rendered including all the children it has thus far, similar to built-in elements? I think "children changed", possibly rate-limited with "request animation frame" depending on what you need to be doing, is still the right approach there.
You might be correct that "rendering what you have" is the right thing to do. But it might also not be the case. As pointed out here:
Some layouts, such as flexbox, shift around as children parse-in, so it can be useful to eg hide elements during that phase.
Another example I can think of (raised by @rniwa here) is declarative shadow dom. It's similar to the issue raised above about hydration. You want to be able to wait until you have parsed all of your child content, including declarative shadow roots, before activating the component.
Generally it seems like there are some real use cases for knowing when parsing is finished for your element and children. People seem willing to go to considerable lengths (see comments above) to achieve this behavior, and in many cases they likely have bugs in their attempts to do this, because it's tough to do correctly or reason about given the parser's yielding behavior. @annevk I therefore don't buy the argument that the best thing is to discourage this common pattern, and force people to create hacks that don't work correctly in all cases.
Well, I think the problem is that an end tag notification doesn't cover all cases. If you cannot handle dynamic insertion and mutation of those insertions, how can you claim to be a proper element?
I think the problem is that an end tag notification doesn't cover all cases
it covers the only case that's not covered, and it shouldn't guarantee anything else but signaling the parser knows that node closed or the parser decided to close it (i.e. open body for sloppy cases).
It's been years we've discussed this (I've asked for this and provided libraries and workarounds) and meanwhile every library used to parse even streamed content is capable of signaling the end of a tag.
What happens internally with that tag is not responsibility of such end tag notification.
The most popular user-land parser has an onclosetag which helps a lot for SSR hydration cases and it's still not clear why the DOM itself cannot provide a way to understand the same: a </close> tag has been encountered (or a </close-element> for custom elements only, that'd be already something as parsedCallback) and I see the usual two three ways custom elements work that could both be useful in all cases:
- the element is being parsed (previously defined as CE, not fully parsed yet):
constructor,attributeChanged(if any observed and changed),connectedandparsedwhenever that happens - the element is already live and parsed and it's being upgraded due lazy CE definition:
constructor,attributeChanged(if any observed and changed),connectedandparsedright after - edit for history sake, the component is being created via its constructor: the
constructorcomes first ,theparsedCallbacktriggers after, allowing libraries or users to understand if the element isthis.isConnectedor not, providing every possible detail any logic needs to work for such case
The parsedCallback would basically be the init method that can't be the constructor itself due this dual nature of already known or declared at distance that Custom Elements are either famous or cursing developers for, making it the perfect place to eventually define a MutationObserver if internal changes are meant to be observed.
It would be a fix them all solution developers need to avoid doing harakiri themselves with intervals, MO that don't really guarantee anything, timers, and all other possible error-prone approaches that still don't guarantee anything ... we've been talking workarounds forever but still debating how to solve this.
Both @rniwa and @jakearchibald gave reasonable solutions ... I'd say pick the easiest to implement and see how that plays for real in the wild, then consider signaling that event to CSS ... where I'd honestly rather use a element:not-parsed instead of element:parsed, simply because element:parsed is what everyone writing CSS already assume is the correct default behavior, and I don't think we want everyone to write more verbose CSS to be sure these are applied only once the element is fully known ... we want escape cases instead for those not-super-common cases the element is huge and its end tag hasn't been reached yet (imho).
This way there is a primitive anyone can optionally hooks some logic into and it's provided by the parser that knows when a tag is closed before dealing with the next one ... but I start wondering if there are technical limitations / issues following this approach, as it seems so obvious (reading how parser libraries work since about ever).
I'd honestly rather use a element:not-parsed instead of element:parsed
I agree, though to avoid confusion with :not(), maybe :unparsed or something. Or would :parsing be more accurate anyway, since iiuc it would only mean it’s not fully parsed?
For my own edification, how would one determine an element is parsed reliably today? I suppose checking nextSibling for each mutation until done? Then I guess also DOMContentLoaded in case your element is the last in the tree? Something like:
function isParsed(el) {
if (el.nextSibling) return true
let node = el
while(node = node.parentNode) if (node.nextSibling) return true
}
const dcl = new Promise(resolve => document.addEventListener('domcontentloaded', resolve, { once: true }))
function whenParsed(el) {
if (document.readyState === 'complete' || isParsed(el)) return Promise.resolve()
return new Promise((resolve) => {
const observer = new MutationObserver(records => {
if(isParsed(el)) {
observer.disconnect()
resolve()
}
}).observe(el, { childList: true, subtree: true })
dcl.then(() => {
observer.disconnect()
resolve()
})
})
}@bathos yes, please don't take my naming suggestions as actual suggestions, it's the concept that matters, I don't have strong opinions for names except it should be not ambiguous, so both your hints would be more than welcome to me.
@keithamus like I've said, this was a workaround from 5+ years ago, when I've already raised the concerns around V1 dropping the createdCallback https://github.com/WebReflection/html-parsed-element#readme
As a matter of fact, it uses indeed a combination of everything you mentioned already, see the source code but it's still a hell of a hack or workaround nobody should or would need if the platform would provide the same without needing any of those indirection as it perfectly knows exactly when a tag is closed or forced closed and the next one in the pipe can go (or is already available).
The current status is that if more than a custom in-house solution lands on pages we have slower pages for everyone and nobody wins + all workarounds are heavier and slower by design but also error prone.
@keithamus there's a logical issue in your example code (I wish it was that simple) ... you are using the dcl.then at the end to resolve but the whole issue here is that nobody wants to wait for DCL if a document is huge and "islands" components / custom elements already landed and are available ... you can see the dance in my workaround is more convoluted to exactly fix that issue what nobody wants to wait DCL on huge streamed documents, all they want is to trigger ASAP the upgrade/update instead in a safe, node fully parsed, way ... I hope this is clear.
P.S. moreover, that dcl would never resolve if the DCL event already triggered, another use case for the defunct document.ready proposal (still @jakearchibald IIRC) ... these are all the footguns such a simple parsed callback would help solving.
@keithamus also worth mentioning that if the DOM would expose a generic Element.prototype.whenParsed().then(callback) API all our issues could be solved in every possible user-land library or use case.
That method would indeed be:
- easy to use on any Custom Element constructor
- not confine the feature to Custom Elements only, as these are already problematic due browsers diversions around the builtin extend topic
- provide a primitive for every non custom elements based library out there (majority of the current market share)
So, if your idea was a proposal for an API, I'd be plus one on that too!
@WebReflection you mention the two parser cases, but you fail to mention the other way an element has to work: through imperative creation and insertion. And for parity with builtin elements that ought to work in whichever way web developers want to do things. It shouldn't impose inserting children before insertion into the document, for instance. Just think about all the ways you can end up with a <select> element and descendants in the tree. Whichever way you go about it, it'll work. That's the bar.
(Also, you posted 4 comments in a 2-hour period. Please amend existing comments instead. There's a lot of people watching this repository. Be mindful of their time and attention.)
@annevk I can be mindful but editing doesn't trigger emails notifications so in this 5+ years discussion I wanted to be sure all the thoughts are well understood ... but I've said everything needed to say, except I am answering your question too in here.
imperative creation and insertion
fair enough, but nothing new neither. the attributeChangedCallback fires before connectedCallback in non-imperative ways component are already part of the DOM, but can actually fire after if an imperative declaration already landed on the live document or not. This means there's no guarantee of callbacks order already in the current specification that has, indeed, 3 ways to consider custom elements, but it doesn't invalidate the parsedCallback idea:
- if imperative, that can trigger right after the constructor, no issues caused to the logic ... it's still the place to add a MutationObserver if needed/meant or the place
this.children.lengthcan be checked (orthis.isConnectedcheck) - in every other cases, it's inevitably after
connectedcallbackbecause aparsedCallbackcannot possibly happen before aconnectedCallbackas the parser is top-down
So yes, the imperative case triggers parsedCallback, if present, before attributeChanged or connected, because that's imperative, which is also the least known/used case out there, but the parsedcallback primitive would ensure the state of the component is well understood (even if imperative) while right now we need ugly dances around.
edit actually, parsedCallback with a this.isConnected === false also hints the component is being declared imperatively, something impossible to understand otherwise, so that'd be more power to users.
That means that parsedCallback is always fired after connectedCallback if the element is live on the DOM, no matter if lazily upgraded or parsed on the fly, while it triggers right after the constructor if the component is declared imperatively, to make the this.isConnected result always reliable with the actual component state, where if live it must be true, hence the connectedCallback is (imho?) logically expected to have already happened ... it's an init method and it must work in all conditions.
If the idea is instead an Element.prototype.whenParsed() solution, none of this concerns matter, as the promise will resolve a tick later, so that attributes changed, or connected callbacks, would eventually already be fired in the meantime.
Well, I think the problem is that an end tag notification doesn't cover all cases. If you cannot handle dynamic insertion and mutation of those insertions, how can you claim to be a proper element?
I agree - dynamic content needs to be handled as well. There are tools for that, e.g. MutationObserver. However, there's no tool to notify a component that parsing is complete, so that first render can avoid flash-of-partial-content problems. These are two different problems (dynamic content vs. initial render/hydration), and if components should be written to handle all of them, then they need a way to detect both of them. Otherwise, hacks will be used (again, see examples above) that will likely be broken in several cases.
I agree that something like <select> should be handled incrementally, although the CSS pseudo-class I mentioned may help with layout shifts.
However, there's also things like <script> and <style> that definitely wait until parsing is complete to perform actions.
@mfreed7 layout happens end-of-task and I believe any kind of tree mutations will have run their notifications by then. So I don't think this has to be a problem in practice. (I think there is a problem to solve ergonomics-wise.)
@jakearchibald I agree that there are some builtin elements that use the end tag notification, but it's not clear to me we should expose that. Certainly the kinds of components people bring up shouldn't require parity with them.
The end tag notification would definitely be useful, when users develop a web component hoping to parse children and define the initial state based of that, then, figure out they can't or have to pick a hack. I've been getting this complain about wc for years myself, glad to see the discussion is still active, hoping for a resolution!
@mfreed7 layout happens end-of-task and I believe any kind of tree mutations will have run their notifications by then. So I don't think this has to be a problem in practice. (I think there is a problem to solve ergonomics-wise.)
Sorry, help me understand this comment. Yes, layout and MutationObservers run when the parser yields, but that could happen before all children are parsed. That case isn't limited to elements with tons of children, it could just happen due to bad luck of placement on the page and the parser's decision making about when to yield.
And for parity with builtin elements that ought to work in whichever way web developers want to do things. It shouldn't impose inserting children before insertion into the document, for instance.
- builtin elements work great, but they aren't constrained by JavaScript API. So builtin elements are not informing us about a good API because we don't know their internals.
- Is that really a responsibility of the Web Components API or should it be a responsibility of the developer?
- Not all web components are going to be something in a public repository that should care about having the cleanest API. I suspect most people here want to use web components in projects which don't expect to share its web components, such as servers that generate HTML for their own purposes.
- Perhaps the funny thing here is that the web component has no idea what to expect in its own subtree, but the developer writing the HTML knows exactly what to expect. So there's this situation where the developer has to write a lot of code inside the web component for the web component to know when its subtree meets the expectations. For the developer the easiest thing would be to know when the subtree is made available by the parser to trigger the subtree-operations. Yes, this does imply that the developer writing the subtree is the same developer that is writing the web component. I think this usage of web components is rather common, is it a use case that should be neglected? Is the only priority
web componentswhose subtree is written by different developers?
@jakearchibald I agree that there are some builtin elements that use the end tag notification, but it's not clear to me we should expose that. Certainly the kinds of components people bring up shouldn't require parity with them.
This is a very good point. While <script> and <style> are arguably fairly "magic", other controls seem like they should provide a good model for web component versions. Several of those do have things that happen only when all children are parsed. Some examples:
I think <script> is an important use cases here, as there are security reasons why you want to make a custom element that behaves like it. I filed an issue for a use case for knowing that a tag is parser inserted, but you may also want to only run initial content and not respond to mutations, like <script>.
@jakearchibald I agree that there are some builtin elements that use the end tag notification, but it's not clear to me we should expose that. Certainly the kinds of components people bring up shouldn't require parity with them.
This is a very good point. While
<script>and<style>are arguably fairly "magic", other controls seem like they should provide a good model for web component versions. Several of those do have things that happen only when all children are parsed. Some examples:
If browsers ate their own dog food, i.e. created the native elements with the CustomElements API, we would see a lot of changes to make custom webcomponents usage much more practical. Unfortunately it would be much slower than the C++ implementation, so it's not going to happen. But in an ideal world, how browser create web components, should be the same way user do, that will ensure feature parity.
For anyone reading this thread looking for an alternative solution for built-in web components, I built a solution that may work for you. You can find it here: https://github.com/alexspirgel/children-loaded
It's a different implementation than the previously suggested html-parsed-element. I hope others find it helpful.
For anyone reading this thread looking for an alternative solution for built-in web components, I built a solution that may work for you. You can find it here: https://github.com/alexspirgel/children-loaded
Thanks for sharing. At a first glance, why on earth would anyone want to use require over native import in 2023?
Hi @franktopel, thanks for your feedback. However I don't want to sidetrack this thread with feedback for specific solutions that are not directly related to the thread topic. You can open an issue on the repository to start a conversation about require vs import (Please do I am genuinely interested about learning more on this subject). Thanks again for checking out my code.
@alexspirgel for what is worth it, I like your solution but I still think this burden shouldn't be on developers shoulders ... we're making components bootstrap heavier and slower for something browser internals know already ... I hope there will be a better option backed in the standard API, still I think your solution is cool 👍
@WebReflection thanks! I agree with you completely.
This was just discussed in the Web Components F2F, and the general consensus was that there should be a way to detect when the closing tag of an element is parsed.
One suggestion was made that might mitigate the concern that this will encourage authors to ignore subsequent changes to children: rather than a new event or callback, add a field to MutationRecord that can be used to detect whether an element's closing tag has been parsed. This would seem to meet the use case, and it would also happily not present any performance issues, since mutation records are delivered asynchronously and the timing is well defined.
As a strawperson proposal:
- MutationRecord.closingTagParsed - Read only boolean, which is
falseonly when the closing tag forMutationRecord.targethas not yet been parsed by the HTML parser. Will betruewhen the closing tag has already been parsed, or in the case that the element was imperatively added (e.g. viaappendNode()).
As stated above, WCCG had their spring F2F in which this was discussed. You can read the full notes of the discussion (#978 (comment)), heading entitled "Reaction Callbacks for Child Node Changes".
I just want to voice that a non-MutationObserver API will be far more ergonomic and developer-friendly, f.e. callbacks or events that are guaranteed to fire in the exact order the reactions happened.
MutationObserver callbacks do not always fire in the exact order of the element reactions they represent, making them extremely difficult to use.
Implementing callbacks for custom elements such as childConnectedCallback, childComposedCallback (when a child is composed to your custom element by being child of your element's ShadowRoot, or via distribution to a slot child of your element's ShadowRoot), etc, has proven to be very difficult because of that issue with MutationObserver.
If we can get an API with a simple intuitive ordering of events, that would be very valuable for developer productivity!
I believe that there are a group of people that try to make vanilla custom elements (or use libs like Lit which don't have childConnectedCallback, etc)), face these difficult issues, then become motivated to go use something else like Solid, React, Vue, Svelte, etc. We can change this story.
I just want to voice that a non-MutationObserver API will be far more ergonomic and developer-friendly, f.e. callbacks or events that are guaranteed to fire in the exact order the reactions happened.
MutationObserver callbacks do not always fire in the exact order of the element reactions they represent, making them extremely difficult to use.
Implementing callbacks for custom elements such as
childConnectedCallback,childComposedCallback(when a child is composed to your custom element by being child of your element's ShadowRoot, or via distribution to a slot child of your element's ShadowRoot), etc, has proven to be very difficult because of that issue with MutationObserver.If we can get an API with a simple intuitive ordering of events, that would be very valuable for developer productivity!
I believe that there are a group of people that try to make vanilla custom elements (or use libs like Lit which don't have childConnectedCallback, etc)), face these difficult issues, then become motivated to go use something else like Solid, React, Vue, Svelte, etc. We can change this story.
Very much agree. Amazon's concept of "eat your own dog food" here applies here. If browsers used native web components to implement thier own native components (e.g. div, span, ul), I'm sure we would have a much better API. Maybe they could have a C version of the JavaScript version that developers use.
Very much agree. Amazon's concept of "eat your own dog food" here applies here. If browsers used native web components to implement thier own native components
We do, for much of the browser UI, on multiple browsers. E.g. https://twitter.com/EisenbergEffect/status/1654516292169113601?s=20