How to build page resources dependency tree by initiator ?

[andrewsezko]

Hi, I am really excited about the puppeteer, you have done really great tool. I am using it from v0.11.0.
I am wondering if it possible to build the page resources dependency tree by their initiators.
For example, to visualize what amount of calls are sent by 3-d party tracking or analytics libs.
Such functionality has Lighthouse (critical request chains in performance audit results) but I need the whole picture.
It would be great if some of you guys could share your thoughts or advice how it could be done with the puppeteer.
As I understood, all information I need could be found in the trace of the page, but maybe there is a better solution than parsing that big and scary JSON file.

[Nathan Wingert]

Related thread / info on upstream blocker: puppeteer/puppeteer#1395

[Nathan Wingert]

@andrewsezko In case it helps, here's a quick snippet I wrote using pptr CDP to access devtools directly and get the initiator info for a similar requirement:

// https://github.com/ChromeDevTools/devtools-protocol/issues/56
// increase CallStackDepth to fix initiator stack issue in chrome
const client = await testPage.target().createCDPSession();
await client.send('Debugger.enable');
await client.send('Debugger.setAsyncCallStackDepth', { maxDepth: 32 });

// https://github.com/GoogleChrome/puppeteer/issues/1395    
// puppeteer request events don't propagate initiator currently
// so register for the event directly with CDP instead
await client.send('Network.enable');
await client.on('Network.requestWillBeSent', (params) => {
  const reqURL = params.request.url;
  logger.info(`request to: ${reqURL}`);
  logger.info(`from initiator: ${getInitiatorUrl(params.initiator)}`);
});

getInitiatorUrl is a little recursive function to search the initiator callstack for the info I need, not included because it's too specific to my requirements. You could look at the object dumps for initiator to write your own.

[Eric Bidelman]

Lighthouse does most of this work for you. I wonder if you could modify the critical request chain audit slightly to include everything rather than just critical requests?

[Nathan Wingert]

@ebidel I honestly haven't looked at Lighthouse very much as I navigated here from puppeteer/devtools searches. I'll take a look there as well, thanks for the suggestion.