brendankenny / vitals-flow

vitals-flow

A Lighthouse plugin and runner to experiment with Lighthouse user flows to simulate user interactions and measure Core Web Vitals in the lab.

This repo is for iterating on user flow ergonomics and the new responsiveness metric—so feel free to file issues or PRs—but in the long term, changes will need to land in Lighthouse core.

This is not an officially supported Google product.

Table of Contents

• Getting Started
• Basic usage
• Plugin with Lighthouse user flows
• Plugin with interaction runner
• Plugin with interaction runner and hacked perf section

Quick start

• git clone [e-mail]:brendankenny/vitals-flow.git
• cd vitals-flow
• yarn
• yarn demo

Basic usage

The lighthouse-plugin-web-vitals plugin adds FID and responsiveness audits so that all the Core Wev Vitals are available in a lab setting. When run, these are found in a "Lab Web Vitals" section in the Lighthouse report.

The plugin is compatible with any Lighthouse run, for instance on the command line:

yarn lighthouse https://example.com --plugins lighthouse-plugin-web-vitals --view

However, because there's no FID or responsiveness without user input and Lighthouse doesn't simulate input by default, these audits won't show any results.

Plugin with Lighthouse user flows

The plugin can be used in conjunction with Lighthouse user flows to capture metrics while user input is being simulated. To run in a Lighthouse timespan, add the plugin to the step's config overrides:

await flow.startTimespan({stepName: 'Interactions', configContext: {
  settingsOverrides: {
    // Bring in web-vitals plugin.
    plugins: ['lighthouse-plugin-web-vitals'],
    onlyCategories: [
      // To trim out the other categories.
      'lighthouse-plugin-web-vitals',
      'performance',
    ],
  },
}});

See a full example in examples/user-flow-report.js.

view live report

To run locally: node examples/user-flow-report.js

A few issues:

• it's difficult to set up throttling in this mode (timespan wasn't really meant for this)
• some metrics aren't available, e.g. LCP currently only works in the navigation mode since it is defined relative to a navigation, but we have to use timespan here

Plugin with interaction runner

This repo also includes an "interaction runner", which creates a custom config, gatherer, and audit, to allow using Lighthouse navigation mode but still yield to user code to interact with the page with puppeteer. Since this is a real Lighthouse navigation:

• all navigation metrics can run
• regular Lighthouse throttling schemes can be used
• Lighthouse criteria can be used to determine when page load is complete

import {createRunner} from './runner/interaction-runner.js';

async function captureReport() {
  const runner = await createRunner();
  await runner.startNavigation({url: 'https://www.khanacademy.org/'});

  /* Interact with the page using `runner.page` pptr endpoint. */

  await runner.endNavigation();
  await runner.saveReport({filepath: 'khan-sample.report.html', view: true});
}

captureReport();

See a full example in examples/interaction-runner-report.js.

view live report

To run locally: node examples/interaction-runner-report.js

Plugin with interaction runner and hacked perf section

With some knowledge of how the Lighthouse performance section is special-cased for rendering, the interaction runner can also make a web vitals output that replaces the normal Performance section with our lab-based "field" metrics. It requires only passing in an extra flag:

diff --git a/examples/interaction-runner-report.js b/examples/interaction-runner-report.js
index 1d29c71..ad25a8d 100644
--- a/examples/interaction-runner-report.js
+++ b/examples/interaction-runner-report.js
@@ -22,7 +22,7 @@ function sleep(time) {
 }
 
 async function captureReport() {
-  const runner = await createRunner();
+  const runner = await createRunner({useHackReport: true});
   await runner.startNavigation({url: 'https://www.khanacademy.org/'});
 
   try {

See a full example in examples/hack-runner-report.js.

view live report

To run locally: node examples/hack-runner-report.js

TODO

• import DevTools interaction recording (will require changes to their Puppeteer export)