microsoft / playwright-python

Python version of the Playwright testing and automation library.

Home Page:https://playwright.dev/python/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[Bug]: Using playwright browser to measure the FCP & LCP caused inaccuracy

shaoerkuai opened this issue · comments

Version

1.41.0

Steps to reproduce

1.Using playwright to execute PerformanceObserver
2.Wait for asyncing promised performanceobserver like this

async () => {
    async function asyncPerformanceObserver() {
        return new Promise((resolve) => {
            const ob = new PerformanceObserver((list) => {
                ob.disconnect()
                const entries = list.getEntries();
                console.log("entries",entries)
                const lastEntry = entries[entries.length - 1];
                resolve(lastEntry.startTime)
            })
            ob.observe({type: "largest-contentful-paint", buffered: true});
        })
    }
  let lcp = await asyncPerformanceObserver()
  return {...}
}

I just found that the page.evaluate executing this script will only catch one largest-contentful-paint, is the loading gif of the page. It cannot indicate the page loading state since a small gif is useless..

But if I execute a PerformanceObserver in the Google Chrome, 2 LCPs will shown in the console.log ,and the last LCP element is the page background image render, which is meaningful.

It seems that the browser launched via Playwright cannot accurately capture PerformanceObserver, it picks a render element at the beginning and then sleept(not sure..), but I need capturing element that is loaded immediately after.
I tried to perform a synchronized PerformanceObserver observe after refreshing the page (using page.pause() and mannuly execute it in console), but it still couldn't capture the correct elements.

At this point, the LCP always equalling to FCP.

On my native Google Chrome, things seem to be working fine.

Expected behavior

Browsers launched through Google Chrome and Playwright have the same PerformanceObserver performance.

Actual behavior

        browser = playwright.chromium.launch(headless=False)
        context = browser.new_context(ignore_https_errors=True)
        context.tracing.start(screenshots=True, snapshots=True)
        page = context.new_page()
        page.set_viewport_size({"width": 1920, "height": 1080})
        page.goto(url,wait_until='networkidle')
        _performance = page.evaluate(performance_js)
        page.pause()

Additional context

No response

Environment

- Operating System: [Windows 10 Pro]
- CPU: [amd 64]
- Browser: [All, Chromium, Firefox, WebKit]
- Python Version: [3.8.6]
- Other info:

I used diconnect to shutdown the watcher,thats.

after remove that here is still a problem that some pages may still be unstable.

I took a little trick:

Implement the Promise timeout with the Promise Race, avoid page.evalute execution sticking, then check the current list of entities inside the Observer to make sure it matches the pattern of my page, and only when I think the entries[entries.length -1].startTime makes sense, it will be returned through resolve.

It doesn't seem like a big deal