When testing multiple URLs, the cache of previous URLs will not be obtained and will not interfere with each other.
Liu6625 opened this issue · comments
Your question
I have a question: I need to get the cookie in the pre-execution, and then test multiple URLs, but I found that the result of the first URL request I tested was complete, and the subsequent URLs in the request all obtained the cache of the previous URL, which reduced A lot of requests have been made, and I want the test results of each URL to not interfere with each other, and to obtain the cookies obtained by pre-execution. Is this okay? Looking forward for your reply, thank you
Hi @Liu6625 I think that should be doable. If you run something like this:
--preScript yourScript https://www.sitespeed.io https://www.sitespeed.io/documentation/ then your script will run before each URL that is tested with a new browser session.
Hi @Liu6625 I think that should be doable. If you run something like this:
--preScript yourScript https://www.sitespeed.io https://www.sitespeed.io/documentation/then your script will run before each URL that is tested with a new browser session.
Thank you for your reply. For some reasons, I am currently using the run method in the directory bin/browsertime.js in the source code to run it. I directly pass multiple URLs in the run method, but I found that the results are different depending on the order of the URLs. , including the number of requests, firstPaint, and TTFB. I don’t know if it’s caused by caching.
Can you show me how you run and the exact result and how it differs so I can reproduce it? Thanks!
// copy from bin/browsertime.js
import lodash from 'lodash';
import { resolve, relative } from 'path';
const { Engine, browserScripts} = require('browsertime');
import intel from 'intel';
import { existsSync, mkdirSync } from 'node:fs';
import { StorageManager } from '../util/storageManager.js';
const { get, set, merge } = lodash;
export function isAndroidConfigured(options: any) {
if (options.android && options.android.enabled === true) {
return true;
}
return get(
options,
'chrome.android.package',
get(options, 'firefox.android.package', false)
)
? true
: false;
}
const log = intel.getLogger('browsertime');
const delay = (ms: number) => new Promise(res => setTimeout(res, ms));
async function parseUserScripts(scripts: any) {
if (!Array.isArray(scripts)) scripts = [scripts];
const results = {};
for (const script of scripts) {
const code = await browserScripts.findAndParseScripts(resolve(script), 'custom');
merge(results, code);
}
return results;
}
async function preWarmServer(urls: any, options: any) {
const preWarmOptions = {
browser: options.browser,
iterations: 1,
xvfb: options.xvfb,
android: isAndroidConfigured(options),
docker: options.docker,
headless: options.headless
};
const chromeDevice = get(options, 'chrome.android.deviceSerial');
const firefoxDevice = get(options, 'firefox.android.deviceSerial');
const safariIos = get(options, 'safari.ios');
const safariDeviceName = get(options, 'safari.deviceName');
const safariDeviceUDID = get(options, 'safari.deviceUDID ');
if (isAndroidConfigured(options) && options.browser === 'chrome') {
set(preWarmOptions, 'chrome.android.package', 'com.android.chrome');
}
if (chromeDevice) {
set(preWarmOptions, 'chrome.android.deviceSerial', chromeDevice);
} else if (firefoxDevice) {
set(preWarmOptions, 'firefox.android.deviceSerial', firefoxDevice);
}
if (safariIos) {
set(preWarmOptions, 'safari.ios', true);
if (safariDeviceName) {
set(preWarmOptions, 'safari.deviceName', safariDeviceName);
}
if (safariDeviceUDID) {
set(preWarmOptions, 'safari.deviceUDID', safariDeviceUDID);
}
}
let engine = new Engine(preWarmOptions);
await engine.start();
log.info('Start pre-testing/warming');
await engine.runMultiple(urls, {});
await engine.stop();
log.info('Pre-testing done, closed the browser.');
return delay(options.preWarmServerWaitTime || 5000);
}
export async function run(urls: any, options: any) {
if (options.debug) {
log.info('Running Browsertime in debug mode.');
}
let result: any = null
try {
if (!options.resultDir) {
let dir = 'browsertime-results';
if (!existsSync(dir)) {
mkdirSync(dir);
}
}
log.info('Browsertime start');
let engine = new Engine(options);
const scriptCategories = await browserScripts.allScriptCategories;
let scriptsByCategory = await browserScripts.getScriptsForCategories(scriptCategories);
if (options.script) {
const userScripts = await parseUserScripts(options.script);
scriptsByCategory = merge(scriptsByCategory, userScripts);
}
try {
if (options.preWarmServer) {
await preWarmServer(urls, options);
}
await engine.start();
result = await engine.runMultiple(urls, scriptsByCategory);
let saveOperations = [];
// TODO setup by name
let firstUrl = urls[0];
// if the url is an array, it's of the form [filename, function]
if (Array.isArray(firstUrl)) {
firstUrl = firstUrl[0];
}
const storageManager = new StorageManager(firstUrl, options);
const harName = options.har ?? 'browsertime';
const jsonName = options.output ?? 'browsertime';
saveOperations.push(storageManager.writeJson(jsonName + '.json', result));
if (result.har) {
const useGzip = options.gzipHar === true;
saveOperations.push(
storageManager.writeJson(harName + '.har', result.har, useGzip)
);
}
if (options.enableProfileRun) {
log.info('Make one extra run to collect trace information');
options.iterations = 1;
if (options.browser === 'firefox') {
options.firefox.geckoProfiler = true;
} else if (options.browser === 'chrome') {
options.chrome.timeline = true;
options.cpu = true;
options.chrome.enableTraceScreenshots = true;
options.chrome.traceCategory = [
'disabled-by-default-v8.cpu_profiler'
];
}
options.video = false;
options.visualMetrics = false;
const traceEngine = new Engine(options);
await traceEngine.start();
await traceEngine.runMultiple(urls, scriptsByCategory);
await traceEngine.stop();
}
await Promise.all(saveOperations);
const resultDirectory = relative(process.cwd(), storageManager.directory);
// check for errors
for (let eachResult of result) {
for (let errors of eachResult.errors) {
if (errors.length > 0) {
process.exitCode = 1;
}
}
}
log.info(`Wrote data to ${resultDirectory}`);
} catch (error) {
log.error('err msg', error);
process.exitCode = 1;
} finally {
log.debug('Stopping Browsertime');
try {
await engine.stop();
log.debug('Stopped Browsertime');
log.info('Browsertime end');
} catch (error) {
log.info('Browsertime error end');
log.error('Error stopping Browsertime!', error);
process.exitCode = 1;
}
}
} catch (error) {
log.info('Browsertime error');
log.error('Error running browsertime', error);
process.exitCode = 1;
} finally {
return result
}
}
let options = {
iterations: 1,
headless: false,
browser: 'chrome',
screenshot: true,
cacheClearRaw: true,
};
// rough execution process
(async ()=>{
let res1 = await run([url1, url2], options);
// The results printed by res1 on the console
// url1 120 requests, TTFB: 913ms, firstPaint: 6.19s, FCP: 6.19s, DOMContentLoaded: 5.49s, LCP: 6.19s, CLS: 0.3457, TBT: 1.18s, Load: 11.83s
// url2 59 requests, TTFB: 342ms, firstPaint: 891ms, FCP: 891ms, DOMContentLoaded: 637ms, LCP: 2.03s, CLS: 0.1445, TBT: 365ms, Load: 2.12s
let res2 = await run([url2, url1], options);
// The results printed by res2 on the console
// url2 109 requests, TTFB: 609ms, firstPaint: 3.77s, FCP: 3.77s, DOMContentLoaded: 3.05s, LCP: 4.72s, CLS: 0.176, TBT: 730ms, Load: 8.05s
// url1 66 requests, TTFB: 404ms, firstPaint: 1.34s, FCP: 1.34s, DOMContentLoaded: 675ms, LCP: 1.34s, CLS: 0.3263, TBT: 606ms, Load: 1.21s
// I just changed their order, and the results printed by res1 and res2 are different.
// These two pages are different dependencies of the same website, but if there are some identical requests when entering for the first time,
// so I suspect there is a cache.
})I think when you run them like that using the "runMultiple" method you will use the same browser session? Multiple in this way means "run multiple URLs after each other in the same session". I think the easiest way to confirm that is if you look at the browser window when you run your test. Is the window closed between the URLS = new session.
I think when you run them like that using the "runMultiple" method you will use the same browser session? Multiple in this way means "run multiple URLs after each other in the same session". I think the easiest way to confirm that is if you look at the browser window when you run your test. Is the window closed between the URLS = new session.
Yes, as you said, the browser window is not closed when testing different URLs, but I still want to call it in this way. Is there a good way to solve it?
You are using the method where you run multiple tests in the same browser sessions. I think you could just change:
result = await engine.runMultiple(urls, scriptsByCategory);
to be
result = await engine.run(urls, scriptsByCategory);
You are using the method where you run multiple tests in the same browser sessions. I think you could just change:
result = await engine.runMultiple(urls, scriptsByCategory);to beresult = await engine.run(urls, scriptsByCategory);
I tried it, but there was a problem with the display. Passing a url would work normally, but passing an array of urls would result in an error. This was not in line with my expectations.
[ERROR] No page that could be associated with the error:urlOrAlias.startsWith is not a function
[ERROR] TypeError: urlOrAlias.startsWith is not a functionI don't know how to deal with it, I hope you can give me some advice, thank you
Then I would try:
const result1 = await engine.run(url1, scriptsByCategory);
const result2 = await engine.run(url2, scriptsByCategory);
Then I would try:
const result1 = await engine.run(url1, scriptsByCategory); const result2 = await engine.run(url2, scriptsByCategory);
thanks for your advice
Closing.