Cannot read property 'url' of null

[FarhadMohseni]

when i want to use the proxy on page with :

if (proxy && proxy.ip && proxy.port) {
        const proxyString = `http://${proxy.ip}:${proxy.port}`;
        await useProxy(page, proxyString);
      }

i'm getting this error:

TypeError: Cannot read property 'url' of null
    at new CookieHandler (/home/nodejs/tiktok_captcha/node_modules/puppeteer-page-proxy/src/lib/cookies.js:67:83)
    at requestHandler (/home/nodejs/tiktok_captcha/node_modules/puppeteer-page-proxy/src/core/proxy.js:12:27)
    at $ppp_request_listener (/home/nodejs/tiktok_captcha/node_modules/puppeteer-page-proxy/src/core/proxy.js:82:15)
    at /home/nodejs/tiktok_captcha/node_modules/puppeteer/lib/cjs/vendor/mitt/src/index.js:47:62
    at Array.map (<anonymous>)
    at Object.emit (/home/nodejs/tiktok_captcha/node_modules/puppeteer/lib/cjs/vendor/mitt/src/index.js:47:43)
    at Page.emit (/home/nodejs/tiktok_captcha/node_modules/puppeteer/lib/cjs/puppeteer/common/EventEmitter.js:72:22)
    at /home/nodejs/tiktok_captcha/node_modules/puppeteer/lib/cjs/puppeteer/common/Page.js:165:100
    at /home/nodejs/tiktok_captcha/node_modules/puppeteer/lib/cjs/vendor/mitt/src/index.js:47:62
    at Array.map (<anonymous>)

puppeteer version : 5.2.1
using puppeteer-extra with the stealth plugin

[Adam]

I'm also facing this issue and a similar one and from what I can tell debugging, the issue seems to occur when a frame is embedded in the page which is "about:blank" or empty.

From what I know, things like AdWords creates dynamic frames which have no URL associated with them, this I assume calls Chrome to default to "about:blank". It seems to be related to #36

What seems to happen is that in requestHandler of proxy.js, it validates that request.url() starts with a proper value, it then continues and passes that to CookieHandler, it doesn't check request.isNavigationRequest()

CookieHandler has this line

this.url = request.isNavigationRequest() ? request.url() : request.frame().url();

If isNavigationRequest() returns false then it takes request.frame().url() which in the case of AdWords seems to return 'about:blank'.

this gets passed to the next line

 this.domain = new URL(this.url).hostname;

Where the error is then thrown.

Maybe I've completely missed the boat, I've not spent a massive amount of time on this yet so maybe I'm wrong.

[Somberor]

I'm also facing this issue and a similar one and from what I can tell debugging, the issue seems to occur when a frame is embedded in the page which is "about:blank" or empty.

From what I know, things like AdWords creates dynamic frames which have no URL associated with them, this I assume calls Chrome to default to "about:blank". It seems to be related to #36

What seems to happen is that in requestHandler of proxy.js, it validates that request.url() starts with a proper value, it then continues and passes that to CookieHandler, it doesn't check request.isNavigationRequest()

CookieHandler has this line

this.url = request.isNavigationRequest() ? request.url() : request.frame().url();

If isNavigationRequest() returns false then it takes request.frame().url() which in the case of AdWords seems to return 'about:blank'.

this gets passed to the next line

 this.domain = new URL(this.url).hostname;

Where the error is then thrown.

Maybe I've completely missed the boat, I've not spent a massive amount of time on this yet so maybe I'm wrong.

This seems to be a right thing, it also happens on Twitter
Any idea how to fix it yet?

[Adam]

@Somberor

I forked the branches and modified proxy.js

You can see it here

https://github.com/undefinedfunction/puppeteer-page-proxy/blob/master/src/core/proxy.js

I didn't do a PR because I wasn't entirely confident I'd figured the issue but the above did seem to work for me

[Somberor]

@undefinedfunction sadly the solution didn't work for me , it doesn't give error but some functions on twitter doesn't work that way :/

[MeikyuuTrader]

tarik0's work around fixed my issue. #36

[Cuadrix]

fixed in v1.2.9