puppeteer / examples

Use case-driven examples for using Puppeteer and headless chrome

Home Page:https://developers.google.com/web/tools/puppeteer/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Crawlsite.js Error: TypeError: Converting circular structure to JSON

dspstudio opened this issue · comments

When I try to crawl with a DEPTH bigger than 2 I get this error:
Error: TypeError: Converting circular structure to JSON
line: await util.promisify(fs.writeFile)(./${OUT_DIR}/crawl.json, JSON.stringify((root), null, ' '));
if i use util.inspect, error is gone but seems like the output is invalid json.

I could eliminate THAT problem bei using an alternative to JSON.stringify, as described here: https://makandracards.com/makandra/28847-dealing-with-typeerror-converting-circular-structure-to-json-on-javascript

Now I "only" have the problem, that node seems not to like the code in general, as it throws a promise-error:

(node:13544) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch().

But that's another story...