Multiple different errors in the theme of "cannot read properties of undefined"
edge-loop opened this issue · comments
I'm using the npm version of this, which I think might be slightly different, but maybe the error will be solvable from here (since the issues of the fork don't seem to be available). I keep getting errors when trying to scrape the later pages of some videos (I think it's longer ones but I'm not sure of the correlation.
TypeError: Cannot read properties of undefined (reading 'continuationCommand')
at getComments ([links n stuff]/@freetube/yt-comment-scraper/src/Youtube-Scraper.js:62:80)
and
data.forEach((node) => {
^
TypeError: Cannot read properties of undefined (reading 'forEach')
at Function.parseCommentData ([links n stuff]/@freetube/yt-comment-scraper/src/htmlParser.js:5:10)
at getCommentReplies ([links n stuff]/@freetube/yt-comment-scraper/src/Youtube-Scraper.js:80:36)
at runMicrotasks (<anonymous>)
Basically I'm just trying to scrape all of the video's comments and replies and put them into an array:
const commentScraper = require('@freetube/yt-comment-scraper');
const id = 'foSAi-olfck'; // penguinz0's video on the rush e meme or whatever doesn't really matter
(async function()
{
var comments = await scrape(commentScraper.getComments, { videoId: id, mustSetCookie: true });
async function scrape(method, payload)
{
console.log('Loading...');
const data = await method(payload);
const result = data.comments;
for(e of data.comments)
if(e.replyToken !== null)
result.push(...await scrape(commentScraper.getCommentReplies, { videoId: id, mustSetCookie: true, replyToken: e.replyToken }));
if(data.continuation !== null)
result.push(...await scrape(commentScraper.getComments, { videoId: id, mustSetCookie: true, continuation: data.continuation }));
return result;
}
console.log(comments.length);
})();
I tried switching mustSetCookie:
out with just setCookie:
since I saw both but neither did anything
Is this a problem with my code or something else?
I'm also using Replit which is sometimes weird about things like this