Want to capture the "div" which is created after scrolling the page
N-pharmarack opened this issue · comments
N-pharmarack commented
Subject of the issue
I want to capture the items from a site, but half of the items are loaded when I scroll the page(link of the page : https://www.91mobiles.com/mobile-price-list-in-india?page=1 ). A new div ( page_scroll_1) is created after I scroll the the page, and the items I want to capture are there in that div.
I have tried ".scrollTo" method from 'x-ray-phantom' but It didn't work
Your environment
- version of node: 16.13.0
- version of npm: 8.1.0
Steps to reproduce
const Xray = require('x-ray');
const puppeteer = require('puppeteer');
var phantom = require('x-ray-phantom');
var phantom_opts = {
webSecurity: false,
images: false,
weak: false
};
var x = Xray().driver(phantom(phantom_opts, function (nightmare, done) {
done
.useragent("Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2125.111 Safari/537.36")
.goto(nightmare.request.req.url)
.scrollTo(0, 500)
.wait(3000)
.scrollTo(0, 1000)
.wait(3000)
.scrollTo(0, 1500)
.wait(3000)
.scrollTo(0, 2500)
.wait(3000)
.scrollTo(0, 3000)
.wait(3000)
.scrollTo(0, 3500)
.wait(3000)
.scrollTo(0, 4000)
.wait(3000)
}));
x('https://www.91mobiles.com/mobile-price-list-in-india?page=1', '.finder_snipet_wrap',[{
mobile_model: '.name',
Price: '.price',
Url: '.name@href'
}])
.write('results.json')
Expected behaviour
It should capture all the item with '.finder_snipet_wrap'
Actual behaviour
Capture's only half items, and skips all the other '.finder_snipet_wrap' that is being created after page scrolling
jmcoder1 commented
Did you figure this out? I have a very similar issue.