matthewmueller / x-ray

The next web scraper. See through the <html> noise.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

nested paginate support

haysclark opened this issue · comments

Is it possible to have nested paginate() calls?

I am trying to use X-ray to crawl from one paginated source to a 2nd paginated source, but I am not having any luck getting it to work. Is this a known limitation?

const x = Xray()
x('some_url', "#table-designer > tbody > tr", [{
        id: "td",
        data: x("td > a@href", {
          title: "h1",
          photo: "#content > img@src",
          models: x("#content > div > div > div > table.table.table-striped > tbody > tr", [{
              name: "td > a",
              url: "td > a@href"
          }])
          // inner paginated content 
          .paginate("#content > div > div > ul.pagination > li:last-child > a@href")
        }),
      }
    ])
    // outter paginated content
    .paginate("#content > div > ul.pagination > li:last-child > a@href");

can you send me a private gist of the nonworking code?

daniel.lathrop@gmail.com