roach-php / core

The complete web scraping toolkit for PHP.

Home Page:https://roach-php.dev

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

How to login and then scrap data from a page that requires auth?

skiopey opened this issue · comments

I want to login into a website which is a Laravel10 app that has basic auth. and then access a page that requires the user to be logged in.

Thank you for this amazing package, you are the best man.

Edit:

Checkout the solution in the next comment. 👇 👇

i figured it out.

public array $downloaderMiddleware = [
    RequestDeduplicationMiddleware::class,
    CookieMiddleware::class,
];

public function parse(Response $response): Generator
{
    $csrfToken = $response->filter('meta[name="csrf-token"]')->attr('content');

    $request = new Request(
        'POST',
        'http://127.0.0.1:8001/login',
        [$this, 'parseContent'],
        [
            'headers' => [
                'X-CSRF-TOKEN' => $csrfToken,
            ],
            'form_params' => [
                'email' => 'admin@demo.com',
                'password' => 'password',
            ],
        ],
    );

    yield ParseResult::fromValue($request);
}

public function parseContent(Response $response): Generator
{
    yield $this->request('GET', 'http://127.0.0.1:8001/billing', 'ParseBilling');
}

public function ParseBilling(Response $response): Generator
{
    dd($response->filter('h3')->text());
}