By Running "php artisan roach:run MySpider" unable to scrap any data.

[farsidd]

I am working in Laravel 9.0. Whenever i am running command I'm not getting anything in Log file. See the attached pic of log file. Request sent successfully but no data scrapped.

Here is the view of log file

here is the view of my spider file

i have installed this version of roach-php: "roach-php/laravel": "^2.0"

Is there anything I'm missing?

[DodgeWhale]

@farsidd Hey! I ran into the same issue you were having. This is what I did to fix it.

From the root of the project, I ran
openssl req -newkey rsa:2048 -new -nodes -x509 -days 3650 -keyout key.pem -out cert.pem
Which generates a key.pem and cert.pem file.

Then, I override the base method initialRequests found in RoachPHP\Spider\AbstractSpider.

/**
 * @return Request[]
 */
protected function initialRequests(): array
{
    return \array_map(
        fn (string $url): Request => new Request(
            $this->requestMethod,
            $url,
            [$this, 'parse'],
            [
                'cert' => base_path('cert.pem'),
                'ssl_key' => base_path('key.pem')
             ]
        ),
        $this->configuration->startUrls
    );
}

base_path reference https://laravel.com/docs/10.x/helpers#method-base-path
openssl req reference https://support.microfocus.com/kb/doc.php?id=7013103

However, if the page you're trying to request relies on JavaScript to load the DOM elements you're selecting, I'm not sure a fix for that yet (using RoachPHP at least).

Hope this helps you, or anyone else having this issue.

[farsidd]

Sure. I will check that thing.

[ksassnowski]

I’m moving this to discussions as this seems more related to scraping a particular page than a bug with the package.