VinciGit00 / Scrapegraph-ai

Python scraper based on AI

Home Page:https://scrapegraphai.com

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

In headless mode, there is content in the browser, but it prompts that there is no HTML content

db005 opened this issue · comments

Describe the bug
In headless mode, there is content in the browser, but it prompts that there is no HTML content

To Reproduce

import os
from dotenv import load_dotenv
from scrapegraphai.graphs import SmartScraperGraph
from scrapegraphai.utils import prettify_exec_info

graph_config = {
    "llm": {
        "api_key": "",
        "model": "gemini-pro"
    },
    "max_results": True
}

smart_scraper_graph = SmartScraperGraph(
   prompt="提取每一条文字信息与url",
   source="https://www.google.com/search?q=gemini+api申请",
   config=graph_config
)

result = smart_scraper_graph.run()
print(result)

Expected behavior
ValueError ValueError: No HTML body content found, please try setting the 'headless' flag to False in the graph configuration.

Screenshots
image

Desktop:

  • OS: win11
  • Browser edge
  • Version v1.5.4

hi, try with this config

graph_config = {
    "llm": {
        "api_key": "",
        "model": "gemini-pro"
    },
"headless": False
    "max_results": True
}