mendableai / firecrawl

🔥 Turn entire websites into LLM-ready markdown or structured data. Scrape, crawl and extract with a single API.

Home Page:https://firecrawl.dev

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Crawl API fails for local installation

mattjoyce opened this issue · comments

After installing locally using Docker, I cannot get crawl to work. "Failed to bill team, no subscription was found"
Scrape does work.

This is a typical response for Crawl via Curl or Python.

{
  "data": {
    "url": "https://github.com/mendableai/firecrawl",
    "mode": "crawl",
    "crawlerOptions": {},
    "pageOptions": {
      "onlyMainContent": true
    },
    "origin": "api"
  },
  "returnValue": {
    "success": false,
    "result": {
      "links": []
    },
    "error": "Failed to bill team, no subscription was found"
  }
}

hey @mattjoyce have you set the USE_DB_AUTHENTICATION to false in the apps/api/.env file?

Yes, without that the response is a 401.
Strange that Scrape works and Crawl does not, I assumed Scrape was a shallow single page crawl.
The specific erros only appears in
runWebScraper.ts

I tried to work back from that, but my JS/TS is weak. :(

The Python binding, requires an API key, but there is no api key for local right, is that relevant?
@rafaelsideguide let me know if any of my config would be useful to see, or if I can help test.

Ok, I just rebuild my local installation, and it works.
So I assume it was something in my docker file.