BuilderIO / gpt-crawler

Crawl a site to generate knowledge files to create your own custom GPT from a URL

Home Page:https://www.builder.io/blog/custom-gpt

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

How to paginate for large JSON files?

chnsh opened this issue · comments

I get an error that the JSON files are too large for GPT. I can use maxTokens, maxFileSize to reduce it but how do I create the next JSON file such that it starts off from the current one?