Using up Google API very quickly

[Yash Bansal]

Whenever I run the search code, the google JSON API key limit gets reached within a minute and opposite to what is claimed, only 5-20 profiles are scraped (I tried it with multiple API keys from diff. accounts). Is there a setting needed somewhere to be done?

[Yash Bansal]

Attaching search payload

[Ethan Baker]

Hi Yash, By default the program automatically stores previously stored search terms and previously indexed accounts in .pkl files to prevent duplicate results. This is intentional as the google results do not update very often, but it is definitely challenging to work around when initially using the tool and setting it up. I think this might be why you are having difficulties. To fix, you could just download the repository again, or go to /search_tool/data and delete all data in the indexed_queries.pkl and indexed_profiles.pkl files. If you would like to remove the duplicate checking functionality entirely, I think that you might be able to just check the repeat queries box in the UI, and change function load_indexed_profiles in the search_tool/indexed_data.py file to just always return []. I hope this helps. If you are still having trouble, try searching again with a completely new search term you have never used before to see if that works, if not, I’d be more than happy to help. Ethan On Aug 22, 2023 at 5:15 AM -0400, Yash Bansal ***@***.***>, wrote: Whenever I run the search code, the google JSON API key limit gets reached within a minute and opposite to what is claimed, only 5-20 profiles are scraped (I tried it with multiple API keys from diff. accounts). Is there a setting needed somewhere to be done? — Reply to this email directly, view it on GitHub<#1>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BATNCXEHTHFUPE57KLZZQ5TXWR2DNANCNFSM6AAAAAA3ZTA3QQ>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>