t9tio / cloudquery

Turn any website to API by several clicks (serverless and support SPA!)

CloudQuery

Turn any website to API by several clicks.

Try online: https://cloudquery.t9t.io

API

• Sample Query: Best 3 posts on HN

Query params:

• url: url of the webpage you want to fetch
• selectors: css selectors describing the elements interests you

Returns:

• innerText: element innerText
• href: element href

Run locally

# install dependencies
npm i
# start backend server(for auto restart when code change)
npm run sb
# start frontend server(for auto restart when code change)
npm run sf

# visit http://localhost:1234

How to deploy your own CloudQuery to AWS lambda

1. Configuration

• update profile(AWS Credential Profiles) in up.json to use your own aws account
• update rateLimit in config.json to set your own rate limit (Default: 5 request/hour)

2. Deploy

# deploy to aws lambda
up

# see the URL of your CloudQuery
up url

# deploy production version(to cloudquery.t9t.io)
up deploy production

Thanks

• up for deploying serverless API to AWS with ease
• serverless-chrome for running chrome on AWS lambda
• finder for making it easy to select elements on webpage