CurryProxy
A fast and performant proxy and aggregator for querying multiple instances of an API spread across globally distributed data centers.
Capabilities
- CurryProxy can handle multiple routes for use against different revisions of an API or for use against completely unrelated APIs. See etc/routes.sample.yaml for an example configuration.
- Simple request forwarding (useful for preserving backwards
compatability with pre-CurryProxy versions of an API hosted in a
single datacenter)
- Example:
GET https://api.example.com/v1.0/foo/bar
forwarded toGET https://1.api.example.com/v1.0/foo/bar
- Example:
- Advanced request forwarding to multiple endpoints
- Example:
GET https://api.example.com/1,2/v1.0/foo/bar
forwarded to the following:GET https://1.api.example.com/v1.0/foo/bar
GET https://2.api.example.com/v1.0/foo/bar
- Requests are made in parallel
- JSON responses received from multiple endpoints are aggregated
and returned to the client
- Example:
{"foo": 1}
received from 1 and{"bar": 2}
received from 2 are aggregated to[{"foo": 1}, {"bar": 2}]
and returned to the client
- Example:
- Rich, meaningful errors logged and returned to the client when a proxied request fails
- Example:
Installation
pip install curryproxy
will do what you expect.
There are a couple of extra options:
pip install curryproxy[tests]
. This will install any packages required to run the tests.pip install curryproxy[frozen]
. This will install curryproxy with explicitly pinned dependencies, as generated bypip freeze
. This is intended as a convenience for those who want the package to provide a reproducible environment. The versions used will be the latest working ones at the time the given Curry version was released.
You will need to provide configuration files before starting Curry.
NOTE: The configuration files have changed in v2.x. routes.json
and logging.conf
have become routes.yaml
and logging.yaml
respectively. See the sample files for examples.
Limitations
- Merging responses from multiple endpoints together occurs only under certain conditions outlined on the Multiple Endpoints Aggregation wiki page.
Testing
Tests are set up to run with tox
.
The default tox
setup creates environments for each known-usable
version of each of our dependencies, and runs the tests in each of them.
That amounts to around a dozen environments at the moment, so you'll
probably want to run them in parallel with tox -p auto
.
If you don't want to be that thorough (perhaps you're smoke testing a
work in progress), you can run against only the most recent supported
versions of dependencies with tox -e py36
.
You can test against specific dependency versions like this:
tox -e py36-requests2.{2,3,4,5,6,7} -p auto tox -e py36-webob1.{2,3,4,5} -p auto tox -e py36-grequests0.{2,3} -p auto tox -e py36-requests.latest
tox -e lint
is also available, and will run various linting checks.
Roadmap
- OData support for server-side ordering and paging
- Response caching for a specified time