Rate limiting bug

[Dave Aitken]

Hi,

I noticed you've put in some client-enforced rate limiting in the client recently:
https://github.com/prismicio/javascript-kit/blob/master/src/utils.js#L176

There appears to be a bug - when trying to build a menu for our site we make 3 calls in parallel to get the contents of a few prismic collections to build various sections. The first two call back OK, but the 3rd one hangs seemingly indefinitely.

Can you explain the reason why there is rate limiting in the client anyway? I've got a few issues with it:

• I'm making 3 requests in parallel which is well below the stated 20 requests per second, but I'm still getting throttled? Our queries are also cached on our side after the initial fetch so don't happen again anyway.
• It's pretty easy to bypass by just setting Prismic.Utils.THRESHOLD to 0 / using a different instance of the api client / passing in a different request handler / using a different API client altogether, so if it's to enforce something in your terms of service it seems like enforcing it on the server-side / monitoring requests from particular users and letting them know would be a better way of dealing with it?

Overall it just seems like it's adding complexity (and bugs as I've discovered) without much benefit, and is just making me uneasy about using your client.

Thanks
Dave

[Erwan Loisant]

Hi Dave,

Some repositories are hitting the server pretty hard with more than 100 requests in parallel. Some times it's better done a different way, but sometimes it's the only way to do it because of limitations in our API. So we're OK with developers launching a lot of requests in parallel but we need to limit the rate to ensure QoS for all users.

We are going to do some throttling server side, but when all requests are started at the same time the only way to protect the server will be to drop some requests - that will cause bugs as the kits won't receive any answer. For that reason we added client-side throttling before rolling out server-side throttling. The server-side throttling should stay pretty low to limit the impact of the page loading of our users' sites.

Additionally we will improve the cache, currently it is invalidated at a reference level meaning that any change in the repository will invalidate all documents. We will implement it at document level so a document stays in cache as long as it should.

This is why we introduced client-side throttling, but we made Prismic.Utils.THRESHOLD configurable on purpose so developers can deactivate or adjust the rate as they want without having to fork the kit. Just be aware that if you're launching hundreds of requests simultaneously without throttling, some requests may be dropped by the server. With 3 requests you'll be fine.

Technically the client-side throttling is spacing all requests starts of at least 50 ms, so indeed if you launch 3 requests in parrallel the third will start after 100ms. However since the content will end up in cache, that will only impact the first request after a change in the repository. It is indeed not very fair to start throttling at the first request, I will see with the server people if we can start the throttle at 20 simultaneous requests. I believe that should be acceptable.

Again if you're sure you will keep a low number of parallel requests, feel free to lower Prismic.Utils.THRESHOLD. It is only there to help you respect rate limitations that will soon be on server side.

---

Now if there's a bug that's something that needs to be fixed, indeed with 3 requests it could take no more than 100ms + the response time of the third. Could you send me a code snippet reproducing the bug? You can send me a private message if you don't want it publicly here.

[Dave Aitken]

Hi Erwan,
Thanks for the reply - that sounds reasonable. This code reproduces the bug when using our endpoint, token, and valid collection names. If you uncomment the line with Prismic.Utils.THRESHOLD = 0 everything completes correctly:

'use strict';
var Prismic = require('prismic.io').Prismic;
var async = require('async'); //(npm install async)

var endpoint = 'https://xxx.cloudfront.net/api';
var token = 'xxx';

//Prismic.Utils.THRESHOLD = 0;

Prismic.Api(endpoint, function (err, client) {
    var ref = client.master();
    var collections = ['collection1', 'collection2', 'collection3'];

    async.map(collections, function (collection, cb) {
        var form = client.form(collection);
        console.log('calling prismic for ' + collection);
        form.ref(ref).submit(function (err, result) {
            console.log('done calling prismic for ' + collection);
            return cb(err, result);
        });
    }, function (err, results) {
        console.log('done');
        console.log(err, results);
    });
}, token);

The collections that complete are collection1 and collection3, so I'm guessing if two requests go down this branch of the if statement here. the second one clears the first https://github.com/prismicio/javascript-kit/blob/master/src/utils.js#L162

[Erwan Loisant]

Hi,

There was a bug indeed.

I just released version 1.0.23 that works by limiting the number of concurrent requests to 20 rather than limiting to 20 requests/seconds. That should be must less impacting to response time, and if you don't launch more than 20 requests simultaneously you shouldn't see any difference at all.

I tested with your snippet and it works.

[Dave Aitken]

Thanks for the quick update - I've upgraded the client and got rid of the THRESHOLD=0 line in our code and everything works as expected now.