digital-science / dimcli

Python client and CLI for scientometrics and research analytics using the Dimensions API.

Home Page:https://digital-science.github.io/dimcli/getting-started.html

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Skip changes limit if I want to start extraction from hits over 50k

marton-balazs-kovacs opened this issue · comments

When I use the iterative query to look up results starting over 50k which is the API limit, the iterative query function rewrites the hard-coded limit value. Thus, it is not possible to start the iteration over the API limit. I suspect the following code chunk is responsible for the behavior:

MAXLIMIT = maxlimit or 50000 flag_last_round = False if skip + limit >= MAXLIMIT: flag_last_round = True if skip + limit > MAXLIMIT: limit = MAXLIMIT - skip

The Analytics API has a built-in upper limit of 50k records for all queries.

So in dimcli auto-iteration utility this is enforced, to avoid triggering an API error.

You can test it out via a simple query, by setting the skip parameter at 50k:

image

PS Let me know if this does not answer your question..