BulkExport Days not returning all records
chadlarsh opened this issue · comments
Issue Summary
When attempting to list the available BulkExport days, the library does not return all records. According to the documentation, if no limit is specified, all records should be returned. It only returns the first 10 records. If you manually specify a page size of 10, the library will return all records.
Code Snippet
This will NOT return all records
client.bulkexports.v1.exports("Messages").days.list()This will return all records
client.bulkexports.v1.exports("Messages").days.list(page_size=10)Exception/Log
No exception occurs.
Technical details:
- twilio-python version: 7.12.0
- python version: 3.8.14
Hi @chadlarsh,
I will be checking the issue. Will get back to you soon. +1s on the issue summary will help it move up the backlog.
I tested:
client.bulkexports.v1.exports("Messages").days.list(). --> returns only 10 records.
This issue seems to be specific to this api only.
This issue has been added to our internal backlog to be prioritized. Pull requests and +1s on the issue summary will help it move up the backlog.
Additional note here: setting limit=1000 results in 30 days being returned - thanks @chadlarsh for the note that setting page_size=10 works around this issue - the bug also impacts client.bulkexports.v1.exports("Messages").days.stream() with the same symptoms (same issue with exports("Calls") too) -
This actually doesn't seem to be a python issue - I'm seeing the same behavior talking to the API via curl. Sending the request with no params to get defaults yields a response with next_page_url: null - supplying the query param ?PageSize=10 (or any other value between 2 and 30 inclusive) works as expected.
(twilio-audit) bernard.gardner@bernard twilio-audit % curl -X GET "https://bulkexports.twilio.com/v1/Exports/Messages/Days" \
-u $TWILIO_ACCOUNT_SID:$TWILIO_AUTH_TOKEN | jq
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1594 100 1594 0 0 6831 0 --:--:-- --:--:-- --:--:-- 7022
{
"meta": {
"page": 0,
"page_size": 50,
"first_page_url": "https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=50&Page=0",
"previous_page_url": null,
"url": "https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=50&Page=0",
"next_page_url": null,
"key": "days"
},
"days": [
{
"friendly_name": "Apr\\-Jun 2023",
"create_date": "2023-10-24",
"day": "2023-04-01",
"resource_type": "Messages",
"size": 1857
},
{
"friendly_name": "Apr\\-Jun 2023",
"create_date": "2023-10-24",
"day": "2023-04-02",
"resource_type": "Messages",
"size": 2172
},
{
"friendly_name": "Apr\\-Jun 2023",
"create_date": "2023-10-24",
"day": "2023-04-03",
"resource_type": "Messages",
"size": 3099
},
{
"friendly_name": "Apr\\-Jun 2023",
"create_date": "2023-10-24",
"day": "2023-04-04",
"resource_type": "Messages",
"size": 3582
},
{
"friendly_name": "Apr\\-Jun 2023",
"create_date": "2023-10-24",
"day": "2023-04-05",
"resource_type": "Messages",
"size": 4584
},
{
"friendly_name": "Apr\\-Jun 2023",
"create_date": "2023-10-24",
"day": "2023-04-06",
"resource_type": "Messages",
"size": 4625
},
{
"friendly_name": "Apr\\-Jun 2023",
"create_date": "2023-10-24",
"day": "2023-04-07",
"resource_type": "Messages",
"size": 4269
},
{
"friendly_name": "Apr\\-Jun 2023",
"create_date": "2023-10-24",
"day": "2023-04-08",
"resource_type": "Messages",
"size": 3504
},
{
"friendly_name": "Apr\\-Jun 2023",
"create_date": "2023-10-24",
"day": "2023-04-09",
"resource_type": "Messages",
"size": 2609
},
{
"friendly_name": "Apr\\-Jun 2023",
"create_date": "2023-10-24",
"day": "2023-04-10",
"resource_type": "Messages",
"size": 3258
}
]
}
(twilio-audit) bernard.gardner@bernard twilio-audit % for i in {1..50}
do
echo -n "$i "; curl -s -X GET "https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=$i" -u $TWILIO_ACCOUNT_SID:$TWILIO_AUTH_TOKEN | jq -Mr .meta.next_page_url
done
1 null
2 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=2&Page=1&PageToken=ENMjAyMy0wNC0wMg%3D%3D
3 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=3&Page=1&PageToken=ENMjAyMy0wNC0wMw%3D%3D
4 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=4&Page=1&PageToken=ENMjAyMy0wNC0wNA%3D%3D
5 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=5&Page=1&PageToken=ENMjAyMy0wNC0wNQ%3D%3D
6 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=6&Page=1&PageToken=ENMjAyMy0wNC0wNg%3D%3D
7 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=7&Page=1&PageToken=ENMjAyMy0wNC0wNw%3D%3D
8 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=8&Page=1&PageToken=ENMjAyMy0wNC0wOA%3D%3D
9 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=9&Page=1&PageToken=ENMjAyMy0wNC0wOQ%3D%3D
10 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=10&Page=1&PageToken=ENMjAyMy0wNC0xMA%3D%3D
11 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=11&Page=1&PageToken=ENMjAyMy0wNC0xMQ%3D%3D
12 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=12&Page=1&PageToken=ENMjAyMy0wNC0xMg%3D%3D
13 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=13&Page=1&PageToken=ENMjAyMy0wNC0xMw%3D%3D
14 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=14&Page=1&PageToken=ENMjAyMy0wNC0xNA%3D%3D
15 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=15&Page=1&PageToken=ENMjAyMy0wNC0xNQ%3D%3D
16 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=16&Page=1&PageToken=ENMjAyMy0wNC0xNg%3D%3D
17 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=17&Page=1&PageToken=ENMjAyMy0wNC0xNw%3D%3D
18 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=18&Page=1&PageToken=ENMjAyMy0wNC0xOA%3D%3D
19 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=19&Page=1&PageToken=ENMjAyMy0wNC0xOQ%3D%3D
20 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=20&Page=1&PageToken=ENMjAyMy0wNC0yMA%3D%3D
21 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=21&Page=1&PageToken=ENMjAyMy0wNC0yMQ%3D%3D
22 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=22&Page=1&PageToken=ENMjAyMy0wNC0yMg%3D%3D
23 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=23&Page=1&PageToken=ENMjAyMy0wNC0yMw%3D%3D
24 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=24&Page=1&PageToken=ENMjAyMy0wNC0yNA%3D%3D
25 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=25&Page=1&PageToken=ENMjAyMy0wNC0yNQ%3D%3D
26 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=26&Page=1&PageToken=ENMjAyMy0wNC0yNg%3D%3D
27 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=27&Page=1&PageToken=ENMjAyMy0wNC0yNw%3D%3D
28 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=28&Page=1&PageToken=ENMjAyMy0wNC0yOA%3D%3D
29 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=29&Page=1&PageToken=ENMjAyMy0wNC0yOQ%3D%3D
30 https://bulkexports.twilio.com/v1/Exports/Messages/Days?PageSize=30&Page=1&PageToken=ENMjAyMy0wNC0zMA%3D%3D
31 null
32 null
33 null
34 null
35 null
36 null
37 null
38 null
39 null
40 null
41 null
42 null
43 null
44 null
45 null
46 null
47 null
48 null
49 null
50 null