DynamoDB pagination issue in GSI with range key

[John Williams]

Description

I think there's an issue with paging using LastEvaluatedKey and ExclusiveStartKey in a GSI with a range key. It works ok in version 5.0.3 but not with the latest (5.0.9)

If the GSI just has a hash key and no range key, it seems to work fine. It's just where it also has a range key.

I've tested it using the following test in test_dynamodb_query.py:

@pytest.mark.aws_verified
@dynamodb_aws_verified(add_range=True, add_gsi_range=True)
def test_query_gsi_pagination_with_string_gsi_range(table_name=None):
    dynamodb = boto3.resource("dynamodb", region_name="us-east-1")
    table = dynamodb.Table(table_name)

    for i in range(3, 7):
        table.put_item(Item={"pk": "the-key", "sk": f"{i}", "gsi_pk": "johndoe", "gsi_sk": "janedoe"})

    for i in range(9, 6, -1):
        table.put_item(Item={"pk": "the-key", "sk": f"{i}", "gsi_pk": "johndoe", "gsi_sk": "janedoe"})

    for i in range(3):
        table.put_item(Item={"pk": "the-key", "sk": f"{i}", "gsi_pk": "johndoe", "gsi_sk": "janedoe"})

    page1 = table.query(
        KeyConditionExpression=Key("gsi_pk").eq("johndoe"),
        IndexName="test_gsi",
        Limit=6,
    )
    assert page1["Count"] == 6
    assert page1["ScannedCount"] == 6
    assert len(page1["Items"]) == 6

    page2 = table.query(
        KeyConditionExpression=Key("gsi_pk").eq("johndoe"),
        IndexName="test_gsi",
        Limit=6,
        ExclusiveStartKey=page1["LastEvaluatedKey"],
    )
    assert page2["Count"] == 4
    assert page2["ScannedCount"] == 4
    assert len(page2["Items"]) == 4
    assert "LastEvaluatedKey" not in page2

    results = page1["Items"] + page2["Items"]
    subjects = set([int(r["sk"]) for r in results])
    assert subjects == set(range(10))

Expected result

the line:
assert page2["Count"] == 4

should succeed.

Actual result

It fails with a:
assert 0 == 4

[Bert Blommers]

Hi @johnwilliams57-nhs, thanks for raising this. I think this is a duplicate of #7751, and fixed with #7775.

Can you reproduce this with the latest dev-release, moto >= 5.0.10.dev32?

[John Williams]

Hi, yes it works with the latest version. Thanks!
i'll close the issue.