Supporting Insert, Update, Delete for DynamoDB

[Ahmed Ali]

Currently execute_statement in the Moto library only supports Select queries, From my understanding Moto uses py-partiql-parser to parse the partiQL statements when calling execute_statement execute_transaction and batch_execute_statement.

It would be nice to be able to handle Insert, Update and Delete in Moto.

I can have a look and work on making the changes if you agree.

[Bert Blommers]

Hi @ajaali!

The py-partiql-parser is currently essentially a readonly library, since Moto sends a copy of the data to the library. So implementing support for modifications to the DynamoDB table needs a new API design.

Out of the top of my head I can see two approaches:

1. send the original table/data to py-partiql-parser, with the implicit understanding that py-partiql-parser may alter the data
2. send the update statement to py-partiql-parser, and the library sends back info about the updates that Moto should make (format TBD)

There is a danger that the first option would make the two modules too closely coupled, but I don't know yet what the second option could look like.

PR's/suggestions are always welcome!

[Ahmed Ali]

Thanks, I was leaning toward no.2 where py-partiql-parser can send moto a dict/json giving details about the operation. and moto can then use that to operate on the table. essentially moto can convert it into an input it can then use to call the non-partial api functions that boto provide. I think the 2 PRs for py-partiql-parser and moto will have to go hand in hand. I'll try to get the PRs raised.

[Bert Blommers]

Sounds good @ajaali - let me know if you want any help!

[Bert Blommers]

Hi @ajaali! Hope you don't mind - but I had a crack at this myself.

py_partiql_parser 0.5.0 now supports the Insert, Update and Delete statements - see #16

This is exposed as of moto >= 4.2.12.dev28, with PR getmoto/moto#7130

Note that the RETURNING keyword is not yet supported - is that something that you need as well? At the moment the execute_statement doesn't return anything, we only update the data in the backend.

[r4m4k0735w4r4r40]

Hi @bblommers with py-partiql-parser==0.5.0 I am getting below error.

query = 'UPDATE users SET first_name = ?, last_name = ? WHERE username= ?'

    @classmethod
    def get_query_metadata(cls, query: str) -> QueryMetadata:
        query = query.replace("\n", " ")
        clauses = re.split("SELECT | FROM | WHERE ", query, flags=re.IGNORECASE)
        from_parser = FromParser(clauses[2])

        IndexError: list index out of range

[Bert Blommers]

Hi @r4m4k0735w4r4r40, that should have been fixed as of 0.5.1.

I'll close this, but let me know if you run into any other issues with the latest release (0.5.4)