deserialize lost data

[Martin Vargas]

Hi!
This is my form post:
MultiDict([('_charset_', 'UTF-8'), ('__formid__', 'deform'), ('token', '2993df2af005fdedea14ea7d79dcde97c7b0e677'), ('__start__', 'sdate:mapping'), ('date', '2021-08-20'), ('dt_date-visible', '20/08/2021'), ('__end__', 'sdate:mapping'), ('__start__', 'estimated_edate:mapping'), ('date', '2021-08-20'), ('dt_date-visible', '20/08/2021'), ('__end__', 'estimated_edate:mapping'), ('name', 'AAA'), ('description', 'DDD'), ('metric', '364'), ('__start__', 'nodes:sequence'), ('nodes', '116'), ('__end__', 'nodes:sequence'), ('__start__', 'clusters:sequence'), ('clusters', '140'), ('__end__', 'clusters:sequence'), ('__start__', 'users:mapping'), ('expert', 'NAVARRO, Teresa (id: NAT)'), ('__start__', 'groups:sequence'), ('__end__', 'groups:sequence'), ('__start__', 'assistants:sequence'), ('__end__', 'assistants:sequence'), ('send_message', 'true'), ('__end__', 'users:mapping'), ('__start__', 'transactions:sequence'), ('__start__', 'transaction:mapping'), ('id', ''), ('id_transaction', '321'), ('__start__', 'recording:sequence'), ('__start__', 'recording:mapping'), ('uid', 'IHWHBX5BDQ'), ('s3_filename', 'logo-Telegram.png'), ('s3_final_url', 'https://bucket.s3.amazonaws.com/45f57055-de02-4136-9122-7b650eadbfd4'), ('s3_mimetype', 'image/png'), ('s3_upload_finished', '1'), ('s3_upload_attempts', '1'), ('__end__', 'recording:mapping'), ('__end__', 'recording:sequence'), ('__end__', 'transaction:mapping'), ('__end__', 'transactions:sequence'), ('Create', 'Crear')])
When there is a form error, I found this in my pstruct:

env/lib/python3.6/site-packages/deform/field.py(786)validate_pstruct()
    785         import ipdb;ipdb.set_trace()
--> 786         try:
    787             cstruct = self.deserialize(pstruct)

ipdb> pstruct
{'_charset_': 'UTF-8', '__formid__': 'deform', 'token': '2993df2af005fdedea14ea7d79dcde97c7b0e677', 'sdate': {'date': '2021-08-20', 'dt_date-visible': '20/08/2021'}, 'estimated_edate': {'date': '2021-08-20', 'dt_date-visible': '20/08/2021'}, 'name': 'AAA', 'description': 'DDD', 'metric': '364', 'nodes': ['116'], 'clusters': ['140'], 'users': {'expert': 'NAVARRO, Teresa (id: NAT)', 'groups': [], 'assistants': [], 'send_message': 'true'}, 'transactions': [{'id': '', 'id_transaction': '321', 'recording': [{'uid': 'IHWHBX5BDQ', 's3_filename': 'logo-Telegram.png', 's3_final_url': 'https://bucket.s3.amazonaws.com/45f57055-de02-4136-9122-7b650eadbfd4', 's3_mimetype': 'image/png', 's3_upload_finished': '1', 's3_upload_attempts': '1'}]}], 'Create': 'Crear'}

ipdb> self.deserialize(pstruct) <-- here the deserialization
{'token': '2993df2af005fdedea14ea7d79dcde97c7b0e677', 'sdate': '2021-08-20', 'estimated_edate': '2021-08-20', 'name': 'AAA', 'description': 'DDD', 'metric': '364', 'nodes': ('116',), 'clusters': ('140',), 'users': {'expert': 'NAVARRO, Teresa (id: NAT)', 'groups': [], 'assistants': [], 'send_message': 'true'}, 'transactions': [{'id': <colander.null>, 'id_transaction': '321', 'recording': [{'filename': '', 'uid': 'IHWHBX5BDQ', 'mimetype': None, 'size': None, 'fp': None, 'preview_url': None}], 'performance': 'false'}]} ipdb>

when the form return the error, the recording key inside the transactions key, lost the values. I resolve it doing somethig like this:
cstruct['transactions'] = pstruct['transactions']

Now, the new cstruct is what i need:
{'token': '2993df2af005fdedea14ea7d79dcde97c7b0e677', 'sdate': '2021-08-20', 'estimated_edate': '2021-08-20', 'name': 'AAA', 'description': 'DDD', 'metric': '364', 'nodes': ('116',), 'clusters': ('140',), 'users': {'expert': 'NAVARRO, Teresa (id: NAT)', 'groups': [], 'assistants': [], 'send_message': 'true'}, 'transactions': [{'id': '', 'id_transaction': '321', 'recording': [{'uid': 'IHWHBX5BDQ', 's3_filename': 'logo-Telegram.png', 's3_final_url': 'https://bucket.s3.amazonaws.com/45f57055-de02-4136-9122-7b650eadbfd4', 's3_mimetype': 'image/png', 's3_upload_finished': '1', 's3_upload_attempts': '1'}]}]}
and when this line runing:
if exc: raise exception.ValidationFailure(self, cstruct, exc)
The form tell me the error and no data is lost

Can it will be an issue to fix? or i have an error in my form post structure??

[Steve Piercy]

I'm not clear why you're messing around with the pstruct.

Usually we only use cstruct or appstruct.

See https://docs.pylonsproject.org/projects/deform/en/latest/serialization.html#deserialization

[Martin Vargas]

I'm using a part of pstruct because cstruct = self.deserialize(pstruct) lost data.

[Steve Piercy]

What happens when you follow my suggestion to use cstruct or appstruct?