PUT,POST,PATCH with original values gives an error
Dragomir-Ivanov opened this issue · comments
Working with PATCH, I hit an issue doing updates onto resources:
This is what I do:
GET http://localhost:8080/api/visits/5a4d69264e2127a7118b4c51
Result:
{
"appointmentDT": "2018-01-05T18:00:00+02:00",
"clientId": "5a4d63fe41095f105dcd49b2",
"createdDT": "2018-01-04T01:37:18.079+02:00",
"duration": 180,
"id": "5a4d69264e2127a7118b4c51",
"name": "Dryer, Refrigerator, Microwave Diagnose",
"serviceId": "5a4d63fe41095f105dcd49b6",
"technitian": "Asparuh Vasilev",
"zipCode": "89107"
}
Then I try to update with the same exact data:
PUT http://localhost:8080/api/visits/5a4d69264e2127a7118b4c51
{
"appointmentDT": "2018-01-05T18:00:00+02:00",
"clientId": "5a4d63fe41095f105dcd49b2",
"createdDT": "2018-01-04T01:37:18.079+02:00",
"duration": 180,
"id": "5a4d69264e2127a7118b4c51",
"name": "Dryer, Refrigerator, Microwave Diagnose",
"serviceId": "5a4d63fe41095f105dcd49b6",
"technitian": "Asparuh Vasilev",
"zipCode": "89107"
}
Result:
{
"code": 422,
"issues": {
"createdDT": [
"read-only"
],
"id": [
"read-only"
]
},
"message": "Document contains error(s)"
}
This shouldn't be the case since, I am putting exactly the same data, and removing read-only
fields from the WEB UI, seems tedious and overkill. I made a simple patch to trace the issue:
func (s Schema) Prepare(......
if found && (!oFound || !reflect.DeepEqual(v, oValue)) {
fmt.Printf("field=%s type=%T value=%#v otype=%T ovalue=%#v \n", field, value, value, oValue, oValue)
changes[field] = value
}
Which gives following trace:
field=appointmentDT type=string value="2018-01-05T18:00:00+02:00" otype=time.Time ovalue=time.Time{wall:0x0, ext:63650764800, loc:(*time.Location)(0xb47380)}
field=serviceId type=string value="5a4d63fe41095f105dcd49b6" otype=bson.ObjectId ovalue="ZMc\xfeA\t_\x10]\xcdI\xb6"
field=id type=string value="5a4d69264e2127a7118b4c51" otype=bson.ObjectId ovalue="ZMi&N!'\xa7\x11\x8bLQ"
field=createdDT type=string value="2018-01-04T01:37:18.079+02:00" otype=time.Time ovalue=time.Time{wall:0x4b571c0, ext:63650619438, loc:(*time.Location)(0xb47380)}
This indicates that for some types, original values are in DB format, but incoming are in text format. Later in func (s Schema) Validate(......
conversion is made, but this is too late.
Can I propose following change:
v, err := def.Validator.Validate(value)
if err != nil {
log.Panic("Cannot validate")
}
if found && (!oFound || !reflect.DeepEqual(v, oValue)) {
changes[field] = v
}
Maybe we can refactor Prepare
to propagate that error
instead of panic.
Did you remove the code I describe the other day which copy read only values into the base? It’s to workaround this exact issue.
Hmm, where is that description? I didn't see anything on GitHub, or on Slack.
Also, it is not just about Read-only values, although rest-layer
explicitly errors on these. Implicitly all fields may be affected, and just considered as updates, calling hooks, etc.
Also some more explanation about non-read-only
values.
Using JSON-Patch protocol RFC6902, I use original values, apply the patch on those, and then get the new value. Doing so, I have whole bunch of fields that are actually the original ones, residing in the "new" values, that need to be discarded as updates, and need to go to base
.
In the example above "appointmentDT" non-read-only
field, has the same value, just in different formats. time.Time
in original
values, and string
in the new values. This is considered as update that will potentially trigger a hook. This is quite a popular use-case. GET-ing values for a form, updating some fields, then PUT-ing the full form back.
Okay, looking more at the problem, it seems that Mongo is storing some types in its internal types such as Date
and ObjectId
that may be need to be converted to string
before supplying to original
. What you think @rs?
I was refering to a different part of the code related to read only but interacting with lookup fields. Ignore my comment.
Yes, the storer engine should guarantee that in == out. We may add that to the doc of the interface of not already.
Okay, I think with storer
that this is already the case. And it seems a bit awkward we are deciding if a field is update or not, even before we validate it. That is why I think my initial proposal is correct, and we actually call FieldValidator.Validate
in schema.Prepare
, and we obtain that "internal" type for the field, then compare that with deepEqual
to the original
one. After that schema.Validate
, which calls again FieldValidator.Validate
on the fields will get already validated fields, and Validator
method will cost us just a type check. Here is example from time.go
:
func (v Time) parse(value interface{}) (interface{}, error) {
if s, ok := value.(string); ok {
for _, layout := range v.layouts {
if t, err := time.Parse(layout, s); err == nil {
value = t
break
}
}
}
if _, ok := value.(time.Time); !ok {
return nil, errors.New("not a time")
}
return value, nil
}
If we supply time.Time
as value
parameter, it will not do the conversion/parsing again.
Read-only fields in the current schema package are handled here (for PATCH):
rest-layer/rest/method_item_patch.go
Line 87 in c3ebffd
and for PUT:
rest-layer/rest/method_item_put.go
Line 64 in c3ebffd
But for your case they are not working?
... it seems a bit awkward we are deciding if a field is update or not, even before we validate it. That is why I think my initial proposal is correct, and we actually call
FieldValidator.Validate
inschema.Prepare
.
I have been considering if it's possible to get rid of Prepare all together... It's early stages, but in
https://github.com/searis/schema/blob/33e73d98b639bd7bb0f1ac3d883cb99d67b9fcfb/schema.go#L9 I have been toying with a possibly design for allowing to ignore Read-Only checks to be omitted without a separate Prepare call. Ignoring ReadOnly would then probably also be accessible to hooks. Don't know if the design will really work yet, and it may change substantially. This is just one evenings worth of work.
As I see it, the code snippets above will work only for id
field, because it is the only one residing in the resource path. Other Read-only
fields will work only if you don't PUT them at all, however if you PUT
them with the old value, rest-layer
will raise an error, where I don't think this is the correct behavior.
Now I am experience this issue, because the way JSON-Patch
works, basically I am updating the whole resource, along with any read-only
fields in it. However this might happen if someone tries to PUT an HTTP form, without stripping read-only
fields first.
Also as I stated, this affects not-changed
, non-read-only
fields as well. They go to changes
instead of base
.