FieldValue.increment() is not working as expected on subfields of a map

Question

FieldValue.increment() is not working as expected on subfields of a map

JamieCurnow opened this issue a year ago · comments

Thanks for stopping by to let us know something could be better!

PLEASE READ: If you have a support contract with Google, please create an issue in the support console instead of filing on GitHub. This will ensure a timely response.

Is this a client library issue or a product issue?
I think it's probably something that should be at-least handled by this library.
Did someone already solve this?

Search the issues already opened: https://github.com/googleapis/nodejs-firestore/issues ✅
Search the issues on our "catch-all" repository: https://github.com/googleapis/google-cloud-node ✅
Search or ask on StackOverflow (engineers monitor these tags): http://stackoverflow.com/questions/tagged/google-cloud-platform+node.js ✅

Do you have a support contract?
Nope

If the support paths suggested above still do not result in a resolution, please provide the following details.

Environment details

OS: Mac
Node.js version: 18.16.0
npm version: 9.5.1
@google-cloud/firestore version: ^6.8.0

Steps to reproduce

The FieldValue.increment() function does not work as expected on subFields of a map value. Given this data structure of the object in the database:

{
  amount: 1,
  map: {
    amount: 1
  }
}

If I run this code:

await docRef.update({
  amount: FieldValue.increment(2),
  sub: {
    amount: FieldValue.increment(2)
  }
})

The data in the database ends up being:

{
  amount: 3,
  map: {
    amount: 2
  }
}

It would seem that incrementing using this object syntax does increment on top-level keys, but "sets" the value of subfields of maps to the exact increment value, rather than incrementing.

To me this feels like the incorrect behaviour - I would firstly expect that map.amount would be 3, OR expect the SDK to throw an error as it does when using FieldValue.delete() on a subfield of a map in this way.

The "workaround" is to use dot-notation strings as keys on the top level of the object like so:

await docRef.update({
  amount: FieldValue.increment(2),
  'sub.amount': FieldValue.increment(2)
})

This does work as expected and the data in the db ends up being:

  amount: 3,
  map: {
    amount: 3
  }
}

This issue has been observed in SO here:
https://stackoverflow.com/questions/56427582/how-to-use-fieldvalue-increment-on-the-property-of-an-object-type-document-fie

It feels like this warrants an issue and a change to how the sdk handles this to bring it in line with how it handles FieldValue.delete() on subfields of a map where for example, this code throws an error:

await docRef.update({
  amount: FieldValue.increment(2),
  sub: {
    amount: FieldValue.delete()
  }
})

Error:

Update() requires either a single JavaScript object or an alternating list of field/value pairs that can be followed by an optional precondition. Value for argument "dataOrField" is not a valid Firestore value. FieldValue.delete() must appear at the top-level and can only be used in update() or set() with {merge:true} (found in field "sub.amount")

It took a wilte to figure out what was going on here, and can be especially confusing when incrementing from 0 because it seems like the first request works, but then subsequent requests do not behave as expected. Eg:

Db is:

{
  amount: 0,
  map: {
    amount: 0
  }
}

Run:

await docRef.update({
  amount: FieldValue.increment(1),
  sub: {
    amount: FieldValue.increment(1)
  }
})

Db is now:

{
  amount: 1,
  map: {
    amount: 1
  }
}

Which seems like it works... But if you run the exact same code again:

await docRef.update({
  amount: FieldValue.increment(1),
  sub: {
    amount: FieldValue.increment(1)
  }
})

Now the data in the db ends up being incorrect:

{
  amount: 2,
  map: {
    amount: 1
  }
}

Mark Duckworth · Answer 1 · Wed Dec 27 2023 04:34:03 GMT+0800 (China Standard Time)

@JamieCurnow,

Thanks for bringing this to our attention. I reproduced the behavior and I agree that it is confusing and the throwing behavior of FieldValue.delete() on a nested property adds to that confusion.

However, this call is technically correct. It's just stating that sub should be replaced with a new object, and then increment the starting value of sub.amount by 1.

await docRef.update({
  sub: {
    amount: FieldValue.increment(1)
  }
});

The solution in your case is to use the dot notation as you already stated.

await docRef.update({
  'sub.amount': FieldValue.increment(1)
});

I'll review this issue with the team after the holidays to see if we can reduce the confusion. The solution might be raising an error, logging a warning, or adding some additional documentation.

Jamie Curnow · Answer 2 · Wed Dec 27 2023 05:42:40 GMT+0800 (China Standard Time)

Hi @MarkDuckworth, that totally makes sense why this is happening now after your explanation, so thank you so much for the reply 🙌

It's super interesting that a set() with a { merge: true } works differently than an update in this situation. I think I've seen another issue that addresses that... 🤔

In the actual code, I'm changing from a set() call to an update call because I want to fail if the document doesn't already exist. So the solution for me is to just convert all nested object changes into dot-notation strings, though in doing that, I loose type-safety.

Is there any support for a { merge: true } on an update() call in the pipeline that would help this transition in the future to bring the two methods more inline? Or is that something that is just technically not possible under the Firestore hood?

Thanks again!

Mark Duckworth · Answer 3 · Wed Dec 27 2023 07:02:50 GMT+0800 (China Standard Time)

I just reviewed set(..., {merge: true}) and I can see that the message sent to the backend is slightly different than when using update(...). The set+merge call is requesting that sub should be merged with object {}, and then increment the existing value of sub.amount by 1.

await docRef.set({
  sub: {
    amount: FieldValue.increment(1)
  }
},
{ merge: true });

Is there any support for a { merge: true } on an update() call in the pipeline that would help this transition in the future to bring the two methods more inline? Or is that something that is just technically not possible under the Firestore hood?

This is not currently supported in the SDK from what I can tell. We will discuss allowing this in the SDK.

So the solution for me is to just convert all nested object changes into dot-notation strings, though in doing that, I loose type-safety.

update() can enforce type safety even with dot notation, except when using indexed properties.

type MyType = {
    sub: {
        amount: number
    }
};
let update: UpdateData<MyType> = {
    'sub.amount': increment(1) // must be number | FieldValue | undefined
};
await docRef.update(update);

An alternative solution would be to use a Transaction that checks for existence with a get and then performs the set. The downside to this is that it involves two operations, a read and a write.