Datastore, ndb, and querying for existence of a repeated field in alias computation.

Question

Datastore, ndb, and querying for existence of a repeated field in alias computation.

michaelkedar opened this issue 2 months ago · comments

Making this issue to document this problem we've run into.

Alias computation was using osv.Bug.query(osv.Bug.aliases != '') to find all bugs that have at least one alias in the aliases field.
The condition was originally being interpreted as aliases < '' OR aliases > ''
In 2.3.0, the ndb library started using server-side != instead
- Datastore emulator does not support this operation, causing the tests to fail when trying to update in #2084
- See issue googleapis/python-ndb#962

Additionally, queries with the OR operation de-duplicates results in ndb (see the note above the linked heading), which is why using != (i.e. < OR >) fixed the 'bug' in #1778

Currently, my solution is to replace osv.Bug.query(osv.Bug.aliases != '') with osv.Bug.query(ndb.OR(osv.Bug.aliases > '', osv.Bug.aliases < '')) to restore the original behaviour.