jmespath / jmespath.py

I'm trying to do a search like:

from jmespath import search
data = {
    "grid": [
        {"col1": "a", "col2": None},
        {"col1": "b", "col2": None},
        {"col1": "c", "col2": "x"},
        {"col1": "d", "col2": "y"},
    ]
}
col1_data = search("grid[*].col1", data)  # gets ["a", "b", "c", "d"]
col2_data = search("grid[*].col2", data)  # gets ["x", "y"], expecting [None, None, "x", "y"]

If the col2 element did not exist, I think the current behavior makes sense. However, I think that search by default shouldn't also do filtering.

For reference:

jmespath.py/jmespath/visitor.py

Lines 134 to 138 in 599b0f7

    
           def visit_field(self, node, value): 
        
               try: 
        
                   return value.get(node['value']) 
        
               except AttributeError: 
        
                   return None

jmespath.py/jmespath/visitor.py

Lines 279 to 280 in 599b0f7

    
           if current is not None: 
        
               collected.append(current)

Thoughts ?

Just came across this myself. Would a PR be welcome?

@zalmane @ntextreme3 you might be interested to learn that JMESPath Community supports this scenario - albeit currently in an indirect way.

@zalmane @ntextreme3 you might be interested to learn that JMESPath Community supports this scenario - albeit currently in an indirect way.

That's completely different that what the OP asked...

The questions was how search can support None values as well, currently they are discarded.

That's completely different that what the OP asked...
The questions was how search can support None values as well, currently they are discarded.

Hum 🤔 then maybe I’m missing something…

search( map(&col2, grid), data ) -> [ null, null, "x", "y" ]

If you need more downstream processing, you can definitely distinguish between missing values vs null values by converting them temporarily for processing:

map(&col2 && col2 || 'value-was-null', grid)-> [ "value-was-null", "value-was-null", "x", "y" ]

Sure, this is kludgey, but that is definitely something that’s possible today.

	def visit_field(self, node, value):
	try:
	return value.get(node['value'])
	except AttributeError:
	return None

Differentiate between missing values and null values