Validate keys

[Martin Klepsch]

Hi! I have some data that looks like this:

{"abc" {:stat "efg"}}
{"xyz" {:stat "123"}}

Now I tried writing a validator for it but failed so far. Failed attempts include:

{(v/string) {:stat (v/string)}}

{:* (v/string), :??? {:stat (v/string)}}

For :* the docs say:

If a specs map contains the wildcard key :*, then that key's spec is applied first to all keys in the data map at the wildcard's parent path.

Now reading that it seems somewhat close to what I want to achieve but how do I designate a validator for all keys in the map?

[Martin Klepsch]

Huh, I found something that works:

(def string-keys
  "Returns validation spec to ensure key is a string."
  (v/validator (fn [k _] (string? (last k))) "must be a string key"))

Feedback would be appreciated. In general I'm surprised there's no general way to say "this maps keys must match this validator".

Thanks for this library and it's very readable code. I actually understand what's going on after reading it.
Magic! ✨

[Karsten Schmidt]

Hi @martinklepsch - this is a case of maybe unfortunate wording... when the docs talk about "applying to a key" it actually means "applying to a key's value"). Will have to reword this, since I now see this isn't immediately obvious. All of the bundled validators in this lib currently only deal with validating the values, not the keys directly. However, you could write a custom validator which instead validates parts of the path (instead of the value). The wildcard :* is simply meant to apply to the values of all keys at the path/cursor the wildcard appears in the validation map. Since the validation map has the same structure as your data map, am not quite sure how you would marry this with key validation itself (in a generic way)...

However, not to throw the towel...

Hah! You just beat me to it with your new reply :) Was just going to propose the same (only using the faster peek (instead of last)). In theory, you could also add a correction part to your validator, which e.g. silently convert keywords into strings, or replace with UUIDs etc. At the moment these corrected values would be used as the new value for the key, not the key itself. TBH, I never needed key validation so far, but I can see how this would be useful indeed. Always happy to receive a PR on that matter... :) Thanks & good luck!

[Karsten Schmidt]

To elaborate a bit more why there isn't any key validation itself. It's simply that I often just use this lib to validate defrecords and basically define a schema for its values. Another major usecase is documented here: https://github.com/thi-ng/trio/blob/master/src/entities.org - there it's about validating & mapping RDF subgraphs to hashmaps...

[Martin Klepsch]

Thanks for the quick reply! I see why key validation probably does not make as much sense and probably will revise my model to not need key validation.

I'll try thinking of a way to add key validation. It would be nice to re-use the existing validators but I don't yet see how that could work.

[Karsten Schmidt]

Just thinking loud... how about adding a special key to a validation map which specifies key validators for that level, e.g.

(v/validate
  {"abc" {:stat "efg"}}
  {::v/keys {:* (v/string)} ;; all keys at that level must be strings
   :*       {:stat (v/string)}})

IMHO that wouldn't be too inelegant and neither would require any changes to existing validators... Thoughts?

[Martin Klepsch]

I was thinking about something along those lines as well. It makes specs a bit harder to read and introduces another "magic" keyword though. I wonder if :* should then also have the same ::v/ prefix and potentially a more descriptive name? This would obviously break existing validators unless there's some deprecation mechanism.

[Karsten Schmidt]

Why would renaming :* break validators? Unless I'm missing something, this should only break existing validation maps in user code, but not the validators themselves...

Another, possibly cleaner, alternative to using ::v/keys would be to just add a validate-keys fn and keep the current impl as is (or rename to validate-values and keep validate as alias w/ deprecation msg). That would also make the error reporting maps easier to produce & understand (since they would also need the ::v/keys...)

In validate-keys the same kind of validation maps are used, but stricly applied to keys only (not their vals). Having these two fns would keep the different concerns nicely separated, but mean slightly more work for users requiring validation of both keys & vals. Could provide some sugar for that, though.

[Martin Klepsch]

Your example in #5 made me think about another situation:

(v/validate
   {:x [nil :b :c]}
   {:x {:* [(v/required) (keyword-or-nil)]}})

Because the wildcard is used here the required validator reaches into each path, reporting an error if any of the values is nil. Admittedly this is a contrived example but it made me wonder if maybe there are other options besides magic keys. Maybe there could be some kind of meta validators that would allow flexible descriptions like (v/all [(v/keyword)]). This would make it harder to accurately reflect the path to the error-causing item in error map but maybe that's a considerable tradeoff?

[Martin Klepsch]

this should only break existing validation maps in user code, but not the validators themselves...

That's what I meant. 😄