Discuss $ref: json-schema patternProperties has no effect

[eKyNoX]

Steps to reproduce

I'm trying to define a collection of boolean values inside an object that can have an arbitrary name:

pairs: {
  type: 'object',
  patternProperties: {
    "^[A-Za-z]+$": {
      type: 'boolean'
    }
  },
  additionalProperties: false
}

This arbitrary name should be composed by latin letters only.
We see an example of patternProperties on feed property in this page:
https://code.tutsplus.com/tutorials/validating-data-with-json-schema-part-1--cms-25343

Unfortunately, when I make an API request, I can put whatever I want in that patternProperty.
Is it intended? Is it implemented?

Thanks!

System configuration

NodeJS version: 10.15.1

Operating System: Ubuntu

[Eddyystop]

Have you added a validation hook to the service?

[eKyNoX]

Yes, other validations works without problems:

(...)
    create: [validateCreate()],
    update: [validateUpdate()],
    patch: [validatePatch()],
(...)

[Eddyystop]

Please check that the patternProperties prop has been copied to the serviceName.validate.js module.

If it does then the common hook validateShema uses the popular "ajv" repo for json validation.

You can check its docs for the support. Maybe even an optional plugin.

Please let me know what you find out.

[eKyNoX]

Hello again.

For some reason, restarted the app, and now validation works on patternProperties.

Thanks a lot!
I'm closing here.

[eKyNoX]

Sorry to re-open here:
I'm facing the same problem with patternProperties but this time on top-level.

On my "service".schema.ts, closed to the top level elements, I need to place a patternProperties keyword for the same purpose, having arbitrary names:

let schema = {
  title: 'Exchanges',
  description: 'Exchanges database.',

  // Added by me
  type: 'object',
    patternProperties: {
      "^[A-Za-z]+$": { type: 'boolean' }
    },
    additionalProperties: false,
  // end

  required: [
    (...)
  ],

  properties: {
    (...)
  }
}

But results are I can pass anything I want in my request. I'm not sure if feathers-plus-generator accept patternProperties on top like this.

Thanks again!

[Eddyystop]

The generated code is

// Define the model using JSON-schema
let schema = {
  // !<DEFAULT> code: schema_header
  title: 'Nedb1',
  description: 'Nedb1 database.',
  // !end
  // !code: schema_definitions // !end

  // Required fields.
  required: [
    // !code: schema_required // !end
  ],
  // Fields with unique values.
  uniqueItemProperties: [
    // !code: schema_unique // !end
  ],

  // Fields in the model.
  properties: {
    // !code: schema_properties
    id: { type: 'ID' },
    _id: { type: 'ID' },
    nedb2Id: { type: 'ID' },
    // !end
  },
  // !code: schema_more // !end
};

Why would you not add your custom code in // !code: schema_definitions // !end ?

[eKyNoX]

Sorry for late reply. Yes I tried:

(...)
title: 'Exchanges',
  description: 'Exchanges database.',
  // !end
  // !code: schema_definitions
  patternProperties: {
    "^[a-zA-Z]+$": {
      type: 'object',
      properties: {
        key: { type: 'boolean' },
      },
      additionalProperties: false
    }
  },
  // !end

  // Required fields.
  required: [
(...)

I tried validation of a key that doesn't respect the regexp, give a key with data type other than boolean and as well added more parameters close to the "key" one (that should be alone), all passed and none of my parameters got validation on it.

I went to see the json-schema ajv guys and read some more docs. Having top level patternProperties is not a problem normally. So they think it can be related to what uses ajv.

It can be problematic as I went back on my project to use a big global json-schema with many levels inside only one service (which work perfectly), but if I try to split logical responsibilities and include two other services that have arbitrary key names, currently, I can't.

Thanks a lot eddy about your thoughts and participation here.

[Eddyystop]

You'll have to post the whole json-schema for your file.

[eKyNoX]

Here is the full schema as it works today from the 'configs' service:

let schema = {
  // !<DEFAULT> code: schema_header
  title: 'Configs',
  description: 'Configs database.',
  // !end
  // !code: schema_definitions // !end

  // Required fields.
  required: [
    // !code: schema_required
    'name',
    'pairs',
    'exchanges',
    'strategies',
    'bot',
    'GUI',
    'ws',
    'createdAt',
    'updatedAt'
    // !end
  ],
  // Fields with unique values.
  uniqueItemProperties: [
    // !code: schema_unique // !end
  ],

  // Fields in the model.
  properties: {
    // !code: schema_properties
    id: { type: 'ID' },
    name: {},
    pairs: {
      type: 'object',
      patternProperties: {
        "^[a-zA-Z0-9-_]+$": {
          type: 'object',
          patternProperties: {
            "^[a-zA-Z0-9-_]+$": {
              type: 'object',
              properties: {
                strategy: {},
                enabled: { type: 'boolean' },
                override: { type: 'object' }
              },
              required: [
                'strategy',
                'enabled',
                'override'
              ],
              additionalProperties: false,
            }
          },
          additionalProperties: false
        }
      },
      additionalProperties: false
    },
    strategies: {
      type: 'object',
      patternProperties: {
        "^[a-zA-Z0-9-_]+$": {
          type: 'object',
          properties: {
            ARBITRAGE_GAIN: { type: 'number' },
            TRADING_FEES: { type: 'number' },
            PANIC_SELL: { type: 'boolean' }
          },
          required: [
            'ARBITRAGE_GAIN',
            'TRADING_FEES',
            'PANIC_SELL'
          ],
          // We will have more params here later, regarding arbitrage currencies (BTC, USDT, USD, etc...)
          additionalProperties: true
        }
      }
    },
    exchanges: {
      type: 'object',
      patternProperties: {
        "^[a-zA-Z0-9-_]+$": {
          type: 'object',
          properties: {
            key: {},
            secret: {},
            masterkey: {},
            mastersecret: {},
            delay: { type: 'number' }
          },
          required: [
            'key',
            'secret',
            'masterkey',
            'mastersecret',
            'delay'
          ],
          additionalProperties: false
        }
      },
      additionalProperties: false
    },
    bot: {
      type: 'object',
      properties: {
        debug: { type: 'boolean' },
        gui: { type: 'boolean' },
        period_storage_ticker: { type: 'number' },
        interval_ticker_update: { type: 'number' },
        timeout_buy: { type: 'number' },
        timeout_sell: { type: 'number' },
        WATCH_MODE: { type: 'boolean' },
        TELEGRAM_ENABLED: { type: 'boolean' },
        TELEGRAM_NICK: {},
        TOKEN: {},
        chat_id: { type: 'number' }
      },
      required: [
        'debug',
        'gui',
        'period_storage_ticker',
        'interval_ticker_update',
        'timeout_buy',
        'timeout_sell',
        'WATCH_MODE',
        'TELEGRAM_ENABLED',
        'TELEGRAM_NICK',
        'TOKEN',
        'chat_id'
      ]
    },
    GUI: {
      type: 'object',
      properties: {
        enabled: { type: 'boolean' },
        start: { type: 'boolean' },
        port: { type: 'number' },
        https: { type: 'boolean' },
        key: {},
        cert: {},
        networktraffic: { type: 'boolean' },
        authentication: {
          type: 'object',
          properties: {
            login: { type: 'boolean' },
            twoFA: { type: 'boolean' }
          },
          required: [
            'login',
            'twoFA'
          ]
        },
        notifications: {
          type: 'object',
          properties: {
            trade: { type: 'boolean' },
            callback: { type: 'boolean' },
            error: { type: 'boolean' }
          },
          required: [
            'trade',
            'callback',
            'error'
          ]
        }
      },
      required: [
        'enabled',
        'start',
        'port',
        'https',
        'key',
        'cert',
        'networktraffic',
        'authentication',
        'notifications'
      ]
    },
    ws: {
      type: 'object',
      properties: {
        port: { type: 'number' },
        clientport: { type: 'number' },
        hostname: {}
      },
      required: [
        'port',
        'clientport',
        'hostname'
      ]
    },
    createdAt: {},
    updatedAt: {}
    // !end
  },
  // !code: schema_more // !end
}

I want to extract three parts into 3 other services from this schema:

1. pairs
2. strategies
3. exchanges

So from this schema, I "extract" these parts. and it becomes:

    pairs: {
      type: 'object'
    },
    strategies: {
      type: 'object'
    },
    exchanges: {
      type: 'object'
    },

Then, this example below, like on my previous comment, show how I proceed inside this 'exchanges' new service:

let schema = {
  // !<DEFAULT> code: schema_header
  title: 'Exchanges',
  description: 'Exchanges database.',
  // !end
  // !code: schema_definitions
  patternProperties: {
    "^[a-zA-Z]+$": {
      type: 'object',
      properties: {
        key: { type: 'boolean' },
      },
      additionalProperties: false
    }
  },
  // !end

  // Required fields.
  required: [
    // !code: schema_required // !end
  ],
  // Fields with unique values.
  uniqueItemProperties: [
    // !code: schema_unique // !end
  ],

  // Fields in the model.
  properties: {
    // !code: schema_properties // !end
  },
  // !code: schema_more // !end
}

For now I keep things simple. Something similar is needed on pairs and strategies services.
Later, it is planed when I call the parent service (configs), I'll use fastJoin to join results from the child services.

[eKyNoX]

I just replied :)

[Eddyystop]

As far as I know JSON-schema only allows individual fields to refer to "definitions" maintained elsewhere. These "definitions" may either be in the same JSON-schema or in external files. Your example does not follow this syntax.

You can read up on this at https://github.com/bojand/json-schema-deref-sync (the repo cli+ uses to deref the service JSON-schemas) and at https://generator.feathers-plus.com/guide/#ref-modularizing-definitions