Prism Client not validating request and response from Server despite setting mock: false.

[Xi Sheng]

Context

I am using Prism Client in a node js application to perform API contract validation. However, I noticed that the prism client always return the mock data from the OAS despite setting mock: false in the client configuration. I believe that this also causes the prism client to validate the mock response/examples generated from the OAS instead of validating the actual responses from the server.

Current Behavior

Prism client is returning the mock responses / examples and validating the mock responses / examples generated from the OAS.
Based on the sample code below, there is an intentional error in the contract whereby used field should be boolean type, not string type. However, the Prism client is returning the mock responses from OAS and also not detecting any contract violation.

Expected Behavior

Prism client should return the actual responses from the server, not mock from the OAS. Prism client should also validate the responses from the server, not the OAS.

Here's an example of the validation error from using Prism CLI (not Prism Client)
Violation: response.body.0.used Response body property 0.used must be string

Possible Workaround/Solution

Steps to Reproduce

Here's a sample code created for Prism Client to purposely detect the contract violation.

Prism client prism.js:

const { getHttpOperationsFromSpec } = require('@stoplight/prism-cli/dist/operations');
const { createClientFromOperations } = require('@stoplight/prism-http/dist/client');
const { URL } = require('url');

async function main() {
    const operations = await getHttpOperationsFromSpec('cat_facts.yaml');
    
    const client = createClientFromOperations(operations, {
        mock: false,
        validateRequest: true,
        validateResponse: true,
        checkSecurity: false,
        errors: true,
        upstream: new URL("https://cat-fact.herokuapp.com/"), 
        upstreamProxy: undefined,
      });
    
    client.request('https://cat-fact.herokuapp.com/facts', { method: 'get' }).then(console.log);
}

main().catch((error) => {
    console.error(error);
});

Sample OAS cat_facts.yaml:

openapi: 3.0.0
info:
  title: Cat Facts API
  version: 1.0.0
servers:
  - url: https://cat-fact.herokuapp.com
paths:
  /facts:
    get:
      summary: Get Cat Facts
      responses:
        '200':
          description: Successful response
          content:
            application/json:
              schema:
                type: array
                items:
                  type: object
                  properties:
                    status:
                      type: object
                      properties:
                        verified:
                          type: boolean
                        sentCount:
                          type: integer
                      required:
                        - verified
                        - sentCount
                    _id:
                      type: string
                    user:
                      type: string
                    text:
                      type: string
                    __v:
                      type: integer
                    source:
                      type: string
                    updatedAt:
                      type: string
                      format: date-time
                    type:
                      type: string
                    createdAt:
                      type: string
                      format: date-time
                    deleted:
                      type: boolean
                    used:
                      type: string
                  required:
                    - status
                    - _id
                    - user
                    - text

Environment

• Version used:
  "@stoplight/prism-cli": "^5.4.0"

[Chelsea Hohmann]

@xeroxis-xs it looks like you're using (and our docs show) outdated configuration options for the Prism Client.

mock config option no longer controls whether the client is in mock or proxy mode. isProxy controls this.

So if you update your prism.js code to be the following, the prism client behaves as you expect:

const { getHttpOperationsFromSpec } = require('@stoplight/prism-cli/dist/operations');
const { createClientFromOperations } = require('@stoplight/prism-http/dist/client');
const { URL } = require('url');

async function main() {
    const operations = await getHttpOperationsFromSpec('cat_facts.yaml');
    
    const client = createClientFromOperations(operations, {
        mock: { dynamic: false },
        validateRequest: true,
        validateResponse: true,
        checkSecurity: false,
        errors: true,
        upstream: new URL("https://cat-fact.herokuapp.com/"), 
        upstreamProxy: undefined,
        isProxy: true,
      });
    
    client.request('https://cat-fact.herokuapp.com/facts', { method: 'get' }).then(console.log);
}

main().catch((error) => {
    console.error(error);
});

I've put up a PR to update our documentation to reflect the correct configuration operations as well.