schemathesis / schemathesis

Supercharge your API testing, catch bugs, and ensure compliance

Home Page:https://schemathesis.readthedocs.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Schemathesis is generating invalid payload when we use examples and value objects in openapi spec

gnanendra-bogireddy opened this issue · comments

Checklist

  • I checked the FAQ section of the documentation
  • I looked for similar issues in the issue tracker
  • I am using the latest version of Schemathesis

Describe the bug

Schemathesis is generating invalid payload when we use examples and value objects in openapi spec
it is considering values and summary, description tags mentioned in examples tag as individual objects and appending to other objects mentioned in values tag.

To Reproduce

🚨 Mandatory 🚨: Steps to reproduce the behavior:

  1. Run this command 'st run openapi.yaml --base-url https://reqres.in/api --request-tls-verify=false --show-trace --cassette-path="cassette.yaml" --debug-output-file="debugOutputFile.yaml" -c all'
  2. See error

body:
encoding: 'utf-8'
string: "{"summary": "summary", "value": {"username": "username1", "email": "john.doe@email.com", "password": "password1"}}"

Please include a minimal API schema causing this issue:

openapi: 3.0.3
info:
  title: Reqres - OpenAPI 3.0
  version: 1.0.0
servers:
  - url: 'https://reqres.in/api'
paths:
  /register:
    post:
      tags:
        - register
      operationId: register
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/Register'
            examples:
              objectExample:
                $ref: '#/components/examples/objectExample'
      responses:
        '200':
          description: Successful operation
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/RegisterResponse'
        '400':
          description: Invalid ID supplied
        '404':
          description: Object not found
        '422':
          description: Validation exception
        '500':
          description: Internal server error
components:
  schemas:
    Register:
      type: object
      properties:
        username:
          type: string
          example: username
        email:
          type: string
          example: john.doe@email.com
        password:
          type: string
          example: password
    RegisterResponse:
      type: object
      properties:
        error:
          type: string
          example: string
  examples:
    objectExample:
      summary: summary
      value:
        username: username1
        email: john.doe@email.com
        password: password1
    objectExample2:
      value:
        username: username2
        email: john.doe@gmail.com
        password: password2

Expected behavior

body:
encoding: 'utf-8'
string: "{"username": "username1", "email": "john.doe@email.com", "password": "password1"}"

Environment

- OS: [ Windows]
- Python version: [e.g. 3.11]
- Schemathesis version: [e.g. 3.29.2]
- Spec version: [e.g. Open API 3.0.3]

Additional context

Cassette test case which has invalid payload


  • id: '3'
    status: 'SUCCESS'
    seed: '125427259992210988850556979699645139052'
    thread_id: 27652
    correlation_id: 'f6f85799f9504d0d837db6c389cf43e9'
    data_generation_method: 'positive'
    elapsed: '0.34498'
    recorded_at: '2024-06-04T11:23:44.739353'
    checks:
    • name: 'not_a_server_error'
      status: 'SUCCESS'
      message: ~
    • name: 'status_code_conformance'
      status: 'SUCCESS'
      message: ~
    • name: 'content_type_conformance'
      status: 'SUCCESS'
      message: ~
    • name: 'response_headers_conformance'
      status: 'SUCCESS'
      message: ~
    • name: 'response_schema_conformance'
      status: 'SUCCESS'
      message: ~
      request:
      uri: 'https://reqres.in/api/register'
      method: 'POST'
      headers:
      "User-Agent":
      • "schemathesis/3.29.2"
        "Accept-Encoding":
      • "gzip, deflate"
        "Accept":
      • "/"
        "Connection":
      • "keep-alive"
        "X-Schemathesis-TestCaseId":
      • "RzQIpj"
        "Content-Type":
      • "application/json"
        "Content-Length":
      • "114"
        body:
        encoding: 'utf-8'
        string: "{"summary": "summary", "value": {"username": "username1", "email": "john.doe@email.com", "password": "password1"}}"
        response:
        status:
        code: '400'
        message: "Bad Request"
        headers:
        "Date":
      • "Tue, 04 Jun 2024 05:53:44 GMT"
        "Content-Type":
      • "application/json; charset=utf-8"
        "Content-Length":
      • "37"
        "Connection":
      • "keep-alive"
        "Report-To":
      • "{"group":"heroku-nel","max_age":3600,"endpoints":[{"url":"https://nel.heroku.com/reports?ts=1717480424&sid=c4c9725f-1ab0-44d8-820f-430df2718e11&s=cEvEWz70%2FpOcnfDDkx50LBky7Oxqxvb1Gir2YxztC8U%3D"}]}"
        "Reporting-Endpoints":
      • "heroku-nel=https://nel.heroku.com/reports?ts=1717480424&sid=c4c9725f-1ab0-44d8-820f-430df2718e11&s=cEvEWz70%2FpOcnfDDkx50LBky7Oxqxvb1Gir2YxztC8U%3D"
        "Nel":
      • "{"report_to":"heroku-nel","max_age":3600,"success_fraction":0.005,"failure_fraction":0.05,"response_headers":["Via"]}"
        "X-Powered-By":
      • "Express"
        "Access-Control-Allow-Origin":
      • "*"
        "Etag":
      • "W/"25-Ik70kof1lbc973ShN5FoVPxYn7A""
        "Via":
      • "1.1 vegur"
        "CF-Cache-Status":
      • "DYNAMIC"
        "Server":
      • "cloudflare"
        "CF-RAY":
      • "88e5aa0c8a6317b0-MAA"
        body:
        encoding: 'utf-8'
        string: "{"error":"Missing email or username"}"
        http_version: '1.1

please check discussion on GitHub - #2225

Thanks for your detailed report! Will be fixed soon