FeatureProbe / server-sdk-specification

Unified Test Cases for All Server Side SDKs

FeatureProbe server side SDK Specification

We want FeatureProbe to be available for each popular programming languages, and still we want each of these SDKs to have a consistent behavior.

This project defined expected result for a given FeatureProbe environment. And each server side SDK implementation should adhere to these specs.

File structure

Specs are located in /spec folder, in JSON format. Each language SDK implementation have to construct a test framework to load and run these specs with their own language and test framework conventions.

Spec description

A spec file has the following shape:

{
  "tests": [
    {
      "scenario": "Name of this scenario",
      "fixture": {
        "segments": {},
        "toggles": {}
      },
      "cases": [
        {
          "name": "name of this test case.",
          "user": {
            "key": "user id",
            "customValues": [
              {
                "key": "email",
                "value": "[e-mail]"
              }
            ]
          },
          "function": {
            "name": "bool_value",
            "toggle": "toggle_to_test",
            "default": true
          },
          "expectResult": {
            "value": true,
            "noRuleIndex": true,
            "reason": "default"
          }
        }
      ]
    }
  ]
}

Fields description:

• tests : list of all test scenarios in this spec file.
  • scenario : each spec file contains several scenarios, this is the name of the scenario.
  • fixture : each scenario share this fixture, fixture represent a FeatureProbe environment, which use the same format as SDK pulled from FeatureProbe server.
    • segments : user segments in this environment.
    • toggles : feature toggles in this environment.
  • cases : test cases in this scenario.
    • name : name of this case.
    • user : use info in this struct to construct a FPUser.
      • key : user id of FPUser.
      • customValues : other key-value pairs to use with FPUser.with.
    • function : FeatureProbe function to test.
      • name : function name.
      • toggle : toggle name, the first parameter to call FeatureProbe function.
      • default : default value, the last parameter to call FeatureProbe function.
    • expectResult : expected result of function call.
      • value : returned variation value.
      • reason : reason in FPDetail, if called _detail functions.
      • ruleIndex : ruleIndex in FPDetail.
      • noRuleIndex : should not return ruleIndex in FPDetail.
      • version : version in FPDetail.