devopstoday11 / xp-ml-experimentation

ML Experimentation Platform

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

xp

License

Overview

XP helps with designing and managing experiment configurations in a safe and holistic manner. At runtime, these configurations can be used to run experiments and generate treatments.

The API is broken down into 2 services:

  • Management Service: Used to configure experiments
  • Treatment Service: Used to obtain the treatment configuration from active experiments

Architecture

The following diagram summarizes the interaction between the various components of the system. The Management service is packaged together with the UI, while the Treatment service is published separately - this is so that the Treatment service, which is lightweight, can be deployed and made to scale independently.

XP Architecture

Why XP?

  • Reliable - Inherent fault-detection rules help create experiments without conflicts.
  • Customizable - Every service has unique requirements. XP allows for defining flexible segmentation and experiment validation rules.
  • Fast - 99p server-side latency (excluding the network latency between the calling service and XP) averages around 1 ms.
  • Observable - Resource utilization, treatment assignment and performance metrics are available on Prometheus

Development Environment

Quick Start

a. Setup MLP API

Instructions as described in README of https://github.com/gojek/mlp.

b. Starting XP

Prior to starting XP, we'll need to ensure correct MLP API is correctly set in the config file, i.e management-service/config/example.yaml, and setting the correct MLPConfig::URL value.

# Start Management Service
make mgmt-svc

# Start Treatment Service
make treatment-svc

# Exploring Swagger-UI
make swagger-ui

make mgmt-svc runs the following:

  1. make local-db
    • Setup a local DB for storing experiment configurations
  2. make local-authz-server

To test authorization for Management Service locally, make the following changes before starting Management Service:

  • A sample policy exists at keto/policies/example_policy.json. It can be modified.
  • Set AuthorizationConfig.Enabled=true in the config file that's being used
  • Issue requests to the app with the header User-Email: test-user@gojek.com

c. Using XP

To use the XP Go modules as API dependencies, replace directives need to be used when consuming both the Management and Treatment API modules since the API modules uses local relative path.

replace github.com/caraml-dev/xp/clients => github.com/caraml-dev/xp/clients v0.0.0
replace github.com/caraml-dev/xp/common => github.com/caraml-dev/xp/common v0.0.0
replace github.com/caraml-dev/xp/management-service => github.com/caraml-dev/xp/management-service v0.0.0
replace github.com/caraml-dev/xp/treatment-service => github.com/caraml-dev/xp/treatment-service v0.0.0

Contributing

XP is still under active development. Please have a look at our contributing and development guides if you want to contribute to the project:

About

ML Experimentation Platform

License:Apache License 2.0


Languages

Language:Go 71.6%Language:JavaScript 23.4%Language:Python 3.0%Language:Makefile 0.7%Language:SCSS 0.4%Language:Shell 0.4%Language:Smarty 0.3%Language:HTML 0.1%Language:Dockerfile 0.1%