How to compare

[ringsaturn]

Hi,

How to compare control and candidate output?

I’m trying use this repo to test refactor.

I didn’t get how to use it. Maybe a complete example will help me a lot.

Here is my code try to compare, I don’t know how to get the Observation instance. So the program raise an error

Traceback (most recent call last):
  File "laboratory_demo.py", line 52, in <module>
    experiment1.candidate))
  File "/usr/local/lib/python3.7/site-packages/laboratory/experiment.py", line 185, in compare
    if candidate.failure or control.value != candidate.value:
AttributeError: 'function' object has no attribute 'failure'

The code:

from laboratory import Experiment


def old_func(x, y, z=None):
    if z is None:
        return x+y
    else:
        return x+y


def new_func(x, y, z=None):
    if z is None:
        return x+y
    else:
        return x+y


x = 1
y = 10
z = None

experiment1 = Experiment()

experiment1.control(
    old_func,
    args=[x, y],
    kwargs={'z': z},
    context={'strategy': 1})
experiment1.candidate(
    new_func,
    args=[x, y],
    kwargs={'z': z},
    context={'strategy': 2})


print('real return:', experiment1.conduct())
print(experiment1.compare(experiment1.control, experiment1.candidate))

[Joe Alcorn]

Hi @ringsaturn

Laboratory compares the results for you when it passes a Result to the experiment's publish method, there's no need to call it yourself. You can override the publish method to print things out the results and you can tell if the comparison matches with result.match. Here are the publishing docs: https://laboratory-python.readthedocs.io/en/latest/publishing.html