avalade / prop-testing

A presentation on property based testing

Property Based Testing Lunch and Learn

Property Based Testing

Writing tests first forces you to think about the problem you’re solving. Writing property-based tests forces you to think way harder.

• Jessica Kerr (@jessitron)

Let’s talk about tests, baby

function symDiff<T>(
  x: Array<T>,
  y: Array<T>
): Array<Array<T>> {
  return [
    _.flatten(x.filter((a) => !_.contains(y, a))),
    _.flatten(y.filter((b) => !_.contains(x, b)))
  ];
}

What unit tests do you write?

• x contains no elements and y contains no elements

• x contains an element and y contains no elements

• x contains no elements and y contains an element

• x contains the same elements as y

• x contains some elements but not all the elements of y

• y contains some elements but not all the elements of x

How many tests do you need!?!?

We could use fixtures

const emptyArrayOfNumbers: Array<number> =
  [];
const arrayWithOneNumber = [1];
const myAdditionalNumber = 2;
const arrayWithAnAdditionalNumber =
  [...arrayWithOneNumber,
   myAdditonalNumber];

Then test

describe('symDiff', () => {
  it('should return two empty arrays if both inputs are empty', () => {
    const [a, b] = symDiff(emptyArrayOfNumbers, emptyArrayOfNumbers);
    expect(a).empty();
    expect(b).empty()
  });
  it('should return all of x if y is empty', () => {
    const [a, b] = symDiff(arrayOfOneNumber, emptyArrayOfNumbers);
    expect(a).eql(arrayOfOneNumber);
    expect(b).empty();
  })
});

Let’s think about this a different way

What are the properties of the function?

What is a property?

• If our y is an empty array, the first element of our return will be x
• If our x is an empty array, the second element of our return will be y
• If our x and y are equal, we get back two empty arrays

Let’s code that

describe('symDiff', () => {
  it('satisfies the identity property for the first argument', () => {
    const anArrayOfNumbers = someArrayOfNumbers();
    const [a, b] = symDiff(anArrayOfNumbers, []);
    expect(a).eql(anArrayOfNumber);
  });
  it('satisfies the identity property of the second argument', () => {
    const anArrayOfNumbers = someArrayOfNumbers();
    const [a, b] = symDiff([], anArrayOfNumbers);
    expect(b).eql(anArrayOfNumbers);
  }));
  it('satisfies the identity property if the two arguments are identical', () => {
    const anArrayOfNumbers = someArrayOfNumbers();
    const [a, b] = symDiff(anArrayOfNumbers, anArrayOfNumbers);
    expect(a).empty();
    expect(b).empty();
  });
});

This property is what we were expressing in three of our unit tests

• x contains no elements and y contains no elements

• x contains an element and y contains no elements

• x contains no elements and y contains an element

• x contains the same elements as y

But what do we do for our someArrayOfNumber()?

Let’s use FastCheck

https://github.com/dubzzz/fast-check

const fc = require('fast-check');

Updated code

describe('symDiff', () => {
  it('satisfies the identity property for the first argument', () => {
    fc.assert(fc.property(fc.array(fc.integer()), (anArrayOfNumbers) => {
      const [a, b] = symDiff(anArrayOfNumbers, []);
      expect(a).eql(anArrayOfNumbers);
    }));
  });
  it('satisfies the identity property of the second argument', () => {
    fc.assert(fc.property(fc.array(fc.integer()), (anArrayOfNumbers) => {
      const [a, b] = symDiff([], anArrayOfNumbers);
      expect(b).eql(anArrayOfNumbers);
    }));
  });
  it('satisfies the identity property if the two arguments are identical', () => {
    fc.assert(fc.property(fc.array(fc.integer()), (anArrayOfNumbers) => {
      const [a, b] = symDiff(anArrayOfNumbers, anArrayOfNumbers);
      expect(a.length).eql(0);
      expect(b.length).eql(0);
    }));
  });
});

What did that just do?

• it just ran 100 tests with different arrays of numbers and checked that our properties held
• it tried long arrays of numbers
• it tried short arrays of numbers
• it tried negative numbers
• it tried HUGE numbers
• it tried empty arrays

So what….

Generators

• knows edge cases and makes sure those are run

Combining Generators

• arrays of numbers fc.array(fc.integer())
• arrays of strings fc.array(fc.unicode())
• arrays of your object fc.array(myAwesomeObjectArbitrary())

Minimizing Test Failures

• long-string-with-a-snowman-☃ fails, it figures out that ☃ is the cause

So what…

• isn’t meant to replace all example tests
• but can be helpful for testing polymorphic code
• helpful way to think about your code

Code Export