Best Practice for Property-Based Testing

This talk aims to introduce property-based testing, why you may want to use it, and real-life use cases for including it in code bases. Property-based testing is an approach to testing that involves specifying statements that should always be true (for example, reversing a list twice will give the original result) rather than relying on specific examples.

This presentation provides an introduction to property-based testing, why you would want to use it, and how you might apply it in the real world. How do you unit test a function? Typically, one might identify different sets of inputs (for example, for an integer you may have sets for positive, negative, and zero) and write a single unit test using a single example from each set. Can you be sure your code is correct for the whole set? Maybe it only works for that particular example?

Here are the steps used to perform property-based testing:

1. Identify properties (invariants)
   • Think about rules that should always hold true for your function or system.
   • Example: For a sorting function, the output should always be in non-decreasing order.
2. Choose a property-based testing framework
   • Popular tools:
     • Hypothesis (Python)
     • QuickCheck (Haskell, Erlang, Elixir)
     • ScalaCheck (Scala)
     • jqwik (Java)
3. Write property tests instead of example tests
   • Instead of writing assert sort([3,1,2]) == [1,2,3], you write:
     • “For any list of integers, sorting it should produce a list where each element is ≤ the next.”
4. Let the framework generate inputs
   • The tool automatically produces random inputs (including edge cases like empty lists, very large lists, negative numbers).
   • It runs your property against hundreds or thousands of cases.
5. Framework shrinks failing cases
   • If a test fails, the framework tries to simplify the input to the smallest failing example.
   • Example: If sorting fails on a huge list, it might shrink it down to [2,1] to show the minimal counterexample.

Video producer: https://ndcoslo.com/