What is property-based testing?

Definition

A property describes behavior expected over a range of inputs, such as sorting preserving the original items. A testing tool generates inputs and checks whether the program obeys that property, often helping reduce a failure to a smaller explanatory example.

Generated tests can reveal unexpected edge cases, but ordinary property-based testing samples executions and is not itself a proof over every possible input. The properties must reflect the intended behavior, and useful input generation remains important.

ELI5

You describe a rule the program should always follow, then test it with many different examples. The tool can try combinations that a person might overlook.

For example, a sorting function should keep all the original numbers and put them in order. A property test can generate empty lists, repeated numbers and negative numbers to look for cases that break those rules.

Is property-based testing the same as formal verification?

No. Typical property-based tests sample generated inputs, while formal verification establishes properties over a precisely modeled set of cases.

Can a bad property miss real bugs?

Yes. A property must describe the behavior you care about; a passing irrelevant property gives little assurance.

  1. Jason Gross, Rajashree Agrawal and Sophia Dew beneath the blue and white headline Prove AI Code Safe on a black background.
Definition card for property-based testing: What is property-based testing?

Property-based testing checks general rules about software behavior across many generated inputs instead of relying only on individually chosen examples.