A counterexample is a case where a general rule produces the wrong result. Automated generation varies situations and searches decision boundaries to find one concrete example that remains allowed when it should be blocked, or blocked when it should be allowed.
The result helps improve a rule because it is specific enough to inspect and add to future tests. It does not automatically reveal the correct policy, especially when the counterexample exposes a genuine conflict between human values.
ELI5
Counterexample generation tries to find one example that breaks an apparently good rule. A single clear exception can show that the rule needs another boundary or a better explanation.
For example, a chatbot constitution may reject obviously forbidden answers but allow a carefully reframed version. Finding that case gives designers something concrete to fix and test again.
Why is one counterexample useful?
One valid counterexample is enough to show that a universal claim or rule does not cover every intended case.
Can counterexample generation settle a moral disagreement?
No. It can expose the disagreement or contradiction, but people may still need to decide which principle should take priority.
