Conventional software

A rule written as a regular expression can pass every test anyone thought to write and still take exponential time on one unlucky input, because a backtracking engine tries an exponential number of paths before it gives up. Reading the pattern does not tell you which inputs do it.

What people are trying

Engine authors are moving to matchers that run in linear time and never backtrack, and static analyzers look for the nested, overlapping repetition that makes a pattern vulnerable. Neither removes the need to check a pattern you inherited.

• Regular expression Denial of Service - ReDoS · OWASP Foundation · read 09/19/2026

  may reach extreme situations that cause them to work very slowly (exponentially related to input size)

Where it bites: When not to use a model

A rule system's real input space is every field's values multiplied together, so testing all of it is out of reach, and there is no general way to say how much of it a test suite covers. Every practical answer rests on an assumption about how failures are spread across combinations.

What people are trying

NIST's combinatorial testing work builds test sets that cover every two-way to six-way combination of parameter values rather than every combination, on the finding that most real failures are triggered by a few interacting factors. Teams pair it with tracking which rules ever fire in production, to find the branches no test reaches.

• Combinatorial Methods for Trust and Assurance: Why do Combinatorial Testing? · NIST, Computer Security Resource Center · read 09/19/2026

  it is nearly always impossible to do exhaustive testing, but we don’t have to test all possible combinations of inputs; we only have to test all of the combinations that trigger faults

Where it bites: When not to use a model

A classifier your code acts on can drift away from the data it was trained on long before any true label arrives to prove it. Measuring accuracy directly means waiting, and by then it may have been wrong for weeks.

What people are trying

Detectors that watch the model's own output distribution instead of waiting for labels, built on statistical process control and set to fold labels in if they ever turn up. Keeping the false alarm rate low at production volume is the part that is not settled.

• Flexible and Efficient Drift Detection without Labels · arXiv · read 09/19/2026

  Controlling for false positives while monitoring the performance of predictive models used to make inference from extremely large datasets periodically, where the true labels are not instantly available, becomes extremely challenging.

Where it bites: When not to use a model