Any filter strict enough to catch the offence will catch innocent cases that contain it, because the rule operates on strings and the offence is in meaning.
Named for the Lincolnshire town whose residents were blocked from registering with an internet service because its name contains an offensive substring. The general problem is that context determines offence and the filter cannot see context, so every increase in strictness buys additional false positives, and the residents of Penistone, Lightwater and Arsenal have all paid them.
In practice
- A profanity filter that blocks a medical discussion has enforced the rule exactly as written.
- A fraud rule tuned until it catches every case is now catching mostly customers.
- Further reading
- Wikipedia
- Subject
- Technology & Software
- Essay
- How technologies spread, scale, decay and frighten us