Everything platforms and marketplaces need to know about the benefits and limitations of profanity filters.
A profanity filter allows online platforms to improve user experience and safeguard communities, yet they are time-consuming to update and leave users vulnerable. You can learn more about profanity filters, and how Spectrum Labs can further enhance your moderation practices and increase protection of your community.
A profanity filter is a type of software that scans user-generated content (UGC) to filter out profanity within online communities, social platforms, marketplaces, and more. Moderators decide on which words to censor, including swear words, words associated with hate speech, harassment, etc. Though profanity filters are limited in their capabilities and don’t examine the surrounding context of words, they are considered to be a good first step for content moderation as they are simple and quick to set up.
Just as the name suggests, profanity filters are designed to scan UGC using a list of blacklisted keywords, and either block the profanity entirely or replace it with special characters (@!%$) using a search and replace method. Users will likely notice the change in their intended message and use filtering evasion tactics like alternate spelling, dashes within the word, L33T speak, or Unicode characters.
Because the language of the internet is always evolving, content moderators should continually update keyword lists.
Profanity filters can be applied to all forms of text-based UGC, including usernames. While most usernames will not contain profanity or hate speech, some individuals will use the account creation process as an opportunity to include words that are offensive or inappropriate.
A profanity filter can be used as a first step for username moderation, and depending on the volume of users and a community’s internal resources, human moderators should also review the usernames as they pass through the profanity filter software. This step can ensure the highest quality results but can also be resource-intensive.
Profanity filters can be useful for platforms that rely on user-generated content. Each type of platform has unique needs for the protection of its members, but profanity filters also present limitations for each industry.
Player communication plays a large role in modern online gaming, creating a lot of potential for abuse. Profanity filters can be applied to text-based messaging in video games, such as private messaging between players and in-game text that is viewable to a wider audience (as in a game’s lobby before and after starting a match).
How profanity filters fail for gaming platforms: They cannot stop extremism and illegal solicitations, and they fail to prevent cyberbullying and hate speech.
Related Reading: Learn how Spectrum Labs is protecting gaming communities
Communication is at the heart of every dating platform, and the quality of communication between members can have a direct impact on engagement, membership, and revenue. Profanity filters can be an effective first step in the protection of your members from some toxic behaviors.
How profanity filters fail for dating platforms: They cannot prevent the offer or solicitation of prostitution, they cannot prevent underage users from joining.
Related Reading: Learn how Spectrum Labs is protecting dating communities
Illegal solicitations within marketplaces run the gamut from weapons and drugs to exotic animals. A keyword filter can help stop communication mentioning these terms, and anything else a marketplace would like to stop from being sold. However, savvy criminals will quickly find ways to circumvent filtering.
How profanity filters fail for marketplaces: They cannot completely stop illegal solicitations within marketplaces or prevent spam from taking their members off the platform through offline transactions.
Related Reading: Learn how Spectrum Labs is protecting marketplace communities
Social platforms must deliberately create a safe and positive user experience if they wish to retain users, attract advertisers, and drive revenue. Content moderation within social platforms presents a great challenge because of the high volume of user-generated content paired with the context and method in which it is presented. Profanity filters can only help identify and remove the most basic forms of offensive speech.
How profanity filters fail for social media platforms: They cannot stop extremism, hate speech, and illegal solicitations, or accurately prevent the promotion of self-harm.
Related Reading: Learn how Spectrum Labs is protecting social media communities
The biggest downside of profanity filter software are that they fail to observe the context in which a word is presented. While a platform may have profanities that it wants to block at every instance, sometimes a word can have different meanings depending on the situation. For example, a gaming platform may find that the word “kill” is often used as harmless in-game banter, not a real-life violent threat. The result is that words that aren’t intended to harm may be needlessly censored, and words that seem harmless within a common context can be used to harass or offend.
This kind of failure from profanity filters is referred to as the “Scunthorpe problem” - which is the name of a town in England. Users were blocked from being able to create AOL accounts due to filters identifying the town name as profane.
Due to the simplistic nature of profanity filters, they commonly create false positives, which result in a poor user experience if it results in unjust punishment. For example, if a video game for kids includes "asses" in its profanity filter, they will also punish children who innocently talk about getting new "sunglasses." And if a dating app wants to filter out any mention of the word "rapers," they are also punishing users who want to set up dates to meet at the top of "skyscrapers" to watch the sunset.
Profanity filters search for strings of characters and nothing more, making it easy for users and bots posting spam to get around them once they notice that the filter is in place. Users tend to get creative and employ evasion tactics like those mentioned above, such as alternate spelling, or Unicode characters.
Communication over the internet evolves rapidly, and depending on the size of the community, moderators may deal with thousands of incident reports daily. Those tasked with monitoring and revising the blacklisted keyword list will need to spend much time making updates as they start to notice toxicity trends via moderation efforts or reports submitted by users.
The most efficient and sophisticated solution to replace and overcome the challenges presented by profanity filters is contextual AI. Contextual AI interprets contextual cues in real-time sent through a variety of media, including text, voice, and chat; and in many languages. By allowing contextual AI to act as the first defense in content moderation, it rapidly identifies infractions that can be automatically actioned without moderator intervention or prioritized for moderators. This saves time and resources for an organization while protecting the mental health of moderation teams.
Spectrum Labs provides AI-powered behavior identification models, content moderation tools, and services to help Trust & Safety professionals safeguard user experience from the threats of today, and anticipate those that are coming. Because every company has different needs when it comes to content moderation, Spectrum Labs has specialized expertise in the fields of gaming, dating, social networks, and marketplaces.
Whether you are looking to safeguard your audiences, increase brand loyalty and user engagement, or maximize moderator productivity, Spectrum Labs empowers you to recognize and respond to toxicity in real-time across languages. Contact Spectrum Labs to learn more about how we can help make your community a safer place.
