Meta’s Oversight Board Addresses AI Deepfake Scams

Scams utilizing AI deepfakes of celebrities have emerged as a significant concern for Meta in recent years. The Oversight Board has weighed in and seemingly confirmed criticisms that Meta is not adequately enforcing its own policies, allowing scammers to exploit its platforms with relative ease.

Oversight Board’s Findings

The board stated, “Meta is likely allowing significant amounts of scam content on its platforms to avoid potentially overenforcing a small subset of genuine celebrity endorsements.” This statement reflects the board’s latest decision, which highlights the inadequacies in Meta’s content moderation practices. The board noted that at-scale reviewers lack the authority to enforce prohibitions against content that creates fake personas or impersonates famous individuals for fraudulent purposes.

Case Study: The Plinko Ad

This conclusion arose from a specific case involving an advertisement for an online casino-style game called Plinko, which featured an AI-manipulated video of Ronaldo Nazário, a retired Brazilian soccer player. The advertisement, which the board indicated displayed clear signs of being fabricated, remained on Meta’s platforms despite being reported as a scam over 50 times. Although Meta eventually removed the ad, the underlying Facebook post remained active until the Oversight Board agreed to review the situation. The ad had garnered over 600,000 views.

Flaws in Content Moderation

The Oversight Board emphasized that this case underscores fundamental flaws in Meta’s approach to moderating content related to scams involving celebrities and public figures. According to the board, Meta informed its members that it only enforces its policy upon escalation, ensuring that the individual depicted in the content did not endorse the product. Furthermore, the board pointed out that individual reviewers’ interpretations of what constitutes a “fake persona” may vary across different regions, leading to inconsistencies in enforcement. Consequently, a “significant” amount of scam content is likely slipping through the cracks.

Recommendations for Improvement

In its sole recommendation to Meta, the Oversight Board urged the company to update its internal guidelines, empower content reviewers to identify such scams, and provide training on the “indicators” of AI-manipulated content. In response, a spokesperson for Meta stated that “many of the Board’s claims are simply inaccurate” and referenced a test, which the company initiated last year to combat “celeb-bait” scams using facial recognition technology.

Meta’s Response to Scams

The spokesperson elaborated, stating, “Scams have grown in scale and complexity in recent years, driven by ruthless cross-border criminal networks. As this activity has become more persistent and sophisticated, so have our efforts to combat it. We’re testing the use of facial recognition technology, enforcing aggressively against scams, and empowering people to protect themselves through various on-platform safety tools and warnings. While we appreciate the Oversight Board’s views in this case, many of the Board’s claims are simply inaccurate, and we will respond to the full recommendation in 60 days in accordance with the bylaws.”

Widespread Issues with AI Deepfakes

The rise of AI deepfake scams has become a major issue for Meta, particularly as AI technology becomes more affordable and accessible. Earlier this year, reports indicated that numerous pages were running ads featuring deepfakes of Elon Musk and Fox News personalities promoting supplements that claimed to cure diabetes. Some of these pages repeatedly circulated hundreds of versions of these ads with seemingly minimal consequences. Although Meta disabled some of the pages following these reports, similar scam ads continue to appear on Facebook.

Actress Jamie Lee Curtis also recently publicly slammed Mark Zuckerberg for failing to remove a deepfaked Facebook ad featuring her likeness. Meta ultimately removed the ad after her public outcry.

Scale of the Problem

The Oversight Board highlighted the extensive nature of the problem in this case, noting that it discovered thousands of video ads promoting the Plinko app within Meta’s Ad Library. Several of these ads featured AI deepfakes, including those of another Brazilian soccer star, Cristiano Ronaldo, and Meta’s own CEO, Mark Zuckerberg.

External Concerns and Regulatory Scrutiny

Meta is not the only entity raising alarms about scams on its platforms. The Wall Street Journal reported recently reported that Meta “accounted for nearly half of all reported scams on Zelle for JPMorgan Chase between the summers of 2023 and 2024” and that “British and Australian regulators have found similar levels of fraud originating on Meta’s platforms.” The report noted that Meta is “reluctant” to add friction to its ad-buying process and has shown hesitance in banning advertisers, even those with a history of conducting scams.

This article originally appeared on Engadget at https://www.engadget.com/social-media/the-oversight-board-says-meta-isnt-doing-enough-to-fight-celeb-deepfake-scams-194636203.html?src=rss