Trust And Safety Archives

The Dark Side of the Moon: Generative AI Spurring Rising Trust and Safety Interventions | Blog

March 17, 2023

The rise of generative Artificial Intelligence (AI) has been nothing short of a technological revolution. From art to advertising, it has transformed content creation and consumption. As we dive deeper into generative AI, we can’t ignore the elephant in the room: How will it impact enterprise Trust and Safety (T&S) policies and operations? Read this latest blog in our series to learn about the impact of generative AI on T&S, potential challenges for enterprises, and recommendations to navigate T&S waters.

“In the age of generative AI, trust brokers will become increasingly necessary and increasingly valuable in the marketplace.” – Hendrith Vanlon Smith Jr., CEO of Mayflower-Plymouth

Generative AI is making headlines as organizations discover exciting breakthroughs by investing in developing their own models. These advanced models have demonstrated remarkable abilities to generate realistic and diverse outputs that can be useful in various applications such as content creation, language translation, image and video synthesis, and even drug discovery. However, valid potential ethical concerns arise around deepfakes, truthfulness (or lack thereof), biased output, plagiarism, and malware, among others. Let’s explore this further.

Generative AI is the future of content but introduces additional complexities for enterprises

Generative AI is a type of artificial intelligence that can create new and diverse data forms, including images, audio, text, and videos. This technology has been around for some time, but recent advancements in models such as ChatGPT, DALL-E 2, Midjourney, and others have brought generative AI into the spotlight. While generative AI has led to an explosion in potential usability across diverse fields, it also presents additional complexities for enterprises to moderate it effectively, as illustrated below.

As generative AI constantly evolves with daily updates and new models contributing to its increasing complexity, heightened protection and care are necessary to avoid lasting negative consequences, such as reduced brand value and loss of customer trust. To ensure customer protection and platform safety, enterprises require dedicated T&S teams more than ever.

Generative AI impacts the entire T&S value chain to varying degrees

Below are some examples of emerging T&S use cases:

Getty Images, a UK-based photo and art gallery, sued Stability AI in the UK and US for copying over 12 million photos without permission, alleging copyright infringement, trademark violation, and unfair competition
Twitch, a streaming platform, temporarily banned the AI-generated sitcom “Nothing, Forever,” a parody of the popular TV show “Seinfeld,” due to the use of transphobic language in an episode

T&S needs to keep pace with generative AI

Current T&S operations are lagging behind the evolution in various content formats and need to step up their game to deal with the nuances of generative AI.

T&S services:

Effectively moderating generative AI content requires human moderators to possess a blend of technical expertise, ethical consciousness, and critical thinking abilities. A lack of these attributes can result in incorrectly flagging or removing content, as evidenced by the artist who was banned from a well-known art community by the moderator on accusations of using AI to create their artwork. Moderation of AI-generated content also can be limited by a lack of support for moderating niche languages.

Moreover, training the AI requires moderators to be exposed to more dangerous and harmful threats over longer periods, which could take a toll on their mental health and productivity. For example, a service provider hired by a tech-based company to train its GPT model terminated the contract due to prolonged exposure to harmful content.

T&S solutions:

Content moderation technology solutions have room to improve their maturity and ethical technology to effectively handle complex content formats such as generative AI content. The technology also needs to become more robust to moderate all types of generative AI content, including audio, video, and livestream. The shortage of training data leading to the lack of explainable AI and contextual understanding also poses a significant hurdle for content moderation that relies on large datasets.

Efforts to create effective AI-generated content detectors are underway, but their accuracy remains a challenge, with current capabilities falling short of achieving 95% reliability. In the race between generative AI creating harmful content and T&S AI moderating it, generative AI seems to be leading its T&S counterpart.

Enterprises need to amplify their FOCUS on T&S to deal with the nuances of generative AI

In this rapidly changing AI world, implementing the FOCUS framework illustrated below can enhance enterprises’ capabilities in handling generative AI complexities to safeguard users.

Strategic providers can help enterprises maintain the FOCUS framework in the following ways:

Upgrade algorithms and training datasets: Partnerships can help maintain the robustness of enterprise T&S platforms by improving algorithm accuracy and including a wide and large variety of data sets to train the T&S technology
Approach innovation collaboratively: Enterprises should look outward for innovation in addition to in-house investments. The right partner can help explore new technologies, such as explainable AI and responsible and ethical AI, to detect patterns and anomalies in generative AI content and assist enterprises in keeping their platforms secure
Provide policy management support: Enterprises also require constant market monitoring in addition to collaboration with regulatory bodies and policymakers to identify triggers, which can potentially impact existing T&S policies
Customize moderation rules: Operationally, generative AI requires customized moderation rules that are specifically geared to identify and highlight AI output that violates enterprise guidelines or other policies
Deliver holistic platform-based T&S solution: Service providers are increasingly partnering with technology providers that offer comprehensive, scalable platforms integrating various content handling technologies and human capabilities into a unified system

Looking forward, T&S for generative AI puts forth many questions:

As generative AI evolves, the following questions need to be addressed before charting the right investment strategy for T&S:

Can generative AI be used to build a T&S solution that replaces current AI models?
Will data privacy regulations and the reluctance of peers to share proprietary data limit companies’ ability to collect and use data for generative AI training?
Will content generated by AI be used to train even more AI models?

As we continue to make progress in the AI field, the possibilities for generative AI are endless, and imagining a future where AI-generated content becomes the norm is conceivable. However, it’s essential to approach this development cautiously and diligently, taking into account all the potential implications. As the future of generative AI unfolds, its success will depend on our ability to balance its power with our responsibility to use and moderate it wisely.

For our other recent blogs on how ChatGPT will impact various industry sectors, see Impact of ChatGPT and Similar Generative AI Solutions on the Talent Market, Can BFSI Benefit from an Intelligent Conversation Friend in the Long Term, and ChatGPT Trends – A Bot’s Perspective on How the Promising Technology will Impact BPS.

To discuss how generative AI will impact the T&S process in detail, please reach out to Abhijnan Dasgupta and Shubhali Jain.

Discover more about generative AI in our LinkedIn Live, Generative AI and ChatGPT: Separating Fact from Fiction.

Rethinking Trust and Safety: Six Strategies for Enterprises to Convert the Recession into an Opportunity | Blog

March 10, 2023

Trust and Safety (T&S) is no longer just good to have, it is one of the most critical business functions for any organization that has a presence in the digital world. Read on to learn about six strategies that can be deployed to bolster your T&S capabilities even as the fear of a recession looms large.

Multiple economic indicators are signaling that a recession is in the cards. Soaring inflation rates, the geopolitical shocks caused by the Russia-Ukraine war, and the lingering impact of COVID-19 continue to take a toll on the global economy. The US economy is expected to contract by 0.5 percent in the fourth quarter, and a more severe impact is predicted for the UK and the European region.

Contrary to previous recessions, the high-tech industry is the most impacted of all segments. Layoffs have been among the many cost optimization measures the industry has adopted to navigate these uncertain times. Over 93,000 employees have been laid off in the US alone so far.

Opportunity to rethink the current T&S practices

Regardless of the macroeconomic situation, T&S remains critical for all businesses that host user-generated content on digital platforms. The recession may be a short-term phenomenon, but it clearly presents enterprises an opportunity to re-examine their current T&S practices and determine how to do the work more efficiently over the long term.

Here are some strategies that enterprises can consider to optimize their T&S/content moderation investments:

Consider outsourcing as an option: Maintaining and managing a large talent pool for T&S can be costly. Companies also need to take into account the incremental costs of creating a wellness infrastructure for employees and continuously training them on the latest regulations. In addition to potential cost benefits, outsourcing enables businesses to manage and moderate user-generated content in various niche and mainstream languages at a global scale, while ensuring accuracy and agility in response. The recession warrants enterprises to evaluate the sustainability of maintaining in-house T&S teams in the long run

Identify the most viable shoring mix: Offshoring T&S is an effective way to cut costs while maintaining the same quality. In a bid to optimize costs during the economic slowdown, certain large tech firms have already shifted their T&S capabilities from the US and Europe to cost-effective locations such as India and South East Asia.

Enterprises even can consider leveraging certain non-traditional delivery regions such as Africa and Latin America, which offer significant cost arbitrage opportunities. Some of these regions have relatively high maturity for the delivery of certain trust and safety services (e.g., Kenya has high maturity in the delivery of data annotation services). Sourcing from certain locations also can fit in well with broader environmental, social, and governance (ESG) initiatives of firms

Adopt a pragmatic approach to digital investments: Large tech firms’ T&S strategies have always been to build the technology in-house and partner with third-party service providers to leverage their talent pool for any human intervention required. As building technology for every new type of threat may not be economically viable, “buying” or “partnering” with the relevant technology providers can be a better long-term alternative. Several digital platforms already have resorted to mergers and acquisitions to boost their T&S capabilities – Microsoft’s acquisition of Two Hat and Reddit’s acqui-hiring of Oterlu being the most notable examples.

Another trend to keenly observe is the growing number of partnerships among service providers and technology providers. This is a move that can unlock Artificial Intelligence (AI)-human synergies and lead to further cost and process efficiencies. Enterprises need to evaluate how these collaborations can be effectively leveraged as part of their T&S strategy.

Leverage alternative talent models for non-sensitive work types within T&S: Alternative talent constructs such as the gig workforce and contractual workforce are cost-effective alternatives to full-time agents, Everest Group research While the gig workforce is not deployed to moderate egregious content due to persisting concerns around data privacy and ensuring the well-being of moderators, these workers can be extensively used to handle non-egregious content moderation and other trust and safety tasks (data annotation, translation, etc.).

In fact, a hybrid model that involves gig workers and full-time employees is becoming more prevalent in the market. From 2019-2021, the share of the gig workforce in translation and localization and transcription services in the service provider workforce has grown 20% and 28%, respectively. At the same time, the full-time employee growth rate for these services has been less than 10% and 3%, according to Everest Group research

Engage in discussions on creative constructs: Many organizations are already deploying proactive threat detection tools and T&S solutions that can minimize human intervention by analyzing the context. Innovation in talent management and well-being facilities can reduce the backfill costs from attrition and create a more resilient workforce. Adopting innovative commercial constructs such as Experience Level Agreements (XLAs) that supplement Service Level Agreements (SLAs) can help enterprises drive business value while optimizing spending. Innovation should be a major criterion for any organization in its T&S investment strategy and when engaging with third-party suppliers
Reduce the portfolio risk: Enterprises can de-risk their portfolio by diversifying the supplier mix. This would require them to identify “at-risk” vendors and to have an active plan for business continuity. Leveraging a healthy mix of global and local providers would help in maintaining a localized flavor in service delivery while optimizing costs.

Trust and Safety outlook for 2023

With newer content formats such as metaverse and generative AI gaining popularity, the volume and variety of malicious content on the internet are expected to surge. Additionally, regulators across the world continue to tighten their grip on social media and hold them accountable for the content on their platforms.

Hence, organizations need to tread carefully and keep growing their T&S capabilities to ensure a safe experience for their users. They should not be overly focused on cost and must weigh the risks arising from making any changes to their T&S function. Given the current trends, the fast-paced growth that the content moderation market has been witnessing in the previous years is bound to continue near term.

To discuss Trust and Safety strategies, please reach out to [email protected] or [email protected].

Learn about the importance of the wellbeing of human moderators in content moderation in this webinar, Sustainable Moderation: The “Impact” of “Sourcing” on Trust & Safety.

Now is the Time to Protect Operational Technology Systems from Cyber Risks | Blog

January 6, 2023

With growing digitalization and interconnected devices, Operational Technology systems that monitor and control industrial processes in critical infrastructure are increasingly vulnerable to cyber attacks. Learn about the OT security concerns enterprises face and key considerations for selecting an OT security provider in this blog.

Historically, IT and Operational Technology (OT) systems have been air-gapped, with little or no spending on Industrial Control Systems (ICS) security by enterprises. Further, most investments in industrial robots, SCADA, and PLC systems were made with a multi-decade horizon. This differs from IT investments, where the horizon is five years, and the technological refresh takes care of the security risk. Enterprises have been unwilling to touch OT systems because these big, monolithic systems ran well for many years, making security vulnerabilities and risks less of a priority to consider.

But OT systems – that power some of the nation’s most critical infrastructure – are at risk.

With the recent pandemic-driven digitization push and enterprises wanting to run resilient supply chains, these large monolithic untouched systems are now interconnected, making them highly prone to cyber attacks. These OT systems have also never been given basic security treatment of frequent patching, regular security updates, and periodic backups, which has further aggravated the issue.

Operational technology systems cyber attacks

Highly public cyber attacks on OT systems have raised awareness about the serious risks these security breaches can have on essential services, as seen in these industry-specific cases:

Manufacturing – This segment had the second-highest ransomware-associated data extortion attacks in 2021. Traditionally, plant machinery and equipment have been designed for performance and not security. But stalled assembly lines or production units directly impact end buyers and can be disastrous for manufacturers
Energy, utilities, and water – Threat actors have been targeting the most crucial elements in this critical infrastructure industry, mandating enhanced cybersecurity controls. Securing critical grid assets, substations, distribution pipelines, meters, etc., must be addressed

Oil and gas – Digitizing operations for improved efficiency has increased the attack surface and has made this area more vulnerable to threats. During the Russia-Ukraine war, states were reportedly involved in sponsoring attacks, leading governments across the world to alter or create industry regulations and guidelines

Operational technology systems key challenges

The increasing connectivity of operational technology with external networks has further exacerbated the many OT security challenges enterprises face. Major OT security concerns include asset identification, misaligned IT and OT functions, OT threat and asset intelligence, patching legacy infrastructures, OT vulnerability management, and network segmentation.

Based on market conversations with more than 100 market participants, Everest Group identified and prioritized the following key enterprise challenges.

Technology vendor snapshot for OT security

Source: Everest Group

Specialist providers can help enterprises navigate security challenges related to OT and ICS. Enterprises should seek technology solutions that allow them to quickly identify vulnerabilities and prioritize actions to reduce and eliminate potential risks.

A provider of choice should offer a single platform for visibility and threat monitoring while ensuring seamless integration with existing enterprise technology investments. The below capabilities illustrate what enterprises should look for when selecting an OT security provider.

By investing in tools that can provide vital intelligence and partnering with providers that offer compatible, industry-specific solutions and a skilled talent pool, enterprises can begin to thwart the growing risks to OT systems – before it is too late.

To discuss Operational Technology Systems and OT security, please reach out to [email protected] and [email protected].

Explore the top five demand themes in technology services – data and AI, cloud, experience, platforms, and security – driving growth for IT service providers in 2023 in our upcoming webinar, IT Service Provider 2023 Forecast: The Top 5 Themes for Growth and Wallet Share.

Announcing our inaugural conference: Everest Group Engage, The Pragmatic Edge: Designing Your Future. Learn more.

Category: Trust and Safety