Tag: Generative AI

Comparing Large Language Models: Everest Group’s AI LLM Assessment Provides a Powerful Framework for Enterprise Decision-Making | Blog

Selecting the right Large Language Model (LLM) is critical for developing the best-suited generative AI (gen AI) solution. However, choosing an LLM on the number of parameters alone can be a costly mistake as larger size alone doesn’t always equate to better performance. Discover a comprehensive framework that evaluates and compares more than a dozen LLMs on 27 key parameters to enhance enterprise decision-making.  

Since the fervor surrounding gen AI started in November 2022, the explosion of LLMs is redefining language understanding and generation boundaries. As more models continue to emerge, evaluating them presents a significant challenge. A more structured and detailed approach is critically needed to evaluate these massive models that goes beyond assessing them solely on their sheer number of parameters.

Given the rising interest in gen AI across diverse applications, the lack of comprehensive research into LLM evaluation is striking. Relying solely on the parameter count when choosing an LLM can be misleading. It neglects crucial performance aspects, increases implementation costs, hinders enterprise readiness, enhances risk, and more. As these LLMs shape interactions and decision-making, an all-inclusive evaluation framework is essential to navigate their impact effectively.

Building on inaugural research in this area, Everest Group has assessed LLMs on multiple parameters and showcased how they rank against each other to help enterprises make the best and most informed decisions.

Introducing Everest Group’s AI LLM Assessment

Everest Group’s AI LLM Assessment presents a comprehensive framework, offering valuable guidance for stakeholders seeking to understand the various elements of LLMs. This assessment meticulously evaluates 13 leading LLMs across 27 distinct dimensions.

The framework evaluates LLMs’ unique capabilities, enabling a deeper understanding of their functionalities. Consequently, enterprises can determine which LLMs are fast, user-friendly, and capable of handling large amounts of input data for practical implementations.

The AI LLM Assessment evaluates various capabilities through such dimensions as the number of tokens they can process, the modalities supported, inference speed, training data quality, and overall market perception. These factors ultimately become differentiators, setting LLMs apart from peers and predecessors.

Below is a snapshot of the Everest Group AI LLM assessment matrix or explore the full framework.


The hype of large language models – is bigger always better?

Recently, we have witnessed numerous technology providers developing their LLMs. Each model aims to outperform the others by emphasizing its larger size compared to its peers and previous iterations. However, it is rarely discussed whether having more parameters and larger datasets actually enhances the ability to deliver value across various use cases.

LLM advancements have highlighted a fascinating trend where smaller models like PaLM 2 have demonstrated superior performance despite being trained on fewer parameters than their predecessors. These compact models not only offer better performance but also deliver faster inference times and reduced processing costs. This underscores that larger models may not always be the only way to achieve the desired outcomes.

Choosing the right LLM

Deciding which LLM is the best fit for enterprise applications and use cases based on LLM capabilities and features is the most crucial step in developing a gen AI solution. After assessing a model for these variables, understanding how easily it can be integrated into enterprise operations is vitally important.

To address this need, the framework takes into account the feasibility of practical implementation, considering factors such as the average implementation cost based on usage and ecosystem readiness. The framework also examines the selected LLMs for potential risks that may hinder enterprise adoption.

By considering capability versus adoption ease, the framework offers enterprises a balanced approach for analyzing LLM attributes and functionalities while also accounting for the associated challenges and considerations to integration and utilization.

The path to enhanced LLM performance and adoption

This framework has the potential to help developers enhance their capabilities over peers in building new LLMs tailored for specific tasks or applications by better understanding competitors’ strengths and weaknesses.

While LLM evaluation is undeniably complex and continuously evolving, this framework provides a vital starting point. As Everest Group continues to track developments in the gen AI landscape, we welcome discussing potential use cases, risk and cost considerations, and the impact of gen AI across various industries.

Please reach out to Priya Bhalla, [email protected], Vishal Gupta, [email protected], Vaibhav Bansal, [email protected], Yukta Sharma, [email protected], or Vatsalya Singhal, [email protected] to discuss generative AI topics further.

The Generative AI Odyssey: A Year in Review and What’s Ahead in 2024 | Webinar

on-demand webinar

The Generative AI Odyssey: A Year in Review and What's Ahead in 2024

In 2023, we witnessed massive investments in generative AI, with the majority coming from the supply side. Despite the initial fervor ignited by ChatGPT, the anticipated surge in enterprise adoption is yet to come. One factor contributing to this delay is the need for real-world implementation examples to make adoption more realistic and attainable.

In this webinar, our expert analysts explored the hype vs. reality of generative AI, showcased actual production-level use cases, addressed challenges head-on, and provided a glimpse into the future of this transformative technology as we move into 2024.

What questions has the webinar answered for the participants?

  • Beyond the hype, what does real generative AI adoption look like in enterprises?
  • What are the key technology trends shaping the generative AI market?
  • What will be the future themes across generative AI in 2024?

Who should attend?

  • Global enterprises
  • CIOs, CTOs, CDOs, Chief AI Officers (CAIO)
  • Technology directors, IT managers
  • Data & Analytics heads
  • ITS/BPS strategy heads
  • Senior AI executives
  • Service providers
  • Technology providers
Bhalla Priya
Practice Director
Vice President

PodChats for FutureCIO: The Future of Custom Applications | In the News

For decades, custom applications were the norm, allowing organizations to create applications designed to meet the unique requirements of the business.

With the growing popularity of citizen developers using local or no-code platforms, is there value for custom applications? Ankit Gupta, Practice Director at Everest Group, spoke with PodChats for FutureCIO to shed light on the future of custom applications.

Check out the PodChat here in FutureCIO.

The Generative AI Revolution: Transforming Customer Experience Management | Blog

Generative Artificial Intelligence (AI) is poised to revolutionize customer experience management (CXM) by creating personalized, empathetic, and more fulfilling experiences that drive brand loyalty and business growth. In this blog, explore examples of early generative AI adoption and learn about the benefits and challenges of this game-changing technology.

Learn more on this topic in the webinar, The Generative AI Advantage in Enterprise CXM Operations.

As contact centers shift their main focus from improving efficiency to creating impactful customer experience, generative AI is leading the charge in this new direction. Recognizing generative AI’s promise to enable the personalized, hyper-contextual interactions customers desire, enterprises are looking to invest and deploy solutions to leverage its transformative potential.

A recent Everest Group survey revealed nearly 60% of enterprises believe generative AI solutions have huge potential to disrupt the current contact center landscape. Additionally, another 37% perceive these solutions as beneficial in some areas.

Transforming the CXM landscape

By mimicking human creativity, generative AI can create nuanced and contextually relevant content. This opens a wide range of possibilities to reshape the way brands engage with customers across various touchpoints and provide the following benefits:

Enhanced customer service

  • Conversational AI: Supports intelligent virtual assistants for natural, contextual interactions, fostering deeper connections and loyalty
  • Personalization support: Analyzes vast customer data, tailoring experiences and providing real-time product support for heightened experience

Contextual marketing

  • Swift content creation: Crafts personalized content and product descriptions quickly, reducing production time and boosting conversions
  • Engaging storytelling: Creates compelling brand stories and personalized campaigns that resonate with specific audiences

Building stronger relationships

  • Personalized recommendations: Recommends products based on individual preferences, fostering trust and repeat business
  • Proactive engagement: Personalizes messages, contributing to lasting customer relationships

Enterprise generative AI adoption

With vast potential applications, enterprises across vertical markets are already reaping the rewards of early-generation AI adoption. Let’s explore some pioneering examples:

  • Virtual experience: A leading global furniture brand has built a generative AI chatbot to guide customers through the customization process, making furniture shopping more intuitive and natural while also offering 3D product configuration
  • Content enhancement: Prime Video has introduced Defensive Alerts, a generative AI feature that tracks the movements of defensive football players before the snap, reads their acceleration, and identifies “players of interest” likely to rush the quarterback. A red circle appears under the potential blitzer, giving fans a heads up, allowing them to place themselves in the coach’s seat and read developing plays
  • Customer support: Dave, a digital banking service, is implementing AI-powered chatbots that can hold natural conversations with customers, answer complex questions, and even resolve certain issues without human intervention
  • Content generation:com is testing its AI Trip Planner, which utilizes generative AI to create personalized offers and travel itineraries based on customer preferences and provide direct booking options to deliver an integrated travel planning experience
  • Itinerary planning and customization: Expedia has integrated ChatGPT into its app to help users make and save travel plans. Customers can ask the AI for recommendations on destinations, accommodations, and transportation as if it were a human travel agent. The app can also save locations so users can easily check availability and book travel
  • Student coaching: Language-learning platform Duolingo uses the technology underpinning ChatGPT-4 to help users practice language skills and understand when they make a mistake. It also uses the technology to allow learners to practice real-world conversation skills with the roleplay feature in the app
  • Dynamic promotion, pricing, and loyalty program: Levi Strauss & Co. has implemented generative AI to increase diversity on its website and expand its loyalty program by offering personalized benefits. This has significantly increased loyalty enrollments to 5 million members worldwide and boosted revenues and app registrations. Generative AI allows for tailored product recommendations, localized discounts, and customized store experiences based on consumer data and mobility insights. AI-driven analytics help optimize stock for various sales events, including mid-season, end-of-season, and Black Friday sales in the U.S. and Europe
  • Agent assist: Advisors at a multinational IT company that provides subscription-based technology support services worldwide access a secure generative AI-based model to easily answer customer queries

Addressing the challenges

While generative AI’s potential benefits are intriguing, addressing the inherent challenges that come with its implementation is critical. Enterprises have expressed a wide range of issues, from regulatory to accuracy, that could arise with generative AI. The top three enterprise concerns to generative AI adoption are:

  1. Data security and privacy: Robust security measures and transparent data usage policies are necessary to utilize customer data. The risk of data leakages during model training or deployment further intensifies the threat to data privacy. The implementation of generative AI exposes vulnerabilities to cyber threats and presents issues related to the secure handling of sensitive information for training the model
  2. Compliance issues: Enterprises are concerned about copyrights and ownership of intellectual property (IP) produced by generative AI while ensuring the solution doesn’t violate other organizations’ IP. With the diverse generative AI applications, sector-specific regulations are crucial. The technology’s evolving nature also calls for dedicated regulations addressing unique challenges and ethical considerations
  3. Accuracy: Organizations are wary of the risk of biased output stemming from training data biases, the potential for unethical responses requiring human oversight, and instances of “hallucinations” – all underscoring the pressing need to refine and enhance model accuracy

Future of CXM with generative AI

The changing landscape of generative AI in CXM is a testament to the transformative power of technology. The generative AI revolution is here, and it’s poised to significantly alter the way brands interact with their customers. By responsibly and strategically embracing this technology, CXM service providers can create personalized, empathetic, and, ultimately, more rewarding customer experiences, leading to stronger brand loyalty and increased business growth.

To discuss generative AI adoption trends in CXM, please contact Chhandak Biswas, [email protected] and Rishav Kumar [email protected].

Discover how enterprises are looking at generative AI-based solutions adoption to improve CX in the webinar, The Generative AI Advantage in Enterprise CXM Operations.

Beyond Filters: Exploring the Impact of Generative AI Influencers on the Marketing Landscape | Blog

By leveraging generative Artificial Intelligence (gen AI), brands can elevate influencer marketing to the next level by creating compelling content that connects more deeply with consumers. In this blog, discover how AI influencers are changing the influencer marketing market and key factors brands should consider.

In an increasingly digital world, consumers seek personalized connections and are drawn to influencers who embody relatable lifestyles and offer trustworthy recommendations. Brands recognize the potential of influencer marketing to enhance visibility, credibility, and engagement. Influencer marketing fosters genuine connections that resonate with today’s consumers and provides brands with a powerful platform to amplify their message in the crowded digital marketplace.

Let’s take a look at how consumers and brands perceive influencer marketing.


Adding gen AI to the mix

Quick cut to the gen AI disruption – where cutting-edge technology meets influencer creativity. With its capability to produce creative text formats, images, and videos, gen AI brings a new opportunity to this market. It has the potential to empower influencers to craft compelling content at scale that uniquely resonates with followers.

Influencers now find themselves equipped with innovative means to captivate audiences, experiment with storytelling formats, and consistently produce engaging content that reflects the pulse of their followers.

Callout: “A survey of consumers across the UK and the US found a majority (60 percent) prefer creator content designed using gen AI. An additional poll of content creators found most (81 percent) reported more favorable audience engagement with content designed using AI technology.”

Let’s look deeper at how gen AI is being used in influencer marketing.

  • Influencers are crafting more personalized and authentic content, easier and quicker using gen AI
  • Gen AI is assisting influencers with audience engagement, based on data and insights from sentiment analysis
  • Influencers are using AI-generated prompts and ideas to spark creativity, ranging from unique storytelling angles to creative challenges
  • Gen AI-powered influencers are gaining popularity on social media platforms enabling conversations and human-like responses in comments and messages


Rise of AI influencers

The use of AI influencers in marketing is a relatively recent development that has gained significant traction. High-profile brands such as Prada, Versace, Red Bull, and Tinder have all activated AI influencers for social media promotions. Although the results driven by AI influencers are similar to those of human creators, the key difference lies in creating a relatable brand presence in consumers’ minds.

In the graphic below, we compare the skill levels of human and AI influencers in important areas:


With their complementing skills, virtual and human influencers can create engaging content at different ends of the same spectrum. As gen AI becomes more prevalent in the influencer industry, balancing authenticity and AI-generated content will be crucial to maintaining genuine connections.

 Key considerations for brands

 While the intersection of gen AI and influencer marketing presents a transformative landscape for brands to connect with their target audience, a few areas of concern still need to be addressed, including:

  • Identifying the right influencers from a crowd of “experts” with genuine followers
  • Managing controversial content or affiliations that conflict with the brand message
  • Safeguarding against the potential risk of copyright infringement due to inspired content from gen AI
  • Measuring the actual impact and return on investment (ROI) of influencer marketing

As influencer marketing evolves, the future is oriented toward adopting an omnichannel and full-funnel strategy. This entails brands leveraging influencer content across diverse marketing channels, from connected television (CTV) ads to opportunity-to-hear (OTH) display ads. By taking this approach, brands can establish influencer-led paid media and integrate it with commerce, removing steps in the customer journey and ultimately driving faster conversions.

To discuss the growing role of AI influencers in the influencer marketing market, contact [email protected] and [email protected].

Join our webinar, The Generative AI Advantage in Enterprise CXM Operations, to learn how enterprises are looking at generative AI-based solutions adoption to improve customer experiences.


Enterprises Aim to Move Beyond Pilots, Accelerate Consumption of AI in 2024—Everest Group, Yates Ltd.

Despite the global economic turndown, enterprises widely adopted AI in 2023, with generative AI playing a substantial role, according to a survey of CIOs conducted by Everest Group and Yates Ltd.


DALLAS, January 18, 2024 — If chief information officers (CIOs) have their way in 2024, expect to see more enterprises making adoption of generative artificial intelligence (gen AI) a strategic priority with an aim to move past small pilots to scaled implementations. This forecast summarizes the sentiments of more than 50 CIOs interviewed by Everest Group in collaboration with Yates Ltd. The survey also revealed that improving the velocity of existing operations is the primary motivation driving enterprise gen AI initiatives.

The interviews conducted with global CIOs between October 23, 2023, and January 24, 2024, also underscore that gen AI is more than a passing trend, having successfully penetrated early enterprise adoption thresholds. Nearly 83% of global enterprises are either actively testing their capabilities through pilot programs or have already adopted gen AI for one or more production-grade use cases.


Key Findings from the Survey:

  • Sixty-one percent (61%) of global enterprises are actively exploring and piloting gen AI and 22% have already deployed gen AI for at least one or more processes. Another 15% plan to pilot gen AI soon.
  • The three top objectives CIOs are trying to achieve through gen AI are:
    • accelerating consumption of existing digital tools
    • reducing the latency of knowledge sharing
    • shortening the product development lifecycle.
  • CIOs identifying their top three challenges to scaling gen AI initiatives most often named lack of clarity on success metrics (73%), budget/cost concerns (68%) and the fast-evolving technology landscape (64%). Additionally, 55% named data security and privacy concerns, while 41% cited talent shortage.


The full report of findings — “Capturing the Generative AI Pulse: An Exploration of the CIO Mindset” — identifies the current state of enterprise generative AI adoption and the key challenges in scaling AI initiatives. The report also showcases three waves of generative AI adoption levels for enterprises and provides guidance to help enterprises advance in their generative AI adoption journey. The full report is available for complimentary download.

“Unquestionably, gen AI hype dominated 2023, but our survey indicates that it is more than a passing trend,” said Abhishek Singh, partner at Everest Group. “Our research clearly documents that most organizations are in what we call ‘Wave 1’ or the pilot phase of gen AI adoption; however, in 2024 and 2025 we fully expect more organizations to advance to the ‘Wave 2’ phase of production-grade deployments.

“Although enterprise adoption of gen AI is far from its anticipated peak, enterprises continue to experiment with unique use cases in a wide variety of industries, ranging from high-tech and financial services to healthcare and retail,” continued Singh. “As more of these initiatives document measurable impact, we’ll see adoption and full-scale implementation of gen AI accelerate considerably.”

Everest Group maintains that this shift from Wave 1 to Wave 2 will demand that enterprise leaders cultivate data-driven cultures and invest in digital and data maturity. Successful transitions will also require a comprehensive approach that incorporates technological advances, organizational readiness and ethical considerations.

“Gen AI is transforming senior executives’ perspectives on efficiency, growth and competitive advantage, and will revolutionize their operational strategies,” stated Charlotte Yates, the founder and CEO of Yates Ltd. She emphasized the need for a forward-thinking blueprint in Wave 2 to effectively implement gen AI use cases: “This blueprint should address a wide range of opportunities, risks, and investments in platforms, operating models, organization design, governance, strategic partnerships and culture.”


About Yates Ltd.

Yates is an IT and business consultancy that partners with senior executives to create the strategy, blueprints, financial mechanisms and execution plans to drive and achieve transformation. Our clients gain measurable cost savings, new capabilities, and the ability to outperform their competition. Our areas of focus include enterprise networks, software, managed services, end user services and automation. Services include strategy, sourcing, program execution, change management, communications and governance. Yates Ltd. is a WBENC-certified woman-owned business.

About Everest Group

Everest Group is a leading research firm helping business leaders make confident decisions. We guide clients through today’s market challenges and strengthen their strategies by applying contextualized problem-solving to their unique situations. This drives maximized operational and financial performance and transformative experiences. Our deep expertise and tenacious research focused on technology, business processes, and engineering through the lenses of talent, sustainability, and sourcing delivers precise and action-oriented guidance. Find further details and in-depth content at www.everestgrp.com.

Distinguishing Gen AI Hype from Real Applications | LinkedIn Live


Distinguishing Gen AI Hype from Real Applications

View the event on LinkedIn, which was delivered live on Wednesday, January 17, 2024.

Watch this insightful discussion on navigating the landscape of Artificial Intelligence (AI). 🚀 Everest Group experts will unravel real-world use cases that go beyond the hype, shedding light on how generative AI is making a tangible impact across industries.

Tune in for a candid exploration of the practical applications shaping the future of intelligent technologies.

What questions does the event answer for the participants?

  • What is the current gen AI landscape?
  • What are recent real-world gen AI use cases that industries are investing in?
  • What does the future of gen AI look like?

Meet The Presenters

How can we engage?

Please let us know how we can help you on your journey.

Contact Us

"*" indicates required fields

Please review our Privacy Notice and check the box below to consent to the use of Personal Data that you provide.