5 best practices for implementing Copilot for Microsoft 365 at scale

By Boris Jovevski, Risham Sahni and Peter Hudačko

Most companies realize they need to adopt generative AI in some fashion, but the way forward is proving far more complicated than flipping a switch and letting employees begin using the technology.

Since Open AI’s Chat GPT was publicly released in late 2022, organizations have clamored to incorporate generative AI into their operations and ready their enterprises for AI everywhere. Yet, for all the buzz generated by the technology, a Kyndryl survey of 109 C-suite executives revealed that only 18% of respondents had generative AI applications live as of May 2024. Fewer still (10%) had a documented strategy for implementing the technology.

When Kyndryl’s path to Copilot for Microsoft 365 began in 2023, we were in the same situation many organizations find themselves in today. So, we believe our experience deploying the technology at scale can provide insights for other companies looking to integrate generative AI into their own digital workplaces. 

Setting the stage for Copilot

In late 2022, Kyndryl was entering the second year of a 24-month global IT transformation. With AI adoption rapidly expanding and our customers and employees increasingly requesting generative AI tools and solutions, we made a strategic decision to invest in the technology and accelerate deployment so we could share our learning with customers.

Our Chief Information Officer (CIO) and Chief Technology Officer (CTO)—in conjunction with our Chief Information Security Officer (CISO) and Chief Privacy and Regulatory Officer (CPRO)—set the stage for the company’s AI journey by developing the Responsible AI at Kyndryl approach. We then analyzed numerous vendors and AI solutions before expanding our relationship with Microsoft to include generative AI services, making Kyndryl one of the earliest adopters of Copilot.

Kyndryl had already moved to the Microsoft 365 platform to sustain our hybrid workplace model, so Copilot for Microsoft 365 was the logical generative AI tool for Kyndryl’s internal adoption. The technology supported a data-centric, cloud-based approach that was in line with our commitment to automation-led processes and a zero trust security model. Copilot’s capabilities also aligned with our goal of improving employee efficiency and productivity.  

After collaborating with Microsoft on a rollout strategy, our Copilot team launched a multi-phase implementation in July 2023. Over the following 12 months, we worked with multidisciplinary teams across Kyndryl to execute and refine our strategy, integrating principles of our responsible AI approach to ensure our data and information architecture were secure before vetting use cases and staging rollouts of Copilot licenses.

By July 2024, we had assigned nearly 20,000 Copilot licenses and amassed more than 600 approved use cases for the technology. Employees quickly reaped the benefits. (See “By the numbers”)

What we learned along the way about Copilot

Kyndryl’s move to Copilot was accelerated yet strategic. Here, we’ve distilled our lessons learned into five best practices to help guide your path to Copilot.

1. Prepare and secure your data to optimize performance

Since data fuels the large language models (LLMs) that power generative AI tools like Copilot, ­your data structure and information architecture must be in pristine condition and adhere to the latest data privacy and cybersecurity standards for the technology to perform optimally. ­­

The Kyndryl experience

Working closely with Kyndryl’s CISO, CPRO and legal teams, the Copilot team spent four months conducting a preflight assessment of Kyndryl’s M365 environment. We reviewed existing M365 data controls, content lifecycles and data classifications, performing inspections and cleanups to prepare the information architecture for Copilot. Key steps included:

• Tightening access controls in SharePoint and reducing the number of internal public sites to approximately 10,000
• Automating deletion of roughly 20,000 inactive SharePoint sites
• Cleaning up approximately 9,000 Teams and 3,000 Yammer communities
• Implementing data loss prevention policies in Microsoft Purview
• Developing a data classification module based on sensitivity
• Tagging content with relevant keywords and organizing the data

The data cleansing and architecture restructuring done during the preflight assessment—facilitated by regular progress meetings with Microsoft—helped to prepare and secure our data environment for Copilot, laying the foundation for Kyndryl’s Copilot governance model.

2. Establish a robust governance model to reduce risk and manage deployment

Governance underpins the responsible use of generative AI tools like Copilot. Without proper guardrails in place, users may misuse the technology or expose sensitive data that can cause significant operational, financial or reputational damage.  

The Kyndryl experience

Unlike governance programs that focus solely on data protection, our Copilot governance model also incorporates license approval and management.

During implementation, the Copilot team collaborated with legal, risk and human resources teams to review each use case request individually to ensure alignment with compliance standards and risk considerations. Later, employees requesting licenses were also allowed to choose from a list of approved use cases to streamline the application process.

The Copilot team also introduced a system for assessing continued business need. Each month, administrators reviewed dashboard reports to measure the frequency of use and reclaimed licenses from employees who hadn’t used the technology within a specified period. Reclaimed licenses were then reallocated to individuals with approved use cases, ensuring employees who would benefit most from the technology had access to it.

Moving forward, data protection and license management will remain central to our governance model as we onboard more Copilot users and integrate generative AI into other areas of Kyndryl’s operations.

3. Grant access incrementally to facilitate use case vetting

With tools like Copilot, it’s better to allocate licenses over time than immediately grant full access to the technology. Staggering implementation lets administrators review use cases and assess associated risks before deploying the technology to your entire workforce. 

The Kyndryl experience

Following the preflight assessment period, we began a two-phase rollout of Copilot. This approach allowed a limited number of Kyndryl employees to experiment with the technology prior to purchasing thousands of licenses, giving our implementation team time to determine which use cases provided the most business value.

• Early access program (October – November 2023) As part of Microsoft’s Early Access Program, Kyndryl was assigned 300 licenses that were allocated to employees with approved use cases. A committee with representatives from Kyndryl’s CIO, legal, data privacy and cybersecurity teams reviewed the use cases for compliance with data-sharing policies, especially since the early access terms didn’t offer the same protections as the enterprise agreement with Microsoft. Administrators also collected feedback from early users to improve the implementation process and foster accountability for license holders.
  
  
• General access release (November 2023 – onward) After the early access period, Kyndryl purchased licenses and introduced a structured process for allocation based on approved use cases. Licenses were then assigned in a phased manner after receiving the appropriate compliance approvals from work councils in various countries. In time, employees who applied for licenses were able to choose from pre-approved use cases, reducing the time for license approval. We also allowed team representatives to submit requests for use cases so multiple team members could be assigned licenses under a single use case. 

4. Provide extensive training and education to accelerate adoption

Education is crucial for maximizing the potential of Copilot, so it’s critical to start training users with the technology early in the implementation process.

The Kyndryl experience

Before we granted employees access to Copilot, they were required to attend training conducted in partnership with Microsoft. These educational sessions covered proper Copilot usage, including how to create effective prompts and ways to properly handle data.

We also worked with Microsoft to conduct role-specific training sessions to address the unique needs of different business units within Kyndryl. For example, Copilot specialists led interactive video training with our finance team to highlight use cases and demonstrate the technology’s capabilities in Excel, showcasing how Copilot can be used for data summarization and analysis in finance scenarios.

After we began assigning licenses, we established an internal Center of Excellence to support ongoing training and provide a central location for employees with licenses to:

• Learn about the features and benefits of Copilot
• Find guidelines and best practices
• Access resources such as videos, documents and forms
• Share feedback, questions and use cases with other Copilot users

As an extension of our Center of Excellence, we created a Copilot Champions network of early adopters to share their knowledge with newer users. The group, which now includes roughly 5,000 Champions in 35 countries, provides general application knowledge in a peer-to-peer format and hosts a Teams channel to seek and gather information, exchange best practices and share success stories with other Copilot users.

Education will remain an integral component of Kyndryl’s Copilot program as we onboard more employees and explore additional applications for the technology.

5. Create robust feedback cycles to enhance business value

It’s essential to collect feedback once employees start using Copilot. Internal administrators and Microsoft product teams can use these insights to fine-tune deployment, increase engagement and enhance the Copilot user experience.

The Kyndryl experience

Our Copilot team established feedback protocols for users to document their experiences in four categories:

• Use-case application: Describe how Copilot was applied to the specific use case for which the license was granted
• Overall quality and performance: Rate specific aspects of Copilot’s accuracy, reliability, usability and helpfulness
• Unauthorized access: Help identify the source of any unauthorized access to confidential information or sensitive data to prevent it from happening again
• Technical questions: Submit issues related to Copilot installation, usage, compatibility or other technical concerns

After three to four weeks of using Copilot, licensed employees had to provide feedback in the first two categories. If users didn’t respond to questionnaires in these areas, we reclaimed their licenses and allocated them to other employees or teams. Mandating feedback ensured licenses were being allotted to active individuals or groups who could benefit most from the technology while providing valuable information that could be incorporated into future change management plans.     

We also empowered Copilot users to improve the quality of our organizational data. For example, if a user discovered outdated or inaccurate information, they were asked to report it to the Copilot team so administrators could correct the data on the SharePoint site where it was published or contact the site owner to address the issue.

What’s next in our transformation with Copilot

Generative AI is rapidly evolving, so early adopters will need to continually refine their systems and processes and offer ongoing education and training to take full advantage of tools like Copilot. As we continue to weave Copilot into Kyndryl’s operations, we’ll turn our attention to:

• Creating a user prompt library. To facilitate the adoption and usage of Copilot, we’re developing a library of ready-to-use prompts based on the top 25 use cases from the rollout. Employees can use these curated questions, statements and instructions to generate higher-quality responses, eliminating the need to create prompts from scratch for every task in Copilot.
  
  
• Conducting retrospective sessions. Through early 2025, we’re collecting user feedback to evaluate the Copilot program’s performance and understand challenges. We’ll use the feedback to schedule training sessions with Microsoft specialists to help increase efficiency and boost productivity with Copilot.
  
  
• Integrating with a single point of digital engagement. Our goal is to provide employees with a unified platform that includes all the digital tools and services that Kyndryl offers. By integrating Copilot into our planned single point of digital engagement, employees can use generative AI for all digital interactions and requests within the company.
  
  
• Tailoring organizational change management to individual roles. To maximize the benefits of Copilot, we'll continue refining our change management strategy to address the unique needs and challenges of individual roles. Focusing on targeted training and adoption enhances user value and experience and allows us to measure the impact of Copilot by each job function.

The bottom line

Even as Kyndryl explores the full power of generative AI, Copilot has already yielded significant time savings and contributed to a more efficient work environment for our teams. Our approach and lessons learned can serve as a blueprint for enabling your employees to use generative AI and other AI-powered tools.

Boris Jovevski is Vice President and Technical Specialist in Kyndryl’s CIO office. Risham Sahni is Associate Director of Software Engineering and Technical Specialist, and Peter Hudačko is an Infrastructure Specialist in the CIO office.