Agile Education Case Study

Case Study   

Summary: Improving Forecasting Accuracy with Scrum at Scale

This case study focuses on a large non-profit health insurance provider based in the Midwest. The organization was on a journey to scale their Scrum practices and improve their digital product portfolio, valued at $20 million. With 100 team members spread across six Scrum Teams and four Scrum of Scrums, they had experienced organic growth and success until encountering several significant challenges. That’s when Giovanna and her team were called in. Right away, it was clear to Giovanna that the organization’s primary challenges centered around Product Ownership. Here’s what they saw:

• Lack of a Product Owner Cycle: The organization struggled with an underdeveloped Product Owner (PO) cycle, which impeded their ability to efficiently manage and prioritize product backlogs.
• Executive Metascrum Issues: The absence of a functional Executive Metascrum (EMS) limited visibility into roadmaps and release plans, which were often non-existent, outdated, or decentralized.
• Inaccurate Delivery Forecasts: Delivery forecasts were largely speculative and unreliable due to inadequate release planning.
• Inability to Forecast Team Capacity Needs: The organization faced challenges in balancing team sizes and adjusting support when demand fluctuated. This led to missed goals and delayed releases.
• Low Team Morale and Unhappy Customers: The delays and mismanagement led to decreased team morale and dissatisfaction among customers who experienced delays in promised functionalities.

The Solution and Results

Giovanna and her team took a strategic and incremental approach to address these issues. They decided to implement an EMS reference model, focusing on a single Scrum of Scrums with seasoned POs who had sound practices in place. They would track progress for 6 months (12 Sprints). In addition to the experimental group, they also tracked the control group’s progress throughout. Here’s what the implementation looked like: 

• 1st Month: The team started with monthly EMS events to test the process and address initial challenges.
• 2nd Month: In month two, they added an ask for a transparent release plan with clear dependencies identified. They stored those in a centralized location to serve as a reference for those not involved in the experiment.
• 3rd Month: After the first few months, shifted to bi-weekly EMS events to enhance planning and coordination. They also began reporting on the accuracy levels of scheduled forecasts. Of note, the experimental group’s forecast accuracy started at 65-70%. In contrast, the control group was at a 30-60% range.
• 6th Month: By month six, the experimental group had achieved 80-85% forecasting accuracy. The control group’s accuracy did not fluctuate meaningfully over the experiement.

Key Learnings and Themes

• Customer Satisfaction: There was a notable 10% increase in customer satisfaction scores in the final two months of the experiment compared to the previous year.
• Leveraging Change Agents: Involving internal change agents in the process significantly increased buy-in and effectiveness compared to using external consultants alone.
• Increased Team Autonomy: Allowing teams more autonomy within the Executive Metascrum framework improved participation and engagement.
• Self-Organizing Teams: Teams that were given the freedom to self-organize were more engaged and successful.

Giovanna leaves us with wise words from Steve Jobs that guided her team during this transition: “It doesn’t make sense to hire smart people and tell them what to do; we hire smart people so that they can tell us what to do.” She shared that while she and her team knew that they were Scrum subject matter experts and knowledgable in the framework and change management, they understood that they couldn’t discount the insights and capabilities of those already doing the work day-to-day. Giving them clear, outlined objectives, parameters, and guidelines offered them the structure to encourage autonomy and to allow the teams themselves to define those specifics and nuances to make the process work for them.

Next Steps

Building on the success of the initial experiment, the organization plans to expand the reference model across additional teams and Scrum of Scrums. They aim to use established metrics to monitor ongoing performance and foster a culture of continuous improvement.

About Giovanna Berry

Registered Scrum Trainer and Scrum@Scale Trainer Giovanna Berry is a PMP, CSP-SM, and MBA. She is an energetic, seasoned project and program management professional and leader. For over a decade, Giovanna has led teams focused on custom software development, application development, developing creative designs, implementations, and migrations. Giovanna has always had a passion for training, coaching, and mentoring others.