CPED Training Module: Improvement Science

Improvement Science & the EdD A CPED Training MODULE

Before you Begin...

Access: You'll need a CPED website login to access the links in this module. If you don't have one already, please submit a request to be added to our website.

How to use this module: This module is self-paced and will teach you about Improvement Science and how can be applied to your CPED-influenced EdD. To get the most out of the module, we recommend visiting it more than once. The module contains many embedded links, videos, readings, and suggestions for exploring aspects of our online resources and connecting with the CPED community. Before beginning, we recommend that you open a new tab with our website and login.

Fundamentals of Improvement

Improvement Science is a process that allows for the analysis of organizational performance and the systematic efforts to improve it. This process includes theories and practices that support individual, social, and organizational change accomplished through disciplined inquiry.

Developed by Edward Deming in the 1940s, Improvement Science originated in industry. In recent decades, it has been adopted by the field of health care (see Institute for Healthcare Improvement) and most recently in the field of education through the efforts of the Carnegie Foundation for the Advancement of Teaching.

At its core, Improvement Science is meant to serve practitioners who seek to improve their organizational systems through collaborative and applied inquiry. In that respect, it is a natural fit for leaders who enter EdD programs seeking to improve their practice and their organizations. We teach improvement science in EdD programs through applying it to a problem of practice (defined by CPED as “a persistent, contextualized, and specific issue embedded in the work of a professional practitioner, the addressing of which has the potential to result in improved understanding, experience, and outcomes”) that the student identifies in their organization.

The Carnegie Foundation has developed six core principles that guide the work of improvement:

Make the work problem-specific and user-centered
Variation in performance is the core problem to address
See the system that produces the current outcomes
We cannot improve at scale what we cannot measure
Anchor practice improvement in disciplined inquiry
Accelerate improvements through networked communities

A detailed discussion of the six core principles can be found in Learning to Improve by Bryk, Gomez, Grunow, and LeMahieu

These principles are essential to consider when thinking about how to improve an educational organization locally through iterative testing of small changes. These tests are guided by the Model for Improvement developed by the Institute for Healthcare improvement and adaptable to education.

The Model consists of three key questions:

What are we trying to accomplish?
How will we know that change is an improvement?
What change can we make that will result in improvement?

PDSA Cycle: Thee key questions guide the Plan-Do-Study-Act cycle of testing a change idea in a practical setting. Together these two components help students put together their theory for improving a problem of practice.

The Improvement Process

Before the PDSA cycle begins, the student needs to understand the problem of practice they seek to address, the local system where the problem is housed, and establish their theory for improving this problem. As a process, improvement science offers steps and tools by which practitioners can learn to diagnose systems and problems and walk them through an iterative problem-solving process.

Understanding the Problem

Students often bring strong practitioner knowledge about problems they face in their organizations. While this knowledge is very important, the goal of understanding the problem is to go beyond this knowledge to learn more about how the problem is a) understood and experienced by members of an organization (users) and b) defined in the broader context (scholarly and professional literature).

Tools to support understanding the problem are:

Fishbone Diagrams (also known as Ishikawa diagrams or Cause and Effect tool) allow students to work together with others in their organization to detail the roots of a problem.

Empathy Interviews are opportunities for students to have informal conversations with organizational members to understand their personal perspectives about a problem. Examples can be found at Hightech High or the Stanford D-School.

Organizational documents are also useful to understanding and documenting the problem.

Seeing the System

Improvement Science is a systems-thinking methodology. In order to address a problem in practice, a student needs to understand the system that produces the problem. Any given system is made up of:

Processes & Tasks
People (Skills, Numbers, Rewards, Fears, Hopes)
Structures & Job Designs
Culture, Values, & Practices
History & What Has Been Tried Before

Tools that support seeing the system are:

Systems Maps are visual drawings of a system –elements in a system (people, practices, divisions, etc) and the causal relationships between these elements.

Process Maps are visual drawings of the step by step flow of a process as a whole.

Organizational documents can help to see how the system works as well.

These tools help the student see the bigger system and the places where the problem might be influenced or impacted.

Establishing an Aim

Once the student has a good understanding of the problem and the system, they can begin to think about solving the problem. The problem is the gap between the existing state in an organization and the desired state. It is the desired state the guides the establishment of an aim. According to the Carnegie Foundation, an Improvement Aim is a goal for an improvement effort that answers the question, "What are we trying to accomplish?" Improvement aims should clearly specify how much, for whom, and by when.

Criteria for a good aim statement force students to be targeted with time and measurable change.

Specific yet representative of overarching goal (Compelling)
By how much? (Measurable, specific numerical goals)
By when? (Timeframe)
For whom? (System to be improved, setting, and population)

Developing a Theory of Improvement

Students develop their theory of improvement using the three questions from the model of improvement and a driver diagram which is a visual display of the aim, places or processes in the system that might “drive” change, and the change idea or intervention.

Identify Measures

Improvement principle #4 states, “We cannot improve at scale what we cannot measure.” Improvement requires practical measurement that serves to determine if a change is working, how and where in the system that change is working, and if the change is improving the system. Quantitative or qualitative data is gathered during the PDSA cycle. Those data inform four measurements: process, driver, leading and lagging outcome, and balance measures.

Test Change

Testing the changes is where the student implements their change ideas in a PDSA cycle. These iterative cycles can last up to 90 days depending on the change idea and timeframe for the dissertation in practice. Additionally, students may do more than one for their final dissertation.

Tools for doing a PDSA include:

PDSA Sheet

PDSA Checklist created by the Improvement Collective

Spread the Change

Spreading the change speaks to the results of continuous iterative cycles and the learning that addresses variability in different contexts. In an EdD program, it can also speak to the student’s leadership agenda and sharing their learning in professional settings and beyond.

The Dissertation in Practice

Why Improvement and the EdD?

Improvement science is an applied methodology that encompasses many tools that a leader can utilize in practice to engage other in the improvement of an organization or pressing problems. In this way, improvement science is a leadership skill teaches EdD students how to:

Bring communicate and collaborate with stakeholders;
Develop an action plan focuses on systems change vs. silver bullet solutions; and
Provide evidence of improvement through discipline data collection.

For the Dissertation in Practice, improvement science can be mapped across EdD programs and methodology courses, providing a framework for developing the dissertation in practice. The process of improvement science can be mapped into a 3-4 year program with each part of the process. For example, understanding the problem and seeing the system might be part of the initial inquiry courses with assignments that include learning the tools listed above and developing the first part of their dissertation in practice. Establishing an aim, theory of improvement and measures might be second level inquiry courses. The PDSA cycle would encompass the implementation and data collection towards the end of the program. Alternatively, PDSA cycles might be taught in all inquiry and/or content courses as laboratories of practice giving student opportunity to build experience while learning theory about improvement science.

An example of how the University of Pittsburgh implements improvement science in their EdD program can be view in this video.

Readings & Resources

Books & Readings:

Bryk, A. S., Gomez, L. M., Grunow, A., & LeMahieu, P. G. (2015). Learning to improve: How America’s schools can get better at getting better. Harvard Education Press.
Crow, R., Hinnant-Crawford, B. N., & Spaulding, D. T. (Eds.). (2019). The Educational Leader's Guide to Improvement Science: Data, Design and Cases for Reflection. Stylus Publishing, LLC.
Hinnant-Crawford, B. N. (2020). Improvement Science in Education: A Primer. Stylus Publishing, LLC.
Perry, J.A., Zambo, D., & Crow, R. (2020). The Improvement Science Dissertation in Practice: A Guide for Faculty, Committee Members, and their Students. Myers Education Press.
Mintrop, R. (2019). Design-based school improvement: A practical guide for education leaders. Harvard Education Press.
Langley, G. J., Moen, R. D., Nolan, K. M., Nolan, T. W., Norman, C. L., & Provost, L. P. (2009). The improvement guide: a practical approach to enhancing organizational performance. John Wiley & Sons.

Additional Learning Resources:

What's Next?

Learn and Share

As you use Improvement Science in your CPED-influenced EdD program, feel free to check out CPED's other support resources.

More questions? The CPED Team is ready to help. Contact us here.

Appreciate