Turning metrics into meaning: How EQI supports continuous improvement across SCTR

While researchers generate discoveries and programs provide services across SCTR, the small but mighty team behind evaluation and quality improvement (EQI) is ensuring that those efforts are effective, efficient, measurable and continually improving. Jillian Harvey, Ph.D., Evaluation Program director, and Rechelle Paranal, director of Planning and Research, help programs to use data to make better decisions, demonstrate impact, allocate resources wisely and tell the larger story of how SCTR improves clinical and translational science.

Harvey comes from a background in nursing home administration and health services research. Her expertise in quality improvement and outcomes naturally led her to program evaluation work within SCTR.

Paranal brings a different but complementary perspective. With past experience in political science, clinical social work and policy research, she was particularly drawn to the position because of its complexity, collaborative structure and long-term vision.

Measuring impact and driving improvement

Harvey and Paranal explained that EQI’s objectives are two-fold: internal evaluation and improvement and national collaboration. The team evaluates SCTR as a whole by measuring program performance and tracking outcomes and impact. They then use this information to identify opportunities for improvement and advise decision-makers in real time.

“A lot of what we do is really intended to assist leadership in making data-driven decisions, allowing them to take that data to set priorities and allocate resources where needed,” said Paranal.

EQI also collaborates with evaluators from across the national Clinical and Translational Science Award (CTSA) consortium, a grant program funded by the U.S. National Institutes of Health National Center for Advancing Translational Science (NIH NCATS). Together, they share evaluation methods, compare approaches, advance evaluation science and disseminate findings throughout the CTSA network.

What impact actually means

The first step in measuring impact is defining it. Harvey explained that EQI uses the Translational Science Benefits Model (TSBM), a framework for evaluating outcomes, which can be either short term or long term. Short-term measures include traditional indicators of research productivity, such as publications, grant funding, research collaborations and participation in research activities. Long-term impact is determined by broader metrics over months or years, including developing new drugs or medical devices, influencing policy, community health outcomes and economic impact.

Turning data into action

Paranal and Harvey were clear about their goals. “We’re here to help. We’re not here to judge.” They explained that oftentimes people hear the word evaluation and assume it means they are being audited, assessed or criticized. But EQI aims to strengthen programs, improve processes and enhance services, not to give a report card.

To carry out their mission, the EQI team works alongside informatics teams to develop metrics that are incorporated into dashboards that allow leadership to monitor activity in real time rather than waiting for quarterly reports.

This live reporting method helps key stakeholders to identify growth trends, see where resources and services are in high demand and tackle issues as they crop up. Paranal noted that the team always asks an important question: “How will this data actually be used?” This utilization-focused approach helps to prevent unnecessary data collection while ensuring every metric serves a meaningful purpose.

EQI also monitors operational metrics, such as consultation volumes, recruitment activity and service utilization. This way, if demand rapidly increases in a specific area, leadership can use the data to determine whether additional resources are needed.

From strategy to results

Although much of EQI’s work happens behind the scenes, its work leads to real-world successes. The team has been able to identify issues, investigate root causes of obstacles and improve service delivery from constructive feedback shared through Investigator Satisfaction Surveys completed after interactions with SCTR services. Additionally, positive feedback can help to recognize outstanding staff and highlight successful services.

EQI also participated in evaluating a supplemental project involving research billing processes at SCTR’s Research Nexus. The team’s expertise assisted in improving workflows and operational effectiveness.

Currently, the team is leading a pilot project that provides training and real-world experiences to clinical research professionals to meet increasing demand for qualitative research expertise. While outcomes are still forthcoming, the initiative highlights EQI’s involvement in shaping the future research workforce.

In an office as large and multifaceted as SCTR, individual teams and programs can sometimes operate in parallel without fully recognizing opportunities to connect and learn from one another. Paranal sees EQI as the thread that ties those efforts together. Through data collection and evaluation, the team illustrates how SCTR’s programs intersect, where collaborations are occurring and where new partnerships and improvements may be possible.

“We have a lot of great collaborators throughout SCTR,” said Harvey. “It’s not something we can do by ourselves.”

To learn more about EQI, visit the Evaluation and Quality Improvement tab on SCTR’s webpage.