Beyond Citations: How Do We Measure Real Impact?

Why Policy Change, Patient Outcomes, and Practice Matter More Than Journal Metrics

Publish or perish, they say. But what happens after we publish?

For too long, academia has treated citations like currency—more is better, h-index is king, and journal impact factor is the holy grail. But when you’re working in real-world health systems, those numbers don’t always tell the full story. You can have a highly cited paper and zero change on the ground. That’s not impact. That’s inertia in a lab coat.

This week, let’s explore what it means to measure real research success.

The Case of the Invisible Implementation

I once reviewed a paper which end up having 500+ citations, a systematic review on NCD management in Africa. Impressive. But when I asked district health managers about it, none had heard of it. Meanwhile, a two-page policy brief from a local NGO had sparked a region-wide review of hypertension protocols.

Lesson: Citations are a signal—but not always of relevance.

What Should We Be Measuring Instead?

Here are four metrics that matter more than your Google Scholar profile:

1. Policy Uptake
   Did your findings inform a national strategy? Were your recommendations cited in a Ministry of Health circular? If your paper changed a guideline, that’s impact. Bonus points if it was used without a lengthy donor-funded policy dialogue.

2. Practice Change
   Has clinical behavior shifted? Are providers doing something differently because of your work—ordering fewer unnecessary labs, improving follow-up, using a new protocol? You may not see it in a graph, but you’ll hear it on the ward.

3. Patient Outcomes
   Did fewer patients get readmitted? Were more diagnoses made earlier? If research can move a number that matters to a patient—like BP, A1c, or survival—that’s the gold standard.

4. Community Engagement
   Was the research communicated back to the community it studied? Did it build trust, change perceptions, or empower local action? Community validation doesn’t come with a DOI, but it’s powerful.

How Do We Track This?

• Policy Tracking: Keep a “policy influence log”—track every meeting, mention, or memo where your research shows up.

• Implementation Dashboards: Use platforms like DHIS2 or Power BI to monitor indicators linked to your work.

• Qualitative Follow-Up: Conduct interviews or focus groups with stakeholders to ask: “What changed because of this?”

• Case Studies: Document real-world stories of how your evidence was used (or not used), and why.

The Future of Research Evaluation

If we want research to matter, we have to stop evaluating it like it’s a popularity contest. Instead of counting citations, let’s start counting consequences. Who was helped? What system improved? What didn’t change—and why?

Because in the end, a perfectly designed study that sits on a server helps no one. But an imperfect one that shifts practice? That’s success.

In summary
Real impact isn’t measured in references—it’s measured in results.

Next Up: Research Translation in Non-Communicable Diseases (NCDs): The Missing Middle
Why NCD research often struggles to move from findings to frontline care—and how we can close that gap.