Towards Core Competencies for Health Policy and Systems Research (HPSR) Training: Results From a Global Mapping and Consensus-Building Process

Background

Health systems everywhere are facing unprecedented challenges as they seek to respond to ageing populations, changing patterns of disease, new technologies and models of care, and catastrophic events such as the coronavirus disease 2019 (COVID-19) pandemic. Health inequities are also persisting. More concerted efforts and collaborative research are needed in order to strengthen health systems and achieve the Sustainable Development Goals (SDGs).^1-3 Health Policy and Systems Research (HPSR) has emerged as a scientific field that seeks to generate and apply evidence to support efforts to strengthen health systems.⁴ HPSR is defined as “… a field that seeks to understand and improve how societies organize themselves in achieving collective health goals, and how different actors interact in the policy and implementation processes to contribute to policy outcomes.”^4,5 HPSR aims to answer questions that inform policy and practice concerns and generate lessons for application in solving health systems challenges; the field draws on many established disciplines such as medicine, economics, anthropology, sociology, and public health and utilizes a range of methods to respond to the research questions identified.^4,6 In recognition of the pressing demands on health systems, the field of HPSR has seen rapid expansion and evolution in recent years with the creation of HPSR courses and teaching programmes worldwide as well as communities of practice. Investments in HPSR training, particularly in low- and middle- income countries (LMICs), by governments and local development agencies has also contributed to this expansion.

Whereas significant gains have been made in HPSR methods and training, there remains large variability in what HPSR researchers do and how they should be trained. Health Systems Global (HSG), a membership organization of researchers and practitioners that seeks to strengthen HPSR, recently conducted an analysis of HSG conference participation and found progress in the breadth of content and geographical equity of participation.⁷ However, other analyses demonstrated that extensive gaps remain.⁸ Importantly, the 55% of the HSG members who responded to the survey were involved in HPSR education activities at the time, bringing to attention the need to strengthen HPSR teaching and training of individual researchers and their organizations.

A 2014 mapping study of global HPSR teaching capacity demonstrated that training opportunities, though numerous, were concentrated in few geographic regions and the scope and modalities of HPSR training and additional assessment show similar irregularities.⁹ This and other studies demonstrated a lack of consistency within the field of HPSR teaching.^9,10 The Collaboration for Health Policy and Systems Analysis in Africa’s (CHEPSAA’s) work¹² and HSG’s repository of teaching and learning resources (https://courses.healthsystemsglobal.org), which includes CHEPSAA, KEYSTONE, and other pioneering HPSR training offerings, demonstrated there is still limited agreement on what the unique characteristics of HPSR are, and about what distinguishes this field from other broader areas such as global health, health services research or public health.^11,12 What is evident is the huge variability as well as richness of HPSR training that currently exists. When also accounting for HPSR programmes in high-income settings, a lack of coherence across the field becomes apparent.^9,13

One route to respond to this capacity gap is to develop a competency framework, or a set of competencies which are often organized under “domains” or categories, for HSPR in order to support the development of HPSR training programmes on a wider scale. A competency is the ability to apply a set of related knowledge, skills, and abilities needed to successfully perform core work functions or tasks.¹⁴ Building on the advances in competency-based education domains for global public health,^14-16 a package of core HPSR competencies can provide a point of reference for establishing high-quality education and training standards. Competencies provide comparability of desired outcomes and articulation of capabilities that researchers who complete training programmes should have across contexts. They can also provide the basis for critical review of curricula and be adapted for specific target audiences. Competency frameworks can be used as a resource and pragmatic starting point to accelerate the development of country-specific HPSR teaching offerings, such as where there are demands for scaling up national training programmes.

This paper seeks to address these issues by providing guidance on the core elements of what HPSR researchers should be able to do. The competencies are aimed at researchers who lead, conduct and communicate HPSR research. As HPSR is usually taught at later stages of graduate or postgraduate training, a working level of research competency is assumed. We seek to make recommendations for a set of HPSR competencies which can underpin country- or institution-specific HPSR training programmes and be tailored to specific contexts and target audiences.

Methods

We undertook a stepwise approach, with data collection and analysis occurring in four phases, resulting in a set of interim products (see Figure). Each phase focused on a different kind of input and revision of the competency framework. First, we explored previously established HPSR-relevant competency frameworks that have been used in different contexts, and then proceeded with a series of phased engagements with an increasingly broad audience to build consensus and validate findings among the Teaching and Learning Thematic Working Group (TWG) within HSG as well as HSG membership and additional HSPR networks. The TWG (https://healthsystemsglobal.org/thematic-groups/teaching-and-learning/) is a group of global researchers and experts with experience and interest in teaching and learning for HPSR, which convenes periodic activities, contributes to HSG’s bi-annual conference, and maintains a listserv and repository of teaching and learning resources for HPSR. Our approach was iterative, enabling dialogue — not only on the focus and structure of the competency framework, but also on how it will be applied in practice to ensure buy-in from key stakeholders. We adapted our approach based on the experiences and results of each phase in order to achieve these goals.

Close

Figure. Summary of the Iterative Review and Analytical Processes Utilized in This Study. Abbreviations: TWG, Thematic Working Group; KIIs, key informant interviews.

Phase 1: Scoping of HPSR Competency Frameworks

We conducted a scoping review of existing competency sets in the fields of global health and public health, which have relevance to HPSR due to the overlap of these fields with HPSR.^15,16 Given our target audience, we looked in particular at existing frameworks geared towards graduate and post-graduate audiences that included a component of research and/or analysis.^15-18 A preliminary HPSR competency domain set was developed through thematic analysis of existing competency frameworks, identification of the convergence and divergence between them as well as knowledge of the field of HPSR training. For the latter, we used the HPSR training database¹² and mapping of HPSR courses and institutions⁹ as a sampling frame to select all HPSR courses that utilized competency-based teaching or assessment. Our mapping study had previously identified that 11% (n = 18) of HPSR courses and trainings included competencies while 76% of institutions (n = 131) indicated that competencies existed at the programme level but were often not specific to HPSR.¹⁴ An initial draft set of global HPSR competency domains was developed to serve as the basis for discussion and input during the following stage. In addition, we employed snowball sampling to identify courses or other training programs at well-recognized centres of expertise in HPSR to identify competencies through posted information as a part of journal and working papers, websites and repositories. These included major HPSR programmes, in particular CHEPSAA, which had previously developed competencies for its own HPSR training programmes.^10,19

Phase 2: In-Depth Interviews With Key Informants

We sought to gain a detailed understanding of the field of HPSR and to solicit insight about integral HPSR competencies through in-depth interviews with diverse leaders who are actively involved in teaching HPSR. Thirteen in-depth interviews with key informants were conducted between August 2017 and March 2018. Informants were purposively selected after a careful search the HSG training database, our prior mapping exercises, and our professional networks to identify respondents with leading roles in designing, teaching and managing HPSR programmes. They were selected based on identifying key centres of expertise regarding competency-based education in fields related to HSPR and delivering institutionalized HPSR training programmes, and also identified by peers who rated them as highly qualified and engaged in HPSR teaching activities.¹² The interviews had two parts: first, respondents were asked about their aspirations for HPSR training programmes including desired impacts and how to address current capacity gaps, how competencies are utilized across programmes that they are involved with, how existing competencies are being utilized in HPSR-related programmes within institutions, and on how a set of competencies for HPSR could or should be differentiated from public health or global health, given areas of overlap identified in Phase 1. During the second part of the interview, the respondents discussed the initial competency domains and provided feedback based on example frameworks as well as their experiences as educators in HPSR.

All interviews were transcribed and reviewed. Two study team members (MS and DB) reviewed and analysed all transcripts and notes thematically and identified key topics associated with the study aims as well as illustrative quotes that best reflected the predominant respondents’ views. Divergent views were included to indicate areas where further consensus and research was required. The analysis compared the perspectives of respondents from high-income and LMIC in order to ensure that we captured any differences in perspectives based on these different contexts. Findings from these interviews, including on the issues that frame the state of the field of HPSR and the more general themes related to the field are presented in the results section. The competency domains were revised based on findings from these interviews.

Phase 3: Expert Consultations

The initial competency domains and draft competencies were further revised following a series of consultations with experts and practitioners in the field of HPSR training to conduct a framework analysis of the drafted competency domains and specific competencies.²⁰ This included a presentation and open discussion of preliminary findings at a session on building capacities for resilience in LMICs at the European Public Health Association conference in November 2017, which included participants from across the European region representing academic institutions and practitioners. Further consultations were held in May 2018 with HSG’s Teaching and Learning TWG’s Strategy Group including the founding co-chairs of the TWG, a representative from the World Health Organization (WHO), representatives of several leading HPSR institutions and networks, including CHEPSAA, and representatives of the HSG society leadership. This group represented different geographical regions—including LMIC and high-income settings—and diverse disciplinary training backgrounds though all had teaching experience in HSPR. The draft competency domains and the organization of the specific competencies were discussed via a conference call during which detailed notes were taken; and each participant provided substantive written feedback after the discussion. All inputs were analysed thematically, and key recurring points were systematically incorporated into a revised draft of competency domains and related competencies under each domain.

Phase 4: Open Webinar and Presentation at HSR Symposium

At the final stage, we sought exposure and feedback from a broader audience of HPSR researchers and educators. In July 2018, a webinar was held for HSG members which involved a self-selected group of participants who were either teaching or leading HPSR training programmes or planning to develop training for HPSR in the future. The webinar was promoted via HSG email groups and listservs, LinkedIn, and the HSG newsletter, and experts/leaders of teaching and learning HPSR were also individually invited. The aim of this final phase was to receive feedback on the competency domains and the proposed specific competencies under each domain in order to highlight the highest priority areas.

The webinar methods drew on modified e-Delphi techniques to establish consensus on the competency domains and initial draft of competencies. This process has a long history in developing a clinical consensus and it is increasingly applied achieve consensus in health policy development.²² This involves typically developing questions through qualitative research and a literature synthesis, and discussion between rounds of anonymous voting.^21,23 The modified Delphi technique used in this study was appropriate because the consensus on some competency domains took time to develop and a significant learning process unfolded for many of the participants as they considered each other’s views. The webinar involved two rounds of survey questionnaires, one before and one after the interactive webinar. The anonymous pre-webinar survey (see Supplementary file 1 for the full survey tool) was distributed to all 59 webinar registrants using Google Forms; 19 responses were recorded. The survey was divided into two parts: (1) the significance of each domain to the field of HPSR and (2) the significance of a list of specific competencies within each domain to the field of HPSR. Participants were asked to rank each domain on a Likert scale and to provide comments under open-ended questions. The pre-webinar survey sought to provide inputs to help revise and restructure the competency set ahead of the webinar and to identify priority topics related to the competency set to be addressed during the webinar. After the discussion, some domains were merged as they were seen as overlapping and others were rephrased.²³ The post seminar survey was shorter and included a more refined version of the competencies – this modification was required to reflect the evolving and diverging views – with many participants forming their views during the engagement process. The inclusion of online ‘live’ discussion was valuable to capture the richness and enable deeper understanding of the justification for the choice of competency domains and competencies and to understand if the consensus was genuine²⁴ In the anonymous post-webinar survey, respondents were asked to: (1) consider the domains and sub-competencies as a set rather than stand-alone, (2) prioritize competencies for mid-level HPSR researchers, and (3) consider each competency as having levels that are programme- or audience-specific. Further revisions to wording and to how the competencies within each domain were prioritized and presented were made. The overall competencies framework was adapted based on the feedback from the webinar and post-webinar survey. Fourteen out of 59 participants submitted post-webinar survey responses.

Results

The final set of core HPSR competencies outlined in Table evolved over the course of the phased research process. In this section, we describe the key findings from each phase, and then discuss the final competency framework that we developed as a result.

HPSR Competencies Scoping (Phase 1)

The only model that fit our requirements of an explicit HPSR competency framework was the set of HPSR leadership capabilities and competencies by the CHEPSAA and specific to African Health Policy and Systems Research and Analysis (HPSR+A).^25,26 It was fully developed and linked to specific elements of the training programmes and materials. The model was designed for an African context, but with wider relevance as it operationalized elements covered in the learning objectives of other HPSR courses. For these reasons it was chosen as a reference model. After canvassing a broad range of potential competency frameworks, two further relevant frameworks were identified: the global health competency model developed by the Association of Schools of Public Health and the Education Committee’s Master’s degree in Public Health core competencies.¹⁶ Although not specific to HPSR, both have significant overlap with HPSR-focused course, CHEPSAA as well other courses (as reflected in the learning objectives).

The three competency frameworks were useful as a reference for developing an initial HPSR competency set primarily due to their similarity in content, and appropriateness given our research aim and target audiences. The emphasis and organization of the domains across the frameworks differed, however, and provided a useful basis for deliberation about what is needed in a framework for HSPR. In all three frameworks, the competency domains and detailed competencies under each domain were organized in a table format and one also had a visual representation. The CHEPSAA framework took a unique approach to organizing the content and presentation of the domains and sub-competencies within domains. This framework was developed to support leadership training for HPSR and therefore emphasized personal qualities including communications and listening skills, writing and teaching skills, networking, and research skills. Similarities in the audiences of the different frameworks served as an additional rationale for selecting the three samples as a basis for the development of the HPSR competencies. While Association of Schools of Public Health’s model is geared towards master’s level public health students, CHEPSAA’s framework is geared towards early career researchers including graduate students and post-graduate professionals.

Qualitative Interviews (Phase 2)

The in-depth interviews provided important context on the state of the field of HPSR and several fundamental considerations to inform how the HPSR competency framework and training strategies going forward fit into the overall growth and evolution of the field of HPSR. They focused on the priorities for the HPSR competency domains in the context of broader considerations for the state of development of the field of HPSR and whether the field itself is “ready” to have a set of competencies. The interviews also discussed the role of HPSR researchers in producing findings but also communicating and engaging with different constituencies to inform policy, and the potential challenges for meaningful and consistent assessment across competencies. Below, we present key emerging themes.

HPSR: An Evolving Field that Requires Modern-day Skills

Multiple respondents perceived the field of HPSR as young and continuously growing, building on two decades of dramatic change. As such, the capacity needs of the field have evolved and will continue to do so in the years to come; thus, the competency framework needs to build from a deeper understanding of the evolution and diverse application and priorities across users and contexts. Perceptions are changing around the value and role of HPSR, though lingering critiques and traditional perspectives remain:

“We had a group of traditional professionals—important researchers. They were old fashioned—from the reform in the 1980s. They were more political than scientific. Now, new researchers can balance these skills better. We had a tension between these groups. After some crisis, we are in the direction of a civilized consensus within the institution. It is good for students to see these different points of view—we don’t want to protect them. It is a good, more consistent direction from a scientific point of view and also considering the...reality as it is now” (LMIC, academic).

As the next generation of HPSR researchers is trained, emphasizing the historical trajectory, the potential for growth, and the value of the HPSR field can help guide continued evolution and growth.

Respondents offered both larger philosophical as well as specific skill-oriented perspectives on what the particular challenges and opportunities of working in HPSR are and the associated skillsets needed. While the field draws on many different disciplines including global health and public health, there was agreement that the kind of questions that HPSR aims to address as well as the kind of impact that it can yield present unique opportunities but also challenges:

“[HPSR] takes quite extraordinary individuals. It is not enough to have excellent epidemiologists with wonderful designs that sit at the computer and deliver great reports published in wonderful papers. There is something more with the best in HPSR, and we need to be very careful with this. It is not people like me, it is people in the new generation. The new generation, we expect to get both solid competence in mixed methods with some focusing on both sides, but you also want them to have communicative competence and leadership attributes” (HIC, academic).

Ultimately, as the HPSR community continues to explore what sets it apart and also how best to integrate with diverse stakeholders and respond to real-world challenges, there should be continued efforts to more clearly articulate the added value of high quality HPSR and the skills and knowledge that are required to deliver it.

The Role of Disciplinary Skills in a Multi-Disciplinary Field

Ultimately HPSR is considered to involve ‘social research’ competencies. However, respondents offered different perspectives on the appropriate role for prior discipline-specific training and specific disciplinary backgrounds that may be well aligned to enable high quality HPSR. This was important as we developed the competency framework and considered what prior knowledge and skills could be assumed for graduate-level and professional learners in HPSR. While having expertise in a particular discipline can be quite valuable, being able to understand, appreciate, and listen to ideas and considerations across disciplines was generally seen as being at the core of effective HPSR:

“There isn’t one blueprint. Everyone brings their own unique background and context with them. There is not a core HPSR training—you “arrive there” from elsewhere and bring that background with you. We cannot be prescriptive about what this prior background should look like because it is very individualized” (LMIC, academic/international advocate).

Respondents also provided differing perspectives on the appropriate or desirable backgrounds for HPSR, but generally saw the value in having an in-depth background in at least one discipline. Furthermore, respondents agreed on the benefits of engaging with colleagues and drawing upon different disciplines in order to most effectively ask questions and understand complex health system problems. One respondent drew a parallel between how a family physician needs to understand enough about other specialties in order to know when to refer a patient for additional tests or treatment, and whether a similar kind of thought process can be applicable to HPSR.

Some respondents argued to avoid pitching different disciplines against each other and instead finding ways to draw on many of these in a training programme, enabling the students to grapple with real-life issues:

“If someone is coming into the field of HPSR, they are often trying to lobby for their piece of the pie or your approach to things. If someone has never come across any economics, for example, before and it has never been part of their training, it is an enormous battle to first of all educate that person and then you’ve got to convince them…If you’re talking to someone with the background who has covered all of the technical pieces, they will say, ‘oh, yes, I can see your point.’ And then you can move to the crux of the matter” (HIC, academic).

Role of Researchers in Communicating and Using Evidence From HPSR

Respondents provided several perspectives on the roles and responsibilities of researchers in engaging with other stakeholders to ensure effective communication and use of evidence produced through HPSR. While there is a broad debate on the importance of closing the gap between generation and use of evidence, the need for the HPSR competencies to include both these aspects came out as a common theme as an essential component of HPSR. Differing perspectives came out in both the key informant interviews and also in the Phase 3 survey results. Some felt that this was a core aspect of HPSR, while others felt it was an important role for researchers in general and not unique to HPSR. There were also differing perspectives on to what degree HPSR researchers should engage in advocacy and what kinds of added value and/or biases this might result in:

“When I was a senior advisor to the minister, frequently we had to welcome people from academia and other places who wanted to show us results from studies, and the conclusion at the end was: ‘So what?’ ‘ So what do you expect us to do with this knowledge now?’…And the most common answer we would get is: ‘I am a researcher. … I just deliver the information’” (LMIC, public sector).

Overall, respondents felt that it was valuable for HPSR researchers to engage in the full cycle of research, including translation into practice and engaging other relevant stakeholders. However, best practices to undertake this process at different levels and the degree to which HPSR researchers have ethical or professional obligations to be advocates and focus on knowledge translation versus generation were areas of continued debate.

Consensus-Building (Phases 3 and 4)

Given the novelty and contestation of the competencies for the field of HPSR, findings from the qualitative interviews suggested a strong interest and enthusiasm for continuing dialogue around competencies. As we solicited perspectives from key stakeholders, our learning through phases 3 and 4 included (1) validation of the utility and the demand for such a general framework to guide individual countries, organizations, and leaders in developing HPSR curricula as well as (2) convergence around several ways of framing the relationships between the competency domains as well as (3) what makes training for HPSR unique. We report the findings from phases 3-4 in tandem as they built upon each other and took place somewhat concurrently.

During our TWG’s strategy group meeting, participants provided feedback regarding definitions and differentiating between competency domains, orienting sub-competencies towards higher levels of Bloom’s Taxonomy appropriate to our target audiences, and considering the ability to assess competencies within each domain. We filled gaps identified by the strategy group and in response to questions and ideas from conference participants. For example, we added additional competencies under the domain related to assessment of health system policies and programs as well as under ethics in order to enhance the focus of the framework on considering the complexities of working with a range of stakeholders to design research and ensure that it would benefit the desired target audiences – particularly vulnerable populations. There was also a focus, particularly with participants at conference sessions, to test direct application of the competencies into contextually relevant HPSR training programmes and continue to revise the framework based on that experience.

The webinar with HSG members was a central point for consensus building as it included a broad base of participants, including educators and programme leaders but also recipients of HPSR training. The webinar resulted in a rich discussion and generated new perspectives and insights, including the option of thinking about the competencies as an iterative cycle that learners and researchers follow from understanding health systems contexts and identifying research questions through to dissemination and uptake of research findings. Another key recommendation was about the value of structuring the competencies not as a list of stand-alone knowledge and skills areas, but in relation to each other and as a part of an inter-related package of competencies needed for effective HPSR work. Illustrative of this evolution was the pre- and post-webinar survey, where respondents did not rate all the competency domains as equally important to the field of HPSR and provided insightful comments as to the reasons for this and the changes in their views before and after the webinar. Respondents were asked about the relevance of each domain to the field of HPSR; responses were divergent as the “lead and mentor” and the “building partnerships and networks” domains were rated as less highly relevant to HPSR in the post webinar survey compared to the pre-webinar survey responses. These findings are in contrast to many of the key informants’ perspectives, which instead put embeddedness, leadership, and collaborative work at the very heart of the field of HPSR.

The different priorities observed in engaging with multiple stakeholders during Phases 3-4 highlights that HPSR competency domains have different relevance across contexts and apply to different types of training leaders and audiences. Thus, the weights and importance of particular domains and their competencies will vary in each setting, and it is not possible to derive a single definite answer that works for all audiences and learners in HPSR. Ultimately, the competency domains identified through this study reflect a consensus on a global reference point, but they can and should be adapted to specific contexts and be flexible to audience needs.

HPSR Competency Domains and Core Competencies: A Final Set

Although each competency may relate to other disciplines and fields in global health or public health, the study participants identified that the HPSR competencies related to specific knowledge, skills and work practices in the field have to be taught and applied as a coherent set that underpins rigorous HPSR.

The final competency framework (Table) was considered to be a set of core HPSR competencies by HPSR training programme leaders and respondents actively involved in designing and providing HPSR offerings. The table provides explanatory descriptions of each domain as well as the set of competencies that fall under each one. An important caveat is that there are differences in what competencies mean for different disciplines and in diverse settings; HPSR training programmes need to adapt their competencies to fit these different contexts. While they will have differing relevance across contexts, they can serve as a reference point for the continued development of HPSR training initiatives. Ultimately, these domains reflect a conceptualization of the role of researcher as either being a detached observer or a leader and influencer of a rapidly developing health systems agenda.

The order of the competencies is not prescriptive, and the domains of the framework often do not get taught or utilized sequentially in practice; for example, leadership and mentoring can be valuable at all stages of a research process, and ethical considerations may arise at any time. The exact focus of the competencies and their sequencing will vary in each context to ensure best fit and outcomes. The framework can be seen as an analytical tool that can be adapted to diverse contexts, and which should follow and inform—and not dictate—the actual starting point, sequence of needs, and priorities for a particular audience.

Particular emphasis was given to the leadership role of HPSR researchers, which enables them to undertake a full cycle of HPSR, including the policy engagement and communication of findings towards generation of new research agendas. However, a common theme was the importance of co-production of research and working together in partnerships and networks. Finally, the communication of findings to a wide range of stakeholders is a key domain of the HPSR competencies set, which in itself leads to knowledge update and a new cycle of generation of policy questions.

Importantly, there are a few overlaps between competencies in each domain and the methods utilized in each phase of the study elicited diverse preferences among the respondents—therefore the grouping of competencies sets under each domain should not be taken to be rigid. For example, some competencies under the Domain 1 on understanding health systems are also relevant for Domain 6 on building partnerships and networks. In addition, Domain 5 on leading and mentorship relates to the final Domain 7 and communicating and translating knowledge as both rely on foundational skills related to capacity strengthening and effectively engaging with peers, junior colleagues, and other stakeholders.

Discussion

This paper sought to identify and describe a competency framework for HPSR drawing on qualitative data enquiry and consensus building exercises. This framework was viewed by participants as critical to the development of HPSR training programmes worldwide.

We acknowledge the following limitations. Given that HPSR training boundaries and priorities continue to be refined and evolved, we started from a very broad understanding of the field and of competency-based education, and purposefully aimed to capture a diversity of perspectives, including leaders in the field of HPSR globally as well as other key HPSR researchers with interest in training and learning. We sought to ensure diversity through conducting online events and providing opportunities for written feedback over periods of time, but may not have captured a full range of critical thinking on HPSR teaching and learning. We also did not capture a level of detail to specify the particular dimensions of current debates related to inequity, racism, sexism and gender inequities, and other social issues around diversity and inclusion that merit attention across educational endeavours. We also recognized that, while HPSR is a multi-disciplinary field, there was limited consideration of multi-sectoral research or the importance of specifically collaborating to answer research questions across sectors. These kinds of research questions are critical to inform the SDG agenda and we see HPSR starting to given attention to these issues.

Each phase of this study process yielded important and different feedback, which contributed to the final version of the competency framework. One of the most challenging aspects of the study was to carefully consider all of the inputs that were often significantly divergent and to work towards agreement. As the field of HPSR continues to grow, these competencies can serve as a basis for advancing debates about what competencies are required across contexts and to inform further research about how best to strengthen capacity for working within this complex field. As we saw during each phase of this study, the need to adapt and contextualize HPSR training in order to be responsive to the priorities of each health system is a central theme that came out even as we engaged with many respondents who had different understandings and experiences of the role and meaning of competencies to guide curriculum development. Further work is needed to refine how we describe and use competencies across contexts, and to also develop levels of achievement within each competency domain, or alternative approaches to assessing capacity strengthening across these domains. As the field of HPSR continues to be constructed and refined, delving deeper into the best ways to assess competence across the diversity of people engaged in HPSR is a challenge that requires additional work and is a valuable effort to ensure that the competencies included in the framework are as relevant and measurable as possible.

Our results demonstrated some important evolutions in thinking and priorities, as did our deliberations at each stage in the study process. At stages one and two, we noticed the limited work specific to HPSR in this space, while also recognizing CHEPSAA’s ground-breaking work on which we have aimed to build. In stage three, our consultations reinforced the demand for such a framework and enabled us to capture additional perspectives, including from teachers and public sector workers who want to enhance HPSR capacity across different contexts. In stage four, we identified that different groups of participants in the webinar and pre- and post-webinar surveys may have had different priorities, particularly regarding the role of leadership in HPSR. Research experts leading HPSR research and capacity building initiatives may be attributing more importance to the leadership of overall HPSR programmes in contrast to HSG members who may be focused on research/training in relation to specific areas of the framework.

Enthusiasm for the framework and recommendations to adapt and use it have been strong from early on in the development process. The final competency domains and the top-ranked sub-competencies for each domain were presented and discussed at the TWG’s organized session at the Health Systems Research Symposium in Liverpool, United Kingdom in October 2018, receiving positive reception and thoughtful questions by participants. In addition, the draft framework was utilized to inform the development of a national training plan for HPSR in Malaysia supported by the WHO. The HPSR domains from November 2017 were considered during a process led by the Institute for Health Systems Research at the Ministry of Health of Malaysia in collaboration with London School of Hygiene and Tropical Medicine (LSHTM) and WHO which sought to develop a plan for a nation-wide HPSR training programme. The set of domains presented in this paper were used to develop HPSR competencies tailored to the national context of Malaysia. These domains are being used to further adapt the competencies and accelerate the development of HPSR in Malaysia and in the Western Pacific region through consultative processes with key stakeholders and communities in 2020-2021. The process will help to test the adaptability and versatility of the framework across different contexts.

Within a young and evolving field that includes a wide diversity of disciplines and contexts where research is being implemented, it can be important to develop competencies and training content for different audiences. The competency framework is not intended as a blueprint for HSPR training, but as a resource and benchmark which can underpin and enable the development of institution- and region-specific competencies as well as competencies with different thematic foci. Thus, the findings presented in this paper are not an end point, but rather a step in an ongoing process of developing an operational competency framework for the field of HPSR. It builds on the pioneering work by CHEPSAA as well as parallel efforts in the field of global public health.^15,16,26 Overall, the findings suggest that we will need to consider capacity strengthening activities that leverage the particular strengths of HSG and other leading organizations and networks engaged in HPSR as a global society of people working in HPSR in diverse settings around the world.

Conclusion

This paper argues that clearly identifying the skills and knowledge that pertain to HPSR and embedding these into a core competency set is critical to supporting HPSR capacity strengthening. While we propose a set of HPSR competencies derived through consensus building processes within key communities of practice, these are a reference point that should be adapted, implemented, and assessed to fit different country contexts. The field continues to evolve, but we believe this set of competencies can enable research leaders and educators to design and implement effective HPSR training that supports learners to produce stronger evidence, effectively engage in the entire cycle of research generation, and inform the strengthening of health systems.

Acknowledgements

This work was undertaken as a collaborative effort within the Thematic Working Group for Teaching and Learning Health Policy and Systems Research, HSG. We would like to thank all of the respondents who engaged in this participatory process – often multiple times during the study period. We appreciate the insightful perspectives, challenging questions, and commitment to helping achieve a product that will have value for the field. We also wish to thank the HSG secretariat and many Teaching and Learning Thematic Working Group members for their support and encouragement. Finally, we appreciate Professor Lucy Gilson’s very helpful feedback and questions on an earlier version of the manuscript. The named authors alone are responsible for the views expressed in this publication.

Ethical issues

London School of Hygiene and Tropical Medicine (LSHTM) ethics committee reviewed and approved this study (LSHTM ethics ref: 14356/RR/8627).

Competing interests

Authors declare that they have no competing interests.

Authors’ contributions

MJS, AR, and DB collected and analysed the data for this paper. MJS and DB wrote the first drafts of the manuscript. All co-authors had substantive inputs to drafts of the manuscript during at least two points in time. AR supported with editing and formatting of the manuscript.

Supplementary files

References

1. Hogan DR, Stevens GA, Hosseinpoor AR, Boerma T. Monitoring universal health coverage within the Sustainable Development Goals: development and baseline data for an index of essential health services. Lancet Glob Health 2018; 6(2):e152-e168. doi: 10.1016/s2214-109x(17)30472-2 [Crossref] [ Google Scholar]
2. Rasanathan K, Diaz T. Research on health equity in the SDG era: the urgent need for greater focus on implementation. Int J Equity Health 2016; 15(1):202. doi: 10.1186/s12939-016-0493-7 [Crossref] [ Google Scholar]
3. Hone T, Macinko J, Millett C. Revisiting Alma-Ata: what is the role of primary health care in achieving the Sustainable Development Goals?. Lancet 2018; 392(10156):1461-1472. doi: 10.1016/s0140-6736(18)31829-4 [Crossref] [ Google Scholar]
4. WHO. Health Policy and Systems Research: A Methodology Reader. In: Gilson L, ed. Geneva, Switzerland: WHO; 2014.
5. The Alliance for Health Policy and Systems Research (AHPSR). What is Health Policy and Systems Research and Why Does it Matter? Geneva: World Health Organization; 2007.
6. Gilson L, Hanson K, Sheikh K, Agyepong IA, Ssengooba F, Bennett S. Building the field of health policy and systems research: social science matters. PLoS Med 2011; 8(8):e1001079. doi: 10.1371/journal.pmed.1001079 [Crossref] [ Google Scholar]
7. Strengthening Capacity for Health Policy and Systems: What You Told Us. Health Systems Global; 2020.
8. Macarayan EK, Balabanova D, Gotsadze G. Assessing the field of health policy and systems research using symposium abstract submissions and machine learning techniques. Health Policy Plan 2019; 34(10):721-731. doi: 10.1093/heapol/czz086 [Crossref] [ Google Scholar]
9. Tancred TM, Schleiff M, Peters DH, Balabanova D. Health policy and systems research training: global status and recommendations for action. Bull World Health Organ 2016; 94(7):491-500. doi: 10.2471/blt.15.162818 [Crossref] [ Google Scholar]
10. Erasmus E, Lehmann U, Agyepong IA. Strengthening post-graduate educational capacity for health policy and systems research and analysis: the strategy of the Consortium for Health Policy and Systems Analysis in Africa. Health Res Policy Syst 2016; 14:29. doi: 10.1186/s12961-016-0097-0 [Crossref] [ Google Scholar]
11. Mirzoev T, Lê G, Green A. Assessment of capacity for Health Policy and Systems Research and Analysis in seven African universities: results from the CHEPSAA project. Health Policy Plan 2014; 29(7):831-841. doi: 10.1093/heapol/czt065 [Crossref] [ Google Scholar]
12. Teaching & Learning TWG. Health Policy and Systems Research Training Database. Tbilisi, Georgia: Health Systems Global; 2018.
13. Tancred T, Schleiff M, Peters DH, Balabanova D. Global Mapping of Health Policy and Systems Research Training. London, United Kingdom: Health Systems Global, Alliance for Health Policy and Systems Research; 2014.
14. Sawleshwarkar S, Negin J. A review of global health competencies for postgraduate public health education. Front Public Health 2017; 5:46. doi: 10.3389/fpubh.2017.00046 [Crossref] [ Google Scholar]
15. Jogerst K, Callender B, Adams V. Identifying interprofessional global health competencies for 21st-century health professionals. Ann Glob Health 2015; 81(2):239-247. doi: 10.1016/j.aogh.2015.03.006 [Crossref] [ Google Scholar]
16. Association of Schools of Public Health (ASPH). Master’s Degree in Public Health Core Competency Development Project. Version 2.3. Atlanta, GA: ASPH; 2006.
17. Calhoun JG, Spencer HC, Buekens P. Competencies for global heath graduate education. Infect Dis Clin North Am 2011; 25(3):575-592, viii. doi: 10.1016/j.idc.2011.02.015 [Crossref] [ Google Scholar]
18. Ablah E, Biberman DA, Weist EM. Improving global health education: development of a Global Health Competency Model. Am J Trop Med Hyg 2014; 90(3):560-565. doi: 10.4269/ajtmh.13-0537 [Crossref] [ Google Scholar]
19. Lembani M, Teddy G, Molosiwa D, Hwabamungu B. Post-doctoral research fellowship as a health policy and systems research capacity development intervention: a case of the CHESAI initiative. Health Res Policy Syst 2016; 14(1):89. doi: 10.1186/s12961-016-0159-3 [Crossref] [ Google Scholar]
20. Gale NK, Heath G, Cameron E, Rashid S, Redwood S. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol 2013; 13:117. doi: 10.1186/1471-2288-13-117 [Crossref] [ Google Scholar]
21. Eubank BH, Mohtadi NG, Lafave MR. Using the modified Delphi method to establish clinical consensus for the diagnosis and treatment of patients with rotator cuff pathology. BMC Med Res Methodol 2016; 16:56. doi: 10.1186/s12874-016-0165-8 [Crossref] [ Google Scholar]
22. Wong ST, Browne AJ, Varcoe C. Development of health equity indicators in primary health care organizations using a modified Delphi. PLoS One 2014; 9(12):e114563. doi: 10.1371/journal.pone.0114563 [Crossref] [ Google Scholar]
23. Havers SM, Martin E, Wilson A, Hall L. Implementation of government-directed policy in the hospital setting: a modified Delphi study. Health Res Policy Syst 2019; 17(1):91. doi: 10.1186/s12961-019-0500-8 [Crossref] [ Google Scholar]
24. Iqbal S, Pipon-Young L. The Delphi method--a guide. Psychologist 2009; 22(7):598-601. [ Google Scholar]
25. Consortium for Health Policy and Systems Analysis in Africa (CHEPSAA). Principles and Practice of Good Curriculum Design. Cape Town: CHEPSAA; 2013.
26. Nxumalo N. Emerging Leaders in Health Policy and Systems Research Analysis in Africa: Developing the Practices of HPSR+A Leadership. Johannesburg: Consortium for Health Policy and Systems Analysis; 2015.