Examining and Contextualizing Approaches to Establish Policy Support Organizations – A Critical Interpretive Synthesis

Background

There are consistent calls worldwide to ensure the use of research evidence in health policy-making to strengthen health systems and to address the ‘quadruple aim’ of enhancing patient experiences, improving population health with manageable per capita costs and positive provider experiences.^1-3 Using research evidence can help policy-makers make more rational, systematic, and transparent decisions throughout the policy development cycle, namely agenda setting, policy formulation, policy implementation, and policy evaluation.^2,4-7 More precisely, evidence can help in clarifying a problem, framing viable options to address a problem, identifying implementation considerations (ie, the potential barriers and windows of opportunities) and developing monitoring and evaluation plans that enable rapid-cycle improvements over time to implemented polices.^4-6 Despite the advantages of evidence-informed policy-making (EIPM) and the worldwide call to increase the use of research evidence in policy-making, several barriers constrain the use of research evidence in health-system policy-making processes.^8-11

One of the challenges that policy-makers face in using evidence is related to the complex nature of the policy process in which research evidence is only one factor among many that needs to be considered in policy decisions.^8-12 Other factors include institutional constraints, interest-group pressure, values, and “external” events (eg, economic recessions).^8-11 Another challenge is that research evidence can be difficult to use and the results are often packaged and presented in a way that appears to be unhelpful for the types of decisions policy-makers face.^11,13-15 Studies have also found mutual mistrust between researchers and policy-makers, and that policy-makers can have a tendency to place little value on research evidence as an input into policy decisions.^9,14 In addition, timely access to high-quality and relevant research and the organizational characteristics of the policy-related organizations further influence the use of research evidence.^16-18

There are different mechanisms to overcoming these barriers to support the use of research evidence in policy-making.^12,15,19-21 For example, knowledge producers or an intermediary group can make it easier for policy-makers to find research evidence when a demand for it arises (eg, through easy-to-search one-stop-shops or clearinghouses for relevant and high-quality research evidence) and to use the evidence they find (eg, by preparing user-friendly summaries of policy-relevant systematic reviews).²² Knowledge users can support the processes and structures that are needed to facilitate the demand for evidence from policy-makers (eg, by creating routine processes in the policy development process to use key sources to find and use research evidence).²² Furthermore, many have highlighted the importance of mechanisms that support knowledge producers (ie, researchers and academics) and knowledge users (ie, managers and policy-makers) to work more closely together.^23,24 For example, knowledge producers and users can have meaningful partnerships that enable them to jointly ask and answer relevant policy questions, such as by convening stakeholder dialogues where policy-makers, stakeholders, and researchers can combine the best-available evidence with their collective insights to spark action to address pressing policy challenges.²² Lastly, such approaches from knowledge producers and users can be integrated and embedded in a knowledge translation platform (KTP) to enable more comprehensive efforts to support EIPM.^22,25

A KTP is a form of organized effort (ie, organization or network) to bring research and policy communities together. A KTP can have five primary objectives: (i) identifying policy needs and priorities; (ii) harvesting local evidence and experience (eg, by building a database of locally produced evidence) and harmonizing it with global knowledge to guide policy development and implementation; (iii) brokering among policy-makers and researchers on key issues; (iv) packaging evidence for target audiences; and (v) strengthening the capacities of researchers to generate better evidence, and of policy-makers to better find and use research evidence.²⁶ This type of approach has been operationalized by the World Health Organization (WHO) through the Evidence-Informed Policy Networks, which has the goal of supporting the process of translating research evidence into policy and action in a number of low- and middle-income countries.²⁵ EIPM initiatives have different forms and have been called different names,^9,27 and recently, some studies discussed the organizational factors that influence the utilization of research evidence by policy-makers and the importance of institutionalizing EIPM efforts.^12,17,18 Although there is a tendency to assign the EIPM efforts to a particular organization, little is known about the process of establishing such organizations.

In this synthesis we are calling the organized effort (this could be a department, a unit, a forum, a network, an organization, or an initiative either external from government or embedded within government) to support EIPM a policy support organization (PSO), and our aim is to develop a conceptual framework that can guide the process of establishing a PSO or similar entities. As more countries are increasingly interested in enhancing their efforts to use research to inform policy-making, a trend has emerged where a particular group, initiative, department, network, or organization is asked to lead efforts to support EIPM at the system level. However, despite the increased interest in establishing PSOs we are not aware of an existing synthesis of evidence or a conceptual framework that focuses on the different approaches used to establish such organizations in different contexts, which takes into consideration the establishment process, the organizational attributes or features, and the contextual factors that affect the process.

This synthesis seeks to address this gap by undertaking a critical interpretive synthesis (CIS) to develop a conceptual framework for establishing a PSO.

Methods

Design

We conducted a CIS, which is a synthesis approach designed to analyze a broad range of relevant sources and use analytical outputs to develop a conceptual framework.^28-31 CIS is a particular form of systematic review that draws both on traditions of qualitative research inquiry and on systematic review methodology.³² A CIS is best suited to study an emerging phenomenon that is currently difficult to define,^33,34 which is the case with processes for establishing PSOs. In conventional systematic reviews, the researcher formulates a precise question that is tightly focused, allowing for pre-identification of the review parameters and the selection criteria. Developing a narrow research question is useful where the phenomenon of interest and relevant populations, interventions, and outcomes are all well-specified.³⁵ In contrast, CIS methodology allows for flexibility to draw from a wide range of relevant sources and is not constrained by including only pre-specified designs or quality of documents. Instead, the relevance of the article is the most critical judgment for article inclusion.^28,30 An additional strength of the CIS approach is that it allows researchers to formulate an initial compass question that can be further iteratively modified and defined as the synthesis progresses.^30,36

Our initial compass question “what are the key features (infrastructure, activities, outputs, outcomes and impacts) of the organization, initiative or network that support EIPM by clarifying problems, selecting options, and identifying implementation considerations, and how are these key features related to political, health system and research system contexts, particularly as they help to support evidence-informed health policy-making?” This question was iteratively refined as the literature search, review, and analysis proceeded. The finalized compass question was as follows: what is the process of developing a PSO, what are the main features/ attributes of PSOs, and what are the contextual factors influencing this process?

The overall goal of both questions is to understand the process of establishing a PSO. However, the initial question aims to identify patterns in the organizations’ features and approach of establishment with particular contextual factors or setting. For instance, we were aiming to understand whether PSOs in low- and middle-income countries have different features or approaches than PSOs in high-income countries.

During the data extraction and synthesis, we realized that this goal could not be achieved due to lack of evidence. Therefore, we modified the research question to focus mainly on understanding the process of establishing a PSO. Modifying the compass question is a unique feature of CIS, where the compass question is iteratively developed throughout the research process because the phenomenon under consideration is emerging and difficult to define.

Eligibility Criteria

We included documents that focused on one or more PSOs (ie, organizations, initiatives, and networks that support evidence-informed health policy-making by clarifying problems, selecting options, and identifying implementation considerations), as well as documents that focus on organizational attributes and contexts. We excluded documents that focused on clinical practice or clinical practice guidelines, public health practice, health technology assessments, and knowledge translation (KT) of decision-making in other sectors (not health). We also excluded documents that did not have an explicit description of a PSO (eg, discussing KT or EIPM in general without pointing to a particular organization(s)). We restricted our search to English publications only, and no regional restrictions applied.

Search Strategy

To identify relevant literature, we used a two-step search strategy that was conducted in October 2018. First, we conducted a preliminary search in Google Scholar for potentially relevant documents in addition to screening documents the research team were familiar with and had highlighted as relevant to the topic. This search identified 38 documents that highlighted 56 different PSOs. Twenty-eight PSOs were excluded after consulting the research team because they did not fit within the inclusion criteria, mainly because they focus on clinical practice guidelines and health technology assessments instead of the health system policies. Of the remaining 28 PSOs, 22 were found to have a web page. The websites of relevant organizations were scanned for descriptors used to discuss PSOs. These descriptors were then utilized to inform the second step, which was the development of a comprehensive database search strategy.

Using these descriptors, we worked with a librarian at the McMaster Health Sciences Library to develop an explicit and structured approach to search the indexed and grey literature using nine databases to cover the broadest possible spectrum of articles related to the establishment of a PSO. The search strategy used a combination of Medical Subject Heading (MeSH) terms (eg, decision-making, policy-making) and keywords developed for OVID MEDLINE and adapted for other databases as necessary. The search terms were derived by identifying synonyms for five domains relevant to the compass question (the input or what the organization uses, the target or what the organization targets, the connection between the input and the target/what the organizations do, the focus of the organization, and organization descriptor (eg, unit, department, network, organization, forum, platform). The Boolean operators OR were used to combine the MeSH terms and keywords within each concept while AND was used to make the connection across the concepts.

We developed and piloted ten search strategies to test for their sensitivity. This included testing all synonyms individually for sensitivity, and synonyms that expanded the results to an unmanageable number (eg, 40 000+) were refined by scanning the first three pages of search results. If nothing relevant was found, the synonym was dropped. If some relevant articles were identified within the first three pages of search results, the synonym was included, but limited to title only. The final search strategy for Ovid MEDLINE is summarized in Supplementary file 1 and has been adjusted to further search OVID EMBASE, PsychInfo, HealthStar, CINAHL, Web of Science, Social Science Abstract, Health Systems Evidence, and ProQuest Dissertations and Theses Global. These searches were then supplemented by a hand-search of each included document’s bibliography and reports on the WHO general website to identify any additional relevant literature.

Selection of Documents

We used referencing software (Endnote version 9) to manage the retrieved documents and to remove duplicates. All documents published up until October 2018 have been considered in our review. To ensure the included documents met the synthesis criteria, two reviewers independently reviewed a randomly generated sample of 20% of the retrieved documents at two stages. First, the principal investigator (PI) reviewed the titles and abstracts of all documents retrieved and classified them as potentially relevant or not relevant (to be excluded), and the second reviewer independently reviewed the title and abstract of 20% of the retrieved documents. The PI conducted a 30-minute meeting with the second reviewer to explain the topic, the inclusion and exclusion criteria, and the strategy to screen the documents. The eligibility criteria were tested by each reviewer independently assessing the first 5% of the search results. Following discussion to reconcile any area of discrepancy, both reviewers then assessed the remaining documents in the sample. Next, the PI reviewed the full text of each document that had been classified as potentially relevant and the second reviewer reviewed 20% of the sample. A Kappa statistic was calculated for the documents reviewed by both reviewers. All discrepancies were resolved by extensive discussion to establish consensus. By reaching a sufficient level of agreement the same inclusion and exclusion criteria were then used to review the remaining 80% of the retrieved documents by the PI.

Data Extraction and Analysis

We developed and piloted a standardized data extraction form that included data about the documents (the year published, document type, methods employed, countries included and concepts covered), the PSO covered in the paper (organization name and attributes, including leadership, governance, human resources, financial arrangements, linkages, infrastructure, program and services), and contextual factors related to the political, health and research system. The extraction form was designed to conceptually map the process of developing a PSO, the influence of the contextual factors on this process as well as on the organizational attributes. The Cochrane KT framework was used to organize the findings about the program and services, which includes six themes: prioritizing and co-producing evidence syntheses, pushing and supporting implementation, facilitating pull, exchanging knowledge and evidence, improving the EIPM climate, and ensuring sustainable KT processes.³⁷

After reading the included full-text documents and extracting findings using the form, data was synthesized interpretively using the constant comparative analysis approach throughout analysis to ensure that the emerging synthesized constructs are grounded in the data. We started by identifying the common themes and concepts with greater attention given to themes that emerged from multiple documents that helped to understand the process of establishing a PSO and how the contextual factors influenced this process. These themes and concepts were then used to develop conceptual constructs that highlight the main stages for establishing a PSO and the contextual factors that influence this process. Finally, the identified constructs were integrated to produce a synthesized argument (conceptual framework) about the establishment of PSOs in relation to the contextual factors and the organizational attributes. This was done with continuous consultation with other members of the research team who have extensive expertise with supporting policy-makers in identifying conceptual gaps and finalizing conceptual framework. During all stages of data extraction and analysis, the principal investigator (SA) kept a memo to track changes in the compass question and any modifications in identifying documents or synthesizing the results.

Results

Our search retrieved 12 890 documents. After removing duplicates and screening titles and abstracts, we identified 176 full-text articles for further appraisal, of which 52 documents were eligible for synthesis inclusion (Figure 1). The Kappa statistic on the samples for the first step (ie, screening titles and abstracts of all retrieved articles) and the second step (ie, reviewing the full text of the articles that had been initially classified as potentially relevant) was 0.62 and 0.82, respectively, and both scores reflect substantial agreement between the two reviewers.

Close

Figure 1. PRISMA Flow Chart for the Review Search Strategy. Abbreviation: PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

The number of included documents increased over time (2003-2010, n = 8; 2011-onward, n = 44). All WHO regions were represented in the documents, although the organizations predominantly studied were from the African (n = 31) region. Organizations from other regions were less frequently studied, including the Americas (n = 13), South East Asia (n = 10), Europe (n = 7), and Eastern Mediterranean and the Western Pacific (n = 6). Documents also focused on a mix of low-income (n = 27), middle-income (n = 30) and high-income countries (n = 17) (note that several documents focused on countries from more than one income classification). Most of the documents (n = 41) were journal articles, with the rest being commentaries (n = 5), reports (n = 4), meeting abstracts (n = 1), and media articles (n = 1). Of the 52 included documents, 38 were empirical studies. Of these, many employed at least two data-collection methods (n = 19), with document analysis (mainly unpublished internal document, eg, policies, meetings notes, and archives) being the most common method utilized (n = 28), followed by interviews (n = 20), surveys (n = 10), non-systematic reviews (mainly using published documents) (n = 5) and systematic reviews (n = 3). Of the 83 organizations mentioned in the 52 included documents, the most common settings for PSOs were within government (n = 32) or academic institutions (n = 28), while those situated as independent organizations were less common (n = 15). Slightly less than half of the documents focused PSOs from at least two different settings (n = 22) and about one third of the documents focused on PSOs in more than one country (n = 18).

On June 24, 2020 we ran the same search strategy (presented in Supplementary file 1) in four databases (ie, Medline, HealthStar, Embase, PsycINFO) to identify the documents released between 2019 and 2020. After removing the duplicates, our search revealed 345 articles out of which three were potentially relevant. However, after reading the full documents, non were eligible for synthesis inclusion.

Conceptual Framework for Establishing a PSO

We developed a framework that outlines the process of establishing a PSO. Figure 2 presents this framework, which includes four main stages in the establishment process: awareness, development, assessment, and maturation. Each of the stages has unique components as well as connections to the other stages. Although the framework is arranged in a sequential way, it is important to emphasize that the process of establishing a PSO is iterative (this is indicated by the double-headed arrows between the stages) and different organizations may go back and forth between the different stages even after reaching the maturation stage given that the process could be repeated when the organization introduces a new service or program or when an organization goes through an assessment that requires major changes. In addition, some organizations may skip one or more steps depending on what is already in place. For example, if the concept of EIPM is well-established, less work will be needed in the awareness stage, while others might skip the assessment stage when there is not enough capacity to do the assessment. Furthermore, as highlighted in the far left in Figure 2, the process of establishing a PSO is influenced by contextual factors that are related to the political, health and research system. The following sections provide a description of the components and features of each stage, the corresponding contextual factors, and the link between the stages.

Close

Figure 2. Four Stage Conceptual Framework for the Establishment of a Policy Support Organization. Abbreviations: EIPM, evidence-informed policy-making; PSO, policy support organization.

Awareness Stage – Providing a Foundation for Why a PSO Is Needed

The awareness stage provides the foundation for establishing a PSO by identifying the motivation that would push the idea of establishing a PSO forward. Two main motivations were identified for this purpose (see Table 1). The first motivation is having a supportive climate for EIPM which is needed to build awareness among policy-makers and researchers about how evidence can inform policy-making.^38-47 The second motivation is identifying the fragmentation between policy and research communities and the need to address it.^46,48 A supportive climate or an identified and an agreed upon need to bridge the gap between policy and research communities can each individually drive the need to establish a PSO, as well as act synergistically to provide a stronger case. The level of awareness built from this synergy subsequently affects the development stage. For example, countries that do not have widespread awareness may need to initially focus on developing and implementing programs that help to further build awareness while also supporting EIPM. These programs might include capacity building workshops, priority setting processes, and opportunities for exchange between policy-makers, stakeholders and researchers that create opportunities to help bring the policy and research communities together.

Development stage – Understanding the Situation and Specifying the Organization’s Attributes 

The development stage is the actual implementation of a PSO. This stage is the first point where KT activities start to be more organized and attributed to a specific organization (eg, department, unit, forum, or network). This stage involves identifying the organization’s features by first understanding the situation and then specifying the organization’s attributes (see Table 2). At the early stage of establishing a PSO, different approaches can be used to either understand who is doing what, why and where (ie, situation analysis approach),^{26,38,40,41,43,49} or to provide a proof of concept for efforts to support EIPM among research and policy communities.^40,49 While a situation analysis approach is better for assessing the relationship between the research and policy communities,^{26,38,40,41,43,49} a proof of concept approach is critical for demonstrating the potential benefit of establishing the organization.^40,49 These two approaches might play a different role in different countries and they can complement each other. For example, a situation analysis can be used to identify the niche, threats, and opportunities for a newly proposed organization in countries where other organizations that support EIPM in different ways already exist (eg, health technology assessment units that provide decision support for whether to provide funding for specific programs, services and drugs, but not about the system arrangements that are needed to get them to those in need).^{26,38,40,41,43,49} In addition, the same approach can be used to understand policy-makers’ priorities and identify the potential collaborators in countries where EIPM initiatives are dispatched.^40,41 In contrast, a proof of concept can be helpful in providing direct evidence about or experience with how a PSO can play a critical role in achieving these priorities.^40,49

Table 2. Description of the development stage and contextual factors relevant for establishing a Policy Support Organization

Once the organization understands the situation, the next step is to identify its attributes, which (as we outline in Table 2) include defining: (i) strong leadership^{26,40,46,48,54,58-60,63,68-70}; (ii) clearly defined organizational structure^{26,40,43,46,48,51,52,59,63,64,67,71}; (iii) sustainable financial arrangements^{39,47,52,59,64,71}; (iv) capable human resources^{19,21,26,51,52,56,58,62,64,66,72,73}; (v) sufficient infrastructure^{21,39,41,42,55,56,58,66-68,74,75}; (vi) strategic organizational linkages^{21,26,39,40,42,49-52,56,59-61,64-67,69,76,77}; and (vii) targeted programs and services.^{26,39-43,47,48,50,52-54,59-61,65-67,69,70,73,77-80} In specifying these attributes, it is important to consider them in the frame of the PSO working as a platform that brings together policy-makers, stakeholders, and researchers to support EIPM. Such partnership building and co-creation can underpin all of the attributes of the PSO to maximize the organization’s ability to support EIPM. For example, the leader of the organization should be of high credibility, skills, and expertise in both the research and policy communities.^{26,40,46,48,54,58-60,63,68-70} At the same time, processes for selecting what programs and services to offer may consider involving policy-makers and researchers, especially since this stage will shape subsequent stages in the organization’s evolution, particularly its sustainability.

Assessment Stage – Monitoring and Evaluating the Organization

The assessment stage consists of monitoring and evaluating either specific programs and services provided by the PSO and/or its overall performance. As we outline in Table 3, we found three different approaches for monitoring and evaluating PSO activities: convening meetings/focus groups or conducting interviews;^38,76 conducting surveys;^38,39,41 and engaging external experts/agencies.^46,62,76 Among the few documents that discuss assessment, the focus was mainly on evaluating a specific activity and its corresponding product(s). Some organizations conducted an assessment on a regular basis (eg, annually), while others did so at key junctures (eg, at the end of a donor funding cycle or specific project, or after a training workshop). Some documents reported the importance of assessing the impact of a PSO (or similar entities) in health policy and policy-making process.^39,44,55,69 Others further reported that such entities had an impact on health policies and policy-making process.^58,64,65,70 However, none of these documents were explicit about what exactly was assessed (or should be assessed) nor were they explicit about how the impact was assessed.

Maturation Stage – Approaching Sustainability

The maturation stage focuses on ensuring long-term sustainability. Four different approaches were identified to attain sustainability: institutionalization of the PSO^{39,41,42,44,45,47,51,59,64}; having a clear legal mandate^43,47,51; having sustainable funding^39,47,59; and having appropriate capacity (see Table 4).^39,62 Although no single document addressed all four approaches, the evidence strongly supported that none of these approaches can solely drive the organization towards the maturation stage. Instead, it is clear from the included documents that all of these components are important for an organization’s stability.

Contextual Factors

There are facilitators and barriers that influence the establishment of a PSO at the political, research and health system level. The political level influences the process of establishing a PSO by determining the availability of resources needed to establish a PSO, the trust between researchers and policy-makers, and the ideas about EIPM. On the other hand, the research and health system influence the establishment of a PSO by determining the availability of resources needed to establish a PSO and the trust between researchers and policy-makers. Tables 1-4 highlight the influence of each contextual factor at each stage in the development of a PSO, and we summarize the main points of influence in relation to political, research and health system.

Political System Factors

There was a diffusion of ideas about KT and EIPM at the national and international level through conferences, training workshops, and funds for research projects that have a KT component.^46,52,60,66 Diffusion of these ideas were reported to play a major role in changing the ideas about EIPM by shifting the beliefs of policy-makers regarding the importance of research in policy-making and creating consciousness about the need for stronger linkages between policy and evidence.^42,46 Therefore, the diffusion of ideas played a role in increasing awareness and building a supportive climate for establishing a PSO.

At the same time, policy-making processes, existing institutions (eg, planning and research departments), and existing policies influence the establishment of PSO by creating incentive and resources to establish a PSO.^{43,46,47,49,52,60,64,75} For instance, if existing policy-making practices emphasize the importance of using research and policy-makers have experience and expertise in doing so, these provide an incentive and resources (eg, through a supportive climate for establishing a PSO and with experience and expertise that can be used to begin to operationalize a PSO).^51,60,64 In addition, government structures that enable and support collaboration (eg, through pre-existing collaborations with research institutions) facilitate the establishment of PSOs by increasing the trust between policy-makers and researchers and promote the availability of resources through collaboration and networking.^{39,44,50,51,55,56}

Having committed interest groups (eg, high-level policy-makers, health professionals, academic institutions, government agencies, and stakeholders) who express support and advocate for EIPM have been found to be an essential facilitator for establishing a PSO.^{19,20,39,43,50,51,70,74} The interest of policy-makers has been found to be particularly influential when managers within government are highly qualified and value the role of research in policy-making.⁵⁴ This appreciation can foster a supportive climate throughout government departments for establishing a PSO.

Research System

Research systems also influence the establishment of PSOs by influencing the trust between researchers and policy-makers, and by influencing the availability of resources for a PSO. Availability of research infrastructure (eg, access to online databases), capable researchers and funding for local research are important inputs and resources for establishing a PSO.^43,46,65,68 On the other hand, the establishment of a PSO can be hindered if the research system is primarily shaped by the priorities of funders and researcher with implications of poor or no uptake of research outputs for decision-making.^{43,46,65,68,70} Moreover, a lack of communication between the research and policy communities can lead to the mistrust between the two communities and make policy-makers more skeptical in using the findings or other services provided by researchers (eg, because of a perception that researchers do not understand their priorities).^44,50,51,76

Health System

Health system arrangements can facilitate or hinder the establishment of a PSO by influencing the trust between researchers and policy-makers and by influencing the availability of resources for a PSO. Having ongoing collaborations and networking between relevant government departments and researchers enhances the trust between them, which can create a more supportive climate for establishing a PSO.^46,51,53 In addition, well-established trust between the research and policy communities can facilitate the PSO’s ability to run its programs and services using a collaborative approach, which further enhances trust and the production of relevant outputs that can be used to inform policy. However, maintaining trust and making resources available for a PSO becomes more challenging in a health system that is highly dependent on donors because of the deviation between policy authority (ie, the government) and implementation facilitators (ie, the donors).^56,55 Because establishing a PSO in a health system that is highly reliant on external funding (eg, donors or external organizations) is more challenging because it is less trusted and more likely to be fragmented. In a situation where a PSO is planned to be embedded within a government organization, the PSO can draw on existing infrastructure and more efficiently mobilize resources (eg, infrastructure, human and financial resources) to establish it and support its functions.^56,39,50

Linkages Between Stages

As we depict in Figure 2 and across Tables 1-4, the four stages for establishing a PSO are interconnected with the actions that take place in one stage affecting the other stages. Most of the connections are centralized around the development stage given that all actions that take place in the development stage have an impact on the other stages. The development stage contributes to the awareness stage in two main ways. First, understanding the situation helps in identifying the motivations (ie, identifying the fragmentation between research and policy communities or raising awareness) needed to create a supportive climate for establishing a PSO.^{26,38,40,41,43,49} The fragmentation might then be addressed through the PSO’s governance approach that involves policy-makers and researchers,^{26,40,46,51,59,63} and by building collaboration with other organizations to run different activities and programs.^{26,39,40,42,51,52,59-61,64-67} Second, the programs and services offered by a PSO can subsequently enhance the climate for the use of evidence in policy-making (ie, it can create a positive feedback loop).^{38,42,44,52,58}

The development stage is also connected to the assessment stage where some of the organization’s attributes might be adjusted after assessing the organization’s performance.^47,52,64 At the same time, the way the organization’s attributes are specified (particularly its governance approach, human resources, linkages and financial arrangements), determine the organization’s ability to conduct monitoring and evaluation from which the results can be used to inform the PSO’s continued evolution.^{46,47,52,62,64,76}

The maturation stage is connected to multiple aspects of the development stage, including having strong institutionalized leadership,^42,46,70 having a clearly defined organizational structure and mandate,^{43,47,51,63,72} identifying sustainable sources of funding,^39,59,64,71 and incentivizing human resources to avoid staff turn-over.^{26,52,64,65,68,70} The way these aspects are defined at the development stage strongly affects the organization’s ability to sustain itself in the long-term.

Discussion

Principal Findings 

Based on the large and growing volume of evidence in the field, we developed a conceptual framework to inform the process of establishing a PSO. Our findings suggest that a PSO undergoes four stages on the pathway towards becoming sustainable. The four stages include: (i) awareness (providing the foundation for why a PSO is needed); (ii) development (the actual implementation of a PSO); (iii) assessment (assessing performance and making adjustments); and (iv) maturation (advancing the organization towards sustainability and institutionalization). While each stage has its unique features and contributes toward the establishment of a PSO, the entire process is iterative and movements between the stages should be expected.

Although the four stages are interconnected and the activities that take place in one stage influence the other stages, the development stage is central in the process of establishing a PSO and it is the one stage that has a direct effect on all of the other stages. Despite an organization’s age, some may not go through each of the four stages. However, among the organizations in the literature we identified, the development stage was never skipped, because it is the actual stage of implementing a PSO. In contrast, the awareness stage was bypassed in many countries where a wide-spread awareness about EIPM already existed. Similarly, the assessment stage was skipped when there was insufficient capacity and/or skills to do the work. We also found that assessments of PSO mainly focused on evaluating the processes and outputs of a PSO with little evidence available on evaluating their impact on supporting EIPM. This might be because the policy process is complicated and research is only one factor among many others that influence it, which makes conclusions about impact of a PSO supporting evidence use difficult to determine.^8-11 Therefore, even if a PSO succeeds in supporting policy-makers with the best available evidence, it would be hard to discern the exact influence of research from the other factors. Regardless, the documents we included still emphasize the importance of evaluation given that it can provide insights about how to strengthen PSO activities through formative evaluations and to document any impacts through summative evaluations either indirectly (eg, by measuring effects on behavioural intentions to use research evidence) or directly (eg, though qualitative studies that gather insights about whether and how PSO activities supported evidence use in efforts to strengthen health systems).^58,64,65,70

Moreover, the four stages should not be viewed in isolation from important contextual factors that can hinder or facilitate the establishment of a PSO. Our findings indicated that the process of establishing a PSO is influenced by contextual factors in political, research and health systems, which determine the availability of the resources and the trust between researchers and policy-makers. The political system further determines the importance placed on EIPM which is central to enabling the process of establishing a PSO. The contextual factors have an impact on each other, and the challenges that arise from one factor can be mitigated by the other factors.

Findings in Relation to Other Studies 

To the best of our knowledge, this is the first comprehensive literature review examining and contextualizing approaches to establish PSOs. The ever-growing number of studies undertaken to inform efforts to support EIPM differ in important ways from what we have done here. First, many examined the process for a specific approach to supporting EIPM (eg, rapid response, clearinghouses, policy brief) and demonstrated the steps of conducting that particular approach.^{44,63,76,84,86} However, none have focused on how to assign that particular approach or activity to an organization that is solely focusing on supporting EIPM. In addition, the majority of the documents do not distinguish between type of decisions (eg, clinical, public health and health-systems topics) that are the focus of this work. Second, one document focused on identifying the steps in developing a KTP, but it was⁸²: (i) not based on a comprehensive literature review; (ii) limited to the experiences of European countries; (iii) did not identify the specific factors that influence each step; and (iv) focused on operationalization and launching rather than the entire process. Third, some work has been done on the process of institutionalization, but it was very specific for units performing one particular service (ie, rapid response service), compared to our focus on an organization that can perform multiple KT activities to support EIPM.⁴⁷

Lastly, the contextual factors that influence the establishment of a PSO were not well-addressed in the literature which limited our ability in making a better distinction about their impact on the different stages of the framework. While literature is available on the contextual factors that affect KT in general or specific KT approaches or activities,^19-21 there is less emphasis given to factors affecting the establishment of PSOs. The level of state centralization and democratization, the influence of external donors and organizations in the health system policies, the organization and function of bureaucracies, the infrastructural resources, research and KT funding, the framing of evidence in relation to social norms and values, and quality and quantity of research targeting health system are among the factors that were reported to influence research utilization in general.^87-89 Some of these factors intersect with our finding as factors that influence the establishment of a PSO. Particularly, the political factors (ie, ideas, interest, and institutions), quality and quantity of health system research, availability of resources (ie, infrastructure, human resources, and financial resources), and the role of funders.

Strengths and Limitations

A key strength of this CIS is that it used systematic and transparent methods, but in a way that allowed for flexibility that enabled rich interpretive analysis to generate the conceptual framework. This highlights another strength, which is that we developed a new conceptual framework for the process of establishing a PSO that fills an important conceptual gap in the literature and that can be used by policy-makers, researchers, and stakeholders from different countries to guide their efforts in establishing PSOs.

A potential limitation is that we focused our synthesis only on organizations that support policy-makers at the health system level, and not on those that support the production and use of other types of decision supports for policy-making (eg, clinical practice guidelines, health technology assessments, or public health practice). The potential limitation stems from the possibility of having missed relevant insights from these areas. However, we identified a large set of relevant documents (n = 52) that provided substantial insight into an area that is not as far advanced conceptually as these other areas. Given this, the tradeoff between breadth and depth in the more specific domain of PSOs that focus on strengthening health systems was important to advance our thinking in the field. Another potential limitation is that while we attempted to identify whether particular features, activities, or establishment approaches are more common in a particular setting (ie, government, academic, independent) or context (eg, country income level or region), this was not possible for two main reasons. First, most of the studies that included more than one organization presented results in an aggregated format, and this did not allow for an in-depth analysis of linked organizational features and contextual factors. Second, the few studies that presented individual cases either did not provide enough contextual background or they did not explain the role of contextual factors in shaping the organization.

Implications for Policy and Research

We have identified several policy implications for those supporting EIPM based on the results of our CIS. For those interested in establishing a PSO, our framework provides a road map for identifying the most appropriate starting point. It also helps in identifying the factors that might influence the establishment process that varies according to the context in which a PSO is to be established. For example, establishing a PSO in a country where sufficient awareness about EIPM already exists will likely not require much effort invested into the first stage. Instead, in such a context, the focus will be shifted to the second stage for evaluating the situation and identifying the organization’s attributes. Furthermore, our findings can be informative for established PSOs. Leaders of such PSOs can use our findings to expand or refine their scope of work, such as by selecting a new program or service to provide and refining monitoring and evaluation plans to include assessments of the impact of their work (if this was not previously included in the scope of monitoring and evaluation efforts).

Given that this framework focuses only on PSOs in the health sector, an important next step for research would be to include other sectors from social systems and identify any additional insight that can enhance the framework we have developed. This CIS also identifies that assessment of PSO performance is not well-established and therefore future research should focus on identifying approaches for evaluating the impact of PSOs. One approach to do so could be through a before and after quasi-experimental design with a set of indicators to assess the impact of a PSO in informing policies with the best available evidence and whether informing policies by evidence has any impact in the efficiency of the policy-making process and ultimately on strengthening the health system. For already established PSOs, evaluations of impact could be conducted using qualitative methods (eg, interviewing policy-makers, researchers and stakeholders about whether and how programs and services offered by the PSO had an impact on evidence use in policy-making) or conducting theory-informed multiple case studies with clearly defined indicators and with a sampling strategy that would ensure that counterfactual cases are included in order to provide insights about the organization’s impact.

Conclusion

This CIS developed a conceptual framework for establishing PSOs and in so doing makes an important contribution to the literature related to supporting EIPM. The framework captures the main features in the process of establishing a PSO, the main organizational attributes that have to be specified during the process, and the contextual factors that might affect this process. The four stages identified in this framework should be carefully considered in relation to the country-specific needs and readiness in adopting an EIPM approach to know what is required in each stage and to know which stage would be the best starting point given local contexts.

Ethical issues

Not applicable.

Competing interests

Authors declare that they have no competing interests.

Authors’ contributions

SAS was responsible for the conception and design, collection of data, analysis and interpretation of data, drafting and reviewing the manuscript. MGW contributed to the conception and design, analysis and interpretation, drafting and reviewing the manuscript. MGW was also responsible for supervising this work. JNL, FEJ and KM provided critical revisions of the manuscript for important intellectual content. MV assisted with the collection of data and analysis and interpretation of data.

Disclaimer

The views expressed in this paper represent the opinions of the authors and not an official position of the institutions of their affiliation.

Authors’ affiliations

¹Health Policy PhD Program, McMaster University, Hamilton, ON, Canada. ²McMaster Health Forum, McMaster University, Hamilton, ON, Canada. ³Department of Health Evidence and Impact, McMaster University, Hamilton, ON, Canada. ⁴Centre for Health Economics and Policy Analysis, McMaster University, Hamilton, ON, Canada. ⁵Department of Political Science, McMaster University, Hamilton, ON, Canada. ⁶Africa Centre for Evidence, University of Johannesburg, Johannesburg, South Africa. ⁷Knowledge to Policy Center, American University of Beirut, Beirut, Lebanon. ⁸Universidad de Antioquia, Medellin, Colombia.

Supplementary files

References

1. Graham ID, Logan J, Harrison MB. Lost in knowledge translation: time for a map?. J Contin Educ Health Prof 2006; 26(1):13-24. doi: 10.1002/chp.47 [Crossref] [ Google Scholar]
2. Strydom WF, Funke N, Nienaber S, Nortje K, Steyn M. Evidence-based policymaking: a review. S Afr J Sci 2010; 106(5-6):17-24. doi: 10.4102/sajs.v106i5/6.249 [Crossref] [ Google Scholar]
3. Wilson DC, Smith NA, Blakey NC, Shaxson L. Using research-based knowledge to underpin waste and resources policy. Waste Manag Res 2007; 25(3):247-256. doi: 10.1177/0734242x07079154 [Crossref] [ Google Scholar]
4. Fierlbeck K. Paradigm freeze: why it is so hard to reform health-care policy in Canada. Can Public Adm 2014; 57(3):478-480. doi: 10.1111/capa.12085 [Crossref] [ Google Scholar]
5. Kingdon JW. Agendas, Alternatives, and Public Policies. 2nd ed. New York, USA: HarperCollins College Publishers; 2003.
6. Fafard P. Evidence and Healthy Public Policy: Insights from Health and Political Sciences. http://www.ncchpp.ca/docs/FafardEvidence08June.PDF. Accessed March 1, 2019. Published 2008.
7. Oxman AD, Lavis JN, Lewin S, Fretheim A. SUPPORT Tools for evidence-informed health Policymaking (STP) 1: what is evidence-informed policymaking?. Health Res Policy Syst 2009; 7(Suppl 1):S1. doi: 10.1186/1478-4505-7-s1-s1 [Crossref] [ Google Scholar]
8. International Development Research Centre (IDCR). The Knowledge Translation Toolkit: Bridging the Know-Do Gap: A Resource for Researchers. Ottawa: International Development Research Centre; 2011.
9. Lavis JN, Lomas J, Hamid M, Sewankambo NK. Assessing country-level efforts to link research to action. Bull World Health Organ 2006; 84(8):620-628. doi: 10.2471/blt.06.030312 [Crossref] [ Google Scholar]
10. World Health Organization (WHO). World Report on Knowledge for Better Health: Strengthening Health Systems. Geneva: WHO; 2004. https://apps.who.int/iris/bitstream/handle/10665/43058/9241562811.pdf?sequence=1&isAllowed=y. Accessed March 5, 2019.
11. Jewell CJ, Bero LA. “Developing good taste in evidence”: facilitators of and hindrances to evidence-informed health policymaking in state government. Milbank Q 2008; 86(2):177-208. doi: 10.1111/j.1468-0009.2008.00519.x [Crossref] [ Google Scholar]
12. Stewart R, Langer L, Erasmus Y. An integrated model for increasing the use of evidence by decision-makers for improved development. Dev South Afr 2019; 36(5):616-631. doi: 10.1080/0376835X.2018.1543579 [Crossref] [ Google Scholar]
13. Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE. Knowledge translation of research findings. Implement Sci 2012; 7:50. doi: 10.1186/1748-5908-7-50 [Crossref] [ Google Scholar]
14. Innvaer S, Vist G, Trommald M, Oxman A. Health policy-makers’ perceptions of their use of evidence: a systematic review. J Health Serv Res Policy 2002; 7(4):239-244. doi: 10.1258/135581902320432778 [Crossref] [ Google Scholar]
15. Oliver K, Cairney P. The dos and don’ts of influencing policy: a systematic review of advice to academics. Palgrave Commun 2019; 5(1):21. doi: 10.1057/s41599-019-0232-y [Crossref] [ Google Scholar]
16. Oliver K, Innvar S, Lorenc T, Woodman J, Thomas J. A systematic review of barriers to and facilitators of the use of evidence by policymakers. BMC Health Serv Res 2014; 14:2. doi: 10.1186/1472-6963-14-2 [Crossref] [ Google Scholar]
17. Jakobsen MW, Eklund Karlsson L, Skovgaard T, Aro AR. Organisational factors that facilitate research use in public health policy-making: a scoping review. Health Res Policy Syst 2019; 17(1):90. doi: 10.1186/s12961-019-0490-6 [Crossref] [ Google Scholar]
18. Jakobsen MW. Evidence-Informed Policymaking: The Role of Research, Stakeholders and Policy Organizations [thesis]. Odense: Syddansk Universitetsforlag; 2019.
19. Lavis JN, Moynihan R, Oxman AD, Paulsen EJ. Evidence-informed health policy 4 - case descriptions of organizations that support the use of research evidence. Implement Sci 2008; 3:56. doi: 10.1186/1748-5908-3-56 [Crossref] [ Google Scholar]
20. Lavis JN, Oxman AD, Moynihan R, Paulsen EJ. Evidence-informed health policy 3 - interviews with the directors of organizations that support the use of research evidence. Implement Sci 2008; 3:55. doi: 10.1186/1748-5908-3-55 [Crossref] [ Google Scholar]
21. Lavis JN, Paulsen EJ, Oxman AD, Moynihan R. Evidence-informed health policy 2 - survey of organizations that support the use of research evidence. Implement Sci 2008; 3:54. doi: 10.1186/1748-5908-3-54 [Crossref] [ Google Scholar]
22. Lavis JN, Lomas J, Hamid M, Sewankambo NK. Assessing country-level efforts to link research to action. Bull World Health Organ 2006; 84(8):620-628. doi: 10.2471/blt.06.030312 [Crossref] [ Google Scholar]
23. Canadian Institutes of Health Research (CIHR). Knowledge Translation at CIHR. http://www.cihr-irsc.gc.ca/e/29418.html#7. Accessed September 8, 2017. Published 2017.
24. Ellen ME, Horowitz E, Vaknin S, Lavis JN. Views of health system policymakers on the role of research in health policymaking in Israel. Isr J Health Policy Res 2016; 5(1):24. doi: 10.1186/s13584-016-0088-1 [Crossref] [ Google Scholar]
25. Hamid M, Bustamante-Manaog T, Truong VD. EVIPNet: translating the spirit of Mexico. Lancet 2005; 366(9499):1758-1760. doi: 10.1016/s0140-6736(05)67709-4 [Crossref] [ Google Scholar]
26. Kasonde JM, Campbell S. Creating a knowledge translation platform: nine lessons from the Zambia Forum for Health Research. Health Res Policy Syst 2012; 10:31. doi: 10.1186/1478-4505-10-31 [Crossref] [ Google Scholar]
27. Contandriopoulos D, Lemire M, Denis JL, Tremblay E. Knowledge exchange processes in organizations and policy arenas: a narrative systematic review of the literature. Milbank Q 2010; 88(4):444-483. doi: 10.1111/j.1468-0009.2010.00608.x [Crossref] [ Google Scholar]
28. Dixon-Woods M, Agarwal S, Jones D, Young B, Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. J Health Serv Res Policy 2005; 10(1):45-53. doi: 10.1177/135581960501000110 [Crossref] [ Google Scholar]
29. Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review--a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy 2005; 10 Suppl 1:21-34. doi: 10.1258/1355819054308530 [Crossref] [ Google Scholar]
30. Dixon-Woods M, Cavers D, Agarwal S. Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups. BMC Med Res Methodol 2006; 6:35. doi: 10.1186/1471-2288-6-35 [Crossref] [ Google Scholar]
31. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O, Peacock R. Storylines of research in diffusion of innovation: a meta-narrative approach to systematic review. Soc Sci Med 2005; 61(2):417-430. doi: 10.1016/j.socscimed.2004.12.001 [Crossref] [ Google Scholar]
32. Dixon-Woods M. Critical Interpretive Synthesis: What it is and Why it is Needed. https://abstracts.cochrane.org/2006-dublin/critical-interpretive-synthesis-what-it-and-why-it-needed. Accessed March 1, 2019. Published 2006.
33. Wilson MG, Ellen ME, Lavis JN. Processes, contexts, and rationale for disinvestment: a protocol for a critical interpretive synthesis. Syst Rev 2014; 3:143. doi: 10.1186/2046-4053-3-143 [Crossref] [ Google Scholar]
34. Boyko JA, Lavis JN, Abelson J, Dobbins M, Carter N. Deliberative dialogues as a mechanism for knowledge translation and exchange in health systems decision-making. Soc Sci Med 2012; 75(11):1938-1945. doi: 10.1016/j.socscimed.2012.06.016 [Crossref] [ Google Scholar]
35. Alderson P, Green S, Higgins JP. Cochrane Reviewers’ Handbook 4.2.2. The Cochrane Library 2004. https://www.iecs.org.ar/cochrane/guias/Handbook_4-2-2.pdf. Accessed December 23, 2018.
36. Eakin JM, Mykhalovskiy E. Reframing the evaluation of qualitative health research: reflections on a review of appraisal guidelines in the health sciences. J Eval Clin Pract 2003; 9(2):187-194. doi: 10.1046/j.1365-2753.2003.00392.x [Crossref] [ Google Scholar]
37. Cochrane. Cochrane Knowledge Translation Framework. http://community.cochrane.org/sites/default/files/uploads/Cochrane%20Knowledge%20Translation%20Framework%281%29.pdf. Accessed July 13, 2018. Published 2017.
38. Brennan SE, Cumpston M, Misso ML, McDonald S, Murphy MJ, Green SE. Design and formative evaluation of the Policy Liaison Initiative: a long-term knowledge translation strategy to encourage and support the use of Cochrane systematic reviews for informing health policy. Evid Policy 2016; 12(1):25-52. doi: 10.1332/174426415x14291899424526 [Crossref] [ Google Scholar]
39. El-Jardali F, Lavis J, Moat K, Pantoja T, Ataya N. Capturing lessons learned from evidence-to-policy initiatives through structured reflection. Health Res Policy Syst 2014; 12:2. doi: 10.1186/1478-4505-12-2 [Crossref] [ Google Scholar]
40. World Health Organization Regional Office for Europe. EVIPNet Europe: Situation Analysis Manual. Copenhagen: World Health Organization Regional Office for Europe; 2017. http://www.euro.who.int/__data/assets/pdf_file/0004/335551/EVIPNet-Europe-Situation-Analysis-Manual.pdf?ua=1. Accessed January 10, 2019.
41. Hawkes S, B KA, Jadeja N. Strengthening capacity to apply health research evidence in policy making: experience from four countries. Health Policy Plan 2016; 31(2):161-170. doi: 10.1093/heapol/czv032 [Crossref] [ Google Scholar]
42. Haynes A, Rowbotham SJ, Redman S, Brennan S, Williamson A, Moore G. What can we learn from interventions that aim to increase policy-makers’ capacity to use research? a realist scoping review. Health Res Policy Syst 2018; 16(1):31. doi: 10.1186/s12961-018-0277-1 [Crossref] [ Google Scholar]
43. Mihalicza P, Leys M, Borbás I, Szigeti S, Biermann O, Kuchenmüller T. Qualitative assessment of opportunities and challenges to improve evidence-informed health policy-making in Hungary - an EVIPNet situation analysis pilot. Health Res Policy Syst 2018; 16(1):50. doi: 10.1186/s12961-018-0331-z [Crossref] [ Google Scholar]
44. Mijumbi RM, Oxman AD, Panisset U, Sewankambo NK. Feasibility of a rapid response mechanism to meet policymakers’ urgent needs for research evidence about health systems in a low income country: a case study. Implement Sci 2014; 9:114. doi: 10.1186/s13012-014-0114-z [Crossref] [ Google Scholar]
45. Sheikh K, Kumar S, Ved R. India’s new health systems knowledge platform-making research matter. Lancet 2016; 388(10061):2724-2725. doi: 10.1016/s0140-6736(16)32391-1 [Crossref] [ Google Scholar]
46. Sriram V, Bennett S, Raman VR, Sheikh K. Developing the National Knowledge Platform in India: a policy and institutional analysis. Health Res Policy Syst 2018; 16(1):13. doi: 10.1186/s12961-018-0283-3 [Crossref] [ Google Scholar]
47. Zida A, Lavis JN, Sewankambo NK, Kouyate B, Ouedraogo S. Evaluating the process and extent of institutionalization: a case study of a rapid response unit for health policy in Burkina Faso. Int J Health Policy Manag 2018; 7(1):15-26. doi: 10.15171/ijhpm.2017.39 [Crossref] [ Google Scholar]
48. van Kammen J, de Savigny D, Sewankambo N. Using knowledge brokering to promote evidence-based policy-making: the need for support structures. Bull World Health Organ 2006; 84(8):608-612. doi: 10.2471/blt.05.028308 [Crossref] [ Google Scholar]
49. Scarlett J, Köhler K, Reinap M. Evidence-informed Policy Network (EVIPNet) Europe: success stories in knowledge translation. Public Health Panor 2018; 4(2):161-169. [ Google Scholar]
50. El-Jardali F, Ataya N, Jamal D, Jaafar M. A multi-faceted approach to promote knowledge translation platforms in eastern Mediterranean countries: climate for evidence-informed policy. Health Res Policy Syst 2012; 10:15. doi: 10.1186/1478-4505-10-15 [Crossref] [ Google Scholar]
51. Keita N, Lokossou V, Berthe A, Sombie I, Johnson E, Busia K. The West African experience in establishing steering committees for better collaboration between researchers and decision-makers to increase the use of health research findings. Health Res Policy Syst 2017; 15(Suppl 1):50. doi: 10.1186/s12961-017-0216-6 [Crossref] [ Google Scholar]
52. Ongolo-Zogo P, Lavis JN, Tomson G, Sewankambo NK. Initiatives supporting evidence informed health system policymaking in Cameroon and Uganda: a comparative historical case study. BMC Health Serv Res 2014; 14:612. doi: 10.1186/s12913-014-0612-3 [Crossref] [ Google Scholar]
53. Uneke CJ, Ezeoha AE, Uro-Chukwu HC, Ezeonu CT, Igboji J. Promoting researchers and policy-makers collaboration in evidence-informed policy-making in Nigeria: outcome of a two-way secondment model between university and health ministry. Int J Health Policy Manag 2018; 7(6):522-531. doi: 10.15171/ijhpm.2017.123 [Crossref] [ Google Scholar]
54. Mwendera C, de Jager C, Longwe H, Hongoro C, Phiri K, Mutero CM. Development of a framework to improve the utilisation of malaria research for policy development in Malawi. Health Res Policy Syst 2017; 15(1):97. doi: 10.1186/s12961-017-0264-y [Crossref] [ Google Scholar]
55. Ongolo-Zogo P, Lavis JN, Tomson G, Sewankambo NK. Climate for evidence informed health system policymaking in Cameroon and Uganda before and after the introduction of knowledge translation platforms: a structured review of governmental policy documents. Health Res Policy Syst 2015; 13:2. doi: 10.1186/1478-4505-13-2 [Crossref] [ Google Scholar]
56. Nabyonga-Orem J, Gebrikidane M, Mwisongo A. Assessing policy dialogues and the role of context: Liberian case study before and during the Ebola outbreak. BMC Health Serv Res 2016; 16 Suppl 4(Suppl 4):219. doi: 10.1186/s12913-016-1454-y [Crossref] [ Google Scholar]
57. Basaza R, Kinegyere A, Mutatina B, Sewankambo N. National framework for the sustainability of health knowledge translation initiatives in Uganda. Int J Technol Assess Health Care 2018; 34(1):120-128. doi: 10.1017/s0266462317004482 [Crossref] [ Google Scholar]
58. Langlois EV, Becerril Montekio V, Young T, Song K, Alcalde-Rabanal J, Tran N. Enhancing evidence informed policymaking in complex health systems: lessons from multi-site collaborative approaches. Health Res Policy Syst 2016; 14:20. doi: 10.1186/s12961-016-0089-0 [Crossref] [ Google Scholar]
59. Berman J, Mitambo C, Matanje-Mwagomba B. Building a knowledge translation platform in Malawi to support evidence-informed health policy. Health Res Policy Syst 2015; 13:73. doi: 10.1186/s12961-015-0061-4 [Crossref] [ Google Scholar]
60. Mansilla C, Herrera CA, Basagoitia A, Pantoja T. The Evidence-Informed Policy Network (EVIPNet) in Chile: lessons learned from a year of coordinated efforts. Rev Panam Salud Publica 2017; 43:e36. doi: 10.26633/rpsp.2017.36 [Crossref] [ Google Scholar]
61. Uneke CJ, Aulakh BK, Ezeoha AE, Ndukwe CD, Onwe F. Bridging the divide between research and policy in Nigeria: the role of a health policy advisory committee. J Public Health Policy 2012; 33(4):423-429. doi: 10.1057/jphp.2012.30 [Crossref] [ Google Scholar]
62. Uneke CJ, Sombie I, Keita N, Lokossou V, Johnson E, Ongolo-Zogo P. An assessment of policymakers’ engagement initiatives to promote evidence informed health policy making in Nigeria. Pan Afr Med J 2017; 27:57. doi: 10.11604/pamj.2017.27.57.9844 [Crossref] [ Google Scholar]
63. Bennett S, Corluka A, Doherty J. Influencing policy change: the experience of health think tanks in low- and middle-income countries. Health Policy Plan 2012; 27(3):194-203. doi: 10.1093/heapol/czr035 [Crossref] [ Google Scholar]
64. Bennett S, Corluka A. Health Policy Analysis Institutes: Landscaping and Learning from Experience. Geneva: Alliance for Health Policy and Systems Research; 2010. https://www.who.int/alliance-hpsr/evidenceinformed/alliancehpsr_healthpolicyanalysisinstitutesreport.pdf. Accessed October 12, 2018.
65. Rispel LC, Doherty J. Research in support of health systems transformation in South Africa: the experience of the Centre for Health Policy. J Public Health Policy 2011; 32 Suppl 1:S10-29. doi: 10.1057/jphp.2011.33 [Crossref] [ Google Scholar]
66. Uneke CJ, Ndukwe CD, Ezeoha AA, Uro-Chukwu HC, Ezeonu CT. Implementation of a health policy advisory committee as a knowledge translation platform: the Nigeria experience. Int J Health Policy Manag 2015; 4(3):161-168. doi: 10.15171/ijhpm.2015.21 [Crossref] [ Google Scholar]
67. Roby DH, Jacobs K, Kertzner AE, Kominski GF. The California health policy research program - supporting policy making through evidence and responsive research. J Health Polit Policy Law 2014; 39(4):887-900. doi: 10.1215/03616878-2743263 [Crossref] [ Google Scholar]
68. Clar C, Campbell S, Davidson L, Graham W. Systematic Review: What are the Effects of Interventions to Improve the Uptake of Evidence from Health Research into Policy in Low and Middle-Income Countries? https://assets.publishing.service.gov.uk/media/57a08ab3ed915d3cfd0008c2/SR_EvidenceIntoPolicy_Graham_May2011_MinorEditsJuly2011.pdf. Accessed December 5, 2018. Published 2011.
69. Yehia F, El Jardali F. Applying knowledge translation tools to inform policy: the case of mental health in Lebanon. Health Res Policy Syst 2015; 13:29. doi: 10.1186/s12961-015-0018-7 [Crossref] [ Google Scholar]
70. Shroff Z, Aulakh B, Gilson L, Agyepong IA, El-Jardali F, Ghaffar A. Incorporating research evidence into decision-making processes: researcher and decision-maker perceptions from five low- and middle-income countries. Health Res Policy Syst 2015; 13:70. doi: 10.1186/s12961-015-0059-y [Crossref] [ Google Scholar]
71. Shaw SE, Russell J, Parsons W, Greenhalgh T. The view from nowhere? how think tanks work to shape health policy. Crit Policy Stud 2015; 9(1):58-77. doi: 10.1080/19460171.2014.964278 [Crossref] [ Google Scholar]
72. Goering P, Butterill D, Jacobson N, Sturtevant D. Linkage and exchange at the organizational level: a model of collaboration between research and policy. J Health Serv Res Policy 2003; 8 Suppl 2:14-19. doi: 10.1258/135581903322405126 [Crossref] [ Google Scholar]
73. Uzochukwu B, Mbachu C, Onwujekwe O. Health policy and systems research and analysis in Nigeria: examining health policymakers’ and researchers’ capacity assets, needs and perspectives in south-east Nigeria. Health Res Policy Syst 2016; 14:13. doi: 10.1186/s12961-016-0083-6 [Crossref] [ Google Scholar]
74. El-Jardali F, Saleh S, Khodor R. An institutional approach to support the conduct and use of health policy and systems research: The Nodal Institute in the Eastern Mediterranean Region. Health Res Policy Syst 2015; 13:40. doi: 10.1186/s12961-015-0032-9 [Crossref] [ Google Scholar]
75. Mutatina B, Basaza R, Obuku E, Lavis JN, Sewankambo N. Identifying and characterising health policy and system-relevant documents in Uganda: a scoping review to develop a framework for the development of a one-stop shop. Health Res Policy Syst 2017; 15(1):7. doi: 10.1186/s12961-017-0170-3 [Crossref] [ Google Scholar]
76. Mijumbi-Deve R, Sewankambo NK. A process evaluation to assess contextual factors associated with the uptake of a rapid response service to support health systems’ decision-making in Uganda. Int J Health Policy Manag 2017; 6(10):561-571. doi: 10.15171/ijhpm.2017.04 [Crossref] [ Google Scholar]
77. Uzochukwu B, Onwujekwe O, Mbachu C. The challenge of bridging the gap between researchers and policy makers: experiences of a Health Policy Research Group in engaging policy makers to support evidence informed policy making in Nigeria. Global Health 2016; 12(1):67. doi: 10.1186/s12992-016-0209-1 [Crossref] [ Google Scholar]
78. Boyko JA. Evidence-informed health policy making in Canada: past, present, and future. J Evid Based Med 2015; 8(4):215-221. doi: 10.1111/jebm.12169 [Crossref] [ Google Scholar]
79. Ongolo-Zogo P, Lavis JN, Tomson G, Sewankambo NK. Assessing the influence of knowledge translation platforms on health system policy processes to achieve the health millennium development goals in Cameroon and Uganda: a comparative case study. Health Policy Plan 2018; 33(4):539-554. doi: 10.1093/heapol/czx194 [Crossref] [ Google Scholar]
80. Wilson M, Lavis J, Grimshaw J. Supporting the use of research evidence in the Canadian health sector. Healthc Q 2012; 15 Spec No:58-62. doi: 10.12927/hcq.2013.23148 [Crossref] [ Google Scholar]
81. Campbell D, Donald B, Moore G, Frew D. Evidence check: knowledge brokering to commission research reviews for policy. Evid Policy 2011; 7(1):97-107. doi: 10.1332/174426411x553034 [Crossref] [ Google Scholar]
82. WHO Regional Office for Europe. Conceptual Background and Case Studies - Introduction to EVIPNet Europe. Copenhagen: WHO Regional Office for Europe; 2017. http://www.euro.who.int/__data/assets/pdf_file/0004/368833/Annual-report-2017-for-EVIPNet-Europe.pdf?ua=1. Accessed February 15, 2019.
83. Majdzadeh R, Sadighi J, Nejat S, Shahidzade Mahani A, Gholami J. Knowledge translation for research utilization: design of a knowledge translation model at Tehran University of Medical Sciences. J Contin Educ Health Prof 2008; 28(4):270-277. doi: 10.1002/chp.193 [Crossref] [ Google Scholar]
84. Moore GM, Redman S, Turner T, Haines M. Rapid reviews in health policy: a study of intended use in the New South Wales’ Evidence Check programme. Evid Policy 2016; 12(4):505-519. doi: 10.1332/174426415x14446635524057 [Crossref] [ Google Scholar]
85. Ziam S. Knowledge brokers and how to communicate knowledge in 2010. Allergy Asthma Clin Immol 2010; 6(4):A3. doi: 10.1186/1710-1492-6-S4-A3 [Crossref] [ Google Scholar]
86. Moat KA, Lavis JN, Abelson J. How contexts and issues influence the use of policy-relevant research syntheses: a critical interpretive synthesis. Milbank Q 2013; 91(3):604-648. doi: 10.1111/1468-0009.12026 [Crossref] [ Google Scholar]
87. El-Jardali F, Lavis JN, Ataya N, Jamal D, Ammar W, Raouf S. Use of health systems evidence by policymakers in eastern Mediterranean countries: views, practices, and contextual influences. BMC Health Serv Res 2012; 12:200. doi: 10.1186/1472-6963-12-200 [Crossref] [ Google Scholar]
88. Liverani M, Hawkins B, Parkhurst JO. Political and institutional influences on the use of evidence in public health policy A systematic review. PLoS One 2013; 8(10):e77404. doi: 10.1371/journal.pone.0077404 [Crossref] [ Google Scholar]
89. Gagliardi AR, Webster F, Brouwers MC, Baxter NN, Finelli A, Gallinger S. How does context influence collaborative decision-making for health services planning, delivery and evaluation?. BMC Health Serv Res 2014; 14:545. doi: 10.1186/s12913-014-0545-x [Crossref] [ Google Scholar]