Stakeholder Perspectives of Attributes and Features of Context Relevant to Knowledge Translation in Health Settings: A Multi-Country Analysis

Background

Internationally, healthcare professionals’ use of research evidence in clinical practice is suboptimal, despite increased awareness of evidence and known gaps in its use.^1-8 In response, there is widespread interest in the field of knowledge translation (KT)/implementation. The KT field comprises KT science (scientific study of methods to promote the uptake of research findings into routine healthcare in clinical, organizational or policy contexts)⁹ and KT practice (act of applying advancements from KT science in practice; for example, using implementation strategies of known effectiveness to support people, organizations, and/or systems to use evidence in decision-making).¹⁰ Researchers have identified several major conceptual and methodological issues facing the KT field; among them is the need to assess and report on the influence of context in KT studies.^11-30 This first requires conceptual clarity of context and particularly, what comprises context.

It is unclear what is meant by ‘context’ or even if authors are referring to the same concept when they use the term. For example, Ovretveit³¹ and Rycroft-Malone³² define context broadly. Ovretveit³¹ defines context as all factors that are not part of the intervention, and Rycroft-Malone³² defines it as “the environment or setting in which the proposed change is to be implemented” (p. 299). May et al,³³ on the other hand, defines context more specifically, as “the physical, organizational, institutional, and legislative structures that enable and constrain, and resource and realize, people and procedures.” In addition to multiple definitions, context has been examined in implementation research from a large variety of perspectives including syntheses on the determinants of innovation adoption,^11,34,35 the role of context in quality improvement,^29,30,36,37 context features associated with research utilization^38-41; as well as its role in KT frameworks ^(eg,42-51) and in the development of instruments to measure context ^{(eg,23,41,51-55)}. While each of these reports suggests that context is vital to KT in health settings, there is little agreement across them on what comprises context and thus, which attributes and features of context are important to assess to optimize KT in the health sector.

Several KT frameworks exist that recognize the importance of context. However, they provide little detail on what are the key attributes and features of context. Rogers’ Diffusion of InnovationsTheory,⁵⁶ first published in 1962, is a commonly used theory in KT studies. It equates innovation adoption with evidence implementation and describes it as being related to a variety of contextual features such as leadership, centralization, complexity, formalization, interconnectedness, and organizational slack. In 1998, Kitson and colleagues⁴⁸ developed the Promoting Action on Research in Health Services ( PARiHS ) framework. In this framework, successful KT is hypothesized to be a function of evidence, context (defined as leadership, culture, and feedback), and facilitation.^32,44,57 The PARiHS framework was revised in 2016 and termed the Integrated - Promoting Action on Research in Health Services ( i-PARiHS ).⁵⁸ The context construct was expanded on in the revised framework, differentiating inner (including both the immediate local setting (eg, unit, hospital department or primary care team) and the organisation within which this unit or team is embedded) from outer context (the wider health system in which the organisation is based and reflects the policy, social, regulatory and political infrastructures). In 2002, Champagne⁵⁹ proposed a framework of factors to consider when producing change in organizations including context: the implementation climate (incentives, physical, human, and cognitive resources; change management strategies) and structural characteristics (organicity, complexity, and integration). The Knowledge to Action Framework,⁶⁰ published in 2006, also highlights the importance of context to successful KT. Included in this framework are processes related to context necessary to implement evidence in healthcare, namely: adapting evidence to the local context and the assessing the barriers and enablers (which include context) to evidence use. In 2009, Damschroder et al³⁸ published the Consolidated Framework for Implementation Research (CFIR). This framework includes eight concepts related to the intervention itself, four related to the outer context (eg, the patient and resources), twelve related to the inner setting or context (eg, culture and leadership), five related to the individual, and eight related to processes (such as planning and reflection). Context as conceptualised in CFIR is reflective of organizational literature, where context is examined from the perspective of levels. In the organizational literature, three levels of context are commonly proposed: macro in which market-type forces are at play, meso in which organizational characteristics are an influence; and micro in which activities in the clinical setting provides a contextual influence.⁶¹ More recently, in 2013, Flottorp and colleagues⁵⁰ published the Tailored Implementation in Chronic Diseases (TICD) checklist, which draws on a consolidation of KT frameworks that include context. This checklist contains 7 broad domains, of which 5 are related to context: patient factors, professional interactions, incentives and resources, capacity for organizational change, and social, political and legal factors.

Despite the existence of many KT theories and frameworks that include context, as illustrated above, when examined closely several issues are evident. First, while each theory/framework includes context, they are limited in the number of attributes and features of context they identify. In our previous research in this area, we identified several features of context that are not captured in commonly used KT frameworks: TICD, CFIR and PARiHS/i-PARiHS. These include: patient population features (eg, patient demographics), healthcare provider features (eg, job autonomy, level of experience), work structure features (eg, workload, continuity of care), and facility features (eg, type of facility, size of facility).⁶¹ Second, these theories/frameworks are inconsistent with each other regarding the attributes and features of context they include; some features and attributes are represented in more than one framework (eg, leadership in TICD, CFIR, PARiHS/i-PARiHS), while others are unique to a single framework (eg, continuing educational system in TICD but not in CFIR, PARiHS/i-PARiHS). Further, no two frameworks contain the exact same set of context features and attributes. This lack of correspondence between frameworks raises questions about the comprehensiveness of existing frameworks with respect to context. Third, health system stakeholders, defined as individuals responsible for the design and implementation of KT strategies, programs, and change processes focused on improving healthcare professionals’ use of evidence in health settings, have seldom been engaged in developing these theories/frameworks. Health system stakeholders include change agents and KT specialists (eg, healthcare decision-makers, knowledge brokers, implementation practitioners, quality improvement specialists) and KT researchers. Such individuals have tacit firsthand knowledge of KT and the effects of context on KT that have not yet been systematically explored. Hence, their knowledge on what attributes and features of context are important to research use by healthcare professionals is necessary to further elucidate the concept of context. As a result, the purpose of this study was to conduct interviews with a variety of health system stakeholders internationally in order to compile a comprehensive list of contextual attributes and their features relevant to KT in health settings. Specifically, we aimed to identify contextual features perceived by stakeholders to: (1) facilitate or hinder healthcare professionals’ use of research in clinical practice settings; and (2) facilitate or hinder KT strategies to improve healthcare professionals’ use of research evidence in clinical practice settings.

Methods

Study Design and Sample

A descriptive qualitative study design was used. The sample consisted of health system stakeholders, defined as individuals who are responsible for the design and/or implementation of interventions, programs, and change processes focused on improving healthcare professionals’ use of research evidence in clinical practice settings (eg, hospitals, primary care). This group includes both change agents (eg, KT and quality improvement specialists) and researchers who are undertaking KT work (hereafter referred to as KT researchers). Participants were purposefully selected from the three Canadian provinces where our research team’s change agent investigators resided (Nova Scotia, Ontario, Alberta) as well as in Australia, United Kingdom, and United States, where our international researcher investigators resided. A purposive list of potential participants from each country was developed. A maximum variation approach across countries was used to optimize diversity in primary role and experience in KT to yield a range of perspectives. All research team members created a list of potential participants (by role – change agents/KT specialists and KT researchers) for their country (in the case of Canada, for their province) by drawing on their extensive professional networks. These lists were then merged by country so that eight lists resulted, two for each country (one for change agents/KT specialists and one for KT researchers). Snowball sampling⁶² (asking recruited individuals to recommend additional participants) was also used in each country. Potential participants were contacted by email. Participants were purposefully selected from each country to ensure we interviewed participants with varying levels of experience (less than 5 years and 5 or more years) in KT and varying roles (change agents/KT specialists and KT researchers). Sample size was informed by the concept of data saturation, interviews were conducted in each country until no new information was offered, meaning no new codes were obtained in the final interview of each country.⁶³ Broadly informed by previous research,^64-66 a minimum of five participants per country was estimated to be needed to achieve data saturation.

Data Collection and Analysis

Semi-structured one-on-one interviews were conducted in person or by phone over a 16-week period (May-August 2016). Interviewers and participants (called key informants) had no prior relationships before study commencement. Key informants were made aware of the goals of the research study prior to the interviews. Interview questions were open-ended and designed to elicit key informants’ tacit knowledge about context (what features did they perceive to comprise context which are relevant to KT in health settings). We did not present a definition of context to our key informants as we wanted to elicit their perception of what comprises context – we asked each key informant to define context, what ‘context’ meant to them, and what features of context were relevant to KT in health settings. Probes were used throughout the interviews, where appropriate, to explore specific features of context identified by the key informant in more detail and to ensure clarity. The interview guide was pilot-tested with three Canadian stakeholders (two hospital-based change agents – one KT specialist and one quality improvement specialist) and one KT researcher. The pilot-test participants were not part of the larger study sample. Based on the pilot interviews, a short series of background questions were added to the interview guide (eg, Can you give me a brief overview of your professional role including previous relevant roles/responsibilities? What education or training have you completed to prepare you for your work?). No other modifications to the interview guide were made.

Interviews were approximately 30 minutes in duration, digitally recorded, transcribed verbatim and verified by the interviewer prior to analysis. Transcripts were not returned to key informants to be verified. The transcripts were imported into NVivo 10 software.⁶⁷ Data were analyzed independently by research team members trained in qualitative analysis (JES, MC, AH, KB, KD, IDG) using inductive qualitative thematic content analysis.^68,69 We used an inductive, rather than deductive, analytic approach because existing context frameworks are limited in the features and attributes of context that they identify.⁶¹ Thus, a deductive approach would have restricted our lens, leading to potentially missed features and attributes of context. Our inductive analysis occurred in four systematic steps: (1) selection of utterances relating to context, (2) coding of features of context, (3) categorizing of features of context into higher level attributes of context, and (4) comparison of attributes and features by: (i) country (was the code present in interviews from multiple countries), (ii) level of KT experience (was the code present in interviews with key informants with more and less KT experience, using five years of experience as the cut point for more vs. less), and (iii) primary role (was the code present in both change agents/KT specialists and KT researchers). Analysis by country allowed us to assess whether the contextual attributes and their features identified were likely transferrable to diverse settings. Analysis by KT experience was conducted to see if more stakeholders with more experience with KT, based on greater tacit knowledge, would identify additional attributes and features of context, beyond those identified by stakeholders with less KT experience. Five years was chosen as the cut-off based on past research. According to Benner’s⁷⁰ Novice to Expert Model, in the acquisition and development of skills, nurses pass through five levels of proficiency: novice, advanced beginner, competent, proficient, and expert. The dividing point between beginners (novice, advanced beginner, competent) and experts (proficient, and expert) is estimated to be between 3 and 5 years.⁷⁰ Accordingly, we classified stakeholders with 0-5 years of KT experience as ‘less’ experienced and those with greater than 5 years of KT experience as ‘more’ experienced. Finally, with respect to primary role, change agents/KT specialists have first-hand tacit knowledge of KT and the effects of context on KT. Hence, we wanted to see if they would identify additional attributes and features of context that are relevant to KT beyond those identified by KT researchers.

Each transcript was first read and key ideas (ie, utterances) that reflected context were highlighted independently by two research team members. These context utterances were assigned a ‘code’ and labelled as a feature of context. Codes (features) were operationally defined in order to be consistently applied throughout the data. Following coding of all interviews, theoretical definitions were developed for each feature by reviewing all data coded to the feature. This was completed by three team members independently, followed by revision and consensus on the definitions. Two team members then merged similar codes (features of context) to create broader categories (attributes of context), which became our main units of analysis in this study. Each category (attribute) of context identified in the transcripts was given a label, definition, and guideline for identification. Theoretical definitions for each attribute were developed by reviewing all features and their definitions assigned to the attribute. This was completed by three team members independently. The team members responsible for this process met and compared their interpretations bi-weekly and jointly selected a label that best represented the category and a definition. All disagreements in the coding, categorizing and definitional phases were discussed and consensus sought. Finally, attributes of context and their features were examined for their presence (or not) by: (1) country (Canada, United States, United Kingdom, and Australia); (2) more (>5 years) or less (0-5 years) experience in KT; and (3) primary role (change agent/KT specialist and KT researcher). Throughout data analysis, codes (features of context) and categories (attributes of context) were discussed and refined within the research team until agreement was reached.⁷¹

Rigor was established following criteria set forth by Lincoln and Guba.⁷² Credibility (to establish confidence that our results, from the perspective of the stakeholders we interviewed, are true, credible and believable⁷²) was established through examining data across multiple countries, having multiple team members discuss the findings and generate and reach consensus on the key categories (attributes of context),⁷³ and through an audit trail. The audit trail was maintained by documenting discussions and decisions made throughout data collection and analysis. Dependability (to ensure our findings are repeatable if the study occurred in the same cohort of key informants, coders and context⁷²) was established by providing a rich description of our study methods and an audit trail. Confirmability (to extend the confidence that our results can be confirmed or collaborated by other researchers⁷²) was established through reflexivity (we implemented regular team analytic meetings which included senior KT stakeholders, where we reflected on our findings) and triangulation (multiple research team members were involved in coding the data from multiple groups of stakeholders (change agents/KT specialists and KT researchers) across four countries). Finally, transferability (to extend the degree to which the results can be transferred to other contexts or settings⁷²) was established through purposeful sampling (by experience and role across countries) and data saturation.

Results

Sample Characteristics

A total of 91 potential key informants were approached, of which 39 agreed to be interviewed for a response rate of 43%. The sample was drawn from four countries, Australia (n = 12), Canada (n = 14), the United Kingdom (n = 6), and the United States (n = 7). The majority of key informants were female (n = 27), worked within a public healthcare model (n = 33) and had greater than 10 years of experience in KT (practice or research) (mean 11.9 years). There was equal distribution in terms of the key informants’ self-identified primary role (employed in a health system as a change agent/KT specialist, n = 19 or as a KT researcher in a higher education setting or research institute, n = 20). Table 1 summarizes the sample’s characteristics.

Complexity and Interrelatedness of Contextual Attributes

While the focus of this study was to identify specific attributes of context and their features that are relevant to KT from the perspective of health system stakeholders, key informants frequently discussed multiple attributes and features simultaneously, illustrating the complexity and interrelatedness of context. To illustrate, we include a quote from a participant from each country, in which they describe different and multiple facets of context.

A participant from Australia refers to context as varying by a range of factors, including jurisdictional policy, geography, health professional groups, the patient population and organisational culture.

“…there can be different policies depending on the jurisdiction that you’re in. Also in terms of geography, whether you’re talking about an urban or rural or a remote location. Within stakeholders, I guess I’m thinking not just about clinicians, who your clinicians, who your health professionals are, but who your patient population is. And also I would put in things like culture, organizational culture. I think that’s a really important part of context as well. I’m pretty sure there is more” (Australia, Public, Participant 2).

A Canadian participant similarly identifies multiple aspects of context: the organisational culture, staff mix and patient population, in addition to the physical environment.

“ So I, I context can mean different things like I sort of categorize it in terms of the organizational culture. The context of the organizational culture the physical environment, the clinical makeup of the staff as well as the clinical makeup of the patients” (Canada, Public, Participant 3).

Identifying different aspects of context, a participant from the United Kingdom illustrated the complexity of context when referring to potential research users’ resistance to change and concerns about shifts in power.

“So, you know, it’s hard to say whether people who say ‘Well the evidence isn’t strong enough.’ If they’re actually making that purely rational decision or whether that’s just their, their rationalization or their story for opposition, which maybe have some other psychological cause let’s say. Like they don’t like it because it’s to do with power and this change particularly putting in place this kind of system shifts power away from clinicians to patients. And no clinician is gonna stand up and say ‘I don’t think patients should be empowered.’ But they might stand up and say ‘I don’t think the evidence is strong enough we don’t want to do this’” (United Kingdom, Public, Participant 4).

From the United States, a participant refers to context as comprising communication networks, access to resources to guide and support implementation, and accountability as part of shared governance.

“And for that they, you know, everybody was—all the hospitals have a chief nurse and the chief nurses get together once/month to talk with the leader of the nurse core who’s a general officer. So that’s one mechanism to get the word out on how is this being implemented and how are you doing? There were implementation guides. There were training manuals. So for example, to do shared governance they didn’t call it governance they called it accountability because it’s not just governance for your unit” (United States, Public, Participant 3).

These quotes illustrate participants’ perceptions of multiple facets of context and that each facet cannot be considered in isolation. Considering these four quotes collectively also highlights that context is likely complex because people perceive it in different ways.

Specific Context Attributes and their Features

Overview

In total, 66 unique features of context perceived to be relevant to KT in health emerged from the 39 interviews with health system stakeholders. All features surfaced in at least two interviews. We grouped the 66 unique features into 16 broader attributes of context: (1) Patient Characteristics, (2) Health Professional Characteristics, (3) Collaboration, (4) Culture, (5) Evaluation, (6) Facility Characteristics, (7) Financial Considerations, (8) Governance, (9) Leadership, (10) Organizational Readiness for Change, (11) Professional Role, (12) Resource Access, (13) System Features, (14) Work Structure, (15) Political Climate, and (16) Regulatory and Legislative Standards. The number of features in each attribute varied from 1 (in the Culture and Political Climate attributes) to 11 (in the Resource Access attribute). Table 2 contains a listing of the 16 attributes of context and a consensus definition for each attribute derived by the research team. Table 3 contains a list of the 66 features identified, organized by attribute, and an illustrative quote for each feature. Tables 4 and 5 summarize the presence of the context attributes and features by country, and level of KT experience and primary role, respectively.

Two of the 16 context attributes were identified in all 39 interviews: (1) Culture, defined as ‘the inherited beliefs, values, and attitudes of a group’ and (2) Resource Access, defined as ‘access to resources of any kind.’ Culture contained a single feature – organizational culture. An illustrative quote for this feature is:

“Context includes … things like the culture of the organization as a whole, but also the cultures within the organization that maybe at a unit or program level and because they’re always different as well” (Canada, Participant 3).

Resource Access contained 11 features: (1) resource access (generally), (2) lack of time, (3) existence of guidelines, (4) formal communication, (5) expert support, (6) technology, (7) organizational training and education, (8) documentation, (9) online resources, (10) equipment, and (11) space as a resource. An illustrative quote for resource access (generally) is:

“You need resources at your disposal in order to collect it and this was the huge challenge across this organisation. There’s very little priority around equipping people with the resources to gather the evidence they need to make their case” (United States, Participant 3).

Context Attributes and Features by Country

Of particular interest in this study was whether and which attributes and features of context varied by country. Table 4 indicates the presence or not of the 16 attributes of context and their features by country. All 16 attributes were discussed by key informants in each country. Most features (n = 52 of 66) were also discussed by key informants from at least three of the four countries. Only 3 of the 66 features were mentioned by key informants in a single country, with no discernable pattern. Two features were mentioned by only Canadian key informants: (1) mentorship (attribute: Leadership) and (2) provincial responsibility (attribute: Regulatory and Legislative Standards). A third feature – departmental approval (attribute: Governance) was discussed only by Australia key informants. Illustrative quotes for these features can be found in Table 3. There were also differences by country in terms of some features being less frequently mentioned by key informants of a particular country. For example, of the 66 total identified features of context identified, 26 of them were not reported by UK key informants compared to only 12, 7, and 6 features not being identified by the United States, Australian, and Canadian key informants, respectively. A summary of identified features by country can be seen in Table 4.

Context Attributes and Features by Level of Experience

As described earlier, key informants were categorized based on whether they had ‘less’ or ‘more’ experience in KT using 5 years as the cut point; key informants with 5 years or less of KT experience were categorized as ‘less experienced’ and those with more than 5 years of experience were categorized as ‘more experienced.’ Table 5 displays the presence or not of all 16 attributes and 66 features of context by level of KT experience (less or more) of the key informants. There was considerable consistency with respect to context attributes and features identified by key informants with less and more experience in KT. All 16 attributes and the vast majority of features (57 of 66) were discussed by both more and less experienced key informants. Only 9 features were present in one group and not the other, all of which were mentioned by stakeholders with more experience and not those with less experience. The 9 features were: quality improvement monitoring (attribute: Evaluation), financial incentives (attribute: Financial Considerations), mentorship (attribute: Leadership), professional development (attribute: Professional Role), equipment (attribute: Resource Access), space as a resource (attribute: Resource Access), workload (attribute: Work Structure), continuity of care (attribute: Work Structure), and provincial responsibility (attribute: Regulatory & Legislative Standards).

Context Attributes and Features by Primary Role

While all key informants in this study were health system stakeholders responsible for the design and implementation of interventions, programs, and change processes, they were asked to self-identify their primary role as either a change agent/KT specialist or KT researcher. Table 5, in addition to level of experience, also displays the presence or not of all 16 attributes and 66 features of context by primary role of the key informants. Similar to country and level of KT experience, all context attributes and most features (n = 61 of 66) were mentioned by key informants in both primary roles. Four features were discussed by KT researchers but not change agents/KT specialists: (1) workflow (attribute: Work Structure), (2) continuity of care (attribute: Work Structure), (3) social interactions (attribute: Collaboration), and (4) departmental approval (attribute: Governance). One feature was discussed by change agents/KT specialists but not KT researchers: provincial responsibility (attribute: Regulatory and Legislative Standards).

Discussion

Summary of Findings

The purpose of this study was to compile a list of contextual attributes and their features relevant to KT in health settings from the perspective of health system stakeholders (change agents/KT specialists and KT researchers). We codified tacit knowledge of these stakeholders in four countries about what constitutes context. In total, we identified 66 unique features of context, categorized into 16 broader attributes. The 16 attributes of context covered all contextual issues identified in all 39 interviews. We found instances of all 16 attributes of context in the interviews irrespective of country, level of experience with KT, and primary role (change agent/KT specialist vs. KT researcher), revealing robustness and transferability of our attributes of context. Some variation in context was identified at the finer-grained feature level (of these attributes) but even then, most features were relevant across countries, KT experience levels and primary role.

New Attributes of Context Identified and their Features

We compared the 66 features of context identified in this study to those contained in the TICD framework checklist,⁶⁰ a recent comprehensive published consolidation of KT checklists based on frameworks which include context. The TICD includes elements that are in popular meta-typologies such as the CFIR,⁴⁴ as well as commonly used conceptual frameworks such as the Theoretical Domains Framework^42,43 and the PARiHS Framework⁴⁸. Almost all of our context attributes (n = 15 of 16) mapped to concepts in the TICD, supporting international consensus for these attributes. The one attribute from our analyses that is not represented in TICD is ‘Facility Characteristics.’ Three features were identified in our study that characterize this attribute: geography, type of facility, and size.

While many of our 16 higher-level attributes of context were known previously, they were not well-defined. Through this study, we synthesized experiential knowledge about context attributes and provide a large range of illustrative features for them. For instance, almost half of the features (n = 27 of 66) identified by key informants in this study are not contained in the TICD checklist. This includes new features in most (n = 12 of 16) attributes. We identified new features of several commonly discussed context attributes: (1) attribute Resource Access (new features – existence of guidelines, technology, documentation, online resources); (2) attribute Leadership (new feature – mentorship); (3) attribute Work Structure (new features – work structure generally, disconnect between research, education, and clinical care, standardization of care, workload, scheduling, continuity of care); (4) Attribute Financial (new feature – funding model); (5) attribute Collaboration (new features –partnerships, social interactions); (6) Attribute Professional Role (new feature – accountability, job autonomy); and attribute System Features (new features –managing change, system complexity, organizational stability). We also identified new features for some less commonly discussed context attributes including: (1) attribute Professional Role (new features – accountability, job autonomy); (2) attribute Evaluation (new feature – routinized audit); (3) attribute Patient Characteristics (new feature – patient demographics); (4) attribute Governance (new feature – departmental approval); and (5) attribute Regulatory and Legislative Standards (new features – provincial responsibility, accreditation).

One reason why many of our features might be lacking from the TICD and other meta-typologies such as the CFIR may be because that these frameworks focus mainly on higher level context attributes rather than the more detailed context features. Our identification of these features is a critical advancement in the KT field. As a result of this work, we advance much needed conceptual clarity in context, which is critical to develop common assessment tools to measure context to determine which specific context features are more or less important in different contexts and for changing different healthcare professional behaviours. The expectation is that such measurement tools could subsequently be used to: (1) tailor KT intervention designs and their delivery, (2) better interpret the effects of KT interventions, and (3) pragmatically guide change agents and researchers in their KT efforts.

Common Attributes of Context

Next we discuss the more commonly identified context attributes that surfaced in our interviews with stakeholders internationally.

Culture

All 39 key informants in this study identified the context attribute Culture as relevant to KT. This finding is consistent with previous studies examining stakeholder perspectives on facilitators of and barriers to KT,^74-80 suggesting broad agreement among stakeholders internationally of the importance of culture. This is important because consensus among organizational leaders on the factors important for KT has been associated with greater success in the implementation of evidence-based practices.⁸¹

The feature ‘organizational culture,’ which refers to the normative beliefs and shared expectations that govern the work behaviour of a clinical team or employees of a healthcare facility, was mentioned by key informants across all four countries in this study. This is consistent with past research. For example, Beidas and colleagues,⁷⁴ in a recent study of stakeholders’ perspectives of the facilitators of and barriers to implementing evidence-based practices in a United States public mental health system, found agencies that were successful in implementation were those that “changed their culture to fit the needs of the evidence-based practice” (p. 897). Similarly, Harvey and colleagues,⁷⁵ who investigated which contextual factors are most influential in mediating performance improvements in the United Kingdom (England and Scotland) public health systems, found that organizations with successful implementation of evidence-based practices were those that had organizational leadership and management that promoted a “can do” culture.

Resource Access

The attribute Resource Access was also identified by all key informants across all countries in our study. The importance of Resource Access was acknowledged in multiple previous studies.^{74,75,77-80,82,83} Specifically, previous studies with health system stakeholders found that lack of appropriate resources such as insufficient staffing,^78-80,83 lack of appropriate educational and training programs,^74,75,78,83 and insufficient support and poor communication^75,83 hindered implementation efforts. In the present study, the most commonly discussed Resource Access feature was lack of time. This is consistent with previous research indicating the need for protected time for KT efforts (eg, supervision, training and education) to ensure consistent and maintained engagement by staff and leaders.^74,79,82,83 The importance of lack of time may also reflect individuals’ unwillingness to partake in uncompensated work tasks or activities that take time away from more pressing patient care demands. This feature may be particularly important for individuals who must balance many competing demands on their time.

While the attribute Resource Access was mentioned by all key informants on our study, it is unclear why certain features of the attribute were viewed as more important (ie, mentioned by more key informants) than other features within the attribute. For example, the feature of lack of time was discussed by the majority of key informants in all four countries, while the features equipment, and space as a resource were both only mentioned by one key informant in each of Australia and Canada. This may reflect that having enough time to implement an innovation is crucial regardless of what the innovation is. Likewise, having sufficient equipment and space may be specific to particular innovations and thus not a universal important component of context necessary for successful KT. However, it may also be that participants in our study had sufficient experience in KT that they did not require special equipment or space and therefore did not view those components as important contextual features.

Organizational Readiness for Change

All three features of Organizational Readiness for Change were commonly mentioned by the key informants in all four countries in our study: (1) buy-in (agreeing with and accepting a suggestion or change (eg, a new policy)); (2) capacity (the organization’s total workload for running current operations and could include conducting change activities); and (3) engagement (the meaningful involvement of staff or stakeholders in the delivery of healthcare services). The identification of these features as important to KT are also consistent with previous reports of stakeholder perspectives.^74,75,80 Beidas and colleagues⁷⁴ found that agencies with early and sustained engagement experienced successful implementation of evidence-based practices more often than did agencies without capacity and engagement. Similarly, lack of buy-in is suggested to impede KT because individuals who are not fully committed to the implementation are thought to be less motivated to contribute to and ensure the success of the project.^74,75,80

Collaboration, Financial Considerations, Leadership, Work Structure

Other more common attributes identified in our study included: Collaboration, Financial Considerations, Leadership, and Work Structure. Previous studies investigating stakeholders’ perspectives of factors important for KT have also identified leadership as a core feature. Specifically, the presence of committed and supportive leaders and managers who are responsive to KT has been noted as important.^75,76,82,83 Similarly, components of Work Structure,^74,80Financial Considerations,^76,77,79,80 and Collaboration between stakeholders^74-76,78-80 have also been identified as key features of context important to KT by diverse stakeholders internationally. With respect to Work Structure, Beidas and colleagues⁷⁴ in a study of stakeholders from the United States found that teamwork (a feature of our context attribute Work Structure) was important to KT in that it facilitates buy-in (a feature of our context attribute Organizational Readiness for Change) and setting organizational goals and priorities (a feature of our context attribute Leadership). Financial features identified in previous studies from several countries were similar to those we identified. Financial features were found to facilitate KT when perceived cost was low or known,^79,80 funding was available for the implementation,⁷⁹ and incentives were offered to encourage buy-in towards the implementation⁷⁶; and were found to hinder KT when cost was high or unknown.⁷⁷ Internationally, multiple studies with stakeholders have also identified the importance of collaboration between stakeholders, both internal and external to the organization.^74-76,78-80 Beidas and colleagues⁷⁴ in their study of United States stakeholders found that coordinated collaboration across stakeholder groups throughout the whole KT process was important to achieve successful implementation of evidence-based practices. Muellmann and colleagues,⁷⁸ in a study of stakeholders across five countries (Belgium, Germany, Ireland, Norway, and Poland) found that in addition to collaboration during KT, continued collaboration and communication between stakeholders (after implementation) was also necessary to facilitate successful KT. Similar to our study, Harvey et al,⁷⁵ Schneider and colleagues,⁸⁰ and Renz and colleagues⁷⁹ in studies with stakeholders in the United Kingdom (England and Scotland) and United States, found specifically that extending collaboration to external organizations/stakeholders was important to successful KT.

Less Commonly Reported Features of Context

While the 16 attributes of context identified in this study varied very little by country, stakeholder level of experience with KT (less/more experienced) and stakeholder primary role (change agent/KT specialist vs. KT researcher), there was some (although limited) variation as expected in the finer grained features of context. While all features were mentioned by multiple stakeholders, 14 of the 66 features were reported less consistently (ie, in fewer than 3 of the 4 countries in our study). The majority of these 14 features were reported by key informants regardless of their level of experience with KT or their primary role. Furthermore, 11 of them were identified by stakeholders in more than 1 country. The only notable trend was that most of the uncommon features (n = 12 of 14) were not identified by stakeholders in the United Kingdom. However, stakeholders from United Kingdom in general, in this study, spontaneously identified fewer features of context (n = 40 features) compared to the United States (n = 54), Australia (n = 59) and Canada (n = 60). We do not know from this study or the literature why some features of context would be identified less frequently overall or by stakeholders in the United Kingdom in particular, or if these less frequent features are less important to KT. Since specific features of context were not specifically probed in our study, it is possible that more stakeholders would have identified the less commonly reported features had they been deliberately asked about them. Therefore, at this stage of our research program, we have retained all context features identified on our list of features in order to not underestimate the importance of any of the features. This study comprised a necessary first step of codifying tacit knowledge of stakeholders about the nature of context, thereby advancing our understanding of context in KT.

Next Steps

In this study we identified context attributes and features perceived by health system stakeholders in four developed countries to be relevant to KT in health settings. We suggest that a similar study to the one reported here be conducted with stakeholders in developing countries to determine if the same or different features of context apply in developing countries. Simultaneously to the study reported in this manuscript, we also identified attributes and features of context by researchers⁵⁹ and healthcare professionals.⁶¹ Our next steps include triangulation across all three studies to produce a unified framework of context for implementation that can be useful to health system stakeholders – both change agents/KT specialists and KT researchers. Our goal is for change agents/KT specialists to be able to use the framework to: pragmatically guide their implementation efforts by identifying the important features of context to consider when choosing, designing and implementing interventions; and to help assess the transferability of successful interventions from other contexts to their own (by identifying contextual features they need to have in place for successful KT).⁸⁴ KT Researchers will be able to use the framework to guide a priori assessments of context (to assist in the design and delivery of their interventions) as well as posteriori assessments of context (to aid in their interpretation of the effects of interventions which can then inform the design and delivery of their subsequent trials).

We acknowledge that the direction and extent of influence of all context features depend on the extent to which the particular feature is lacking or sufficiently available (eg, time) and the degree of importance of that particular feature to a specific KT initiative/strategy. In this study, we did not set out to establish whether a feature has a positive or negative effect and under what conditions; instead our purpose was to develop a comprehensive list of features of context. Future research should focus on trying to elucidate under what conditions the context features from this study positively or negatively empirically influence KT. A realist review approach might be well-suited to investigating under what conditions a feature has a specific effect in what kind of KT strategy.

Limitations and Strengths

Our main limitation is that we asked stakeholders about their perceptions about what contextual factors influence KT to surface their experiential (tacit) evidence on this topic. We do not know at this stage if the identified factors are important moderators of KT strategies or if they have a positive or negative effect on KT and under what conditions. Second, we did not ask stakeholders about specific context features, therefore we cannot state that selected features, if not mentioned by stakeholders in a specific country, are not important in their country, only that they did not spontaneously mention them as important context features. Third, we only interviewed stakeholders from clinical practice settings (acute care, primary care); stakeholders from other health settings (eg, community, public health) may provide differing views. Fourth, while each of the countries reflected in our sample have dominantly a publicly funded system, each also has a private component to their healthcare system. However, our sample largely reflects stakeholders from the public systems (n = 33 of 39). Fifth, despite conducting a relatively large number of interviews, our sample was limited to stakeholders in four high-income countries. Incorporating the views from stakeholders from low- and middle-income countries may provide a potentially more complete or nuanced picture of the attributes and features of context important for KT success internationally. Given these limitations, the attributes and features of context presented in this paper should be considered provisional until further research is done to confirm these findings in other settings.

The main strength of this study is our sample, which included stakeholders from multiple countries and different healthcare systems. This allowed us to identify attributes and features of context relevant to a broader range of stakeholders and settings than in previous studies.

Conclusion

Through this study, we identified 66 unique features of context, grouped into 16 broader attributes of context, perceived as relevant to KT in health settings by a variety of health system stakeholders internationally. There was considerable consistency in the 16 attributes and 66 features identified irrespective of the stakeholder’s country, amount of KT experience, and primary role (change agent/KT specialist or KT researcher), suggesting transferability of the attributes and features of context identified in this study. This is the largest study to date that we are aware of with health system stakeholders that has resulted in the identification of a large number of attributes and their features of context perceived to be relevant to KT. Further work is needed to confirm these findings across a broader range of countries, including low and middle-income countries.

Ethical issues

This study was approved by the Ottawa Health Science Network Research Ethics Board (affiliated with the University of Ottawa, Ottawa, ON, Canada) and the Deakin University Health Ethics Advisory Group (affiliated with Deakin University, Burwood, VIC, Australia). Ethics approval was only sought in Canada and Australia as this is where the interviews were conducted and data stored and analyzed. Ethics approval was not sought from the United States or the United Kingdom as participants from these countries were contacted by phone by research team members in Canada. All participants provided written informed consent.

Competing interests

Authors declare that they have no competing interests.

Authors’ contributions

All authors (JES, AMH, MC, KB, JC, JMG, KD, LA, JB, JJF, NI, JL, SM, MH, TN, JV, and IDG) participated in conception of the study. JES, AMH, JC, JMG, JB, JJF, NI, JL, SM, MH, TN, JV, and IDG participated in securing funding for the project. All authors (JES, AMH, MC, KB, JC, JMG, KD, LA, JB, JJF, NI, JL, SM, MH, TN, JV, and IDG) participated in designed the data analysis plan. KB, KD, and AH conducted the interviews. KB and KD led the coding of the data. JES, IDG, KB, KD, LA, MC, and AMH participated in the analysis of the coded data and all authors (JES, AMH, MC, KB, JC, JMG, KD, LA, JB, JJF, NI, JL, SM, MH, TN, JV, and IDG) provided critical input into the analysed data. JES, AMH, KB, MC, and LA drafted the manuscript. All authors (JES, AMH, MC, KB, JC, JMG, KD, LA, JB, JJF, NI, JL, SM, MH, TN, JV, and IDG) provided input and critical feedback on the manuscript and approved the final manuscript. Consent for publication was obtained from all co-authors (JES, AMH, MC, KB, JC, JMG, KD, LA, JB, JJF, NI, JL, SM, MH, TN, JV, and IDG).

References

1. Lang ES, Wyer PC, Haynes RB. Knowledge translation: closing the evidence-to-practice gap. Ann Emerg Med 2007; 49(3):355-363. doi: 10.1016/j.annemergmed.2006.08.022 [Crossref] [ Google Scholar]
2. Lauer MS, Skarlatos S. Translational research for cardiovascular diseases at the National Heart, Lung, and Blood Institute: moving from bench to bedside and from bedside to community. Circulation 2010; 121(7):929-933. doi: 10.1161/circulationaha.109.917948 [Crossref] [ Google Scholar]
3. McGlynn EA, Asch SM, Adams J. The quality of health care delivered to adults in the United States. N Engl J Med 2003; 348(26):2635-2645. doi: 10.1056/NEJMsa022615 [Crossref] [ Google Scholar]
4. Schuster MA, McGlynn EA, Brook RH. How good is the quality of health care in the United States?. Milbank Q 1998; 76(4):517-563. doi: 10.1111/1468-0009.00105 [Crossref] [ Google Scholar]
5. Grol R. Successes and failures in the implementation of evidence-based guidelines for clinical practice. Med Care 2001; 39(8 Suppl 2):II46-54. doi: 10.1097/00005650-200108002-00003 [Crossref] [ Google Scholar]
6. Villar J, Carroli G, Gülmezoglu AM. The gap between evidence and practice in maternal healthcare. Int J Gynaecol Obstet 2001; 75 Suppl 1:S47-54. [ Google Scholar]
7. Mellis C. Evidence-based medicine: what has happened in the past 50 years?. J Paediatr Child Health 2015; 51(1):65-68. doi: 10.1111/jpc.12800 [Crossref] [ Google Scholar]
8. Straus SE, McAlister FA. Evidence-based medicine: a commentary on common criticisms. CMAJ 2000; 163(7):837-841. [ Google Scholar]
9. Implementation Science. About. 2020. https://implementationscience.biomedcentral.com/about.
10. Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci 2015; 10:53. doi: 10.1186/s13012-015-0242-0 [Crossref] [ Google Scholar]
11. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q 2004; 82(4):581-629. doi: 10.1111/j.0887-378X.2004.00325.x [Crossref] [ Google Scholar]
12. Backer TE. Knowledge utilization: the third wave. Knowledge 1991; 12(3):225-240. doi: 10.1177/107554709101200303 [Crossref] [ Google Scholar]
13. Landry R, Amara N, Lamari M. Climbing the ladder of research utilization: evidence from social science research. Sci Commun 2001; 22(4):396-422. doi: 10.1177/1075547001022004003 [Crossref] [ Google Scholar]
14. Rich RF, Oh CH. . The utilization of policy research 1994.
15. Dogherty EJ, Harrison MB, Graham ID. Facilitation as a role and process in achieving evidence-based practice in nursing: a focused review of concept and meaning. Worldviews Evid Based Nurs 2010; 7(2):76-89. doi: 10.1111/j.1741-6787.2010.00186.x [Crossref] [ Google Scholar]
16. Kimberly J, Cook JM. Organizational measurement and the implementation of innovations in mental health services. Adm Policy Ment Health 2008; 35(1-2):11-20. doi: 10.1007/s10488-007-0143-x [Crossref] [ Google Scholar]
17. Mitton C, Adair CE, McKenzie E, Patten SB, Waye Perry B. Knowledge transfer and exchange: review and synthesis of the literature. Milbank Q 2007; 85(4):729-768. doi: 10.1111/j.1468-0009.2007.00506.x [Crossref] [ Google Scholar]
18. Squires JE, Estabrooks CA, O’Rourke HM, Gustavsson P, Newburn-Cook CV, Wallin L. A systematic review of the psychometric properties of self-report research utilization measures used in healthcare. Implement Sci 2011; 6:83. doi: 10.1186/1748-5908-6-83 [Crossref] [ Google Scholar]
19. Squires JE, Estabrooks CA, Gustavsson P, Wallin L. Individual determinants of research utilization by nurses: a systematic review update. Implement Sci 2011; 6:1. doi: 10.1186/1748-5908-6-1 [Crossref] [ Google Scholar]
20. Squires JE, Hutchinson AM, Boström AM, O’Rourke HM, Cobban SJ, Estabrooks CA. To what extent do nurses use research in clinical practice? a systematic review. Implement Sci 2011; 6:21. doi: 10.1186/1748-5908-6-21 [Crossref] [ Google Scholar]
21. Glaser EM, Abelson HH, Garrison K. Putting Knowledge to Use: Facilitating the Diffusion of Knowledge and the Implementation of Planned Change. San Francisco: Jossey-Bass; 1983.
22. Lewis CC, Fischer S, Weiner BJ, Stanick C, Kim M, Martinez RG. Outcomes for implementation science: an enhanced systematic review of instruments using evidence-based rating criteria. Implement Sci 2015; 10:155. doi: 10.1186/s13012-015-0342-x [Crossref] [ Google Scholar]
23. Chaudoir SR, Dugan AG, Barr CH. Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level measures. Implement Sci 2013; 8:22. doi: 10.1186/1748-5908-8-22 [Crossref] [ Google Scholar]
24. LaRocca R, Yost J, Dobbins M, Ciliska D, Butt M. The effectiveness of knowledge translation strategies used in public health: a systematic review. BMC Public Health 2012; 12:751. doi: 10.1186/1471-2458-12-751 [Crossref] [ Google Scholar]
25. Reichenpfader U, Carlfjord S, Nilsen P. Leadership in evidence-based practice: a systematic review. Leadersh Health Serv (Bradf Engl) 2015; 28(4):298-316. doi: 10.1108/lhs-08-2014-0061 [Crossref] [ Google Scholar]
26. Wozney L, McGrath PJ, Gehring ND. eMental healthcare technologies for anxiety and depression in childhood and adolescence: systematic review of studies reporting implementation outcomes. JMIR Ment Health 2018; 5(2):e48. doi: 10.2196/mental.9655 [Crossref] [ Google Scholar]
27. Li SA, Jeffs L, Barwick M, Stevens B. Organizational contextual features that influence the implementation of evidence-based practices across healthcare settings: a systematic integrative review. Syst Rev 2018; 7(1):72. doi: 10.1186/s13643-018-0734-5 [Crossref] [ Google Scholar]
28. De Vries H, Bekkers V, Tummers L. Innovation in the public sector: a systematic review and future research agenda. Public Adm 2016; 94(1):146-166. doi: 10.1111/padm.12209 [Crossref] [ Google Scholar]
29. Kaplan HC, Brady PW, Dritz MC. The influence of context on quality improvement success in health care: a systematic review of the literature. Milbank Q 2010; 88(4):500-559. doi: 10.1111/j.1468-0009.2010.00611.x [Crossref] [ Google Scholar]
30. Lau R, Stevenson F, Ong BN. Achieving change in primary care--causes of the evidence to practice gap: systematic reviews of reviews. Implement Sci 2016; 11:40. doi: 10.1186/s13012-016-0396-4 [Crossref] [ Google Scholar]
31. Ovretveit J. Change Achievement Success Indicators (CASI). Stockholm, Sweden: Karolinska Institute Medical Management; 2004.
32. Rycroft-Malone J. The PARIHS framework--a framework for guiding the implementation of evidence-based practice. J Nurs Care Qual 2004; 19(4):297-304. doi: 10.1097/00001786-200410000-00002 [Crossref] [ Google Scholar]
33. May C, Finch T, Mair F. Understanding the implementation of complex interventions in health care: the normalization process model. BMC Health Serv Res 2007; 7:148. doi: 10.1186/1472-6963-7-148 [Crossref] [ Google Scholar]
34. Berta W, Ginsburg L, Gilbart E, Lemieux-Charles L, Davis D. What, why, and how care protocols are implemented in Ontario nursing homes. Can J Aging 2013; 32(1):73-85. doi: 10.1017/s0714980813000081 [Crossref] [ Google Scholar]
35. Fleuren MA, Paulussen TG, Van Dommelen P, Van Buuren S. Towards a measurement instrument for determinants of innovations. Int J Qual Health Care 2014; 26(5):501-510. doi: 10.1093/intqhc/mzu060 [Crossref] [ Google Scholar]
36. McDonald KM, Schultz EM, Chang C. Evaluating the state of quality-improvement science through evidence synthesis: insights from the closing the quality gap series. Perm J 2013; 17(4):52-61. doi: 10.7812/tpp/13-010 [Crossref] [ Google Scholar]
37. Dixon-Woods M. The Problem of Context in Quality Improvement. London: The Health Foundation; 2014.
38. Estabrooks CA. Translating research into practice: implications for organizations and administrators. Can J Nurs Res 2003; 35(3):53-68. [ Google Scholar]
39. Hutchinson AM, Mallidou AA, Toth F, Cummings GG, Schalm C, Estabrooks CA. Review and Synthesis of Literature Examining Characteristics of Organizational Context that Influence Knowledge Translation in Healthcare: Technical Report. Edmonton, Alberta: University of Alberta; 2010.
40. Meijers JM, Janssen MA, Cummings GG, Wallin L, Estabrooks CA, R YGH. Assessing the relationships between contextual factors and research utilization in nursing: systematic literature review. J Adv Nurs 2006; 55(5):622-635. doi: 10.1111/j.1365-2648.2006.03954.x [Crossref] [ Google Scholar]
41. French B. Contextual factors influencing research use in nursing. Worldviews Evid Based Nurs 2005; 2(4):172-183. doi: 10.1111/j.1741-6787.2005.00034.x [Crossref] [ Google Scholar]
42. Cane J, O’Connor D, Michie S. Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci 2012; 7:37. doi: 10.1186/1748-5908-7-37 [Crossref] [ Google Scholar]
43. Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A. Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care 2005; 14(1):26-33. doi: 10.1136/qshc.2004.011155 [Crossref] [ Google Scholar]
44. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009; 4:50. doi: 10.1186/1748-5908-4-50 [Crossref] [ Google Scholar]
45. Champagne F. Discussion Paper No.39: The Ability to Manage Change in Health Care Organizations. Commission on the Future of Health Care in Canada; 2002.
46. Graham ID, Logan J, Harrison MB. Lost in knowledge translation: time for a map?. J Contin Educ Health Prof 2006; 26(1):13-24. doi: 10.1002/chp.47 [Crossref] [ Google Scholar]
47. McNulty T, Ferlie E. Reengineering Health Care: The Complexities of Organizational Transformation. New York: Oxford University Press; 2002.
48. Kitson A, Harvey G, McCormack B. Enabling the implementation of evidence based practice: a conceptual framework. Qual Health Care 1998; 7(3):149-158. doi: 10.1136/qshc.7.3.149 [Crossref] [ Google Scholar]
49. Kitson AL, Rycroft-Malone J, Harvey G, McCormack B, Seers K, Titchen A. Evaluating the successful implementation of evidence into practice using the PARiHS framework: theoretical and practical challenges. Implement Sci 2008; 3:1. doi: 10.1186/1748-5908-3-1 [Crossref] [ Google Scholar]
50. Flottorp SA, Oxman AD, Krause J. A checklist for identifying determinants of practice: a systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implement Sci 2013; 8:35. doi: 10.1186/1748-5908-8-35 [Crossref] [ Google Scholar]
51. Brennan SE, Bosch M, Buchan H, Green SE. Measuring organizational and individual factors thought to influence the success of quality improvement in primary care: a systematic review of instruments. Implement Sci 2012; 7(1):121. doi: 10.1186/1748-5908-7-121 [Crossref] [ Google Scholar]
52. Estabrooks CA, Squires JE, Cummings GG, Birdsell JM, Norton PG. Development and assessment of the Alberta Context Tool. BMC Health Serv Res 2009; 9:234. doi: 10.1186/1472-6963-9-234 [Crossref] [ Google Scholar]
53. Helfrich CD, Li YF, Sharp ND, Sales AE. Organizational readiness to change assessment (ORCA): development of an instrument based on the Promoting Action on Research in Health Services (PARIHS) framework. Implement Sci 2009; 4:38. doi: 10.1186/1748-5908-4-38 [Crossref] [ Google Scholar]
54. McCormack B, McCarthy G, Wright J, Slater P, Coffey A. Development and testing of the Context Assessment Index (CAI). Worldviews Evid Based Nurs 2009; 6(1):27-35. doi: 10.1111/j.1741-6787.2008.00130.x [Crossref] [ Google Scholar]
55. Hartveit M, Hovlid E, Nordin MHA. Measuring implementation: development of the implementation process assessment tool (IPAT). BMC Health Serv Res 2019; 19(1):721. doi: 10.1186/s12913-019-4496-0 [Crossref] [ Google Scholar]
56. Rogers EM. Diffusion of Innovations. 5 ed ed. New York: Free Press; 2003.
57. Rycroft-Malone J, Kitson A, Harvey G. Ingredients for change: revisiting a conceptual framework. Qual Saf Health Care 2002; 11(2):174-180. doi: 10.1136/qhc.11.2.174 [Crossref] [ Google Scholar]
58. Harvey G, Kitson A. PARIHS revisited: from heuristic to integrated framework for the successful implementation of knowledge into practice. Implement Sci 2016; 11:33. doi: 10.1186/s13012-016-0398-2 [Crossref] [ Google Scholar]
59. Squires JE, Graham I, Bashir K. Understanding context: a concept analysis. J Adv Nurs 2019; 75(12):3448-3470. doi: 10.1111/jan.14165 [Crossref] [ Google Scholar]
60. Wensing M. The Tailored Implementation in Chronic Diseases (TICD) project: introduction and main findings. Implement Sci 2017; 12(1):5. doi: 10.1186/s13012-016-0536-x [Crossref] [ Google Scholar]
61. Squires JE, Aloisio LD, Grimshaw JM. Attributes of context relevant to healthcare professionals’ use of research evidence in clinical practice: a multi-study analysis. Implement Sci 2019; 14(1):52. doi: 10.1186/s13012-019-0900-8 [Crossref] [ Google Scholar]
62. Goodman LA. Snowball sampling. Ann Math Statist 1961; 32(1):148-170. doi: 10.1214/aoms/1177705148 [Crossref] [ Google Scholar]
63. Francis JJ, Johnston M, Robertson C. What is an adequate sample size? operationalising data saturation for theory-based interview studies. Psychol Health 2010; 25(10):1229-1245. doi: 10.1080/08870440903194015 [Crossref] [ Google Scholar]
64. Sibley KM, Roche PL, Bell CP, Temple B, Wittmeier KDM. A descriptive qualitative examination of knowledge translation practice among health researchers in Manitoba, Canada. BMC Health Serv Res 2017; 17(1):627. doi: 10.1186/s12913-017-2573-9 [Crossref] [ Google Scholar]
65. Kuzel AJ. . Sampling in qualitative inquiry 1999.
66. Squires JE, Graham N, Coughlin M. Barriers and enablers to organ donation after circulatory determination of death: a qualitative study exploring the beliefs of frontline intensive care unit professionals and organ donor coordinators. Transplant Direct 2018; 4(7):e368. doi: 10.1097/txd.0000000000000805 [Crossref] [ Google Scholar]
67. QSR International Pty Ltd. NVivo Qualitative Data Analysis Software (Version 10). 2012.
68. Sandelowski M. Whatever happened to qualitative description?. Res Nurs Health 2000; 23(4):334-340. [ Google Scholar]
69. Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res 2005; 15(9):1277-1288. doi: 10.1177/1049732305276687 [Crossref] [ Google Scholar]
70. Benner P, Tanner C, Chesla C. Expertise in Nursing Practice: Caring, Clinical Judgment, and Ethics. New York: Springer Publishing Company; 2009.
71. Gibbs GR. Analyzing Qualitative Data. London: Sage; 2008.
72. Lincoln YS, Guba EG. But is it rigorous? trustworthiness and authenticity in naturalistic evaluation. New Directions for Program Evaluation 1986; 1986(30):73-84. doi: 10.1002/ev.1427 [Crossref] [ Google Scholar]
73. Patton MQ. Enhancing the quality and credibility of qualitative analysis. Health Serv Res 1999; 34(5 Pt 2):1189-1208. [ Google Scholar]
74. Beidas RS, Stewart RE, Adams DR. A multi-level examination of stakeholder perspectives of implementation of evidence-based practices in a large urban publicly-funded mental health system. Adm Policy Ment Health 2016; 43(6):893-908. doi: 10.1007/s10488-015-0705-2 [Crossref] [ Google Scholar]
75. Harvey G, Jas P, Walshe K. Analysing organisational context: case studies on the contribution of absorptive capacity theory to understanding inter-organisational variation in performance improvement. BMJ Qual Saf 2015; 24(1):48-55. doi: 10.1136/bmjqs-2014-002928 [Crossref] [ Google Scholar]
76. Hinchcliff R, Greenfield D, Westbrook JI, Pawsey M, Mumford V, Braithwaite J. Stakeholder perspectives on implementing accreditation programs: a qualitative study of enabling factors. BMC Health Serv Res 2013; 13:437. doi: 10.1186/1472-6963-13-437 [Crossref] [ Google Scholar]
77. Holtkamp KC, Vos EM, Rigter T, Lakeman P, Henneman L, Cornel MC. Stakeholder perspectives on the implementation of genetic carrier screening in a changing landscape. BMC Health Serv Res 2017; 17(1):146. doi: 10.1186/s12913-017-2083-9 [Crossref] [ Google Scholar]
78. Muellmann S, Steenbock B, De Cocker K. Views of policy makers and health promotion professionals on factors facilitating implementation and maintenance of interventions and policies promoting physical activity and healthy eating: results of the DEDIPAC project. BMC Public Health 2017; 17(1):932. doi: 10.1186/s12889-017-4929-9 [Crossref] [ Google Scholar]
79. Renz AD, Conrad DA, Watts CA. Stakeholder perspectives on the implementation of shared decision making: a qualitative data analysis. Int J Healthc Manag 2013; 6(2):122-131. doi: 10.1179/2047971912Y.0000000027 [Crossref] [ Google Scholar]
80. Schneider JL, Davis J, Kauffman TL. Stakeholder perspectives on implementing a universal Lynch syndrome screening program: a qualitative study of early barriers and facilitators. Genet Med 2016; 18(2):152-161. doi: 10.1038/gim.2015.43 [Crossref] [ Google Scholar]
81. Palinkas LA, Schoenwald SK, Hoagwood K, Landsverk J, Chorpita BF, Weisz JR. An ethnographic study of implementation of evidence-based treatments in child mental health: first steps. Psychiatr Serv 2008; 59(7):738-746. doi: 10.1176/ps.2008.59.7.738 [Crossref] [ Google Scholar]
82. Hamilton AB, Brunner J, Cain C. Engaging multilevel stakeholders in an implementation trial of evidence-based quality improvement in VA women’s health primary care. Transl Behav Med 2017; 7(3):478-485. doi: 10.1007/s13142-017-0501-5 [Crossref] [ Google Scholar]
83. Ritchie MJ, Parker LE, Edlund CN, Kirchner JE. Using implementation facilitation to foster clinical practice quality and adherence to evidence in challenged settings: a qualitative study. BMC Health Serv Res 2017; 17(1):294. doi: 10.1186/s12913-017-2217-0 [Crossref] [ Google Scholar]
84. Squires JE, Graham ID, Hutchinson AM. Identifying the domains of context important to implementation science: a study protocol. Implement Sci 2015; 10:135. doi: 10.1186/s13012-015-0325-y [Crossref] [ Google Scholar]