Scaling-Up eConsult: Promising Strategies to Address Enabling Factors in Four Jurisdictions in Canada

Background

The challenge of effectively scaling up promising innovations beyond pilot projects often hinders efforts to improve healthcare.^1-4 A healthcare innovation is a new product, intervention, care pathway, or service that significantly benefits patients and health systems.^5-8 Scaling up innovations involves iterative decisions, events, and actions to tackle infrastructure issues that arise across health system levels while expanding innovations to full scale to benefit targeted populations.^1,4,9

Canada faces persistent difficulties in scaling-up healthcare innovations,^1,2,10-14 leading to its reputation as the “country of perpetual pilot projects.”¹¹ Scaling up healthcare innovations in Canada presents a complex and multifaceted challenge that remains poorly understood. Health system leaders and experts have identified several potential barriers, including resources allocated primarily towards pilot projects rather than expansion, inadequate support for clinician-innovators, limited engagement of diverse actors including patients, a risk-averse culture, opaque decision-making processes, insufficient data infrastructure and access, fragmented health systems, ineffective change management, and a general lack of knowledge regarding the most effective strategies for scaling up innovations.^11,13,14

A growing body of evidence on the science of scaling up healthcare innovations has identified several enabling factors.^{1,4,9,10,15-17} These factors include (1) identifying population needs and system problems; (2) dedicating human, financial, and technical resources; (3) creating relational and political connections and actively involving key actors; (4) strong health professional leadership; (5) adapting to the political context and integrating with existing services; and (6) measuring and disseminating innovation and scale-up outcomes.

However, two significant gaps persist in the current literature on scaling up healthcare innovations. Firstly, most studies focus on low- and middle-income countries, which provide valuable information but may differ in terms of resources, policies, governance, and infrastructure compared to high-income contexts.^1,4,17-19 Secondly, there is a notable dearth of evidence on the operationalization of scale-up and strategies that key actors can adopt to address enabling factors.²⁰ Thus, there is a need to study empirical scale-up processes and strategies within the context of high-income countries. Such evidence may provide much-needed guidance on expanding proven healthcare innovations to improve equitable health outcomes in universal health systems.

eConsult is one of the most promising health innovations piloted in Canada to improve access to specialist care.^21-24 eConsult is an asynchronous electronic communication platform used to transmit medical advice between a primary care professional and specialists in fields such as dermatology, cardiology, pediatrics, oncology, psychiatry, and geriatrics.^21-24 Studies have demonstrated that eConsult reduces wait times, avoids face-to-face referrals, is associated with positive patient and professional experiences, reduces health system costs, and improves equity.^21-23

The Building Access to Specialists through eConsultation (BASE^TM) eConsult model was developed and piloted in one region of the Canadian province of Ontario in 2009,²⁴ then spread throughout Ontario and further expanded to other Canadian provinces. This evidence-based innovation began as a small proof-of-concept pilot project that was spread and is being scaled up in other Canadian provinces. eConsult is trademarked by the founders of BASE^TM eConsult, who charge minimal fees for service maintenance but do not charge for its use by other provinces. eConsult is free-of-charge for physicians and professionals. Specialists received a pro-rated fee for each eConsultation as part of a research project. Figure 1 illustrates the eConsult innovation implemented in the four provinces under study. The fact that the four provinces are at different stages of the scale-up process offers a unique learning opportunity.²⁵

Our study aims to identify the key factors that enabled the scale-up of eConsult and the strategies used to address them in four provinces in Canada. As part of this study, we define the factors enabling scale-up as the key areas identified to facilitate scale-up, whereas the strategies for scale-up are the actions taken by actors to address these factors.

Close

Figure 1. The eConsult Innovation Implemented in Four Canadian Provinces. Abbreviations: EMR, electronic medical record; PCP, primary care professionals.

Methods

Design

The research used a narrative case study design.²⁶ This design involved an in-depth examination of eConsult implementation by triangulating data from multiple sources and contexts and reconstructing each studied case in narrative form.²⁷ The study included four Canadian provinces where eConsult scale-up was underway²⁸: Quebec, Ontario, Manitoba, and Newfoundland & Labrador. In Canada, the health care system is publicly funded, but provinces are responsible for planning, administering, and delivering health care services, resulting in provincial differences in laws, regulations, and structures. The provinces under study had participated in a national Partnership for Health System Improvement grant and in the Canadian Foundation for Health Improvement’s (CFHI’s) Connected Medicine Initiative (2017–2018).^28,29 Each province was at a different stage of the scale-up process, providing a richer understanding of the various stages of implementation.^26,28

Data Collection

Between August 2018 and April 2021, we collected data for each jurisdiction as well as data from national organizations (across provinces). Our data sources included non-participant observations, reviews of internal documents, and interviews (see Table 1). We continued data collection until we reached empirical data saturation within each jurisdiction.

Non-participant Observations

We observed provincial eConsult steering committee meetings (n = 65) selected based on agenda items. Participants of the steering committee meetings included primary care professionals, specialists, policy-makers, researchers, information technology (IT) experts, and patients. These meetings focused on the spread and provincial scale-up of eConsult and helped us to better understand the contexts and gather monthly data on what was happening in each province. We collected data using an observation form we developed based on the key enabling factors for scale-up identified in the literature^1,4,15,30 as well as elements deemed essential by knowledge users for the implementation of eConsult at a provincial scale. We completed the observation form, which has been previously published elsewhere,²⁵ live during the meetings. Two PhD-trained research professionals (MAS and CLL) conducted initial observations to test the form and achieve consensus on its use. CLL conducted the remaining observations. In addition, we used the same form to observe three National eConsult Forums (2018 – in person, 2019 – in person, 2020 – virtual) where 75 to 150 provincial and national actors (eg, policy-makers, patient partners, family physicians, specialists, researchers, partner organizations) discussed scale-up strategies.

Internal document review

We collected internal documents (n = 93), such as meeting minutes, PowerPoint presentation slide decks, working documents, policy documents, and scale-up plans, either directly from key actors or by observing committees.

Semi-structured interviews

We conducted semi-structured interviews (n = 40) with key actors in each province (eg, researchers, primary care professionals, specialists, policy-makers, patient partners). To capture a variety of perspectives, we recruited participants with varied roles in the scale-up process and experience with and knowledge of eConsult. We recruited all key actors participating in provincial eConsult committees and those involved in eConsult implementation at the regional, provincial, and national levels. The researchers and knowledge users on our team helped us identify key actors to approach for participation. We adapted the interview guide from a previously published guide²⁵ and tailored it according to each participant’s role and province. Two researchers with expertise in qualitative interviewing (MAS and CLL) who had not played a role in the scale-up of eConsult conducted the interviews to ensure confidentiality and minimize the influence of the interviewers on the answers provided. We digitally recorded and transcribed all interviews verbatim.

Data Analysis

We based our analysis on the main factors identified in the literature that enable the scale-up of health innovations. Our initial thematic analysis³¹ drew inspiration from Roger’s foundational work on the diffusion of innovations,³⁰ the 20 scale-up success factors identified by Milat et al,⁴ factors identified in a literature review published by Ben Charif and al,¹ and factors identified in a study on factors influencing the scale-up of eConsult published by Moroz and al.¹⁵ We imported transcripts of the 40 interviews (lasting 60 minutes on average) into NVivo 12. Observation notes informed our analysis. First, CLL and MAS coded four transcripts, discussing divergences in their codes after each one. Once they reached coding agreement, CLL coded the remaining transcripts. Second, based on the codebook, CLL wrote case narratives for each province detailing context, strategies (eg, events, decisions, and actions), and enabling factors and their influence on the scale-up process. Two other team members (MB and MAS) reviewed these narratives for clarity and completeness. Three researchers on the team (MB, MAS, and CLL) then analyzed strategies across cases, grouping similar strategies and categorizing them within the key enabling factors identified in the literature review. Finally, in May 2021, we held two 60-minute virtual meetings to share, discuss, and validate the key enabling factors and related strategies with team members from the four provinces who were either involved in scaling up eConsult or had expertise in scaling up innovations. We organized two meetings based on the same content to reach as many participants as possible. At these meetings, we used interactive methods (eg, chat box, virtual sticky notes, and live discussions) to facilitate a structured discussion around the trustworthiness and confirmability of the preliminary findings.

Results

We studied the process of scaling up eConsult in four provinces. Table 2 lists the characteristics of each of the jurisdictions.

Based on case narratives and team discussions, we identified six common enabling factors across the four provinces, which we refer to as key factors enabling eConsult scale-up: (1) multi-actor engagement; (2) relative advantage; (3) knowledge transfer; (4) strong evidence base; (5) physician leadership; and (6) resource acquisition. Below, we present each factor in turn, explaining why it is key to the scale-up of eConsult and describing the strategies used to address it. Figure 2 summarizes the factors and strategies.

Close

Figure 2. Summary of Key Factors Enabling eConsult Scale-Up and Strategies Used to Address Them.

Multi-actor Engagement

Why Is This Enabling Factor Key to the Scale-Up of eConsult?

Ongoing engagement of multiple key actors — including patient partners, policy-makers, primary care professionals, specialists, IT experts, and researchers — helped to strategically align the implementation of eConsult with provincial health system priorities and adapt to changing policy contexts. Additionally, engaging various actors from a broad range of organizations and with varied perspectives allowed for early identification of potential policy and infrastructure issues that needed to be addressed and helped to navigate complex policy processes.

What Strategies Were Used?

In each province, the engagement of key actors began with the physician leaders who first piloted the innovation. They recruited a team of knowledge users and patient partners through their personal and professional networks to prepare and apply for one or more research grants. Research funds facilitated the assembly of these teams, and collaborating on a concrete project in this way helped to build their interest and commitment. Motivated and action-oriented strategic actors received invitations to participate in research grants. As eConsult was spread and scaled up, steering committees enlisted additional members to provide insights into emerging issues and recruited specific high-level policy-makers to participate on the committee and act as representatives of their organizations (eg, ministries of health, medical associations). Provincial eConsult physician leaders chaired these meetings. Throughout the scale-up process, provincial eConsult steering committees that met once or twice per month maintained actor engagement for the most part. At these meetings, physician leaders and researchers provided updates on evaluation results (eg, eConsult volumes, wait times), members whose organizations were directly involved in the scale-up process updated the committee on their progress, and committee members discussed emerging challenges, brainstormed solutions, and advised high-level actors on strategic orientations. Beyond these meetings, the research team ensured ongoing engagement of key actors by providing communication tools they could use to advocate for eConsult (eg, presentation slide decks, policy briefs, and letter drafts). Moreover, they celebrated eConsult milestones and successes with all those involved to help build a sense of ownership among them. Furthermore, the Ontario team invited actors from around the country to participate in National eConsult Forums held in collaboration with the Royal College of Physicians and Surgeons of Canada. Participants perceived these forums as fostering further engagement, particularly in provinces that were at the earlier stages of scaling up eConsult. At these forums, 75–150 individuals discussed the potential value of scaling up eConsult. Inviting national-level policy-makers (eg, College of Family Physicians of Canada, Royal College of Physicians and Surgeons) to these forums provided them with an opportunity to discuss national issues, such as interjurisdictional licensing and inclusion of eConsult in medical curricula, to brainstorm solutions, and to stay up-to-date on the progress of scale-up in each province.

Various actors played different complementary roles in the scale-up of eConsult. High-level actors (eg, policy-makers, representatives of medical associations, and physician leaders) participated as spokespeople who endorsed eConsult and advocated for its scale-up among influential actors. Patient partners encouraged the use of several strategies, such as evaluating eConsult from the patient perspective, acting as strong advocates for eConsult, drafting jargon-free promotional materials, and including rich patient stories to promote eConsult.

“I just think that the patient reps and the stories that they bring are the most important part of the development project because we need to be listening to them. We need to be learning from them” (Newfoundland & Labrador).

Multiple types of actors spread a similar message about the need to scale up eConsult through their respective communication channels, generating momentum and excitement across the country.

Table 3 presents a summary of the strategies used in each province to address multi-actor engagement.

Relative Advantage

Why Is This Enabling Factor Key to the Scale-Up of eConsult?

Our results suggest that, in all provinces, a variety of key actors perceived eConsult to be highly useful and to have a strong relative advantage over the status quo or other innovations. This positive perception facilitated its widespread adoption by health service professionals and, hence, its scale-up.

What Strategies Were Used?

eConsult was designed to be as simple as possible for professionals to use. Several respondents highlighted the importance of developing a simple intervention that facilitated its implementation and use. Participants perceived eConsult as being easy for potential adopters and policy-makers to understand and for professionals to use in everyday practice. Furthermore, it was seen as being easy for organizations to manage changes required for its implementation. In particular, participants noted that being a simple, streamlined innovation that integrates well into professionals’ workflows set eConsult apart from other innovations.

“What makes it different is how it is embedded in the workflow. So, it’s convenient…” (Manitoba).

For example, respondents felt that the use of a standardized form with only four mandatory questions facilitated its integration into the workflows of primary care professionals. Although many specialists requested that the form be tailored to their specific needs, the Ontario team decided, based on their experience and user feedback, to use one standardized form with only four fields applicable to any medical specialty. They felt that this would be less burdensome for primary care professionals and, therefore, increase the likelihood of them using it. Our results show that this simplicity helped to avoid delays in scale-up because the design focused on core elements and limited “scope creep,” particularly in terms of IT developments and integration with electronic medical records. This idea of simplicity achieved consensus among the interviewees.

“…developing and scaling integrated things is much harder than developing and scaling dead simple things. So, the trade off, I think it’s easier to scale something dead simple…” (Ontario).

“…The conclusion that I have made is that, even if it’s not advanced computing, [...] the technical solution has to be simple and user friendly and just easy to use and adopt” (Manitoba).

One of the main strategies for development, spread, and scale-up was to begin by addressing users’ needs and to subsequently adapt to their preferences while maintaining a functional eConsult service.

To ensure a relative advantage in terms of usability and ease of implementation, and to reduce adoption burden on professionals, the design of eConsult mirrored traditional referral processes as much as possible. For example, mandatory close-out surveys (all provinces) collected user feedback, and a user advisory committee (Manitoba) adapted the service to user feedback (eg, problems, frustrations, and suggested improvements) to make it user-friendly and ensure integration into the existing workflows of specialists and primary care professionals. Research teams regularly analyzed responses and used them to continually improve the innovation.

Refinement of the eConsult model largely took place during the stages of the pilot project in Ontario. The other provinces followed this model on the BASE^TM platform (Manitoba, Newfoundland and Labrador) or on a local platform managed by a private company (Quebec). In the three other provinces, changes made to eConsult related mainly to technical issues or requests to add new specialties to the service.

In all provinces studied, regional coordinators provided training and ongoing support to help professionals with the onboarding process to the eConsult platform and to troubleshoot issues. In some cases, clinical administrative staff were trained or a train-the-trainer approach was used to ensure access to local support. Training videos and guides, remote virtual training sessions, and in-person training further facilitated adoption and implementation.

Integrating eConsult with other existing services was another strategy used to ensure its relative advantage. Although the eConsult platform started as a standalone service, both Newfoundland & Labrador and Ontario integrated eConsult with existing services within electronic health records or with other telemedicine services. Newfoundland & Labrador’s use of a single electronic health record facilitated eConsult’s integration across the province. Ontario integrated eConsult into targeted electronic health records as a pilot project. In Quebec, in 2021, work was underway to integrate eConsult within regional dispatch centers for referrals to specialists.

To ensure potential adopters were aware of eConsult’s relative advantages (eg, simplicity, effectiveness, ease of use for professionals, and usefulness for patient care), information about eConsult was promoted through a variety of channels, including institutional newsletters, social media, news media (eg, newspaper advertisements and press coverage), eConsult websites, information booths at relevant events, word of mouth (eg, physician champions and patient partners), webinars, presentations, policy briefs, infographics, and promotional material. In Ontario, traditional referral requests that could be answered by eConsult were often returned to the primary care professionals with a recommendation to send the request as an eConsult, thereby helping to promote the service. Promotional messages describing eConsult and promoting its relative advantages were evidence-based, focused, and consistent. For example, Canada Health Infoway created short videos featuring professionals’ testimonials about their positive experiences with eConsult for publication on their website. Patients’ stories of the perceived benefits of eConsult also promoted eConsult to policy-makers. Finally, the favouring of images over words (eg, infographics and images) increased the accessibility and retention of the promotional messages.

Table 4 presents a summary of the strategies used in each province to address relative advantage.

Knowledge Transfer

Why Is This Enabling Factor Key to the Scale-Up of eConsult?

Transferring learnings from Ontario to the other provinces helped to accelerate some aspects of the scale-up process by enabling them to benefit from other’s experiences. Additionally, it generated excitement about eConsult among key actors and increased national momentum for its spread and scale-up. Finally, this key enabling factor created positive peer pressure among provincial actors.

What Strategies Were Used?

Ontario was the first province to begin scaling up eConsult, and they freely shared their evidence and tools (eg, business case, policy brief, implementation guide, training tools; see https://www.champlainbaseeconsult.com/) with the other three provinces. For instance, Ontario leaders shared the eConsult platform they developed with other jurisdictions for a small system maintenance fee. Ontario leaders also shared lessons learned, experiences acquired from spreading and scaling up eConsult, tools, and strategies used to address the challenges they encountered. In doing so, the provinces were able to benefit from the work others had done and adapt it to their jurisdiction, saving them the time and effort of starting from scratch. As a result, they drafted plans to avoid some of the challenges others had faced and devised creative solutions to overcome known barriers to scaling up eConsult. The National eConsult Forums also played a fundamental role in supporting knowledge transfer. At each edition of the forum, participants not only shared learnings from their jurisdiction but also collectively brainstormed potential solutions to challenges.

“I think one of the first turning points for me was the national gatherings – the forum. There were a lot of ‘ ahas ’ in the room. People were curious. [...] The forum is a great opportunity to showcase success, discuss challenges but also to sell the concept. […] For those who are less sure about it, to be in a room, to talk about it for a day and a bit, it’s difficult to walk away not thinking that this is a good idea” (National).

eConsult was part of the CFHI’s Connected Medicine Collaborative, which engaged key actors from different provinces in a formal program aiming to help spread remote consultation services (eg, eConsult and phone consultations). The work of the CFHI collaborative included hosting webinars, developing business cases, establishing governance and scale-up plans, providing coaching from innovation leads, and organizing regular meetings to foster the sharing of lessons learned across provinces.

“I think it was really good to be involved from the get-go with the other folks that are doing similar things across the country. Having that opportunity to collaborate and to meet in person, in particular to learn what others are doing, was very valuable. It helped us to figure out what kind of metrics we should be looking at from the get-go” (Newfoundland & Labrador).

Ontario’s eConsult leads also acted as mentors to those in other provinces. They regularly participated in eConsult steering committee meetings and in discussions with high-level policy-makers by presenting evidence and offering suggestions. This enabled the leads to remain informed of progress in each jurisdiction and facilitated the exchange of strategies used among jurisdictions. For instance, they shared insight into various strategies to build a convincing rationale for scaling up eConsult and for securing funds from alternative sources to help transition eConsult from a research project to a sustainable innovation.

Table 5 presents a summary of the strategies used to address knowledge transfer.

Strong Evidence Base

Why Is This Enabling Factor Key to the Scale-Up of eConsult?

Research funds initially supported the development and piloting of eConsult, and research played an important role throughout all stages of eConsult’s development. The fact that eConsult was embedded within a research infrastructure fostered the creation of a strong evidence base. Several domains contributed data to the evidence base, including the effectiveness (number of non-urgent in-person visits avoided, patient satisfaction, increased access to specialized care), cost-effectiveness (cost per eConsult, cost per eConsult avoided), feasibility (growth projections for volume of eConsults), acceptability (primary care professionals and specialist satisfaction), adoption (volume per professional, number of active professionals), and potential reach (number of eConsults completed per clinic and region) of the eConsult pilot projects. This large evidence base, built over 12 years, lent credibility to the innovation and demonstrated the benefits and value of eConsult to policy-level actors and funding organizations.

“I feel that the publications that Clare and Erin (co-founders) have done over the years add credibility and provide a lot of expertise into informing the model. […] I think it’s instrumental to informing the policy decisions and I think it was a key success factor in making the case for eConsult to be a provincial program” (Ontario).

The well-established evidence base also helped address actors’ concerns regarding patient safety and acceptance of the service. For instance, actors initially questioned the safety of an eConsult compared to an in-person appointment. However, the evidence indicating that only a small proportion of eConsults led to in-person appointments with a specialist, despite this not having been initially planned by the primary care professionals, reassured them. Additionally, in each province, the current evidence base generated by the research team informed policy discussions and negotiations. For example, economic evidence from Ontario on the cost per eConsult and volume data informed remuneration negotiations in other provinces. Needs assessments also helped eConsult teams establish spread and scale-up priorities, such as implementing eConsult for specialties with long wait times or in areas where access to specialists was limited.

What Strategies Were Used?

Our results suggest that embedding eConsult within a research infrastructure helped to create a continuously-growing evidence base. Beginning with the pilot studies in the provinces, a research infrastructure was created and maintained throughout the spread and scale-up of eConsult. Research grants obtained from local, provincial, and national funding bodies financed the human resources needed to implement the eConsult service and maintain it until funding was institutionalized, covered fees for specialists’ time, and allowed for evaluations and needs assessments to be conducted to explore further implementation (eg, in correctional and long-term care facilities) and uses (eg, continuing medical education). Furthermore, negotiations with provincial health authorities regarding long-term funds for sustainability resulted in a portion of the budget being dedicated to the continuing evaluation of the service.

The collection and analysis of relevant data (eg, audits, mandatory close-out survey, and patient and professional surveys to assess effectiveness, cost-effectiveness, feasibility, acceptability, adoption, and potential reach) using quantitative and qualitative research methods built the evidence base. A variety of knowledge translation strategies shared the results of the research projects and the evidence base to inform scale-up. Over 100 scientific publications were produced, primarily by the Ontario team, providing a credible, peer-reviewed body of literature to build a convincing rationale to scale-up eConsult. Other provinces perceived the data collected in Ontario as being transferable to their own contexts.

“I think [the evidence base] really informed and shaped how they (the co-founders) do deployment, how they frame the benefits of the system... It’s much easier, I think, for people to go to a doctor’s office and say, ‘You should use this system. It’s really valuable,’ when they have evidence to back that up” (Ontario).

Table 6 presents a summary of the strategies used in each province to address a strong evidence base.

Physician Leadership

Why Is This Enabling Factor Key to the Scale-Up of eConsult?

Two physician leaders in Ontario, one family physician and one specialist, pioneered the initial development and piloting of eConsult. We defined a physician leader as a physician who played a key role in deploying the intervention. Originally, eConsult emerged from clinical needs and was adapted to the reality and workflow of professionals (ie, innovation users). In addition, the engagement of physician leaders was a key factor in identifying priorities for expanding eConsult to other specialties. For example, in all provinces studied, primary care professionals determined the order in which the eConsult service integrated specialties based on their observations and experiences regarding patient needs as well as specialist care services with the longest wait times. Strong physician leadership also helped to gain valuable support from provincial and federal medical associations for scale-up and to secure their participation in the process. Physician leaders recruited local eConsult users as clinical champions to promote the innovation among their colleagues. This facilitated the recruitment of additional family physicians and specialist users and lent credibility to the innovation, thereby increasing its uptake and adoption among professionals.

What Strategies Were Used?

Ontario and Quebec formalized the role of physician leader in the eConsult scale-up process. Specifically, the Ontario eConsult Center of Excellence named the two initial physician leaders as co-executive directors, and Quebec’s Ministry of Health and Social Services appointed the main physician leader in Quebec as a medical advisor on eConsult. Manitoba and Newfoundland & Labrador used this strategy of formalizing the role of physician leaders less often.

The CFHI’s Connected Medicine Collaborative engaged several physician leaders in each province, providing training, webinars, and tools designed to encourage and support clinicians to take on key roles in the spread and scale-up of eConsult in their province.

During the eConsult pilot projects in each province, physician leaders used their personal networks in a strategic way to build a clinical “coalition of the willing” (ie, enthusiastic professionals who were convinced of the benefits of the innovation) to advocate for eConsult among their colleagues. This provided momentum to increase the use of eConsult and recruit additional professionals to join the coalition.

Physician leaders recruited local physician champions (ie, clinicians who believed in the innovation and were willing to help recruit their local colleagues) as volunteers from among the attendees of their eConsult presentations at scientific and medical conferences (ie, Family Medicine Forum, North American Primary Care Research Group) and at regional tables. Physician champions shared their eConsult experience with their peers, encouraged them to become users, and acted as a resource to help resolve minor issues that users faced. They did not receive incentives for their role as champions.

Physician leaders also sought the endorsement of provincial and federal medical associations to help grow the provincial and national momentum surrounding eConsult and provide support for scale-up efforts (eg, funding, negotiations, political influence, and promoting eConsult to members).

“I think [the medical associations’] biggest role in the collaborative was helping get the word out and putting their stamp of approval. Like saying this is something that we can get behind even before they put it as a standard of practice” (National).

Table 7 presents a summary of the strategies used in each province to address physician leadership.

Resource Acquisition

Why Is This Enabling Factor Key to the Scale-Up of eConsult?

We identified the final key enabling factor as resource acquisition, which include human and financial resources to coordinate and manage eConsult, an adequate electronic platform, remuneration for primary care professionals and specialists, and continuous evaluation of eConsult. Our results suggest that it is essential to secure resources for three phases: (1) development and pilot testing; (2) transitioning from pilot to scale-up (to maintain achievements and spread eConsult to new contexts while policy issues are being addressed); and (3) ensuring sustainability (eg, recurring resources, including government funding, physician remuneration, and human resources to manage and oversee eConsult, as necessary, at the health system level).

What Strategies Were Used?

Since 2009, the eConsult team leads have applied for many research grants and innovation funds from various local, provincial, and national funding bodies (eg, Canadian Institutes for Health Research) to pilot and evaluate eConsult in various settings (eg, interdisciplinary chronic pain care and long-term care) and with various actors (eg, allied health professionals). Physician leaders, in close collaboration with professionals, patient partners, policy-makers, and researchers, wrote detailed study protocols included in grant applications. The CFHI’s Connected Medicine Collaborative provided key multi-jurisdictional funding to support pilot projects and to spread eConsult to new regions across different provinces.

“[The CFHI] really provided the infrastructure and leadership and funding for people to join the connected medicine collaborative, which has meant to really increase capacity across the country as well as implementation” (Ontario).

Our findings suggest that obtaining transition resources is important for maintaining achievements and spreading eConsult to new contexts while policy issues are being addressed. The transition period begins when initial research and innovation funds run out and teams must be creative to find sources of funding. However, obtaining transition resources can be difficult, as traditional Canadian funding bodies do not typically fund transition projects. In Quebec, Manitoba, and Ontario, eConsult teams and their partners advocated for eConsult to their respective provincial governments and obtained temporary transition funds to pay for specialists’ time (pro-rated fee used in research projects) and coordinate and manage the service when the research grant ended. This ensured the maintenance of eConsult services for onboarding professionals and for growth through a limited amount of spread until the establishment of fee-for-service tariffs. Non-traditional funding sources, including small funds obtained through innovation challenges or contests launched by the CFHI, medical associations, and family medicine departments, as well as in-kind resources from key partners (eg, regional organizations, research infrastructure, and medical associations) also supported the service until it was institutionally funded.

Establishing fee-for-service tariffs or alternate funding for specialists and primary care professionals secured, for the most part, sustainable resources. In Ontario, the Ministry of Health and Long-Term Care funded the remuneration of specialists through a prorated fee of $200 CAD/hour. This rate was used in pilot projects in Ontario (beginning in 2009) and in the other three provinces for specialists paid through a fee-for-service payment model. Newfoundland & Labrador and Quebec approved fee-for-service tariffs for specialists in May 2020 and June 2021, respectively. Ontario and Quebec also secured provincial remuneration for primary care professionals from provincial payers. Members of the provincial steering committees and eConsult advocates participated in remuneration negotiations, leveraging the eConsult evidence base. Additionally, to advocate for the establishment of provincial physician remuneration, eConsult steering committee members used a combination of complementary strategies, including sending letters to the Minister of Health, requesting a fee code from provincial payers, and encouraging professionals to contact their medical association and request a tariff. New or existing provincial infrastructure also secured sustainable human resources. For instance, in Ontario the newly created Ontario eConsult Centre of Excellence took on the management of the eConsult service, including physician remuneration. In Newfoundland & Labrador, the existing provincial payer managed physician remuneration and monitored the number of eConsults per physician to ensure that previously established limits per professional (400 eConsults per year) were respected.

Table 8 presents a summary of the strategies used in each province to address resources for scale-up.

Discussion

This study provides an in-depth analysis of the strategies used to scale up a promising innovation and related key factors based on data from four rich cases. For each key enabling factor, we described its importance to the scale-up of eConsult and identified and described the strategies used to address it. To our knowledge, our study is the first to distinguish between factors and strategies. By delving into the strategies used to address enabling factors and providing guidance on how to implement them, our paper makes a substantial and original contribution to the science of scale-up. Furthermore, as our study is one of the first conducted in a high-income primary care context like Canada,⁴ our results are unique within the literature.

In this study, we identified a total of 31 strategies related to six key factors that enabled the scale-up of eConsult in four Canadian provinces: (1) multi-actor engagement; (2) relative advantage; (3) knowledge transfer; (4) physical leadership; (5) a strong evidence base; and (6) resource acquisition. These six identified factors are similar to those identified in the literature,^{1,4,9,10,15,16} although the labels and categories vary among articles.

Similar to other studies,^1,15 our results show that a key enabling factor is the engagement of a range of actors. We add to this by identifying six strategies that address this factor. These strategies include creating multi-actor steering committees to oversee eConsult pilot projects and spread and providing actors with ready-to-use communication materials they can use to advocate for eConsult.

Another key factor identified as enabling the scale-up of eConsult was transferring learnings from Ontario to the other provinces. Four strategies addressed this factor, including sharing evidence, experience, knowledge, and tools, holding National eConsult Forums to facilitate shared learning, and mentoring other provinces. A previous study on key policy strategies for the spread and scale-up of eConsult highlighted the importance of building on current strategies and existing policies in the context of eHealth.¹⁵ Although that study did not report on shared learning, the authors proposed similar strategies to those we identified as enabling this factor in the present study.

We identified physician leadership as a key enabling factor that was addressed by strategies such as formalizing the role of physician leaders, engaging physician champions, and seeking the endorsement of medical associations to grow momentum and support scale-up efforts. Similarly, Moroz et al¹⁵ highlighted the importance of the support provided by “change” champions.

Additionally, we further refined the key enabling factor of evaluation and monitoring, which Milat et al⁴ found to be important, by suggesting that a strong evidence base is needed to enable scale-up and that this can be developed by embedding the innovation in research.

We found that resource acquisition was a key enabling factor addressed by different strategies during the pilot and testing, transition, and sustainability phases. A recent study identified different funding models to move beyond initial financial resources.³² Long-term funding was reported as a key factor to enable sustainability. The authors proposed to include long-term fundings in proposals or plan agreements for funding renewal under certain conditions of innovation performance.³² In addition, the existing literature highlights commonly reported scale-up barriers and facilitators related to human, financial, and technical resources, the capacity and will to scale-up, relational and political connections, active engagement of diverse actors and organizations, leadership, approaches tailored to context, and evaluation.^{1,3,4,9,10,33-39}

Identifying enabling factors at play in different contexts is useful to understand how they influence the scale-up of innovations. However, knowing the necessary enabling factors is not sufficient. A better understanding of these strategies, including concrete actions that actors can take to support scale-up efforts and address key enabling factors, is crucial to advance our ability to scale-up promising health innovations. The strategies and factors we have identified and described are grounded in both empirical evidence and theoretical frameworks on the scaling up of innovations^1,4,30 as well as key enabling factors identified in the literature.¹⁵ To our knowledge, this study is one of the first to provide ideas for actions that different types of actors with varying roles (eg, professionals, researchers, and policy-makers) and at multiple levels of governance (eg, regional, provincial, national) can take to scale up an innovation.

In this research, we studied the scale-up of an innovation with demonstrated effectiveness that was expanded at the healthcare system level in four jurisdictions, each with its own health system. Dedicated teams implemented and supported many strategies, but they also described these strategies as very time and resource intensive. Many actors noted that the scale-up of innovations remains incredibly complex and is a constant “uphill battle” that is poorly supported by our health systems. Promising innovations need support to improve health systems through scale-up. Creating supportive infrastructures and programs may help innovators overcome the challenges of scaling up innovations.⁴⁰ For instance, creating more national infrastructures that promote learning, increasing funding, and facilitating structured interactions between multiple actors across jurisdictions may enable scale-up of other innovations. To achieve this, one potential approach could involve the creation of “scale-up accelerators” that catalyze the process of aligning innovations with health system priorities, facilitate connections with relevant actors, provide scale-up coaching (eg, business case, scale-up plan), and help innovators navigate policy (eg, IT requirements, legislation, and resources).²⁰ Laur et al⁴¹ proposed to support innovators and transition innovations from pilot projects to larger scale-up by creating capacity-building programs. These programs could facilitate the development of strong and long-term partnerships between key actors.⁴¹

Strengths and Limitations

In terms of limitations, it is worth noting that we may not have captured all of the relevant strategies employed in the provinces, particularly strategies for securing sustainable resources, as some of the provinces had not completed the scale-up process during the study period. Certain provinces started the scale-up process earlier, which might have influenced strategies used in other provinces. Furthermore, given that Canada provides universal healthcare, with distinct publicly-funded health systems in each province, the results observed in this study may not necessarily be replicable in other contexts. It is necessary to take contextual factors such as these into consideration when adapting these findings to other settings.

We undertook several strategies to ensure the trustworthiness of our findings. The study design, which covered multiple contexts and involved the triangulation of data collected over 3 years, supports the credibility of our results, as does the peer debriefing conducted with all co-authors during two 60-minute virtual meetings. Moreover, we ensured confirmability by, for example, involving multiple co-authors in the data analysis, as described above.

Conclusion

The key strategies for scaling up an innovation that we identified and described offer concrete ideas for action that may help policy-makers, researchers, patients, and professionals who are interested in scaling up promising innovations. Scaling up innovations is context dependent, and our study provides strategies used in four jurisdictions in a high-income country to scale up an innovation. Further studies are needed to build an evidence base for key strategies that will help those who wish to scale up innovations while making efficient use of resources. Moreover, identifying and describing strategies that fail to address scale-up factors and why may contribute to understanding the complexity of scaling up innovations and how to ensure their sustainability. Building a comprehensive scale-up knowledge base will help actors bring their promising innovations to the masses for the benefit of all.

Acknowledgements

The authors would like to acknowledge the contribution of all stakeholders from the four provinces and thank them for accepting to collaborate on the study. We also wish to thank the Canadian Institutes of Health Research, the McGill Observatory on Health and Social Services Reforms and all organizations that provided matching funds and in-kind support.

Ethical issues

All participants gave written consent to participate in the study. This study was approved by the St. Mary’s Hospital Center Research Ethics Committee (Quebec, #SMHC-18-15), the Bruyère Continuing Care Research Ethics Board (Ontario, #M16-19-006), the Newfoundland & Labrador Health Research Ethics Board (NL, #2019.070), and the Bannatyne Campus Health Research Ethics Board (Manitoba, #H2019:307).

Competing interests

Clare Liddy and Erin Keely are co–executive directors of the Ontario eConsult Centre of Excellence, funded by the Ontario Ministry of Health. They co-founded the Champlain BASE^TM (Building Access to Specialists through eConsultation) eConsult service but do not retain any proprietary rights. Erin Keely answers eConsults through the service, fewer than one per month.

Funding

This study was funded by the Canadian Institutes of Health Research (#402867), Réseau-1 Québec, Fonds de recherche Québec-Santé, Research Centre - Charles-Le Moyne-Saguenay-Lac-St-Jean sur les innovations en santé, Initiative Patient Partenaire from the University of Sherbrooke, the Manitoba SPOR Primary and Integrated Health Care Innovation Network, C.T. Lamont Primary Health Care Research Centre, and the McGill Observatory on Health and Social Services Reforms. The funders played no role in the design of the study, collection, analysis, or interpretation of the data, or writing of the manuscript.

Disclosure of Relationships and Activities

Research grants from the Canadian Institutes of Health Research [grant number #402867], Réseau-1 Québec, Fonds de recherche Québec-Santé, Research Centre - Charles-Le Moyne-Saguenay-Lac-St-Jean sur les innovations en santé, Initiative Patient Partenaire from the University of Sherbrooke, the Manitoba SPOR Primary and Integrated Health Care Innovation Network, C.T. Lamont Primary Health Care Research Centre, and the McGill Observatory on Health and Social Services Reforms were received to support this work.

References

1. Ben Charif A, Zomahoun HTV, LeBlanc A. Effective strategies for scaling up evidence-based practices in primary care: a systematic review. Implement Sci 2017; 12(1):139. doi: 10.1186/s13012-017-0672-y [Crossref] [ Google Scholar]
2. Hutchison B, Abelson J, Lavis J. Primary care in Canada: so much innovation, so little change. Health Aff (Millwood) 2001; 20(3):116-131. doi: 10.1377/hlthaff.20.3.116 [Crossref] [ Google Scholar]
3. World Health Organization (WHO). Nine Steps for Developing a Scaling-Up Strategy. WHO; 2010.
4. Milat AJ, Bauman A, Redman S. Narrative review of models and success factors for scaling up public health interventions. Implement Sci 2015; 10:113. doi: 10.1186/s13012-015-0301-6 [Crossref] [ Google Scholar]
5. Kelly CJ, Young AJ. Promoting innovation in healthcare. Future Healthc J 2017; 4(2):121-125. doi: 10.7861/futurehosp.4-2-121 [Crossref] [ Google Scholar]
6. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q 2004; 82(4):581-629. doi: 10.1111/j.0887-378X.2004.00325.x [Crossref] [ Google Scholar]
7. Länsisalmi H, Kivimäki M, Aalto P, Ruoranen R. Innovation in healthcare: a systematic review of recent research. Nurs Sci Q 2006; 19(1):66-72. doi: 10.1177/0894318405284129 [Crossref] [ Google Scholar]
8. Greenhalgh T, Papoutsi C. Spreading and scaling up innovation and improvement. BMJ 2019; 365:l2068. doi: 10.1136/bmj.l2068 [Crossref] [ Google Scholar]
9. Willis CD, Riley BL, Stockton L. Scaling up complex interventions: insights from a realist synthesis. Health Res Policy Syst 2016; 14(1):88. doi: 10.1186/s12961-016-0158-4 [Crossref] [ Google Scholar]
10. Côté-Boileau É, Denis JL, Callery B, Sabean M. The unpredictable journeys of spreading, sustaining and scaling healthcare innovations: a scoping review. Health Res Policy Syst 2019; 17(1):84. doi: 10.1186/s12961-019-0482-6 [Crossref] [ Google Scholar]
11. Bégin M, Eggertson L, Macdonald N. A country of perpetual pilot projects. CMAJ 2009; 180(12):1185, e1188-1189. doi: 10.1503/cmaj.090808 [Crossref] [ Google Scholar]
12. Zelmer J. Beyond Pilots: Scaling and Spreading Innovation in Healthcare. Healthc Policy 2015; 11(2):8-9. [ Google Scholar]
13. Naylor D, Girard F, Mintz JM, Fraser N, Jenkins T, Power C. Unleashing Innovation: Excellent Healthcare for Canada: Report of the Advisory Panel on Healthcare Innovation. Ottawa, ON: Health Canada; 2015.
14. Collier R. Author urges scale-up of health care innovation. CMAJ 2017; 189(6):E253-E254. doi: 10.1503/cmaj.1095385 [Crossref] [ Google Scholar]
15. Moroz I, Archibald D, Breton M. Key factors for national spread and scale-up of an eConsult innovation. Health Res Policy Syst 2020; 18(1):57. doi: 10.1186/s12961-020-00574-0 [Crossref] [ Google Scholar]
16. Yamey G. Scaling up global health interventions: a proposed framework for success. PLoS Med 2011; 8(6):e1001049. doi: 10.1371/journal.pmed.1001049 [Crossref] [ Google Scholar]
17. de Carvalho Corôa RC, Gogovor A, Ben Charif A, et al. Evidence on scaling in health and social care: an umbrella review. Milbank Q. 2023. 10.1111/1468-0009.12649.
18. James HM, Papoutsi C, Wherton J, Greenhalgh T, Shaw SE. Spread, scale-up, and sustainability of video consulting in health care: systematic review and synthesis guided by the NASSS framework. J Med Internet Res 2021; 23(1):e23775. doi: 10.2196/23775 [Crossref] [ Google Scholar]
19. Bitanihirwe BK. Scaling-up innovation as a means of tackling health inequalities. Commentary. Ann Ist Super Sanita 2016; 52(2):143-145. doi: 10.4415/ann_16_02_01 [Crossref] [ Google Scholar]
20. Breton M, Smithman MA, Lamoureux-Lamarche C. Strategies used throughout the scaling-up process of eConsult - multiple case study of four Canadian provinces. Eval Program Plann 2023; 100:102329. doi: 10.1016/j.evalprogplan.2023.102329 [Crossref] [ Google Scholar]
21. Liddy C, Keely E. Using the quadruple aim framework to measure impact of heath technology implementation: a case study of eConsult. J Am Board Fam Med 2018; 31(3):445-455. doi: 10.3122/jabfm.2018.03.170397 [Crossref] [ Google Scholar]
22. Liddy C, Joschko J, Guglani S, Afkham A, Keely E. Improving equity of access through electronic consultation: a case study of an eConsult service. Front Public Health 2019; 7:279. doi: 10.3389/fpubh.2019.00279 [Crossref] [ Google Scholar]
23. Liddy C, Drosinis P, Deri Armstrong C, McKellips F, Afkham A, Keely E. What are the cost savings associated with providing access to specialist care through the Champlain BASE eConsult service? A costing evaluation. BMJ Open 2016; 6(6):e010920. doi: 10.1136/bmjopen-2015-010920 [Crossref] [ Google Scholar]
24. Liddy C, Maranger J, Afkham A, Keely E. Ten steps to establishing an e-consultation service to improve access to specialist care. Telemed J E Health 2013; 19(12):982-990. doi: 10.1089/tmj.2013.0056 [Crossref] [ Google Scholar]
25. Breton M, Smithman MA, Liddy C. Scaling up eConsult for access to specialists in primary healthcare across four Canadian provinces: study protocol of a multiple case study. Health Res Policy Syst 2019; 17(1):83. doi: 10.1186/s12961-019-0483-5 [Crossref] [ Google Scholar]
26. Stake RE. The Art of Case Study Research. SAGE Publications; 1995.
27. Creswell JW. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. 4th ed. London: SAGE Publications; 2013.
28. Canadian Foundation for Health Improvement. Connected Medicine Collaborative – Participating Teams. https://www.cfhi-fcass.ca/sf-docs/default-source/collaborations/connected-medicine-team-descriptions-e.pdf?sfvrsn=3c54a944_4. Accessed October 19, 2018.
29. Canadian Foundation for Health Improvement. Connected Medicine: Enhancing Primary Care Access to Specialist Consult - A 15-Month Quality Improvement Collaborative – Prospectus. 2017. https://www.cfhi-fcass.ca/sf-docs/default-source/collaborations/connected-medicine-prospectus-e.pdf?sfvrsn=4. Accessed October 16, 2018.
30. Rogers EM. Diffusion of Innovations. Simon and Schuster; 2010.
31. Miles MB, Huberman AM, Huberman MA, Huberman M. Qualitative Data Analysis: An Expanded Sourcebook. SAGE Publications; 1994.
32. Lee K, van Nassau F, Grunseit A. Scaling up population health interventions from decision to sustainability - a window of opportunity? A qualitative view from policy-makers. Health Res Policy Syst 2020; 18(1):118. doi: 10.1186/s12961-020-00636-3 [Crossref] [ Google Scholar]
33. Norton WE, McCannon CJ, Schall MW, Mittman BS. A stakeholder-driven agenda for advancing the science and practice of scale-up and spread in health. Implement Sci 2012; 7:118. doi: 10.1186/1748-5908-7-118 [Crossref] [ Google Scholar]
34. Shaw J, Shaw S, Wherton J, Hughes G, Greenhalgh T. Studying scale-up and spread as social practice: theoretical introduction and empirical case study. J Med Internet Res 2017; 19(7):e244. doi: 10.2196/jmir.7482 [Crossref] [ Google Scholar]
35. Milat AJ, King L, Newson R. Increasing the scale and adoption of population health interventions: experiences and perspectives of policy makers, practitioners, and researchers. Health Res Policy Syst 2014; 12:18. doi: 10.1186/1478-4505-12-18 [Crossref] [ Google Scholar]
36. Yamey G. Scaling up global health interventions: a proposed framework for success. PLoS Med 2011; 8(6):e1001049. doi: 10.1371/journal.pmed.1001049 [Crossref] [ Google Scholar]
37. Mangham LJ, Hanson K. Scaling up in international health: what are the key issues?. Health Policy Plan 2010; 25(2):85-96. doi: 10.1093/heapol/czp066 [Crossref] [ Google Scholar]
38. Wutzke S, Benton M, Verma R. Towards the implementation of large scale innovations in complex health care systems: views of managers and frontline personnel. BMC Res Notes 2016; 9:327. doi: 10.1186/s13104-016-2133-0 [Crossref] [ Google Scholar]
39. Hirschhorn LR, Talbot JR, Irwin AC. From scaling up to sustainability in HIV: potential lessons for moving forward. Global Health 2013; 9:57. doi: 10.1186/1744-8603-9-57 [Crossref] [ Google Scholar]
40. Smithman MA, Dumas-Pilon M, Campbell MJ, Breton M. Evaluation of a Dragons’ Den-inspired symposium to spread primary health care innovations in Quebec, Canada: a mixed-methods study using quality-improvement e-surveys. CMAJ Open 2022; 10(1):E247-E254. doi: 10.9778/cmajo.20200251 [Crossref] [ Google Scholar]
41. Laur C, Corrado AM, Grimshaw JM, Ivers N. Trialists perspectives on sustaining, spreading, and scaling-up of quality improvement interventions. Implement Sci Commun 2021; 2(1):35. doi: 10.1186/s43058-021-00137-6 [Crossref] [ Google Scholar]