Skip to main content
Download PDF

Optimizing the implementation of a population panel management intervention in safety-net clinics for pediatric hypertension (The OpTIMISe–Pediatric Hypertension Study)

Implementation Science Communications volume 1, Article number: 57 (2020) Cite this article

Abstract

Background

Though clinical practice guidelines are available, the diagnosis of pediatric hypertension (HTN) is often missed. Management may not follow guidelines due to the measurement challenges in children, complexity of interpreting youth blood pressure standards that are dependent on height, age, and sex, familiarity with diagnostic criteria, and variable comfort with management of pediatric HTN among providers. Evidence suggests that wide adoption and adherence to pediatric HTN guidelines would result in lower cardiovascular disease and kidney damage in adulthood. The proposed project will develop an implementation strategy package to increase adherence to clinical practice guidelines for pediatric HTN within safety-net community health centers (CHCs). The centerpiece of which is a provider-facing population panel management (PPM) tool and point-of-care clinical decision support (CDS). Prior research indicates that multiple discrete implementation strategies (e.g., stakeholder involvement, readiness planning, training, ongoing audit and feedback) are needed to institute practice- and provider-level adoption of such tools.

Methods

Using participatory research methods involving stakeholders from a practice-based research network of CHCs, with input from scientific advisors, the project aims to (1) employ user-centered design methods to tailor an existing CDS tool for use at the point of care and optimize cohort management with a PPM tool to support adherence to the latest pediatric HTN guidelines, and (2) use a stakeholder-driven method for selecting implementation strategies that support tool adoption and increase guideline-adherent physician behaviors. Multilevel process evaluation using surveys and key informant interview data will assess the acceptability, adoption, appropriateness, cost, and feasibility of the PPM tool and its multicomponent implementation strategy package. Usability testing will be conducted with the PPM tool to iteratively refine features and ensure proper functionality.

Discussion

The proposed research has the potential to improve identification, diagnosis, and management of HTN in primary care settings for high-risk youth by assisting healthcare providers in implementing the American Academy of Pediatrics’ 2017 guidelines using an EHR-integrated PPM tool with CDS. Should the strategy package for PPM tool adoption be successful for pediatric HTN, findings will be translatable to other settings and PPM of other chronic cardiovascular conditions affecting overall population health.

Peer Review reports

Background

Target organ damage, especially left ventricular hypertrophy associated with heightened risk for cardiovascular events in adults, is detectable in youth with primary hypertension (HTN) [1, 2]. Risk factors for adult HTN and concomitant target organ damage from HTN have been shown to begin in childhood [3,4,5]. Hypertensive children are more likely to have adult hypertension and metabolic syndrome [5, 6]. Primary prevention is critical given estimates that almost 10% of HTN in adults could be prevented if high blood pressures (BP) in childhood were recognized and treated [7,8,9,10]. The need to diagnose and manage HTN is further magnified due to increasing rates of childhood obesity [11,12,13] that indicate disproportionate effects on those who are disadvantaged economically and socially [14,15,16]. Guidelines for pediatric HTN were issued by the American Academy of Pediatrics (AAP) in 2017 [17] yet the diagnosis of pediatric HTN is often missed, and management may not follow the guidelines due to the complexity of interpreting youth BP standards, diagnosing HTN, and variable familiarity with managing pediatric HTN among pediatric healthcare providers [18].

A recent study in community health centers in the Chicago area found only 6.1% among 1478 children (age ≥ 3 and < 18 years) who met criteria for abnormal BP—based on BP values from clinical encounters recorded in the electronic health record (EHR)—were correctly diagnosed [19]. Evidence suggests that wide adherence to the HTN guidelines would result in lower risk for target organ damage to the heart and the kidneys and HTN-related cardiovascular disease in adulthood [17]. There is a pressing need for effective implementation strategies to ensure adherence to the pediatric HTN guidelines in primary healthcare systems, particularly those that serve high rates of children and adolescents at greatest risk for HTN due to the health disparities associated with obesity and or low birth weight [20,21,22].

Pediatric HTN and co-occurring cardiovascular conditions are a public health concern

In the USA, nearly 1 in 9 children exhibit serious cardiometabolic symptoms, including HTN, type 2 diabetes, insulin resistance, non-alcoholic fatty liver disease, asthma, and obstructive sleep apnea, all of which accelerate the risk for cardiovascular disease and mortality [23]. Additionally, cardiometabolic risk factors in youth tend to cluster [24]. Both the Bogalusa Heart [4] and Fels Longitudinal [5] studies clearly demonstrated that greater numbers of individual elevated BP measurements in childhood confer increased risk of adult HTN. Two recent cross-sectional studies further indicate that target organ damage is also detectable in adolescents with pre-HTN as well as HTN [25, 26].

About 3% of the general population has HTN, while about 25% of youth with obesity (BMI ≥ 95th%) have HTN [27]. A study examining childhood HTN and overweight/obesity in school children saw that 2.2% of the sample had HTN, and 37% of HTN cases could be attributed to overweight/obesity [28]. In a meta-analysis examining cardiovascular risk factors, compared with normal weight children, systolic BP was higher by 4.54 mmHg (99% confidence interval 2.44 to 6.64; n = 12,169, 8 studies) in overweight children, and by 7.49 mmHg (3.36 to 11.62; n = 8074, 15 studies) in children with obesity [29].

Additionally, the economic impact of pediatric HTN is substantial. There is evidence that elevated BP among children is associated with higher care costs: in a cohort of children and adolescents aged 3 to 17 years who received preventative care services in two states, youth with HTN had significantly higher annual health care costs ($1972 average) than those with normal BP ($736) or pre-HTN ($945), after adjusting for body mass index [30].

Cardiovascular health is an indicator of health disparities

According to the most recent estimates from the 2015–2016 National Health and Nutrition Examination Survey, 18.5% of all 2- to 19-year-old youth are obese and that a disproportionate number are disadvantaged economically and socially [31]. Obesity and its cardiovascular health consequences are disproportionately distributed across the USA; Mexican Americans, American Indians, and African Americans have the highest prevalence rates [32,33,34,35]. Low-income youths bear the burden, with risk factors at family and neighborhood levels [36, 37]. Stress and discrimination, both acutely experienced by minority and low-income communities in the USA, have been linked with cardiovascular health [38]. To address this, effective implementation of best practices for preventive care and chronic disease management is needed in safety-net health systems.

Pediatric HTN is underdiagnosed with incomplete management

Unlike the definition of adult HTN, which is linked to a single BP level, pediatric HTN until age 13 is dependent upon multiple factors, including age, gender, and stature [17]. Knowledge of the threshold values for acceptable BP recordings up until age 13 is therefore dependent on the clinician taking the BP reading and finding the corresponding BP or HTN category for that child. Diagnosis of HTN is further dependent upon taking multiple BP readings, whose interval of repetitive measurements is dependent upon the initial readings themselves. About 48.2% of stage I HTN diagnoses are correct, and only 35.0% of stage II HTN diagnoses are correct [39]. Thus, follow-up appointments must be carefully coordinated to assure accuracy.

The ideal methods of BP recording require having the patient in a quiet room, sitting, and without verbal interactions [17]. The majority of BP readings performed in an office setting initially use an oscillometric device that is often inaccurate compared to a mercury-based sphygmomanometer [17]. These specifications place a high burden on pediatric clinicians and their assistants in typical practices. Management of suspected HTN based on the new guidelines further requires a 24-h ambulatory BP study as well as an echocardiogram, but most general pediatric clinicians do not have such resources easily available to them, requiring referral to specialists. Last, most clinicians caring for children are unfamiliar with pharmacologic management of HTN—again requiring a referral to a sub-specialist. As a result, abundant evidence indicates that pediatric HTN is underdiagnosed.

The 2017 AAP guidelines for pediatric HTN are challenging to implement

The newest guidelines for clinical care of pediatric HTN [17] (referred to in the guidelines as elevated BP) has 30 Key Action Statements and 20 Tables for clinicians to understand and implement, making it very difficult to satisfy at a single office visit in which a BP reading may be above a reference threshold. As a result, many studies show primary care providers are not adherent, particularly related to screening and identification of abnormal BP [16, 40]. To reiterate, challenges of the latest guidelines include calling HTN “elevated blood pressure”, and the issues presented in Fig. 1.

Fig. 1
figure 1

Challenges for implementing current AAP guidelines for youth hypertension

Full size image

Health information technology to increase guideline adoption and adherence

This project focuses on optimizing cohort management with a population panel management (PPM) tool and tailoring an existing clinical decision support (CDS) tool for use at the point of care. PPM tools consist of a primary care practitioner or staff identifying patients who have unmet preventive and chronic care needs using panel-based health information technology tools [41]. At the point of care, CDS can then provide reminders and prompts to assist providers in managing their patient’s conditions, such as cardiovascular disease, obesity, or HTN. Immunizations are prototypic of a successful CDS system [42].

In the scope of single encounter, it is challenging for providers to review all historical data to make an accurate diagnosis. One of the values of a PPM tool is the ability to visualize data for a panel. As such, PPM tools could remind clinicians to schedule visits [17] and help clinicians identify children who merit a BP re/measurement [17]. A real-time alert tool embedded in EHRs, along with provider education, has been shown to increase recognition of elevated BP in children [43], and providers are more likely to correctly abide by the multiple BP readings protocol and order appropriate tests when CDS is available [44]. In a randomized trial, there was a significantly (p < .001) higher rate of correct identification of patients with HTN (54.9%) among clinics with CDS compared to clinics with none (21.3%) [45]. Despite these opportunities, implementation of eHealth tools for PPM and CDS are affected by the technology itself, the inner and outer setting characteristics, and individual health professionals’ readiness to change [46]. Addressing these challenges merits rigorous implementation research and forms the basis of this project.

Adoption of and adherence to the guidelines for pediatric HTN would improve all aspects of underrecognized and inconsistently managed condition that continues to rise amidst the childhood obesity epidemic. The complexity and variation in practice of these guidelines implies that their adoption hinges on effective implementation. PPM tools are a promising strategy that has yet to be evaluated in this context. To speed discovery of an effective practice, data- and stakeholder-driven development of new tools and multicomponent implementation strategies is needed.

The overarching aim of the proposed project is to develop and then optimize an effective and feasible implementation strategy package to increase adherence to clinical practice guidelines for pediatric HTN diagnosis and management and to understand contextual barriers and facilitators. We will target children beginning at age 3, when the AAP guidelines indicate onset of universal screening [17]. Prior research indicates that multiple specific implementation strategies are needed to institute practice- and provider-level adoption of PPM, CDS, and similar health information technology tools [47,48,49]. We will use a low-cost adaptation [50] of the Expert Recommendations for Implementing Change (ERIC) protocol [51, 52], consisting of a panel of stakeholders and key implementation leaders, to identify an initial implementation strategy package comprising feasible, discrete strategies for the adoption of the PPM and CDS tools and the guidelines starting from the four broad strategy types identified by systematic review: stakeholder involvement, readiness planning, training, and ongoing audit and feedback [46].

Methods

Study aims

Aim 1: Employ user-centered design methods to refine health information technology tools that address multilevel implementation barriers

With a stakeholder panel (e.g., pediatric providers, practice managers), led by the project team and with input from an external scientific advisory board, we will employ a user-centered design (UCD) approach [53, 54] to tailor an existing CDS tool for use at the point of care and optimize cohort management with a PPM tool, specifically to support adherence to the AAP guidelines for pediatric HTN and meet the needs of stakeholders.

Aim 2: Develop a multicomponent implementation strategy package

Using the adapted ERIC process, a community stakeholder panel, again with input from our scientific advisory board, will develop the initial implementation strategy package to address known and probable barriers to adoption of the tool and its effectiveness in changing guideline-relevant physician behavior related to identification and diagnosis of HTN. Health economic methods [55] will be used to evaluate the budget impact of the proposed development activities.

Study design

The proposed project involves three primary entities: Northwestern University Feinberg School of Medicine, Lurie Children’s Hospital of Chicago, and AllianceChicago Community Health Services. Within AllianceChicago are four CHC organizations in the Chicago area that have a long history of involvement in research and practice change initiatives using the AllianceChicago’s shared health information technology infrastructure. These organizations have more than 40 total clinics (range: 9 to 17 clinic locations per organization) and serve predominantly racial/ethnic minority patients (49% Black; 37% Hispanic/Latino; 4% Asian; 8% Non-Hispanic White; 2% Other) with 83% of patients below 100% of the poverty line and 25% uninsured. The demographics of these patients align with the health disparities in HTN per epidemiologic data [27].

The proposed study uses a community-engaged implementation research approach with a user-centered, stakeholder-driven approach when developing the PPM and CDS tools for pediatric HTN and the implementation strategy package. We focus on the impact of specific implementation strategies on provider behaviors as this is consistent with the purpose and targets of the AAP guidelines for pediatric HTN. The link between these provider practices in the guidelines and the clinical benefit to children was determined by the AAP when specifying the guidelines, based on the best available evidence [17]. The implementation process is guided by the adapted ERIC protocol [50] and the recommendations of Ross et al. [46]; assessment of determinants (barriers and facilitators) is guided by the Consolidated Framework for Implementation Research (CFIR) [56]; and evaluation of outcomes is guided by the Proctor et al. [57] taxonomy.

Drawing on principles of community-based participatory research [58] and NHLBI-endorsed community-engaged implementation research methods [59], this project follows the Out-reach, In-Reach, In-Translation, and Out-Translation (OIIO) model [60] (Fig. 2), developed by Co-Author Davis [60], to ensure a strong process of engagement with our partner CHC partners and community stakeholders by ensuring a bi-directional and recursive process of engaging community stakeholders in the development and evaluation activities in the proposed project. The OIIO model will guide the process of the stakeholder panel meetings for PPM implementation strategy development by first eliciting members’ perspectives (“in-reach”) and then sharing findings as they emerge (“out-reach”). The initial out-reach step begins by connecting the research team with the members of the stakeholder panel to respectfully and actively promote communication and engagement. The research team will elicit information regarding the needs and knowledge gaps from these diverse stakeholder advisors, which in-turn forms the in-reach. In particular, the team will lead in-translation of stakeholder needs and implementation gaps into scientific research/scientific concepts, which then guides the research team in framing out-translation. The research team shares quantitative and qualitative findings with the stakeholder panel via out-reach—bringing together academic-stakeholder knowledge as recursive OIIO cycles iterate over the project period.

Fig. 2
figure 2

OIIO Stakeholder Engagement Model

Full size image

Implementation preparation (study months [M] 1–12)

The current project consists of the development and planning activities prior to testing the implementation strategy and the PPM tool with pediatric patients in a subsequent study.

Recruitment and composition of the stakeholder panel

The stakeholder panel, comprising 6 to 10 members from four CHC organizations, will be recruited in M1 in collaboration with the leadership of each participating organization and through presentations and individual contacts by research staff. Members will be primarily practitioners who see pediatric patients, including physicians, advanced practice providers, and nurse practitioners; those with clinical leadership roles (e.g., chief medical officer) will also be eligible.

Tailoring the PPM and CDS tools for pediatric HTN and AAP guidelines

This phase will utilize UCD procedures tailoring an existing CDS tool for use at the point of care and optimize cohort management with a PPM tool (developed by Health Catalyst) already in use within AllianceChicago for adults with diabetes, HTN, HIV, and Medicare chronic care management. UCD is an iterative development process involving multiple cycles of evaluation and refinement of prototypes through deep engagement with relevant stakeholders [61]. AllianceChicago is well-versed in UCD procedures for health information technology tools. To facilitate engagement, the stakeholder panel will convene for a 2-h co-design workshop in M2 and M4 to discuss and provide feedback on the tailoring of the PPM and CDS tools’ initial features, functions, and required data inputs. The initial design will be used to start another cycle of co-design workshops. We anticipate that the majority of adaptation changes will happen within the first three iterations [54]. Thus, we plan to conduct at least three iterations of UCD meetings (four are planned; M3-M6) to ensure we capture all the major adaptations for the PPM tool while adhering to the AAP guidelines.

As features and functions are identified by the stakeholder panel, AllianceChicago will begin to build the tools (M3–M7). This is followed by two phases of usability testing, performed following the same procedures. In M7–M8, 10 participants from participating CHCs will evaluate the tools for 30 min in lab-based usability testing. Ten participants are typically required to identify 95% of usability problems [62]. Participants will be videotaped while using a “think aloud” framework as they use the tools to complete a set of basic use tasks [63]. At the conclusion of the testing session, participants will complete a modified version of the System Usability Scale [64] to assess the technology and will be asked to engage in brief user feedback interviews to identify (1) if user goals were met, (2) if any problems or difficulties were encountered, and (3) suggestions for improvements. Following lab-based usability testing of the tools, there will be an integration period with each of the CHC organizations’ EHR systems and functionality testing (M9–M10). Although the tools will be the same across all organizations because of the shared AllianceChicago infrastructure, patient and provider data for PPM will need to be specified for each organization. Once the PPM and CDS tools are installed, the 10 participants will engage in the same usability procedure and assessment once again. All major problems identified during both usability tests will be fixed prior to testing the tool in practice in a subsequent study.

Developing and piloting the implementation strategy package for the health information technology tools

The standard practice when AllianceChicago implements a new health information technology feature is to activate the associated EHR tools, provide written instructions and webinars that explain the tool to its members, and email members to alert them that new tools are in place. A large body of research suggests that such strategies alone without commensurate site-level training are unlikely to result in efficient and complete adoption of the PPM and CDS tools [46], and even intensive training is insufficient to achieve high rates of adoption and use of such tools [46]. This study will develop a package of implementation strategies (often referred to as a toolkit) to support uptake of the tools and use with fidelity.

We propose to follow the protocol of Go et al. [50], a pragmatic adaptation of the ERIC protocol [51, 52, 65]. In M5, the stakeholder panel will begin to develop the initial implementation strategy package following the procedures in Fig. 3. We will use a sequential mixed methods approach involving brief surveys (Step 6 in Fig. 3) of the stakeholder panel members followed by key informant interviews. Results will form the basis of a matrix used to identify common threads and contrasts across and within stakeholder levels. This matrix of common barriers and facilitators is then used in panel meetings to identify the specific type and intensity of implementation strategies to address them. The ERIC matrices are used to advance our understanding of the specific barriers and facilitators to adoption, to inform the process of prioritization with stakeholders across organizations, and to identify feasible implementation strategies through voting and an iterative dialog with our stakeholder panel. Rather than a single occurrence, we will repeat the adapted ERIC procedures to capture emergent barriers and tailor the strategies.

Fig. 3
figure 3

The adapted ERIC procedure

Full size image

To begin the modified ERIC process from an evidence-based foundation, we plan to use the results and recommendations from a recent systematic synthesis of 44 published reviews on eHealth implementation [46], focused on EHR, CDS, and PPM tools in primary care (Fig. 4). The strategy selection and incorporation process are described in detail in Additional file 1. Briefly, the development of the PPM tool for pediatric HTN will follow co-design workshops (Aim 1) explicitly focusing on these aspects. Strategies 2 and 3 are already in place and facilitated by AllianceChicago’s shared health information technology infrastructure. Strategies 4–7 will be the focus of the ERIC process for optimization. Concerning Strategy 4, we have designed our study activities to prospectively involve stakeholders and champions in the design of the tool and the initial implementation strategy package. Planning (Strategy 5) is also built into the study design and the process of developing and optimizing the implementation strategy and can also be modified with each successive cluster rollout. For training and education (Strategy 6), numerous discrete implementation strategies exist which offer significant adaptability for the needs of the intervention [65]. Finally, for Strategy 7, we will follow best practices for audit and feedback [66,67,68]. We will use established guidelines [69,70,71] to ensure accurate and detailed specification of the strategies used within the package as part of the mixed methods analytic plan.

Fig. 4
figure 4

Recommendations for the implementation of eHealth innovations

Full size image

Beyond Ross et al.’s summary, we anticipate using strategies unique to a PPM tool. The primary functionality of a PPM tool that distinguishes it from CDS and other eHealth interventions is the ability to examine cohorts of the clinic population for “missed opportunities” in guideline adherence. This population-based snapshot, as opposed to a single patient (i.e., point-of-care) perspective, allows clinic staff to examine recent and upcoming patient visits, as well as patients inactive in care, for indicators of need for outreach to re-engage in care for attention to HTN-related concerns. The precise manner in which these unique aspects of the PPM tool will be used in this study are unknown given the dearth of such tools for children.

The adapted ERIC process of barrier assessment, strategy selection, and optimization will be further organized through the use of the Implementation Research Logic Model [72, 73] as a process evaluation tool to depict iterative changes over time, and causal pathway diagrams [74]. Both of these methods specify the associations between identified barriers, specific implementation strategies, the mechanism(s) by which the strategy addresses the barrier, and the desired outcomes.

Assessment strategy, measures, and data analysis

The proposed study incorporates surveys and stakeholder interviews/meetings with multiple respondents (e.g., providers, key stakeholders). Mixed methods approaches for implementation research [75,76,77] are central to evaluating the tailoring and optimization of the PPM and CDS tools (Aim 1) and the implementation strategy package (Aim 2).

Aim 1: Tailor CDS and optimize PPM tools that addresses multilevel implementation barriers

The co-design workshops (n = 2) and stakeholder panel meetings (n = 2 meetings), and the “think aloud” portion of usability testing sessions (n = 20 total), will be evaluated through a sequential mixed methods approach involving rapid qualitative analysis of video recordings alongside quantitative results of the System Usability Scale [64]. In this project, the CFIR [56] interview guide will be used to semi-structure the stakeholder panel meetings. To ensure that qualitative results can be processed within our timeframe, we use a quick and comprehensive qualitative analysis strategy called Framework-Guided Rapid Analysis [78] in which a structured template based on the CFIR [56] analysis template is generated. To further reduce the necessary time for qualitative data processing, we will (1) identify themes and conduct content analysis directly from audio recordings, which can be done without the need for time-consuming transcription [79], and (2) provide “summary templates” rather than the typically lengthy results from intensive coding. Implementation researchers have used this method successfully when quick turn-around is required [80]. These qualitative analysis methods have been found to be reliable while requiring only about 13% more time than the recording [79], and some have found that a rapid approach is comparable to thematic analysis in terms of identifying key actionable findings [81]. Use of the CFIR interview guide and analysis template ensure that aspects of the domains and subdomains mentioned by the panel are captured, and uniform terminology and the definitions of other implementation researchers are used, which will increase rigor and reproducibility.

All qualitative data processing and analysis will be done by a trained Research Associate and a Research Assistant, overseen by JS. For reliability, 20% of panel meetings and “think aloud” tasks will be double-coded in order to calculate reliability [82]. Disagreements in coding will be resolved via expert consensus among the investigative team led by JS, who has led studies using qualitative and mixed methods [83,84,85] and rapid qualitative analysis methods and the CFIR tools. The mixed methods analytic approach in this aim will be a “merge the data” approach described by Palinkas and Cooper [75], which involves bringing together quantitative and qualitative data through complementarity [76, 77].

Aim 2: Develop an implementation strategy package

Evaluation of the strategy development will be mixed methods involving qualitative data analysis from key informant interviews and the stakeholder panel meetings and the results of quantitative surveys.

Qualitative analysis of the optimization process

Procedures for the collection, rapid analysis, and reliability of qualitative data are described in Aim 1. To ensure detailed reporting of the implementation strategy package, we will follow published guidelines [69] and use the Implementation Research Logic Model [72, 73]. This is critical for interpretation and rigor and reproducibility of the study, as well as costing of these activities (see “Cost analysis” section below).

Quantitative analysis of the development process

Surveys administered during development activities of this project focus on usability of the tool (see Aim 1) and the feasibility, appropriateness, and acceptability of the tool and its implementation strategy package assessed via brief, validated surveys developed by Weiner et al. [86] after tool and strategy package development.

Mixed methods analysis

The mixed methods analytic approach is sequential (“Connect the dots”) [76, 77] for integrating the qualitative data with the quantitative data. This approach will provide a detailed evaluation of the development process and the specific strategies involved in the package.

Cost analysis

Economic analyses are central to the implementation evaluation and for providing data of use to potential adopters (e.g., administrators, policy makers). Our cost evaluation within this project will focus on estimating the costs associated with implementing each discrete strategy and the package of strategies developed by the stakeholder panel. This cost analysis will be conducted from the perspective of a provider organization that may wish to adopt the multicomponent strategy developed in this study. An ingredient-based cost analysis procedure, including costs for tool refinement and optimization and integrating it with the EHR, training providers and other stakeholders, time of provider usage, and other personnel time to support implementation, will be employed to estimate costs associated with each discrete strategy. Strategy costs will be aggregated to estimate total package costs. Consistent with the aims of this project, we will estimate total cost of implementation preparation activities (that is, the activities required to get ready for implementation prior to any patient benefit) as has been done in prior implementation studies led by NJ and JS [87, 88]. Using an activity-based costing approach will allow us to value activities both locally (Chicago, Illinois) and from national data sources (e.g., Current Population Survey, US Department of Labor), providing estimates relevant for scale up in new pediatric care settings nationwide.

Normalization

Defined as the perception of potential standardization of the PPM and CDS tools, normalization will be assessed using the NoMAD instrument [89, 90]. The NoMAD assesses staff perceptions of factors relevant to embedding health information technology that changes typical work practices. Based on Normalization Process Theory [91], the 23-item NoMAD is concerned with: implementation (bringing a practice or practices into action), embedding (when a practice or practices may be routinely incorporated into everyday work), and integration (when a practice or practices are reproduced and sustained in the social matrices of an organization) [89]. The NoMAD is particularly relevant for evaluating practice changes in primary care involving health information technology interventions [92,93,94]. It will be administered to all involved stakeholders (physicians/providers/practice leaders) in M11.

Protocol modifications and current status

Based on feedback from our scientific advisory board, we have made three modifications to the protocol. First, to provide contemporary baseline adherence rates to the guidelines prior to implementation, we will perform two secondary data analyses of EHR data from the AllianceChicago network of CHCs to characterize the prevalence of correct diagnosis of pediatric HTN and elevated BP. These analyses will be conducted on two time periods. The first will be 1 year prior to the guideline changes (visits occurring between December 2016 and December 2017), and the second period will be the 2 years after the guidelines changed and minimal EHR capacities were added (e.g., automatic calculation of age-specific percentiles). Second, consistent with the goal of Aim 1 (to refine a PPM tool that addresses multilevel implementation barriers), we will invite pediatric patients at risk for HTN and their caregiver(s) to participate in a brief interview to better understand their needs and perspectives around the assessment, diagnosis, and management of pediatric HTN to inform implementation strategies concerning raising awareness and motivating adherence with guideline-based care, such as returning for a repeat BP measurement after a first elevated BP. Third, also aligned with Aim 1, pediatric primary care providers will be invited to complete an online survey to better understand their perspectives around identifying and managing pediatric HTN. Feedback from the patient-caregiver interviews and provider surveys will help us better design and implement the PPM tool and associated implementation strategies to support guideline-adherent care for HTN.

At submission of this protocol, the project is well underway. Specifically, we have recruited the stakeholder panel and the meeting dates have been scheduled; developed and pilot-tested the provider survey; and convened a meeting of the Scientific Advisory Board. Further, the first set of secondary data analyses of EHR data have been conducted and a manuscript is currently being prepared for submission.

It is worth mentioning that we are currently in the middle of the COVID-19 pandemic in the state of Illinois, which includes a stay-at-home order. AllianceChicago CHCs, like most healthcare systems across the nation, are shifting their focus to managing the pandemic. Accordingly, we have temporarily suspended administration of the provider survey and will hold meetings of the stakeholder panels virtually. Our team will continue to modify the protocol as required to meet the shifting needs of our stakeholders and collaborators in the healthcare system during this pandemic, while also adhering to public health guidelines, guidance from Northwestern University, and recommendations from the Center for Disease Control and Prevention and the Illinois and Chicago Departments of Public Health.

Discussion

The ultimate goal of the proposed project is to develop a feasible implementation strategy package to increase adherence to clinical practice guidelines for pediatric HTN diagnosis and management. We will accomplish this aim using a PPM tool and CDS at the point of care. This will broaden the reach of HTN diagnosis and management by providing a tool that assists providers and benefits an entire patient population.

The protocol will extend the existing literature in two novel ways. First, we will design and implement a PPM tool to support the adoption of AAP guidelines for the diagnosis and treatment of pediatric HTN. No such sophisticated, EHR-integrated tool currently exists for pediatric HTN, despite evidence that PPM tools are effective at improving diagnosis and management of multiple chronic health conditions (including HTN in adults), and PPM tools are unique and underutilized tools for guideline implementation in pediatrics. Additionally, while the majority of existing PPM tools are for adults, this innovation will focus on children and adolescents. Second, we will leverage an innovative stakeholder-driven process for tool development using UCD methods and development of an implementation strategy package based on best available evidence from the field of implementation science and a low-cost adaptation of the ERIC process. Third, we will conduct a prospective implementation cost evaluation informed by stakeholders to ensure that critical questions informing economic sustainability and incremental cost-benefits are answered. Inadequate cost evaluation is a top reason for failure to implement [95], and an insufficient proportion of implementation research studies include prospective economic evaluation [96]. Cost information is highly informative for potential adopter organizations and can be used to quanitify the monetary investment required to achieve adoption and the related clinical outcomes of interest.

We anticipate two specific challenges while conducting this study. First, the HTN guidelines currently require repeated measurement of pediatric patients 'BP. As discussed, this guideline receives low adherence among pediatric healthcare professionals who are managing large workloads and competing priorities. Further, it places a large burden on families to attend multiple medical appointments. The activities in Aim 2 will actively address this challenge through understanding the barriers specific to providers and families around this aspect of the AAP pediatric HTN guidelines. Second, accurate assessment of BP in pediatric patients is critical. This speaks to the need of accessible training for pediatric healthcare providers and availability of correct BP assessment tools. Incorporating training in simply measuring BP accurately for younger children (ages 2 to 5 years) and helping pediatric clinics obtain arm cuffs that are properly sized for youth with obesity might become necessary.

Identifying youth with HTN is the gateway to getting them into effective treatments that can prevent the deleterious progression of cardiovascular disease risk. Should the implementation strategy package for PPM tool adoption and guideline adherence be successful for pediatric HTN, findings will be translatable to other settings and for PPM of other chronic heart, lung, blood, and sleep conditions among children and adults, thereby decreasing the health care burden of target organ damage in the cardiovascular system, and other complications, in the adult years and improving overall population health.

Availability of data and materials

Data and materials are available upon request to the corresponding author.

Abbreviations

AAP:

American Academy of Pediatrics

BP:

Blood pressure

CDS:

Clinical decision support

CHC:

Community health center

ERIC:

Expert Recommendations for Implementing Change

HTN:

Hypertension

OIIO:

Out-reach, In-Reach, In-Translation and Out-Translation model

PPM:

Population panel management tool

UCD:

User-centered design

References

  1. Daniels SR, Loggie JM, Khoury P, Kimball TR. Left ventricular geometry and severe left ventricular hypertrophy in children and adolescents with essential hypertension. Circulation. 1998;97:1907–11.

    CAS  PubMed  Google Scholar 

  2. Sorof JM, Cardwell G, Franco K, Portman RJ. Ambulatory blood pressure and left ventricular mass index in hypertensive children. Hypertension. 2002;39:903–8.

    CAS  PubMed  Google Scholar 

  3. May AL, Kuklina EV, Yoon PW. Prevalence of cardiovascular disease risk factors among us adolescents, 1999–2008. Pediatr. 2012;129:1035–41.

    Google Scholar 

  4. Bao W, Threefoot SA, Srinivasan SR, Berenson GS. Essential hypertension predicted by tracking of elevated blood pressure from childhood to adulthood: The Bogalusa Heart Study. Am. J. Hypertens. 1995;8:657–65.

    CAS  PubMed  Google Scholar 

  5. Sun SS, Grave GD, Siervogel RM, Pickoff AA, Arslanian SS, Daniels SR. Systolic blood pressure in childhood predicts hypertension and metabolic syndrome later in life. Pediatr. 2007;119:237–46..

    Google Scholar 

  6. Juhola J, Magnussen CG, Viikari JS, Kahonen M, Hutri-Kahonen N, Jula A, Lehtimaki T, Akerblom HK, Pietikainen M, Laitinen T, et al. Tracking of serum lipid levels, blood pressure, and body mass index from childhood to adulthood: the Cardiovascular Risk in Young Finns Study. J Pediatr. 2011;159:584–90.

    CAS  PubMed  Google Scholar 

  7. Kelly RK, Thomson R, Smith KJ, Dwyer T, Venn A, Magnussen CG. Factors affecting tracking of blood pressure from childhood to adulthood: the childhood determinants of adult health study. J Pediatr. 2015;167:1422–8 e1422.

    PubMed  Google Scholar 

  8. Magnussen CG, Smith KJ. Pediatric blood pressure and adult preclinical markers of cardiovascular disease. Clin Med Insights Blood Disord. 2016;9:1–8.

    PubMed  PubMed Central  Google Scholar 

  9. Song P, Zhang Y, Yu J, Zha M, Zhu Y, Rahimi K, Rudan I. Global prevalence of hypertension in children: a systematic review and meta-analysis. JAMA Pediatr. 2019:1–10.

  10. Jackson SL, Zhang Z, Wiltz JL, Loustalot F, Ritchey MD, Goodman AB, Yang Q. Hypertension Among Youths - United States, 2001-2016. MMWR Morb Mortal Wkly Rep. 2018;67:758–62.

    PubMed  PubMed Central  Google Scholar 

  11. Franks PW, Hanson RL, Knowler WC, Sievers ML, Bennett PH, Looker HC. Childhood obesity, other cardiovascular risk factors, and premature death. N Engl J Med. 2010;362:485–93.

    CAS  PubMed  PubMed Central  Google Scholar 

  12. Juonala M, Magnussen CG, Berenson GS, Venn A, Burns TL, Sabin MA, Srinivasan SR, Daniels SR, Davis PH, Chen W, et al. Childhood adiposity, adult adiposity, and cardiovascular risk factors. N Engl J Med. 2011;365:1876–85.

    CAS  PubMed  Google Scholar 

  13. Muntner P, He J, Cutler JA, Wildman RP, Whelton PK. Trends in blood pressure among children and adolescents. JAMA. 2004;291:2107–13.

    CAS  PubMed  Google Scholar 

  14. Ogden CL, Carroll MD, Kit BK, Flegal KM. Prevalence of childhood and adult obesity in the United States, 2011-2012. JAMA. 2014;311:806–14.

    CAS  PubMed  PubMed Central  Google Scholar 

  15. McNiece KL, Poffenbarger TS, Turner JL, Franco KD, Sorof JM, Portman RJ: Prevalence of hypertension and pre-hypertension among adolescents. The Journal of Pediatrics 2007, 150:640-644.e641.

  16. Hansen ML, Gunn PW, Kaelber DC. Underdiagnosis of hypertension in children and adolescents. JAMA. 2007;298:874–9.

    CAS  PubMed  Google Scholar 

  17. Flynn JT, Kaelber DC, Baker-Smith CM, Blowey D, Carroll AE, Daniels SR, de Ferranti SD, Dionne JM, Falkner B, Flinn SK, et al. Clinical practice guideline for screening and management of high blood pressure in children and adolescents. Pediatrics. 2017.

  18. Bello JK, Mohanty N, Bauer V, Rittner SS, Rao G: Pediatric hypertension: Provider perspectives. Global Pediatric Health 2017, 4:2333794X17712637.

  19. Rao G, Naureckas S, Datta A, Mohanty N, Bauer V, Padilla R, Rittner Sarah S, Tilmon S, Epner P: Pediatric hypertension: diagnostic patterns derived from electronic health records. In Diagnosis, vol. 5. pp. 157; 2018:157.

  20. Edvardsson VO, Steinthorsdottir SD, Eliasdottir SB, Indridason OS, Palsson R. Birth weight and childhood blood pressure. Current Hypertension Reports. 2012;14:596–602.

    PubMed  Google Scholar 

  21. Mhanna MJ, Iqbal AM, Kaelber DC. Weight gain and hypertension at three years of age and older in extremely low birth weight infants. Journal of neonatal-perinatal medicine. 2015;8:363–9.

    CAS  PubMed  Google Scholar 

  22. Crump C, Howell EA, Stroustrup A, McLaughlin MA, Sundquist J, Sundquist K. Association of preterm birth with risk of ischemic heart disease in adulthood. JAMA Pediatrics. 2019.

  23. Grundy SM, Pasternak R, Greenland P, Smith S Jr, Fuster V. Assessment of cardiovascular risk by use of multiple-risk-factor assessment equations: a statement for healthcare professionals from the American Heart Association and the American College of Cardiology. Journal of the American College of Cardiology. 1999;34:1348–59.

    CAS  PubMed  Google Scholar 

  24. Goodman E, Dolan LM, Morrison JA, Daniels SR. Factor analysis of clustered cardiovascular risks in adolescence: obesity is the predominant correlate of risk among youth. Circulation. 2005;111:1970–7.

    PubMed  Google Scholar 

  25. Urbina EM, Khoury PR, McCoy C, Daniels SR, Kimball TR, Dolan LM. Cardiac and vascular consequences of pre-hypertension in youth. The Journal of Clinical Hypertension. 2011;13:332–42.

    PubMed  Google Scholar 

  26. Falkner B, DeLoach S, Keith SW, Gidding SS. High risk blood pressure and obesity increase the risk for left ventricular hypertrophy in African-American adolescents. The Journal of Pediatrics. 2013;162:94–100.

    PubMed  Google Scholar 

  27. Shatat IF, Brady TM. Editorial: Pediatric Hypertension: Update. Frontiers in Pediatrics. 2018;6.

  28. Chiolero A, Cachat F, Burnier M, Paccaud F, Bovet P. Prevalence of hypertension in schoolchildren based on repeated measurements and association with overweight. Journal of Hypertension. 2007;25:2209–17.

    CAS  PubMed  Google Scholar 

  29. Friedemann C, Heneghan C, Mahtani K, Thompson M, Perera R, Ward AM. Cardiovascular disease risk in healthy children and its association with body mass index: systematic review and meta-analysis. BMJ. 2;345.

  30. Gilmer TP, O’Connor PJ, Sinaiko AR, Kharbanda EO, Magid DJ, Sherwood NE, Adams KF, Parker ED, Margolis KL. Impact of hypertension on health care costs among children. Am J Managed Care. 2014;20:622–8.

    Google Scholar 

  31. Hales CM, Fryar CD, Carroll MD, Freedman DS, Ogden CL. Trends in obesity and severe obesity prevalence in us youth and adults by sex and age, 2007-2008 to 2015-2016. JAMA. 2018;319:1723–5.

    PubMed  PubMed Central  Google Scholar 

  32. Broussard BA, Johnson A, Himes JH, Story M, Fichtner R, Hauck F, Bachman-Carter K, Hayes J, Frohlich K, Gray N. Prevalence of obesity in American Indians and Alaska Natives. Am J Clin Nutr. 1991;53:1535S–42S.

    CAS  PubMed  Google Scholar 

  33. Messiah SE, Arheart KL, Lopez-Mitnik G, Lipshultz SE, Miller TL. Ethnic group differences in cardiometabolic disease risk factors independent of body mass index among American youth. Obesity. 2013;21:424–8.

    CAS  PubMed  Google Scholar 

  34. Wang Y, Beydoun MA. The obesity epidemic in the United States—gender, age, socioeconomic, racial/ethnic, and geographic characteristics: A systematic review and meta-regression analysis. Epidemiologic Reviews. 2007;29:6–28.

    CAS  PubMed  Google Scholar 

  35. Ogden CL, Carroll MD, Lawman HG, Fryar CD, Kruszon-Moran D, Kit BK, Flegal KM. Trends in obesity prevalence among children and adolescents in the United States, 1988-1994 through 2013-2014. Jama. 2016;315:2292–9.

    CAS  PubMed  PubMed Central  Google Scholar 

  36. Fradkin C, Wallander JL, Elliott MN, Tortolero S, Cuccaro P, Schuster MA. Associations between socioeconomic status and obesity in diverse, young adolescents: variation across race/ethnicity and gender. Health psychol. 2015;34:1–9.

    PubMed  Google Scholar 

  37. Greves Grow HM, Cook AJ, Arterburn DE, Saelens BE, Drewnowski A, Lozano P. Child obesity associated with social disadvantage of children’s neighborhoods. Soc Sci Med. 2010;71:584–91.

    PubMed  Google Scholar 

  38. Brewis AA. Stigma and the perpetuation of obesity. Soc Sci Med. 2014;118:152–8.

    PubMed  Google Scholar 

  39. Koebnick C, Mohan Y, Li X, Porter AH, Daley MF, Luo G, Kuizon BD. Failure to confirm high blood pressures in pediatric care—quantifying the risks of misclassification. J Clin Hypert. 2018;20:174–82.

    Google Scholar 

  40. Brady TM, Solomon BS, Neu AM, Siberry GK, Parekh RS. Patient-, provider-, and clinic-level predictors of unrecognized elevated blood pressure in children. Pediatrics. 2010;125(6):e1286–93.

    PubMed  Google Scholar 

  41. Neuwirth EB, Schmittdiel JA, Tallman K, Bellows J. Understanding panel management: a comparative study of an emerging approach to population care. Permanente J. 2007;11:12–20.

    Google Scholar 

  42. Garg AX, Adhikari NJ, McDonald H, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005;293:1223–38.

    CAS  PubMed  Google Scholar 

  43. Brady TM, Neu AM, Miller ER, Appel LJ, Siberry GK, Solomon BS. Real-time electronic medical record alerts increase high blood pressure recognition in children. Clinical Pediatrics. 2015;54:667–75.

    PubMed  Google Scholar 

  44. Lobach D, Sanders GD, Bright TJ, Wong A, Dhurjati R, Bristow E, Bastian L, Coeytaux R, Samsa G, Hasselblad V. Enabling health care decisionmaking through clinical decision support and knowledge management. Evid Rep Technol Assess (Full Rep). 2012;203:1–784.

    Google Scholar 

  45. Kharbanda EO, Asche SE, Sinaiko AR, Ekstrom HL, Nordin JD, Sherwood NE, Fontaine PL, Dehmer SP, Appana D, O’Connor P. Clinical decision support for recognition and management of hypertension: a randomized trial. Pediatrics. 2018;141.

  46. Ross J, Stevenson F, Lau R, Murray E. Factors that influence the implementation of e-health: a systematic review of systematic reviews (an update). Implementation Science. 2016;11:146.

    PubMed  Google Scholar 

  47. Holtrop JS, Rabin BA, Glasgow RE. Dissemination and implementation science in primary care research and practice: Contributions and opportunities. The Journal of the American Board of Family Medicine. 2018;31:466–78.

    PubMed  Google Scholar 

  48. Bao Y, Druss BG, Jung H-Y, Chan Y-F, Unützer J. Unpacking collaborative depression care: examining two essential tasks for implementation. Implementation Science. 2015;10:A33.

    Google Scholar 

  49. Bloomfield GS, Wang TY, Boulware LE, Califf RM, Hernandez AF, Velazquez EJ, Peterson ED, Li JS. Implementation of management strategies for diabetes and hypertension: from local to global health in cardiovascular diseases. Global Heart. 2015;10:31–8.

    PubMed  PubMed Central  Google Scholar 

  50. Go VF, Morales GJ, Mai NT, Brownson RC, Ha TV, Miller WC. Finding what works: identification of implementation strategies for the integration of methadone maintenance therapy and HIV services in Vietnam. Implementation Science. 2016;11:54.

    PubMed  Google Scholar 

  51. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, Proctor EK, Kirchner JE. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10.

  52. Waltz TJ, Powell BJ, Chinman MJ, Smith JL, Matthieu MM, Proctor EK, Damschroder LJ, Kirchner JE. Expert recommendations for implementing change (ERIC): protocol for a mixed methods study. Implement Sci. 2014;9.

  53. Norman DA. Design of everyday things: Revised and expanded. New York, NY: Basic Books; 2013.

    Google Scholar 

  54. Nielsen J. Iterative user-interface design. Computer. 1993;26:32–41.

    Google Scholar 

  55. Ritzwoller DP, Sukhanova A, Gaglio B, Glasgow RE. Costing behavioral interventions: a practical guide to enhance translation. Annals of Behavioral Medicine. 2009;37:218–27.

    PubMed  Google Scholar 

  56. Damschroder L, Aron D, Keith R, Kirsh S, Alexander J, Lowery J. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation Science. 2009;4:50.

    PubMed  Google Scholar 

  57. Proctor EK, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, Griffey R, Hensley M. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health Ment Health Serv Res. 2011;38.

  58. Israel B, Eng E, Schulz A. Methods in community-based participatory research for health. San Francisco: CA: Jossey-Bass; 2005.

    Google Scholar 

  59. Mensah GA, Cooper RS, Siega-Riz AM, Cooper LA, Smith JD, Brown CH, Westfall JM, Ofili EO, Price LN, Arteaga S, et al. Reducing cardiovascular disparities through community-engaged implementation research: a National Heart, Lung, and Blood Institute workshop report. Circulation Research. 2018;122:213–30.

    CAS  PubMed  PubMed Central  Google Scholar 

  60. Davis MM, Sandberg E, Miranda M. OIIO: a novel framework for community-engaged research. Chicago, IL: In American Public Health Association Annual Meeting; 2015..

    Google Scholar 

  61. Balancing iterative design and planning for mobile games [https://leantesting.com/resources/iterative-design-for-mobile-games/]. 5 Mar 2020.

  62. Faulkner L. Beyond the five-user assumption: benefits of increased sample sizes in usability testing. Behav Res Methods Instrum Comput. 2003;35:379–83.

    PubMed  Google Scholar 

  63. Virzi RA, Sorce, J. F., & Herbet, L. B. : A comparison of three usability evaluation methods: heuristic, think-aloud, and performance testing. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 1993: 309-313.

  64. Brooke J. SUS-A quick and dirty usability scale. Usability evaluation in industry. 1996;189:4–7.

    Google Scholar 

  65. Waltz TJ, Powell BJ, Matthieu MM, Damschroder LJ, Chinman MJ, Smith JL, Proctor EK, Kirchner JE. Use of concept mapping to characterize relationships among implementation strategies and assess their feasibility and importance: results from the Expert Recommendations for Implementing Change (ERIC) study. Implementation Science. 2015;10:109.

    PubMed  Google Scholar 

  66. Mattox T, Kilburn MR. Supporting effective implementation of evidence-based practices: a resource guide for child-serving organizations. Santa Monica, CA: RAND Corporation; 2016.

    Google Scholar 

  67. Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, O'Brien MA, Johansen M, Grimshaw J, Oxman AD. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database of Systematic Reviews. 2012.

  68. Colquhoun HL, Carroll K, Eva KW, Grimshaw JM, Ivers N, Michie S, Sales A, Brehaut JC. Advancing the literature on designing audit and feedback interventions: identifying theory-informed hypotheses. Implementation Science. 2017;12:117.

    PubMed  Google Scholar 

  69. Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci. 2013;8.

  70. Campbell M, Katikireddi SV, Hoffmann T, Armstrong R, Waters E, Craig P. TIDieR-PHP: a reporting guideline for population health and policy interventions. BMJ. 2018;361.

  71. Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, Altman DG, Barbour V, Macdonald H, Johnston M, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ. 2014;348.

  72. Smith JD. An implementation research logic model: a step toward improving scientific rigor, transparency, reproducibility, and specification. Implementation Science. 2018;14:S39.

    Google Scholar 

  73. Smith JD, Li D, Rafferty MR: The implementation research logic model: a method for planning, executing, reporting, and synthesizing implementation projects. Submitted for publication 2020.

  74. Lewis CC, Klasnja P, Powell BJ, Lyon AR, Tuzzio L, Jones S, Walsh-Bailey C, Weiner B. From classification to causality: advancing understanding of mechanisms of change in implementation science. Frontiers in Public Health. 2018;6.

  75. Palinkas LA, Rhoades Cooper B. Mixed methods evaluation in dissemination and implementation science. In: Brownson RC, Colditz GA, Proctor EK, editors. Dissemination and implementation research in health. 2nd ed. New York, NY: Oxford University Press; 2017.

    Google Scholar 

  76. Palinkas LA, Aarons GA, Horwitz S, Chamberlain P, Hurlburt M, Landsverk J. Mixed method designs in implementation research. Adm Policy Ment Health. 2011;38:44–53.

    PubMed  Google Scholar 

  77. Aarons GA, Fettes DL, Sommerfeld DH, Palinkas LA. Mixed methods for implementation research: application to evidence-based practice implementation and staff turnover in community-based organizations providing child welfare services. Child Maltreatment. 2012;17:67–79.

    PubMed  Google Scholar 

  78. Gale RC, Wu J, Erhardt T, Bounthavong M, Reardon CM, Damschroder LJ, Midboe AM. Comparison of rapid vs in-depth qualitative analytic methods from a process evaluation of academic detailing in the Veterans Health Administration. Implementation Science. 2019;14:11.

    PubMed  Google Scholar 

  79. Neal JW, Neal ZP, VanDyke E, Kornbluh M. Expediting the analysis of qualitative data in evaluation:a procedure for the Rapid Identification of Themes From Audio Recordings (RITA). American Journal of Evaluation. 2015;36:118–32.

    Google Scholar 

  80. Hamilton AB: Qualitative methods in rapid turn-around health services research. In VA Women's Health Research Network; 2013.

    Google Scholar 

  81. Taylor B, Henshall C, Kenyon S, Litchfield I, Greenfield S. Can rapid approaches to qualitative analysis deliver timely, valid findings to clinical leaders? A mixed methods study comparing rapid and thematic analysis. BMJ Open. 2018;8:e019993.

    PubMed  PubMed Central  Google Scholar 

  82. Cicchetti DV, Sparrow SS. Developing criteria for establishing the interrater reliability of specific items in an inventory. Am J Ment Retard. 1981;86:127–37.

    CAS  Google Scholar 

  83. Berkel C, Rudo-Stern J, Villamar JA, Wilson C, Flanagan E, Lokey F, Dishion TJ, Smith JD: Recommendations from community partners to promote sustainable implementation of evidence-based programs in primary care. Journal of Community Psychology 2020. https://onlinelibrary.wiley.com/doi/10.1002/jcop.22317.

  84. Smith JD, Egan KN: Trainee and client experiences of therapeutic assessment in a required graduate course: a qualitative analysis. Journal of Personality Assessment 2016:Available ahead of print.

  85. Smith JD, Berkel C, Rudo-Stern J, et al. The Family Check-Up 4 Health (FCU4Health): applying implementation science frameworks to the process of adapting an evidence-based parenting program for prevention of pediatric obesity and excess weight gain in primary care. Public Health Front. 2018;6:293. https://doi.org/10.3389/fpubh.2018.00293.

  86. Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, Boynton MH, Halko H. Psychometric assessment of three newly developed implementation outcome measures. Implementation Science. 2017;12:108.

    PubMed  Google Scholar 

  87. Saldana L, Chamberlain P, Bradford WD, Campbell M, Landsverk J. The cost of implementing new strategies (COINS): a method for mapping implementation resources using the stages of implementation completion. Children and Youth Services Review. 2014;39:177–82.

    PubMed  Google Scholar 

  88. Smith MJ, Graham AK, Sax R, Spencer E, Razaano L, Smith JD, Jordan N. Costs of preparing to implement a virtual reality job interview training program in a community mental health agency: a budget impact analysis. Journal of Evaluation in Clinical Practice. 2019.

  89. Finch TL, Girling M, May CR, Mair FS, Murray E, Treweek S, McColl E, Steen IN, Cook C, Vernazza CR, et al. Improving the normalization of complex interventions: part 2 - validation of the NoMAD instrument for assessing implementation work based on normalization process theory (NPT). BMC Medical Research Methodology. 2018;18:135.

    PubMed  PubMed Central  Google Scholar 

  90. Rapley T, Girling M, Mair FS, Murray E, Treweek S, McColl E, Steen IN, May CR, Finch TL. Improving the normalization of complex interventions: part 1 - development of the NoMAD instrument for assessing implementation work based on normalization process theory (NPT). BMC Medical Research Methodology. 2018;18:133.

    PubMed  PubMed Central  Google Scholar 

  91. May C, Finch T. Implementing, embedding, and integrating practices: an outline of normalization process theory. Sociology. 2009;43:535–54.

    Google Scholar 

  92. Tierney E, McEvoy R, O'Reilly-de Brún M, de Brún T, Okonkwo E, Rooney M, Dowrick C, Rogers A, MacFarlane A. A critical analysis of the implementation of service user involvement in primary care research and health service development using normalization process theory. Health Expectations. 2016;19:501–15.

    PubMed  Google Scholar 

  93. May CR, Cummings A, Girling M, Bracher M, Mair FS, May CM, Murray E, Myall M, Rapley T, Finch T. Using normalization process theory in feasibility studies and process evaluations of complex healthcare interventions: a systematic review. Implementation Science. 2018;13:80.

    PubMed  Google Scholar 

  94. Vis C, Ruwaard J, Finch T, Rapley T, de Beurs D, van Stel H, van Lettow B, Mol M, Kleiboer A, Riper H, Smit J. Toward an objective assessment of implementation processes for innovations in health care: psychometric evaluation of the Normalization Measure Development (NoMAD) Questionnaire among mental health care professionals. J Med Internet Res. 2019;21(2):e12376.

    PubMed  PubMed Central  Google Scholar 

  95. Glasgow RE, Emmons KM. How can we increase translation of research into practice? Types of evidence needed. Annual Review of Public Health. 2007;28:413–33.

    PubMed  Google Scholar 

  96. Raghavan R. The role of economic evaluation in dissemination and implementation research. In: Brownson R, Colditz G, Proctor EK, editors. Dissemination and implementation research in health: Translating science to practice. 2nd ed. New York, NY: Oxford University Press; 2017. p. 89–106.

    Google Scholar 

Download references

Acknowledgements

The authors wish to thank our collaborators from the community health centers in Chicago; our dedicated research teams at Northwestern University, Lurie Children’s Hospital, and AllianceChicago; and the members of our scientific advisory board (Rachel Gold, PhD, Daniel Johnson, MD, and Goutham Rao, MD) and scientific consultants on this project (Allison Hamilton, PhD, MPH and Byron J. Powell, PhD, MSW) for providing sage guidance.

Funding

This study is supported by grant R56HL148192 from the National Heart, Lung, and Blood Institute of the National Institutes of Health, awarded to Justin Smith. Additional support was provided by grant P30DA027828 from the National Institute on Drug Abuse, awarded to C. Hendricks Brown. The opinions expressed herein are the views of the authors and do not necessarily reflect the official policy or position of the, the National Heart, Lung, and Blood Institute, the National Institute on Drug Abuse, or any other part of the US Department of Health and Human Services.

Author information

Authors and Affiliations

  1. Departments of Psychiatry and Behavioral Sciences, Preventive Medicine, Medical Social Sciences, and Pediatrics, Northwestern University Feinberg School of Medicine, Chicago, IL, USA

    Justin D. Smith

  2. Department of Pediatrics, Northwestern University Feinberg School of Medicine, Chicago, IL, USA

    Nivedita Mohanty & Roxane Padilla

  3. Stanley Manne Children’s Research Institute, Ann & Robert H. Lurie Children’s Hospital of Chicago, and Departments of Pediatrics, Medicine, Medical Social Sciences, and Preventive Medicine, Northwestern University Feinberg School of Medicine, Chicago, IL, USA

    Matthew M. Davis

  4. Department of Psychiatry and Behavioral Sciences, Northwestern University Feinberg School of Medicine, Chicago, IL, USA

    Ashley A. Knapp

  5. Department of Medicine, Northwestern University Feinberg School of Medicine, Chicago, IL, USA

    Yacob G. Tedla

  6. Department of Psychiatry and Behavioral Sciences, Northwestern University Feinberg School of Medicine, Chicago, IL, USA

    Allison J. Carroll & Juan A. Villamar

  7. Stanley Manne Children’s Research Institute, Ann & Robert H. Lurie Children’s Hospital of Chicago and Department of Pediatrics, Northwestern University Feinberg School of Medicine, Chicago, IL, USA

    Heather E. Price

  8. Department of Psychiatry and Behavioral Sciences, Northwestern University Feinberg School of Medicine, Chicago, IL, USA

    Neil Jordan

  9. Departments of Psychiatry and Behavioral Sciences, Preventive Medicine, and Medical Social Sciences, Northwestern University Feinberg School of Medicine, Chicago, IL, USA

    C. Hendricks Brown

  10. Ann & Robert H. Lurie Children’s Hospital of Chicago and Department of Pediatrics, Northwestern University Feinberg School of Medicine, Chicago, IL, USA

    Craig B. Langman

Authors
  1. Justin D. Smith
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nivedita Mohanty
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Matthew M. Davis
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ashley A. Knapp
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yacob G. Tedla
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Allison J. Carroll
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Heather E. Price
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Juan A. Villamar
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Roxane Padilla
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Neil Jordan
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. C. Hendricks Brown
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Craig B. Langman
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

JDS, NM, and CBL conceived of the study. JDS, NM, CBL, MMD, and AAK designed the study. NJ, YGT, HEP, JAV, AJC, RP, and CHB participated in different phases of the study. JDS and AAK collaborated in drafting the manuscript. All authors read, edited, and approved the final version of the manuscript.

Corresponding author

Correspondence to Justin D. Smith.

Ethics declarations

Ethics approval and consent to participate

The study is designed in accordance with the basic ethical principles of autonomy, beneficence, justice, and non-maleficence and will be conducted in accordance with the rules of Good Clinical Practice outlined in the most recent Declaration of Helsinki. Written informed consent of patients will be required. Data confidentiality and anonymity will be ensured, according to the provisions of United States law, both during the implementation phase of the project and in any resulting presentations or publications. The project has been favorably evaluated by the Institutional Review Board of Northwestern University on March 23, 2020 (Protocol STU00210809). All other institutions participating in this research provided signed reliance agreements ceding to the Institutional Review Board of Northwestern University. The informed consents completed by all relevant human subject participants of this trial are provided as Additional file 3. Any important protocol modification will be communicated to the Institutional Review Board of Northwestern University.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1.

Additional file 2.

Notice of Award. Notice of award from NHLBI – Revision # 1 – Removal of Human Subjects Restrictions.

Additional file 3.

IRB Approval. Notice of IRB approval from Northwestern University.

Additional file 4.

Informed Consent. Informed consent documents for study components involving human subjects.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Smith, J.D., Mohanty, N., Davis, M.M. et al. Optimizing the implementation of a population panel management intervention in safety-net clinics for pediatric hypertension (The OpTIMISe–Pediatric Hypertension Study). Implement Sci Commun 1, 57 (2020). https://doi.org/10.1186/s43058-020-00039-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s43058-020-00039-z

Keywords

Implementation Science Communications

ISSN: 2662-2211