PDF
/ 
ePUB
2. Department of Technical Communication, Mercer University, USA.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
The purpose of this pilot study was to assess and refine a Web-based EBP educational intervention focused on improving EBP competence in BSN-prepared pediatric bedside nurses, and to examine its feasibility, acceptability, and usability. Using a two-group experimental embedded mixed methods design, a convenience sample of 29 BSN-prepared nurses was recruited from a pediatric hospital in the Southeastern United States. Participants were randomized into an intervention (n=14) or attention control group (n=15) and both received approximately 2 hours of educational content. The intervention group (IG) reviewed the Web-based EBP module and the attention control group (ACG) reviewed a Webbased module on Munchausen's Syndrome by Proxy. Quantitative data were collected online from both groups using the Evidence-Based Practice Beliefs (EBPB) scale and the adapted Fresno test for Pediatric Nurses. Qualitative data were collected via telephone interview. Data were collected online utilizing SurveyMonkey© technology. Quantitative analyses were conducted using parametric and non-parametric statistics, and effect size calculations. Qualitative data were analyzed using methods described by Creswell and Plano Clark (2011). Demographic characteristics of the two groups were similar. A comparison of the IG and ACG post-intervention indicated there was not a statistically significant difference in the groups on EBP competence. However, the mean of the IG was higher and the effect size estimate of Hedges' g was small to borderline medium in magnitude. EBPB scale scores post-intervention were statistically significantly different with the IG demonstrating statistically significantly more positive beliefs about EBP. The estimated effect size for this difference was borderline medium in magnitude. Participants' interview data supported the intervention as being moderately feasible, acceptable, and usable for improving EBP competence in BSN-prepared pediatric nurses. This study provides preliminary data on Web-based methodologies that can be helpful in improving EBP beliefs and EBP competence among pediatric bedside nurses.
Keywords: Nursing, web-based, evidence-based practice, education, mixed-methods research, EBP competence
The Institute of Medicine (IOM) has set a goal that by 2020, 90% of all healthcare clinical decisions in the U.S. will be supported by accurate, timely, up-to-date clinical information, and will reflect the best available evidence [1]. This emphasis on evidence-based practice (EBP) in healthcare delivery has increased the expectation that nurses utilize research findings to make informed clinical decisions in a constantly changing and increasingly complex healthcare environment. Clinical practice based on the best available evidence is considered the expected standard of care in nursing [2] and endorsed by influential organizations throughout the world [1,3-6].
EBP is inherent in excellent nursing practice as the literature consistently purports that clinical outcomes are improved when nursing implements an EBP approach to care [7-11]. For example, a seminal study within nursing revealed researchbased nursing practice was found to offer patients better outcomes than routine, procedural nursing care [9]. Within the field of oncology, nurses used an EBP approach to care that improved cancer-related dyspnea [7], reduced fatigue and promoted functional well-being in patients experiencing cancer-related fatigue [10], and helped prevent, manage, and treat chemotherapy-induced nausea and vomiting [11]. Within the specialty of cardiac surgery, nurses were key drivers of patient-centered change and integral to enhancing postoperative recovery and reducing the risk of premature mortality in patients following open heart surgery [8].
As noted, the research shows when nurses use an EBP approach, the quality of care and patient outcomes is improved. However, the literature also indicates a large proportion of nurses lack the knowledge and skill sets to provide evidencebased care [12-14]. Furthermore, several reports have demonstrated the current state of EBP competence and use is relatively suboptimal. For example, multiple studies conducted in the the U.S. and other countries have demonstrated (a) nurses continue to use colleagues and personal experience to base their clinical decision-making [13,15-18]; (b) inconsistencies in whether nurses value research [19-23]; (c) nurses reasons for not implementing EBP may be due to individual and/or organizational barriers [24,25]; and (d) EBP competence and use is particularly problematic within pediatric nursing [26-34]. With respect to the interventions currently available to promote EBP in nursing, they mostly focus on nurses who work in specialties other than pediatric populations [35-40]. These interventions often vary in format and/or dose across settings, and usually are institutionally driven. While online or Web-based continuing education for practicing nurses is growing [41-49], accessible EBP training for pediatric nurses is warranted.
Purpose of study
The purpose of this mixed methods pilot study was two-fold. The first was to assess and refine a Web-based EBP educational intervention on improving EBP competence in BSN-prepared pediatric bedside nurses. The second was to examine the feasibility, acceptability, and usability of implementing a Web-based EBP intervention with respect to intervention content and delivery method, measures, and data collection procedures. Effect sizes obtained in this study, along with information about feasibility, acceptability, and usability have the potential to inform a larger intervention study focused on improving EBP competence in pediatric nurses.
Research questions
This study addressed the following research questions:
Conceptual framework
There were two theories that guided this study. The first theory was Patricia E. Benner's novice to expert theory [50]. This theory defines degrees of competence and uses a process of skill acquisition. Specifically, it describes how knowledge acquisition over time can influence change. For this study, the development of EBP competence was the change in cognitive skills expected and measured. The second theory to guide this study was Icek Ajzen's theory of planned behavior [51]. This theory explains how social cognitive factors influence behavior change; particularly, the social cognitive factors of attitude, perception of social norms, and perceived behavioral control, which fall under the umbrella of belief. For this study, components of the theory of planned behavior were used to account for the social cognitive factors that impact the nurses' capacity for improving EBP competence. Thus, EBP belief was measured pre-and post-intervention in this study. Figure 1 shows the integration of these theories as a conceptual guide for this study.
Figure 1 : Diagram illustrating the integration of benner’s and ajzen’s theories.
Design
A two group experimental embedded mixed methods design was used [52]. The secondary qualitative strand was used to explore feasibility, acceptability, and usability factors of implementing a Web-based EBP educational intervention with respect to intervention content and delivery method, measures, and data collection procedures. As noted in this design [52], the inclusion of the qualitative strand to a primarily quantitative study can be a means to overcome some of the perceived challenges associated with a pilot study as well as enhance the interpretation of findings from the quantitative component of the study. Thus, the integration of a qualitative strand into the overall design of this pilot study enhanced the understanding of the (a) fidelity of the intervention, (b) adherence by participants, and (c) strength of the methods and data collection based on the experience of the participants. As such, quantitative data was collected using Web-based data collection techniques at baseline and following completion of the intervention period. In addition, qualitative data around the feasibility, acceptability, and usability of the intervention content and delivery method, as well as around the tools used for data collection was obtained. Figure 2 illustrates the various components of this design and their interrelationships.
Figure 2 : Implementation of a web-based EBP educational intervention: a concurrent embedded model.
Setting and sample
The setting for this study was an acute care free standing pediatric hospital located in a large city in the Southeastern United States. This particular hospital has three main campuses and 20 neighborhood sites, including five Urgent Care Centers. Nurses were recruited from this hospital using a combination of Web-based and face-to-face recruitment strategies, with the exclusion of one main campus. Nurses who worked on the excluded campus were not eligible to participate in this study because an earlier version of the EBP module had been piloted on this campus.
Eligible participants for this study included BSN-prepared pediatric nurses who self-reported (a) having access to a computer that met the basic requirements for retrieving the modules, (b) having basic computer literacy skills, (c) having access to the Internet, (d) not working on the excluded campus of this pediatric hospital, (e) not participating in any formal EBP training program during the study timeline, and (f ) not having participated in the earlier version of the Web-based module piloted at this hospital. A convenience sample of thirty nurses (n=15 in each group) who met the criteria for this study was needed. However, of the 58 nurses recruited for the study, only 29 nurses completed the study in its entirety.
Description of intervention
Nurses in the intervention group received EBP education via the Web-based EBP module (n=14) while nurses in the control group received an attention control module on Munchausen's Syndrome by Proxy (n=15). Table 1 provides a comparative view of the modules used with the EBP intervention group and the attention control group, and the content within the modules. The Web-based EBP educational intervention was a 2-hour module comprised of four units that were delivered via the Internet, in which participants were encouraged to review within 4 weeks. The Web-based EBP educational intervention was designed by the primary author and informed by examples in her practice as a pediatric nurse in an acute care setting. Additionally, information was extrapolated from noted experts in EBP to further develop its content and delivery (i.e., Bernadette Melnyk, Ellen Fineout-Overholt, Nancy Burns, Susan Grove, Kathleen Stevens, Denise Polit, and Cheryl Tatano Beck). The basis for the primary author's decision on EBP competencies that were expected of BSN-prepared bedside nurses was extrapolated from the Essential Competencies for Evidence-Based Practice in Nursing [53]. These competencies were highly influenced by the IOM's recommendations and represent a national consensus on essential EBP competencies by education level [53]. Thus, the application of competencies for nurses educationally prepared and working at the baccalaureate level was used in the development of the Web-based EBP educational intervention.
Table 1 :Overview of modules used with the EBP intervention group and the attention control group.
The attention control module was comprised of three Power Point presentations on content unrelated to EBP but relevant to the scope of pediatric nursing. The purpose of the attention control module was to provide a similar amount of attention to the control group as those in the intervention group, without the specific content on EBP. The content in the module was developed by the primary author and other pediatric nurses for Nursing Grand Round learning events at their institution (i.e., Munchausen's Syndrome by Proxy). This particular module was one of several developed by the primary author and other pediatric nurses at their institution. The module was approximately two hours in length, requiring a similar time commitment from the attention control group as was required from participants in the intervention group.
Nurses accessed the study modules via CourseSites by Blackboard™. CourseSites by BlackBoard™ was the online learning environment that was used to house all study-related material. This included the modules and active links to the instrumentation used within this study. Through CourseSites™, nurses were able to access modules as often as they needed to, once they consented to the study and completed the registration process.
Data collection
Demographic data was collected from nurses pre-intervention using a demographic data form developed for this study. The demographic information collected included: gender, race/ethnicity, age, school enrollment status, years in nursing, years as a pediatric nurse, year in which they obtained their BSN degree, amount of EBP training received during nursing school, years of employment at current organization, perception of EBP resources available at organization, and previous experience with EBP. There was also an area on this form where nurses were asked to provide their email addresses, phone numbers, and unit/campus where they currently worked. The email addresses were used for nurses to initially access the study and the phone numbers were used to collect qualitative data from nurses concerning the acceptability and usability aspects of the study.
The Evidence-Based Practice Beliefs (EBPB) scale was used in this study pre- and post-intervention to measure nurses' beliefs about the value of EBP and their ability to implement it. The EBPB scale was developed by Melnyk and Fineout-Overholt in 2002 and is a 16-item Likert scale, with responses that range from 1 (strongly disagree) to 5 (strongly agree). Higher scores indicate stronger beliefs about the value of EBP and the ability to implement it. Scoring for the EBPB scale consists of summing responses to the 16 items for a total score that can range between 16 and 80.
EBP competence was measured post-intervention using an adapted version of the Fresno test. To date, the original version of this tool is the only validated and reliable instrument found that measures all five stages of EBP [54,55]. However, the original Fresno is a discipline-specific instrument that had to be modified for use in this study. Permission was obtained from the developers of the original Fresno test to adapt the questionnaire. The adapted version of the Fresno used in this study includes three case scenarios relevant to the scope of pediatric nursing. The case scenarios were followed by 11 open-ended questions, with the exception of question #8. Question #8 presented a generic scenario in which participants answered five sub-items constructed as a mix of open- and closed-ended questions. Questions #1 through #7 were answered based on the initial three case scenarios presented. The first case scenario presented a problem with adolescent Cystic Fibrosis (CF) patients and their lack of adherence to current diet and respiratory therapy regimens. The second case scenario presented an infant with bronchiolitis and the use of respiratory nebulizer treatments versus suctioning without nebulizer treatments. Finally, the third case scenario presented a toddler with Type 1 Diabetes with an extreme case of needle anxiety and the proposal of an Insuflon port for administration of insulin. The possible range of scores for this adapted version of the Fresno test were the same as the original Fresno test that ranged from zero to 212 and higher scores indicating greater EBP competence.
The acceptability and usability survey was developed for this study by the primary author and used at the end of the study to determine nurses' perspectives and experience with the entire study. This information was collected via telephone interview by the primary author after nurses completed the adapted Fresno test for pediatric nurses. A combination of open-ended questions and Likert-scale items were used to assess participants' (a) ease of accessibility, use, clarity, understandability, and helpfulness of the Web-based educational modules; (b) perception of content that should be deleted or added to the Web-based educational modules, and (c) overall rating of their experience with the Web-based modules.
The final data collection tool used in this study was the follow-up survey. This tool was used only with nurses who withdrew from the study. This survey was sent to nurses via email and included a checklist of reasons for withdrawing from the study and an open-ended question inviting nurses to comment on their experience.
Procedures
After IRB approval was obtained, a combination of face-toface and Web-based recruitment strategies were conducted through staff meetings with the four primary patient care areas (PCA). Initially, an email invitation was sent directly to the nurse managers of the four primary care units to forward to their nurses. This email invitation contained a link to the Web-based informed consent and demographic data form. However, due to low enrollment rates, an amendment to the IRB application was made to expand the inclusion criteria to (a) include nurses who worked beyond the four PCA and (b) advertise the study on study site's Intranet. After the amendment, the sample pool expanded to include intensive care units (ICUs) and non-ICUs. Nurses were randomized to the intervention group or attention control group by unit (i.e., Intervention Group: PCA2, PCA3, non-ICUs; Attention Control Group: PCA1, PCA4, and ICUs). The decision to randomize nurses by unit versus individually was done to reduce the risk of intervention contamination. Data collection occurred over 10 weeks completely online (i.e., email and CourseSites™) with the exception of the phone interview and nurses were given a monetary incentive of $75 for completing the study.
Demographics
With respect to sample demographics, the mean ages for nurses between the intervention and attention control groups were similar at 36.9 and 36.3, respectively. In addition, the means were found to be similar between both groups with respect to (a) years as a nurse, (b) years since BSN obtained, (c) years as a pediatric nurse, and (d) years employed at study site. The majority of the sample was females who reported they were not currently enrolled in school, and their ethnicity as White/Caucasian (see Table 2).
Table 2 : Comparison of intervention and attention control groups on demographic variables (N=29).
Nurses were also asked about their perception of the EBP resources available to them at this institution. The groups differed substantially on their perceptions on the provision of (a) medical librarian services, (b) a shared decision making environment, and (c) EBP mentors. In these areas, the attention control group was found to have stronger perceptions regarding the availability of these EBP resources.
Nurses were also asked about their experience with EBP during nursing school and since graduating from nursing school. With respect to nurses' EBP experience while in nursing school, more than two-thirds of the sample reported a moderate to complete emphasis of EBP in their nursing curriculum. With respect to nurses' EBP experience since graduating from nursing school, both groups reported limited formal EBP training. However, almost 30% of nurses from the intervention group reported having formal EBP training, whereas only 13% of nurses from the attention control group reported having formal EBP training since graduating from nursing school (see Table 3).
Table 3 : Comparison of intervention and attention control groups on herceptions of available EBP resources pre-intervention (N=29).
EBP competence
The adapted Fresno test for Pediatric Nurses was used in this study to measure EBP competence. An independent samples t-test was conducted and the adapted Fresno test score was the dependent variable. Table 4 shows nurses who received the EBP educational intervention demonstrated higher EBP competence scores compared to nurses who received the attention control group content. However, this difference was not statistically significant but the observed effect size bordered on a small to medium effect with Hedges' g of 0.47.
Table 4 : Independent samples t-test comparison of adapted fresno test scores by intervention group (N=29).
EBP beliefs
The EBPB scale was used in this study to measure EBP beliefs. A one way ANCOVA was conducted and the EBPB scale post-test score was the dependent variable. Preliminary data analysis indicated there was a significant difference between groups pre-intervention. Based on this difference, the EBPB score pre-intervention was controlled in this analysis. Table 5 shows nurses who received the EBP educational intervention reported statistically significantly stronger beliefs about the value of EBP and their ability to implement it compared to nurses who received the attention control content. The effect size of Hedges' g on the post-intervention EBPB scores was 0.49. This degree of difference between the intervention and attention control group with respect to EBP beliefs post-intervention was reflective of a borderline medium effect.
Table 5 : Comparison of post-intervention EBP belief scores by intervention group controlling for pre-intervention belief scores (N=29).
Feasibility, acceptability and usability
An extensive study log was maintained to capture the enrollment, retention, and attrition rates of participants every two weeks over the course of the study. Figure 3 illustrates how participants progressed through the study. A total of 63 nurses expressed interest in the study and five were excluded because they either failed to meet the inclusion criteria or failed to complete baseline data. This left 58 nurses who were randomized into the intervention or attention control group. The largest number of nurses were lost because they never logged into the CourseSites by Blackboard™ website (n=16). In addition, there were a number of nurses who logged into the CourseSites by Blackboard™ website but failed to review their assigned module (n=8). This left 34 nurses between the groups that were eligible for follow-up and six of those 34 were lost to follow-up because they failed to take the adapted Fresno test. This left a final analytic sample of n=14 for the intervention group and n=15 for the attention control group. This meant 29 nurses were retained and 29 nurses withdrew and/or did not complete the study, yielding an attrition rate of 50%. In consideration of the high attrition rate in this study, an independent samples t-test and chi-square analysis were conducted to compare the two groups (i.e., those who completed the study versus those who did not complete the study) on age, gender, race/ethnicity, years as a nurse, years as a pediatric nurse, and perceptions of EBP resources. There were no significant differences found between completers and non-completers among these variables.
Figure 3 : CONSORT diagram showing participants’ progression through study.
In addition, a follow-up survey was sent via email to nurses who withdrew and/or did not complete the study (n=29).
Eighteen nurses responded and provided information as to why they could not complete the study. Reasons reported by nurses for withdrawing from or not completing the study included (a) time constraints with work schedule (n=11), (b) life events/stressors (n=4), and (c) technical issues (n=3). Additionally, there were feasibility factors concerning the Respondus Lockdown Browser and the time restriction of 1 hour when taking the adapted Fresno test. With respect to the Respondus Lockdown Browser, as part of the study protocol, nurses had to download this custom browser that locked down the testing environment within CourseSites by Blackboard™. This meant when nurses used this browser, they were unable to print, copy, go to another URL, or access other applications. Some nurses reported not being able to download the custom browser while others reported not being able to complete the adapted Fresno test within the 1 hour time limit.
An analysis of the narrative comments was conducted for both the intervention and control groups. Over 60% of nurses from both groups reported their assigned module was useful in promoting EBP in BSN-prepared pediatric nurses and would recommend to nurses who want to learn about EBP. However, the groups differed in their perception of having the knowledge and skill sets in EBP to implement it in their practice. For example, nurses in the intervention group reported the module as a "primer" to EBP implementation while nurses in the attention control group expressed confusion between what was reviewed in the module and EBP implementation. The response from the control group was expected since the module they reviewed had nothing to do with EBP knowledge and skill building. Nurses were also asked to provide additional comments. Nurses in the intervention group reported the module having good information and being a foundation to begin EBP while other nurses in the same group reported the module lacked interaction. Lastly, some nurses in the attention control group reported the module being engaging and relevant, user-friendly, and with good examples.
Finally, web-statistics analyses were conducted to determine the usability of the CourseSites by Blackboard™ website for the delivery of content between the groups (see Table 6). The key point derived from this analysis was that although there were apparent differences between the groups with respect to the total time spent in the course, average time in the course per user, the total logins, average login per user, etc., these differences were not that large. This further demonstrated the attention control group content did what it was supposed to do and kept nurses engaged in this group for a comparable amount of time as nurses in the intervention group.
Table 6 : Coursesites by blackboard web statistics for intervention and attention control groups (N=29).
EBP training
Nurses in this pilot study reported they received limited EBP training after graduating from nursing school. These results are consistent with publications that report a lack of continued EBP education and skills building for bedside nurses [17,45,56-58]. In addition, a recent survey of practicing nurses in the U. S. confirmed nurses are still (a) struggling with using EBP and (b) practicing in environments that are non-conducive to EBP despite longstanding recommendations for EBP use in nursing education and nursing practice [59].
EBP competence
The effect size reported in this pilot study supported the intervention group had greater EBP competence scores with a small to borderline medium effect (i.e., Hedges' g=0.47). This observed effect size is consistent with intervention studies that have used the Fresno test or a revised version of the original Fresno as the instrument for measuring changes in knowledge and skills [60-63]. However, these reported effect sizes were interpreted from EBP competence studies in disciplines other than nursing that used alternative approaches to EBP training, over variable periods of time. Consequently, the small to baseline medium effect size reported in this study is the first reported for an intervention study focused on EBP competence in pediatric nursing.
EBP beliefs
There were two key occurrences in this study with respect to EBP beliefs. The first occurrence was nurses in the intervention group, who began this study with weaker EBP beliefs reported stronger EBP beliefs post-intervention. These results are consistent with other studies that used the EBPB scale pre- and post-intervention and found EBP beliefs can increase with educational interventions [64-66]. The second occurrence was nurses in the attention control group, who began this study with stronger EBP beliefs than the intervention group, experienced a substantial decrease in their EBP beliefs postintervention. These findings were similar to a pilot study conducted to evaluate the preliminary effects of implementing an EBP model on several variables (i.e., ARCC model) [64]. The authors provided no explanation about this decline in their article. However, for this study, the fact that the attention control group demonstrated a noticeable decline in their beliefs about EBP suggests other factors may have influenced their perceptions toward EBP. For example, the adapted Fresno test deals with practical application of EBP principles. It is possible that in completing the adapted Fresno test they recognized their limited understanding of EBP and consequently, this had a negative effect on their EBP beliefs. Qualitative data in this study supported this assumption as nurses, particularly in the attention control group, expressed frustration after taking the adapted Fresno test.
Feasibility, acceptability and usability
The Web-based EBP educational intervention implemented in this pilot study was found to be feasible. However, acceptability and usability factors showed the content and delivery of the intervention must be addressed prior to conducting a larger study. For example, some nurses reported the content was too much to absorb, or the module needed to be more interactive, and more emphasis was needed on complex content. There were also unfavorable feedback about the Respondus Download Browser and the 1 hour time limit for testing. However, the majority of nurses in this study identified the Web-based approach to learning about EBP as an effective method and these findings support the use of Web-based technology for the education of busy working nurses [41-49].
This pilot study was a novel and useful attempt to assess and refine a Web-based EBP educational approach for improving EBP competence in BSN-prepared pediatric bedside nurses. The preliminary data from this study demonstrated the Web-based approach for EBP education (a) was feasible for practicing nurses, (b) demonstrated a non-significant but borderline medium effect in improving EBP competence, and (c) demonstrated a statistically significant and borderline medium effect in improving nurses' beliefs about EBP. Refinements of EBP knowledge and skills for BSN-prepared pediatric nurses are still needed to facilitate the acceleration of EBP competence in order to maximize patient outcomes and improve the quality of care. Additionally, creative strategies that promote a culture of clinical inquiry and motivate nurses to use the EBP resources available are warranted. Ultimately, improving EBP competence in this population will increase the capacity of pediatric nurses in achieving the IOM's 2020 goal of basing 90% of clinical decisions made on the best evidence.
The authors declare that they have no competing interests.
| Authors' contributions | NL | SC |
| Research concept and design | √ | -- |
| Collection and/or assembly of data | √ | -- |
| Data analysis and interpretation | √ | -- |
| Writing the article | √ | -- |
| Critical revision of the article | √ | √ |
| Final approval of article | √ | √ |
| Statistical analysis | √ | -- |
The author would like to acknowledge her (a) dissertation committee (i.e., Laura P. Kimble, Tanya Sudia-Robinson, Susan Codone, and Bernadette M. Melnyk), (b) managers and directors at the study site, (c) study participants, and funding sources that financed her doctoral training (i.e., Jonas Nurse Leader's Scholarship, Georgia Baptist College of Nursing that matched the funds from the Jonas Scholarship, and Nurse Faculty Loan Program).
Editors: Pamela gail Hawranik, Athabasca University, Canada.
Jean Ivey, The University of Alabama, USA.
Received: 23-May-2014 Final Revised: 12-Sep-2014
Accepted: 25-Sep-2014 Published: 01-Oct-2014
Copyright © 2015 Herbert Publications Limited. All rights reserved.
