Simulation as a Learning and Evaluation Modality: A Concept Analysis

Simulation as a Learning and Evaluation Methodology: A Concept Analysis
Simulation as a teaching-learning modality in nursing education has been increasing over the past several years [5]. Most programs use simulation as a learning modality; however, some are beginning to use it as an evaluation method as well [18]. There are currently two conceptual definitions in the literature regarding simulation, one focuses on simulation as a learning methodology while the other focuses on simulation as a summative evaluation methodology. Minimalizing formative evaluation in the conceptual definition of simulation in nursing education can give rise to confusion and inconsistencies in research [23]. It is vital to address this issue to close the gap in knowledge and practice.This report will offer a new definition of the concept of simulation based on literature exploring simulation as a learning methodology and as a formative and summative evaluation method. The Walker and Avant [38] modified version of Wilson’s [39] concept analysis plan was used to guide this analysis.

Background

Nursing educators are tasked with providing students a clinical experience that will enhance the application of theoretical knowledge learned in the classroom [30]. The current health care system is extremely complex and continually evolving to meet patient needs [42]. This presents many challenges for nursing education. Clinical sites and hours available to practice are limited, increased security provides charting access challenges, and there is a shortage in nursing faculty [10]. These constrained clinical opportunities coupled with shortened patient stays limit student practice with real patient care situations which can affect the students’ ability to cultivate clinical competence and capacity to care for patients as well as apply theoretical knowledge learned in the classroom [41].

In nursing education, simulation is an active learning process in which students usually participate in groups to provide care for a patient. This may occur in the form of a manikin, actor, or standardized patient [17]. A variety of manikins exist for nursing education simulation purposes, they range from simple task trainers to medium-and high-fidelity manikins [16]. The level of fidelity must be matched with the learning goals of the students [5]. Students take on a designated role in an evolving case study to address issues and learn first-hand the effects of their decisions [27]. Immediately following the simulation activity, a period of debriefing takes place. The debriefing period consists of a structured discussion led by the facilitator to guide students in self-reflection and analysis of the experience; this is an integral part of the learning process [5].

Simulation provides an opportunity for students to bridge the gap between classroom knowledge and clinical practice in a safe and realistic learning environment [17]. Providing a realistic environment in which there is no impact or threat to actual patients reduces student anxiety about causing patient harm [21] and creates a safe environment where students can concentrate efforts on personal learning [25]. The armed services have used simulation for many years as a method of teaching “students complex skills and build team and individual knowledge” [7]. Simulation allows students to actively engage in unpredicitable, time-sensitive, and challenging clinical situations, rather than just act in the role of a passive observer which occurs in many real-life situations. This allows students to learn from mistakes without patient harm, practice skills, participate in interprofessional communication and teamwork, and provide the opportunity for formative and summative assessments [19].

Review

Significance to Nursing Education
Simulation as a part of the nursing curriculum is becoming more and more common [30]. In a recent study, the National Council of State Boards of Nursing (NCSBN) found that over 900 nursing programs in the United States were using medium- to high-fidelity patient manikins in the curriculum [13]. Simulation is considered a recommended teaching-learning strategy in nursing education to aid in the development of clinical skill [37].

Simulation has shown to have positive effects on nursing student confidence and self-efficacy [18]. Chiang and Chan [6] found that simulation led to a significant increase in student critical thinking and another study found an increase in student knowledge acquisition and retention after a simulation experience [2]. Other studies have found inconclusive evidence regarding simulation and student clinical judgment, warranting further investigation. In addition to serving as a teaching-learning methodology, simulation is also being used as both a formative and summative evaluation method [3,18].

It is important for simulation facilitators to be well versed in simulation as a learning modality. If a facilitator is unaware of the current standards of practice, the simulation experience may not provide an adequate learning opportunity for students. Therefore, the rigor of simulation must be upheld in order for students to benefit from this form of instruction [13]. Unfortunately, this is not seen across the United States in every program that utilizes simulation as a learning methodology [28]. The current standards for simulation experiences are set forth by the International Nursing Association for Clinical Simulation and Learning [14]. If facilitators are not versed in this educational modality, students may not have the optimal experience. It is often not reported in studies if these standards are being met.

Consequences
The effects of an incomplete conceptual definition of simulation can be seen throughout the literature. In the absence of a complete definition, a lack of rigor and objective evaluation in the research regarding simulation has been observed [20]. Yuan, Williams, and Fang [41] claim learning outcomes of simulation research are inconsistent and vary in rigor and focus. Moreover, this concern also applies to the small but growing set of studies using simulation as a means of evaluation. There is also a lack of simulation training for faculty and staff [28], which is required to meet the INASCL standards of best practice. It is vital that nursing research regarding simulation education practices be reliable and valid. There is a current expectation for schools of nursing to design curricula from evidence-based educational modalities [15]. The education of nurses needs to be informed by evidence just as clinical medicine is informed by evidence [8].

Due to the confusion and toll this incomplete definition has taken on simulation research, action needs to be taken. Meleis [23] argues that a concept analysis is an excellent strategy for developing concepts. Walker and Avant, [38] agree with this thought, claiming concept analyses can be used to refine the concepts in a theory. Once a concept analysis is completed, it cannot be considered the end of the road [4], as knowledge is fluid and ever expanding. Both concepts and attitudes may change over time, thus requiring a new concept analysis [38]. Based upon these arguments, a new concept analysis was needed for simulation as a learning and evaluation methodology.

Study
Aim
The need for a broader definition of simulation warrants a review of the concept. The gap created by this definition can lead to confusion [23] and insufficient use of simulation in nursing curricula. The aim of this analysis is to explore, clarify, and define the concept of simulation as an evaluation method. The goals of this concept analysis are to better enhance simulation in nursing education as learning strategy, evaluation method, and thereby improve nursing education and research in this area.

Method
The phenomenon of simulation as an evaluative method needs to be clarified and redefined to encompass its many elements. While implementing this undertaking, it is important to include the philosophical basis throughout the project in order to assess the current knowledge of simulation as an evaluative method and construct new and improved concepts to clarify the definition [26].

This concept analysis was accomplished utilizing the Walker and Avant [38] modified version of Wilson’s [39] classic concept analysis procedure. The goals of this concept analysis are to better enable simulation in nursing education as learning strategy, evaluation method, and thereby improve nursing education and research. Walker and Avant’s modified version of Wilson’s method of concept analysis was utilized. In Wilson’s classic method, eleven steps of the process are outlined. This modified version is found to be more user-friendly and appropriate for nursing [23]. Walker and Avant have modified engaging in a concept analysis to eight steps that adequately applies the process.These steps include the selection of a concept, determining the reason for analysis, identifying all uses of the concept, determining the defining attributes, finding a model case as well as borderline, related, contrary, invented and illegitimate cases, identifying antecedents and consequences, and defining the empirical referents [38].

This author performed the concept analysis under the historicist view of science, specifically utilizing Toulmin’s idea of science. In order to gain understanding of scientific activity, historicism classically places greater importance on the processes and contexts utilized. Historicism proclaims that in order to fully understand a concept or phenomenon, one must first consider its place in historical development [26]. Tolumin takes this a step further and places great emphasis on concepts and the continuing expansion of knowledge and science [36].

Data sources
A broad approach was utilized to complete a thorough search in order to explore the literature fully and identify all uses of the concept and not limit the outcome [38]. A computer-assisted search was conducted using multiple databases including CINAHL, OVID, ERIC, ProQuest, and Cochrane. Nursing and education books, dictionaries, and thesauruses were also reviewed to determine the nature of simulation. The search terms nurse or nursing combined with assess or evaluate and simulation and outcomes were used to distinguish studies relevant to the aim of the review. Exclusion criteria included studies that were not in English, published before 2010, or dissertations. Studies were also excluded if they used case study, role-play simulation, simulated interviews, or only examined student confidence or satisfaction. The terms nurse or nursing were also removed to include definitions from other disciplines.

Primary studies addressing simulation as an evaluation method were included. This comprised of experimental, quasi-experimental, descriptive, qualitative, and mixedmethods studies. Meta-analyses and systematic reviews were also included. The participants of the studies included in the analysis comprised of nursing students who participated in a simulation experience with human patient simulators.

Results

The first two phases of concept analysis, selecting a concept and determining the reason for analysis were completed simultaneously. Simulation in nursing education was chosen after the tasks of soul-searching and a review of the literature were completed by the author to find a topic that was interesting, important, and manageable, as Walker and Avant [38] suggest. Based on the review of the current literature, the aim of the analysis was determined to be performing a concept analysis that describes simulation as a teaching-learning strategy and a summative and formative evaluation method. This is vital to keep the analysis on track and focused [38].

Uses of the concept
This analysis is attempting to define evaluation through simulation; therefore, both the concepts of simulation and evaluation will be discussed. It is important to identify all uses of the concept available [38]. Examining the concepts from the nursing, medical, educational, military, and general perspectives provided a diverse base. The search for uses of simulation began with dictionary definitions. Some general definitions of simulation include “to imitate or reproduce the appearance, character, or condition” [31] or “the act or process of pretending” [32]. Medically, simulation has been referred to as “imitation of symptoms of one disease by another” [34] and “a reproduction or representation” [33]. To gain perspective from the military, Page and Smith state “modeling and simulation refers to the use of models, including emulators, prototypes, simulators, stimulators, either statically or over time, to develop data as a basis for making managerial or technical decisions. The terms “modeling” and “simulation” are often used interchangeably” [24].

Turning to textbooks and the available literature provides a more concise definition of simulation in the realm of nursing education. Jeffries [16] describes simulation as “a student or group of students providing care for a patient who is represented by a manikin, an actor, or an SP (standardized patient), depending on the clinical situation” and an activity “observed by faculty” (p. 3). Caputi [5] claims simulation is a “technique or device that attempts to create a realistic representation of the real world” (p. 30). While descriptive, these definitions do not fully explain the concept of simulation. Hansen and Bratt [12] define “competence acquisition in (simulated learning experiences)… as a summative student demonstration of essential behaviors, safety, clinical judgment, knowledge application, and psychomotor skill occurring in a structured, controlled, and authentic environment, as evaluated by a trained instructor based on predetermined competency objectives” (p. 106). This definition only looks at evaluating simulation in a summative manner.

Identifying the use of the concept of evaluation occurred in the same fashion as simulation. Evaluation is defined as the process of judging “the value or condition of (someone or something) in a careful and thoughtful way” [11]. Taber’s [34] describes evaluation as a judgment and the final step in the nursing process, which can be formative or summative [5].

Examining the literature provided further clarification of the use of evaluation. Hansen and Bratt [12] define evaluation as the “outcome of the assessment process” (p. 104), while Billings and Halstead [3] describe it as “next level of judging the value and quality of performance at a defined end point” (p. 441). Another definition offered by McDonald [22] provides guidance to nurse educators stating that evaluation is “a value judgment that attaches meaning to the data obtained by measurement and gathered through assessment. It is guided by professional judgment and involves interpreting what the accumulated information means and how it can be used” (p. 13). Formative evaluation is an on-going process throughout the learning experience and allows the instructor to identify where the student is strong and weak. Summative evaluation occurs at the end of a formal learning experience, such as at the end of a semester or just prior to graduation [5].

Defining attributes
Walker and Avant [38] describe determining the defining attributes as the core of an analysis. Several characteristics were identified with both simulation and evaluation that formed themes across the available resources. These are the fewest possible attributes that clearly depict the concept [38]. The defining attributes identified regarding simulation include: a realistic situation, active participation, reflection, and trained facilitators. The defining attributes identified regarding evaluation include: demonstration, assessment, and objectives.

Providing a realistic learning environment for students emerged as a defining attribute of simulation. Tubaishat and Tawalbeh [35] describe a realistic environment as a valuable aspect of the simulation experience. This is partly due to the fact that a realistic learning environment can supplement an inadequate clinical experience [42]. A realistic environment in a simulation experiences provides the opportunity for students to actively participate in a situation they may not be legally, physically, or logisticallyable to in clinical with real patients[5]. A realistic environment also allows students to assess, intervene, and evaluate patient outcomes [30].

Active participation is also necessary in a simulation experience. Students must be interactive in the experience to enhance knowledge and skills [10]. Simulations provide active experiential learning experiences, which enhance knowledge gained through didactic lecture or written materials [5]. In order for simulation experiences to be successful, students must interact and engage with the learning activity first hand [4], which can better prepare students for clinical practice [25].

Reflection, usually in the form of debriefing, is another defining attribute of simulation. Reflection and support can facilitate learning [29]. Engaging in self-reflection in a nonthreatening environment allows students to maximize the effects of learning [1]. Debriefing is often looked upon as one of the most important aspects of simulation, and where the majority of learning occurs.It aids in constructing confidence and should be included in every simulation experience [17].

Appropriately trained staff was the final defining attribute of simulation that emerged. Trained faculty and standardization of simulation is essential to producing high-quality simulation experiences [28].

Demonstration is the act of showing how to do something [3] and necessary in evaluation through simulation [12]. It is a tool of the evaluation process [22]. Visibly demonstrating a skill or process aids in retention [3]. The demonstration of skills leads to competency-based nursing curriculum [40].

Assessment and evaluation are two separate entitles [22]. Assessment is understood to be the process of evaluation [12] or the act of gathering information with a specific purpose in mind [3]. Billings and Halstead state the main purpose of assessment is to understand and improve student learning. Assessments are the building blocks of evaluation [22].

The final defining attribute identified with evaluation is the presence of objectives. Clear objectives are a vital aspect of evaluation [17]. Objectives allow the student to understand what is expected of them so they can strive for success [3]. The instructional process from learning to assessment and evaluation are guided by the clearly defined learning objectives [22]. The objectives of an evaluation will determine if it is formative or summative.

Model case
Meleis [23] states that a model case is an example of what is, most certainly, the concept. These cases may be found in research or in reality. However, the author may also construct a model case for the purpose of clarifying the concept [38]. For this concept analysis, the author constructed two fictional model cases. The example of a case is one in which students participate in a simulation experience for evaluation purposes follows these guidelines mid-semester as a formative evaluation experience. The second case is one in which students participate in a simulation experience for evaluation purposes follows these guidelines at the end of the semester as a summative evaluation experience.

After learning about the concept of stroke evaluation and care in a lecture class, the students engage in a simulation regarding that topic. They enter the simulation center and receive pre-briefing and report on a patient. Once the students enter the patient room, they perform a patient assessment and recognize the signs and symptoms of a stroke. The students then contact the doctor and recommend the appropriate tests. After receiving the test results, students once again contact the doctor to obtain medication orders. They then administer the medication and provide instruction to the patient and family.

While this is occurring, the instructor trained in simulation education is observing the student behavior and as they concurrently assess a list of pre-determined objectives the students should follow. They are concurrently assessing the student performance as well using a valid and reliable tool. Upon completion of the scenario, the students and instructor engage in debriefing of the scenario to reinforce learning and discuss mistakes. The debriefing process should last at least half as long as the simulation scenario and be facilitated by a person competent in the field of debriefing. The process and questions should be based on a theoretical framework and structured to be in agreement with the objectives and outcomes of the simulation experience (INACSL, 2016).

Other cases
Walker and Avant [38] state it is important to identify other cases as well, such as borderline or contrary cases. A borderline case is one that contains most of the defining attributes [38]. An example of this would be a scenario as described in the model case, except the students do not engage in reflection at the end of the experience. Reflection is an integral step in simulation, and a scenario that does not include this aspect cannot be considered a simulation experience [16].

According to Walker and Avant [38], a contrary case is a scenario that is a clear depiction of what the concept is not. A contrary case for evaluation through simulation would be considered a simulation experience in which the students receive continuous feedback and guidance throughout the simulation. Student performance or knowledge is not evaluated throughout or at the completion of the simulation.

Antecedents and consequences
The next step identified by Walker and Avant [38] is recognizing antecedents and consequences of the concept. Antecedents are defined as “those events or incidents that must occur or be in place prior to the occurrence of the concept” [38]. While performing the concept analysis of simulation as an evaluation method, several antecedents were identified. They include the suspension of disbelief, an open and safe environment, student and faculty investment in simulation, faculty training, clearly defined objectives and student learning outcomes as well as a structured grading rubric.

Consequences are defined as “those events or incidents that occur as a result of the occurrence of the concept [38]. Several consequences were identified while performing the concept analysis. They include confidence, identification of student competence or the need for remediation, and the transfer of knowledge.

Empirical referents
The final step in Walker and Avant’s [38] concept analysis strategy is defining the empirical referents. Empirical referents encompass phenomena that demonstrate the occurrence of the concept [38]. The presence of simulation evaluation tools and student gains in knowledge, confidence, and satisfaction provide recognition to the analyst that evaluation through simulation did occur.

Definition
Based upon the concept analysis performed, the author of this paper presents the following definition of evaluation of nursing students through simulation:

Simulation as an evaluation method is a process ininvolving a realistic patient scenario and safe environment provided by an instructor trained in simulation education, in which students participate in active learning through demonstration of patient care and reflection. While the student is participating in the simulation, the trained instructor is observing and assessing student performance of cognition, psychomotor skills, clinical judgment, or critical thinking or any combination of these elements based on pre-determined objectives and using a valid and reliable tool for formative or summative evaluation.

This new definition allows nursing educators to utilize simulation as a teaching methodology and an evaluation method while highlighting the need for adequate simulation training. Including evaluation allows the educator to appropriately assess the students’ learning outcomes in the simulation experience and in the overall course.

Arguments
Although there is a wealth of information regarding simulation, most of it does not examine student learning outcomes [18]. Although some researchers are now beginning to explore the realm of student outcomes. For example, Shin, et al. [30] state simulation improves learning outcomes in nursing students. Some nursing educators are now using simulation to evaluate learning outcomes as well [3]. Despite overall student satisfaction with simulation as a learning modality, Schlairet [29] discovered students expressed lower enthusiasm of simulation when it was used as an evaluation method. Mc- Donald [22] states that testing is not an appropriate time to teach. Evaluation through simulation combines learning and evaluation, which may be unfair to students. This increased anxiety demonstrates that educators need to express exact expectations to students and state if the simulation is to be a formative or summative assessment or a learning activity.

Limitations
While this analysis presented a thorough search and proposed a new conceptual definition of simulation, some limitations of the analysis remain. The literature search was limited to articles and other sources written in English, published after 2010, and did not include dissertations. A new definition may have been excluded. The definition was primarily examined from a nursing perspective. This may have limited the richness of the meaning by not fully exploring the definition through other disciplines.

Conclusion

Simulation is a valuable teaching-learning modality in the clinical learning of nursing students [6] and a recommended component of curriculum [37]. While simulation is usually employed as a learning strategy, its use as an evaluation method is increasing [18]. This new use does not fit the current conceptual definition of simulation as set forth by Hansen and Bratt [12]. Incomplete conceptual definitions can give rise to confusion and inconsistency in research [38]. This can be seen by the call for further research [9] that is found to be valid and reliable [20]. To address this issue, this author proposes a new definition after conducting a thorough concept analysis.

Competing interests

The author declares that he has no competing interests.

Publication history

Editor: David Reiss, Imperial College London, UK.
Received: 10-Dec-2019 Final Revised: 25-Jan-2020
Accepted: 29-Jan-2020 Published: 16-Feb-2020