A preliminary reliability study of a qualitative scoring system of limb alignment during single leg squat

Introduction

It has been proposed that functional movements, such as the single-leg squat (SLS), can be measured to assess predisposition to common degenerative and traumatic musculoskeletal injuries of the lower limb [1-4]. For example, evidence indicates that an excessive valgus angle at the knee during functional tasks is a risk factor for noncontact anterior cruciate ligament injury [1,5,6] and is also associated with overuse injuries such as patellofemoral pain [7]. Preventing a medial position of the knee is suggested to reduce the risk of ACL injuries [8,9] and forms an integral component of ACL rehabilitation through neuromuscular training interventions [10]. A knee-medial tofoot position, i.e., when the knee is not aligned over the ankle in the frontal plane, is related to an increased risk of anterior cruciate ligament (ACL) injury [11,12], and is related to poorer patient-reported function after knee injury [13].

Three dimensional motion capture is regarded as the gold standard for assessment of movement [2,14]. But, the use of these measurement methods to detect abnormal movement patterns during functional activities are expensive, time cons-uming and technically complicated, and difficult to repli-cate these types of movement analyses in the clinical setting [2,15]. A number of studies have attempted to undertake visual assessment from observation of video, which proved to be effective and pragmatic tools to measure and provide immediate feedback to the patient during the performance of functional tasks [16]. Recently, greater emphasis has been placed on visually analyzing movement patterns during functional tasks to identify candidates for knee injury prevention programs or neuromuscular control interventions during knee rehabilitation [17-20].

The SLS test is said to be a simple functional task and offers a safe clinical examination in comparison with single leg landing [8]. It also provides an attractive and clinically-efficient means of identifying undesirable movement patterns during screening and rehabilitation [21].

In recent years, there has been an increasing amount of literature on the visual evaluation of limb alignment including SLS tasks. These studies investigated the reliability of visual assessment of SLS or single limb mini squat tests or the small knee bend (SKB) which have disclosed inconsistent results (inter-observer =slight to excellent; intra-observer=fair to excellent) [22]. Ageberg et al., [10]; Weir et al., [23]; Crossley et al., [24]; Pousen et al., [25]; Örtqvist et al., [26]; Weeks et al., [21] and Whatman et al., [20,27] reported the use of a variety of protocols that included differences in the amount of knee flexion, foot position, arm position, head position and movement tempo. Some studies used a small box to squat from [24] or allowed finger tip balance [10]. The different protocols may present different challenges to neuromuscular control and result in different movement patterns that influence reliability. Therefore generalisation of reliability reported from studies is not appropriate unless the protocol conditions are similar.

To standardise the performance of SLS, several studies have relied on monitoring the amount of knee flexion [28,29]. This is likely to not be the major clinical concern compared to knee valgus for example. Furthermore, regulation of knee flexion angle is not likely to be achievable in the clinic due to the extra time and equipment required. Ageberg et al., [10] had participants looking at the position of the anterior aspect of their knee, relative to tape on the floor to try to standardise the amount of knee flexion. While eye focus on a target is a simple technique that can assist clinical use, and probably reliability (although this was not reported), it alters the natural trunk/head posture and consequently may well not be the most relevant assessment of movement quality/dynamic alignment [22]. Variation in agreement on rating is likely to be due to differences in the functional tests themselves as well as those rated by the population, the methods used for rating, variations in the amount of training and experiences for those responsible for rating, and differences in analysis. There is evidence that visual ratings are the most accurate at determining differences in 2D kinematics (pelvis and knee) [10,16,27].

There are several of studies that have attempted to undertake screening of lower limb movement using observational (video) analysis of knee alignment and control. But the inter and intrarater reliability and accuracy of visual rating of lower extremity movement quality, particularly of various body segments in subjects with current or potential musculoskeletal disorders, has still not been well defined [22].

The objective of this paper is to assess the inter- and intra-observer reliability of the new assessment tool, to determine if it shows similar or better reliability than other qualitative assessment methods of limb alignment during SLS tasks. The aim of this study is to test the hypothesis that the qualitative scoring system of limb alignment during single leg squat will show excellent inter- and intraobserver reliability as evidenced by minimal differences in scores between examiners and during the session. If the test proves to have strong reliability then it gives the clinician and researcher another testing option when looking for methods to assess lower limb alignment control, muscle strength and endurance.

Methods

Subjects
Four observers, all expert musculoskeletal physiotherapists (PhD or Masters qualified and all senior physiotherapist with an average of 12 years clinical experience) independently viewed and scored 4 recorded videos of the performance of the SLS test. All the participants videoed were free from lower limb, pelvis or spinal injury and gave informed consent to participate in the study which was approved by the university research ethics committee. The participant group comprised of two male and two female subjects all recruited from the postgraduate sports science course (mean age 25.6+/-1.3years; height 1.76+/-.18m; weight 78.6+/-10.1kg), who were all physically active participating in a minimum of 3 hours or more of aerobic exercise per week.

Visual assessment procedure
All ratings were recorded on a standardised rating sheet specifically designed for this study (Figure 1). The independent assessors received a CD ROM containing the video clips to be rated. The participants used their dominant leg only (based on preferred kicking leg), because the use of the contralateral knee as a reference may be inappropriate, as strength deficits are also seen contra-laterally following a knee joint injury [30]. All individuals performed the SLS test, they wore their comfortable sport shoes, with their arms held relaxed by their side and wore a tight fitting sleeve-less shirt (rolled up to expose their lower trunk/upper pelvis) and a pair of tight fitting shorts. All individuals on the video were given standardised verbal instructions prior to each test and the researcher demonstrated the test in a standardised manner. Stensrud et al., [16] demonstrated that a two-dimensional video analysis method has been shown to successfully screen subjects at increased risk of ACL injury arising from large valgus motions and kappa showed good to excellent agreement between 2D video analysis and subjective assessment for SLS. Therefore, previous work of Herrington et al., [31], was chosen as a method of capturing 2D knee motion by using a digital video camera (model DCR-HC40; Sony Electronics, Inc., Oradell, NJ, USA) that was positioned on a tripod with height set to equate with the participant's waist. It was positioned approximately 2 meters away in front of the subject to record a frontal plane view of the SLS. The image was adjusted so that the subject was visible in at least two thirds of the viewing area when the person was in a neutral standing position. The participants were unaware of what was being assessed during the test. All participants were allowed a maximum of three practice attempts. In accordance with previous investigations [21,24,32], squat depth was not standardised in keeping with an approach that most closely resembles clinical practice.

Qualitative assessment tool
The qualitative analysis of single leg squat (QASLS) is a new scoring system designed to identify segmental sub optimal behaviour following performance of a single leg squat. A qualitative scoring system was devised by one of authors (LH) based on the previously reported scoring systems of Crossley et al., [24] and Whatman et al., [27]. It involved dichotomous scoring of the movement strategy occurring in individual body regions (arm, trunk, pelvis, thigh, knee, foot). Scoring was defined as a zero for appropriate strategy and one for inappropriate movements, for each region with best overall score being 0 and worst 10 points. The scoring sheet is shown in Figure 1. The qualitative scoring system used was based on those previously reported in the literature which had attempted to analyse single leg squat and had shown good to excellent intra and inter tester reliability [24,27]. The scheme incorporated the region criteria similar to that used by both Crossley et al., [24] and Whatman et al., [27], following the assertion from both Chmielewski et al., [3], Onate et al., [33] and Whatman et al., [27] that this increased content validity. The scheme used was modified from those studies to also take into account trunk and pelvis motion which Crossley et al., [24] and Whatman et al., [27] regarded as significant factors in the alteration of lower limb alignment and load. Similarly, a dichotomous scale was used when classifying motion within each of the regions which has been shown to increase reliability by Whatman et al., [27].

The QASLS scoring system is a segmental method of analysing and a set of tests to rate the single leg loading specifically, which focus on the knee impairments (ACL injury and prevention, ACL rehabilitation and control through neuromuscular training interventions, and patellofemoral pain) [10-12]. Therefore it is not a global balance test as the Berg Balance Scale [34].

Movement analysis is subdivided into six categories-arm strategy, trunk alignment, pelvic plane, thigh motion, knee position and steady stance. Pelvic plane, thigh motion, knee position and steady stance each have two performance points. One point is given for each sub-optimal behaviour that the patient demonstrates. The patient is scored between 0-10, with a higher score indicating a higher risk of injury or a poorer performance. The QASLS scoring sheet is provided in Figure 1.

Statistical analysis
Four observers independently viewed and scored 2 female & 2 male subjects' recorded video performance of the SLS test. For the scoring performance, in each film only the frontal plane view was viewed three times at normal speed and the score then marked. The investigator compared the scores and analysed the scores for each participant. The four observers then re-examined the same videos one month later, blinded to the original scores. The scores were analysed for percentage of agreement (PA) [PA=(agreed/agreed + disagreed)x100] and Cohen's Kappa for both inter- and intraobserver reliability. Calculation of Cohen's

k = Pr(a) – Pr(e) / 1 – Pr (e)

Where Pr(a) represents the actual observed agreement, and Pr(e) represents chance agreement, using the observed data to calculate the probabilities of each observer randomly saying each category. If the observers are in complete agreement then k=1. If there is no agreement among the observers other than what would be expected by chance (as defined by Pr(e), k=0 [35].

Theoretically, the confidence intervals are represented by subtracting the kappa from the value of the desired CI level times the standard error of kappa. Given that the most frequent value desired is 95%, the formula uses 1.96 as the constant by which the standard error of kappa (SEκ) is multiplied. The formula for a confidence interval is:

Κ - 1.96 x SEκ to k + 1.96 x SEκ [35]

The level of inter-observer agreement based on initial ratings only. The overall percentage agreement and the kappa coefficient were used in this study due to this categorical data. Based on a scale proposed by Landis and Koch [36]: 0.01-0.20=slight; 0.21-0.40=fair; 0.41-0.60=moderate; 0.61-0.80=good/substantial; 0.81-1.0=almost perfect/excellent. Statistical analyses were performed using the Statistical Package for the Social Sciences (SPSS) version 20.0 for Windows (SPSS, Chicago, IL, USA).

Results

Overall
Inter-observer
Average percentage exact agreement (PA) between the four observers across all scoring criteria for all subjects was excellent (range 83-100%) (Table 2). All observers were in absolute 100% agreement in 5 out of 10 of all of the scoring criteria of all subjects. The kappa measure of Agreement ranged from 0.63 to 1.00 which is good to almost perfect (Table 2). In three of the scoring criteria the observers disagreed by a single point once on Q1, Q7 & Q8, and disagreed by two points on Q2 and disagreed by four points on Q3.

Male subjects
Inter-observer
Average percentage exact agreement (PA) between the four observers across all scoring criteria for male subjects was which good to almost perfect (range 80-100%). All observers were in absolute 100% agreement in 8 out of 10 of all of the scoring criteria of male subjects. In one of the scoring criteria the observers disagreed by a single point on Q7 and three points on Q3. The kappa measure of agreement was ranged from 0.62 to 1.00 which is substantial to excellent (Table 3).

Intra-observer
The average PA for observer 3 across the two viewing sessions was 100% for male subjects with kappa measure of agreement was ranged from 0.80 to 1.00 which is good to almost perfect agreement (Table 1) For observers 1, 2 & 4 there were a 95.0%, 90.0 % & 95.0% respectively average PA, the kappa measure of agreement were k=0.89, 0.80 & 0.90 which is very good/excellent (Table 1).

Female subjects
Inter-observer
Average percentage of the exact agreement (PA) between the four observers across all scoring criteria for female subjects was 88.0% (range 70-100%) (Table 3). All observers were in absolute 100% agreement in 6 out of 10 of all the scoring criteria of female subjects. In three of the scoring criteria the observers disagreed by one point once on Q1, Q3 & Q8 and two points on Q2. The kappa measure of agreement with values of k=0.48-1.00 which is moderate to almost perfect across all tests (Table 3).

Intra-observer
The average PA for observers 2 & 3 across the two viewing sessions was 100% for female subjects with kappa measure of agreement was k=1.0 for observer 3 only which is excellent agreement, for observers 1&4 it were 95% disagreeing on two points for subject two Q1 & Q6 respectively. The kappa measure of agreement for observers 1, 2 & 4 were k=0.90 for each, which is very good/excellent across all tests (Table 1).

Discussion

The objective of this study was to assess the inter- and intraobserver reliability of the new assessment tool, to determine if it shows similar reliability to other qualitative assessment methods of SLS tests.

The level of inter-observer agreement achieved by the observers was good to excellent in total, PA=83 to 1.0% and kappa coefficient ranged from K=0.63 to 1.0, but in males it is greater than in females. This findings are greater than most related studies testing the SLS performance visually [20,21,23,25-27] and only agreed with a study conducted by Ageberg et al., [10] who reported high level of agreement PA=96% with K=0.92. The study conducted by Ageberg et al., [10] provided explicit guidelines and training for the two experienced physiotherapists who on a single occasion rated 25 healthy subjects (18-37 yrs) on their medio-lateral knee position during a single limb mini squat. However they used a simple dichotomous rating scale of one body segment and anatomical references (knee relative to foot). All these factors and the experienced nature of the small number of physiotherapists probably helped to achieve the high level of agreement.

Mean intra-observer agreement was almost perfect for all physiotherapists. This was reached without teachings or training, using a evaluation system prepared to reflect recent clinical practice, suggesting this level of agreement is reachable in the clinic [22] and considered adequate for clinical use (≥0.75) [37], when assessed with a 10-point visual evaluation score. Although not easy to compare, due to differences in purpose and analysis [22] the level of intra-observer reliability of the qualitative scale measures in this study was excellent. This is higher than the previous studies including those from Chmielewski et al., [3]; Weir et al., [23] and Örtqvist et al., [26] that used visual scales of SLS test for healthy subjects. Chmielewski et al., [3] used a similar segmental rating method (with less detailed criteria and including a rating of segment oscillation) and reported a lower agreement (κ=0.35 to 0.53, 32% to 48% agreement) for visual ratings for movement quality during a unilateral squat task. These ratings were made 10 weeks apart. This study's results were comparable to other studies that used different visual measures to assess the SLS which reported acceptable levels of reliability such as: The highest agreement (κ=0.61 to 0.80, 73-87% agreement) was reported recently by Crossley et al., [24] in a study using experienced physiotherapists, where example ratings were provided as training prior to repeat ratings made one week apart. Providing examples and training in rating have probably contributed to the substantial agreement reported. As the authors referred to digital images it was unclear whether the ratings were made from still images or videos and how many times the videos were viewed. Nevertheless the rating method used involved a relatively complex evaluation of the trunk, pelvis and knee in a manner similar to common clinical practice. This study also showed that two physiotherapists with musculoskeletal postgraduate qualifications and more experience, achieved a higher agreement than a graduate physiotherapist. This influence of experience on intra-observing agreement was also reported by Pousen et al., [25] intraobserver ranged from 0.38 to 0.94 when determined through the generalized quadratically weighed kappa coefficient. Whatman et al., [20] used a similar segmental method to rate a range of movements (SKB, single leg SKB, lunge, hop lunge) over three to four weeks, 33 physiotherapists showed a wide range of intra-rater agreement (AC1=0.01 to 0.96). Our finding is in agreement with Weeks and colleagues [21] who demonstrated similarity in ratings between observers for the SLS test, showing that intra-observer reliability was excellent for physiotherapists (ICC=0.81). Whatman et al., [27] reported similar agreement but with more variation in a group of 26 physiotherapists (without additional training) rating young athletes (AC1=0.14 to 0.92). Agreement in these studies when rating children and young athletes was similar to that achieved with adults intra-observer PA: 79% to 88%, AC1=0.60 to 0.78.

Finally, our results suggest that clinicians and researchers can use the qualitative scoring system of limb alignment during single leg squat test with confidence to identify undesirable movement patterns, at least in generally healthy individuals. Although, the value of this kind of visual assessment tools from observation of videos going to increase due to the variety and availability of monitor devices like; computer screen, martphone, tablet etc. The variation of these video screen sizes and pixel density may present different challenges to judge a performance and result in non visible movement patterns that influence reliability.

Limitations
The major limitations of this study were the rating of small number of healthy subjects and that ratings were made via video, from an anterior view only. Most use of movement assessment in the clinical setting is likely to be through watching patients move in a live situation. Therefore, the implications for reliability of movement in the clinical setting remain unknown. Despite this limitation, using video images was the only method that could limit the introduction of error that might occur due to variation in what was being assessed by the observer. Exclusion of participants with lower limb pathology may be considered a limitation.

Further studies are required to assess the use of these tests in identifying readiness for return to sport and progress during rehabilitation with injured subjects determining the value of this clinical measure. Thus, additional studies are needed for generalisation to people with musculoskeletal pathology and injury.

In conclusion, the test is feasible and easy to administer in the clinical setting and in research to address lower extremity movement quality. However, both intra-observer and inter-observer reliability of the qualitative scale measures successfully exceeded levels necessary for application of this measurement method in the clinical setting and research.

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

Acknowledgement

The authors gratefully acknowledge the physiotherapists and student participants at Salford university for sharing their time with us.

Publication history

Editor: Giorgio Ferriero, Institute of Care and Research, Italy.
Received: 24-Feb-2014 Final Revised: 12-May-2014
Accepted: 21-May-2014 Published: 31-May-2014