Exploring the use of negative binomial regression modeling for pediatric peripheral intravenous catheterization

Introduction

For pediatric hospital patients in need of fluids quickly, successfully inserting a peripheral intravenous catheter, commonly known as an IV, is often a medical necessity and critical first step in treatment. The process involves a hollow, metal needle that is beveled at one end and sheathed in a plastic catheter tube [1]. Once the needle has been successfully placed in the lumen, or interior, of the vein, the catheter is pushed over the end of needle. The metal component is then removed from the vein, leaving the catheter in place. This process is necessary to infuse needed fluids and/or medications into the body while preventing swelling in the patient.

The process of IV placement is unique in that there are very few alternatives to it, so in virtually all cases the process must be performed until it is successful. For pediatric patients, the procedure can be particularly difficult and time consuming. Factors such as patient age and unfamiliarity with the process present difficulties in pediatric IV placement. It is not uncommon for the process to require several separate needle stick attempts for a single patient.

Previous studies have been conducted to research the factors that impact successful IV insertion. Gorelick [2] studied 615 children aged 0-21 years who underwent peripheral intravenous catheter placement by staff nurses in a pediatric emergency department. Information on predictor variables was obtained before attempting intravenous placement, as well as whether the outcome was successful on the first attempt. Backward stepwise logistic regression was used to identify factors independently predictive of success. The study concluded that patients exhibiting prematurity, under the age of 1, or between the ages of 1 and 2, could be assessed different difficulty scores.

Larsen [3] and Goff [4] studied 592 children ages 0-18 who underwent pediatric IV catheter insertions in the inpatient setting at two southeastern US teaching hospitals. Larsen [3] sought to identify the predictive variables for successful IV insertion. The data was analyzed using a combination of a logistic regression model for success on first attempt, repeated measures logistic regression across multiple stick attempts, and a proportional hazards model on time to success. Larsen's results indicated that shift (day versus night), health professional perception of difficulty of the IV insertion, and the level of cooperation of the child all significantly contributed to the odds of a higher than usual number of IV attempts.

The published regression models for the Larsen study may not have fully utilized all aspects of the data. Specifically, none of the aforementioned models considered the data under the notion of a counting process. It is possible that other effects could be discerned with a second analysis of the data. The data were truncated into success/not success bins for either the first stick attempt or the general overall process. This truncation does not fully take into account the structure of the collected data as a multiple attempt process. Furthermore, it is unclear whether the use of count regression modeling provides superior results from traditional ordinary least squares regression modeling. The goal of this paper is to determine the appropriateness of a negative binomial assumption and to use a negative binomial regression model to take a second look at identifying factors which lead to successful IV placement.

Using a negative binomial assumption
Negative binomial data models a process that will be repeated until there is a single or multiple successes. Hilbe [5] has provided a seminal text on dealing with negative binomial regression models, and states that the traditional negative binomial model "is a Poisson-gamma mixture model in which the gamma distribution is used to adjust the Poisson in the presence of overdispersion". Poisson data, which is typically used to model count data, assumes that the mean and variance are equal. Data with a variance that is greater than the mean is termed Poisson overdispersed data [5]. Indeed the negative binomial is often parameterized as the mixture model described above with mean μ and variance μ+μ²α, where μ is the mean from a Poisson process and α follows a gamma distribution. This parameterization is commonly referred to as the 'NB2' model, in contrast with the classic 'NB1' parameterization involving k failures until r successes with common probability of success p. Under this formulation the mean value, μ, can be represented as µ=pk/ (1-p). The negative binomial distribution can also be derived as a member of the single parameter exponential family of distributions, known as generalized linear models, or GLMs [5]. Specifically we will model E(Y|X)= μ=log (β₀+β₁ X₁+...+β_m X_m) for m unique predictors (either continuous covariates, indicator functions, or a mixture of the two) and some dispersion value so that the mean and variance relationship occurs as described above. By using the GLM interpretation, various Goodness of Fit tests and residual analyses can be applied to negative binomial data that cannot be well utilized used with the mixture model [5].

The NB2 alternative parameterization is likely popular due to the lack of real data that follow a true negative binomial distribution. A careful examination of Hilbe's text [5] finds four main examples that utilize negative binomial regression models studying the following outcomes: number of extramarital affairs, length of hospital stay of a sample of cardiac patients, Titanic survival data, and number of physician visits per year. None of these examples have an intuitive NB1 formulation in terms of a sequence of failures until 1 or more successes. Therefore instances with real data that follow this parameterization would be useful to individuals hoping to make advances in the theory of such models.

An important assumption regarding traditional negative binomial data is that each trial, or attempt, in the process is independent of the other trials. In other words, the chance of success for each trial must be the same, regardless of what happened during previous trials. It seems reasonable that pediatric catheter insertion data can be modeled using a negative binomial setting, since the process must be attempted until an IV is successfully in place. However, the idea of independence is counterintuitive to the clinical setting. According to Ogden-Grable [6], there is a hierarchy of preferential venipuncture sites, based on factors such as how well a vein is anchored, how prominent it is, and how easy it is to locate. Thus, certain veins present less challenges than others. Previous analysis indicates that the experience of the nurse also plays a role in whether or not a particular attempt is successful [3]. For many patients in this study, after the first two or three attempts failed, a more experienced nurse was usually brought in to attempt the following placement(s). Thus, the initialization scenario clinically changed from attempt to attempt. A common assumption may be that the probability of a success may also change based on whether the previous attempt was a failure.

Patient distress in IV placement
Patient comfort is of concern in IV placement, particularly for pediatric patients [7,8]. The site for the IV attempt is selected based not only on the chance of a successful IV start at the site, but with patient comfort and likelihood the IV will remain in place in mind as well.

During this study, information regarding pain assessment was collected via the Observational Scale of Behavioral Distress (OSBD) [9]. The OSBD, which is a modification of the Procedural Behavioral Rating System, is used to monitor pediatric patients' behavioral reactions to painful procedures. To create an OSBD score, 11 operationally-defined behaviors which indicate distress were observed and recorded. The categories were weighted, according to standard practices, from 1.0-4.0 [9]. OSBD were measured for each IV attempt and mean score over the entire process for each of these distress measures, OSBDM, was used for the analysis. Higher scores on OSBDM indicate higher levels of distress.

For this analysis patient comfort was considered secondary to IV insertion success. OSBDM is not treated as a direct variable that impacts IV success rates but rather as a potential confounder. This was done to account for the notion that there is often little that can be done in the clinical setting to directly alter a pediatric patient's comfort with the IV insertion process.

Methods

Checking the assumptions of a negative binomial model involved two steps: examining whether the data violated any independence assumption and verifying that the observed outcomes were a reasonable fit to a negative binomial distribution. To verify whether the assumption of independence was violated, binomial proportions, along with Clopper- Pearson exact 95% confidence intervals for each stick attempt were examined and compared under the assumption that there may be some common underlying probability of success that is irrespective of attempt. Clopper-Pearson exact confidence intervals were selected because a standard, conservative, exact test was desired that was readily available in standard software [10].

To verify that the observed data values were similar to the expected values under a negative binomial model, the current data was examined using a Goodness-of-Fit test as outlined by Morel and Neerchal [11]. Fitting the data to an intercept only negative binomial regression model, observed and expected counts were output and compared using Goodness of Fit criteria. Since negative binomial models include the possibility of observing a success on the zero count, a small change in the way the data was counted was necessary in order to truly compare the observations to the expectation [5]. For the process of IV insertion, at least one attempt to start an IV must be made. Therefore, rather than counting the actual number of the attempt, the data was modeled on the number of additional stick attempts beyond the first. Thus, those patients who experienced a successful IV placement on the first stick were represented as a success with zero additional stick attempts, or zero failures. For thoroughness, the data was fit under zero-inflated negative binomial and Poisson models as well.

After exploring the assumptions and utility of modeling the data under the negative binomial model, the results of the analysis indicated that the data itself should be modeled using the negative binomial distribution directly. Therefore, the GLM interpretation was be used to fit a negative binomial regression model. Those results were be compared to ordinary least squares (OLS) output in terms of inferences and model fit. It is worth noting that the use of negative binomial models is a fairly recent innovation and historically this type of count data would be modeled using OLS methods. It seems obvious that should an individual decide to model the data under a GLM, that the negative binomial would provide a more natural fit than other models, such as the Poisson. A seemingly appropriate question of interest is whether there are gains by modeling this data as a GLM in terms of model fit and general clinical inferences. Negative binomial regression uses a log-link function, making it difficult to directly compare parameter estimates with an OLS model. Hence, direct comparison to OLS parameters is not recommended. Instead, comparing the model based adjusted means (based on maximum likelihood estimates) for each factor was be preferable. Such comparison will indicated the impact for the factor, having adjusted for the contribution of other variables in the model. Comparisons of model fit were be made by examination of Akaike's Information Criterion (AIC) with smaller values purporting a better fitting model to the observed data [12]. All statistical analyses and simulations were performed using SAS^® version 9.4 [13]. Data cleaning and quality control evaluations were performed using JMP^® Pro 11 [14] software.

Simulation
While it is important to determine whether this data fit well under a negative binomial model, it is important to also assess and compare the choice of models with such data. The idea of a negative binomial generating process for this data seems intuitive, but it is unclear that there are gains from using a count regression framework over OLS modeling. A simulation study was conducted to examine the impact of modeling data generated from a negative binomial model to traditional OLS modeling. Here the main idea was to examine whether there would be gains in statistical power, model fit (via AIC), or estimating of model effects via model based adjusted means while simultaneously verifying good coverage in negative binomial models for data generated from a negative binomial process.

Since models would be compared mostly by model based adjusted means in the next section, we restricted our models to an instance where the covariates are binary. Three simulations were performed with samples of size 50, 200, and 600, respectively. Four binary variables were generated for the aforementioned sample sizes with a common correlation of 0.20. The binary variables were generated to each have a univariate distribution matching four of the covariates used in the models of the next section (shift, difficulty, cooperation, and dehydration). The negative binomial data was generated via the algorithm proposed by Hilbe [5].

1,000 Monte Carlo samples were run for each sample size and the results can be found in Table 1. There one can see the true model parameters and dispersion. Note that the negative binomial model attained appropriate coverage (for 95% confidence intervals) at samples of size 200 while samples of size 50 show somewhat weak coverage. Since the data was generated via a negative binomial model, there was no 'true' value with which to compare in OLS models. Power is measured by the proportion of parameter estimate 95% confidence intervals that do not contain zero; here we see that there were a high proportion of samples that did not contain zero for the n=600 simulations. It was interesting to see that the estimates of power were similar in the OLS and negative binomial models suggesting that for this scenario there was little gain in power by utilizing the GLM framework.

While power may not be impacted, there are vast differences in fit and estimates of effects; across the different sample sizes we found reductions in AIC ranging from 14.3–21.6%. Likewise, across all three sets of simulations we saw that the OLS models overestimated the model adjusted means. This is seen by looking at the difference in adjusted means and their associated confidence intervals, for which the OLS models were consistently higher across virtually all covariate levels and sample sizes. Note that both the n=200 and n=600 illustrated that the coverage of negative binomial parameter estimates were close to the truth, which imply that the estimates of adjusted means are also close to their true value. Thus the OLS values consistently overestimated the true values.

Results

The patients involved in the study had a mean age of 5.26 years (SD=5.73), with just over half (53.04%) of participants aged 2 or older. The mean weight was 24.75 kilograms (SD=25.36). A slight majority (55.57%) of participants were female. Most of the patients were either African American (47.80%) or Caucasian (42.05%). Overall, the mean number of sticks required for successful IV placement was 2.14, the median number of sticks was 2, and the variance in the number of sticks required was 2.35 (SD=1.53). It should be noted that the variance is not much higher than the mean, which indicates that there is not much overdispersion in the data (if viewed as a Poisson process). The majority of patients were observed during the day shift (58.5%); medical personnel assessment had assessed 30% of patients as difficult IV initiations and 53% of children as cooperative. 19% of children were listed as dehydrated upon admittance.

Binomial proportions, along with Clopper-Pearson exact 95% confidence intervals (Table 2, Figure 1) were calculated for each stick attempt, and then compared. It is important to note that once an observation experienced a success, that data point was excluded from the calculation of future proportions. Thus, the sample sizes vary widely among the stick attempts, with attempts greater than 5 having sample sizes that were less than 30. The analysis demonstrated that the confidence intervals for the proportion of success per stick attempt overlapped. The values between 42.14% and 43.18% were observed in every confidence interval.

Table 3 illustrates the observed frequencies of successes by attempt, as well as the expected frequencies under different models. The results seemed to show that the Poisson model is a poor fit for the data, while the negative binomial and zeroinflated negative binomial data appear to fit the data well. A Goodness of Fit test under the null hypothesis that the data fit a Poisson model yielded an observed chi-squared value of 273.9 (p<0.0001), confirming that the Poisson distribution is not appropriate for the data under consideration. By contrast, using a negative binomial model, a Goodness of Fit test yielded a Chi-Squared value of 9.80 (p=0.2002). So the assumption that the data is negative binomial would not be violated. Under the null hypothesis that the data fit a zero-inflated negative binomial distribution, a chi-squared value of 9.61 was observed (p=0.1421). While it is not appropriate to reject the null-hypothesis in this instance, the visual evidence does not clearly support the use of the zero-inflated negative binomial model over the use of the standard negative binomial model (see Table 3). To illustrate this information further, the observed data was graphed against the expected data of the negative binomial model. (See Figure 2) It is clear from the graph that the data fit the negative binomial assumption well.

Most (58.45%) of the procedures occurred during the day shift. Researchers observed IV initiations and recorded a battery of information at each stick attempt including: experience and competence of the health professional attempting the IV initiation, time of the attempt, perceived cooperation and difficulty at each attempt, and whether the attempt was successful or not [3]. An ordinary least square (OLS) regression was run on the data; factors identified as significant in the model were: shift (day or night) during which the procedure was performed (p=0.0002), perceived difficulty of the child (p<0.0001), whether the child was dehydrated (p<0.0001), the cooperativeness of the child (p=0.0004), and whether the nurse had less than 1 year versus one year or more of experience (p=0.0013), while OSBDM score did not have a significant effect (p=0.0804). The model based adjusted means and corresponding standard errors for each factor (except OSBDM) were also calculated (Table 4). The AIC for the OLS regression model was 1919.

A negative binomial regression model was also fit; all of the factors identified in the OLS model as significant were also significant in the negative binomial model: shift (p=0.0001), difficulty (p<0.0001), dehydrated (p<0.0001), cooperativeness (p=0.0003), nurse experience (p=0.0012); while OSBDM score was a not a significant predictor (p=0.2492). The model based adjusted means and corresponding standard errors for each factor were also calculated (Table 4). The AIC for the negative binomial regression model was 1531, which was 20% lower than the AIC for the OLS model.

To get an idea of the impact of the effects under each model, we considered the difference in the model based adjusted means. Procedures performed during the day shift would be expected to require an average of 0.44 additional sticks under the OLS model, and 0.58 additional sticks under the negative binomial model. A dehydrated pediatric patient would be expected to require 0.80 additional sticks on average under the OLS model, and 0.98 additional sticks under the negative binomial model. For every other significant factor the observed impact of the effect was greater under the negative binomial model with difference less than 0.10 units. The full results for both the OLS and negative binomial regression models can be found in Table 4. The continuous nature of OSBDM scores do not allow for comparison of the model based adjusted means. The parameter estimate for OSBDM under the negative binomial regression was -0.0478, which indicated a negative association between OSBDM scores and the number of sticks required for successful IV placement.

As an alternative to modeling counts, the outcome was dichotomized in two ways (agreeing with Larsen et al.,); first by splitting on successful first attempt and then splitting on success for less than or equal to two attempts. While no output is being provided, it is important to note that each of the following predictors were not significant in a first attempt success logistic regression model: nurse experience (p=0.080) and OSBDM (p=0.86). However, each variable (except OSBDM) was significant in the dichotomization at less than or equal to two sticks (p<0.015 for all variables).

Using the overdispersion estimates from our model and working backwards we estimated that the common probability of a success under this model was 0.4378, which mostly agrees with the analysis of the confidence intervals for each stick attempt (Figure 1).

Discussion

Based on the confidence intervals for each stick attempt, as well as the observed values being similar to the expected values of negative binomial data, the negative binomial assumption is a good fit for modeling the IV insertion process data. The smaller AIC of the negative binomial regression model indicates that it is a better fit than the OLS regression model. Additionally, the model based adjusted means for the significant factors are generally smaller with smaller standard errors under the negative binomial regression model. Thus, the negative binomial regression model appears to give more precise effects for each significant factor. It is interesting to note that the general clinical inferences implied by both models are the same, that is to say, the variables that had a significant impact on number of IV placement attempts under a negative binomial model were also identified in the OLS model.

The finding regarding independence may be hard to accept from a clinical perspective. An alternative explanation is that the data indicates that the clinical differences observed between stick attempts, regarding nurse experience and IV site, work to offset one another. A less experienced nurse may be more likely to use a location that should be easier to get an IV started, but once the attempt is unsuccessful, that site can no longer be used. Thus, a more experienced nurse would be forced to attempt the IV at a more difficult location. Alternatively, it may be that these differences exert very minor influences on the successfulness of the process as a whole. We have some indirect evidence that this may occur by examining the association between OSBDM and outcome. The negative parameter estimates suggest that higher OSBDM is associated with lower stick counts, clearly a non-intuitive result unless one considers the notion that less comfortable placement sites may be chosen for patients under high stress. Regardless, it is clear that, from a statistical perspective, the assumption of independent data is reasonable.

Had an assumption of independence not worked well for this data, there would be very little that could have been done at this time. A reasonable alternative may have been to explore a General Estimating Equation framework (GEE) by which one could specify a working correlation matrix to continue to work with the data. Work progresses in this area but as Hilbe [5] stipulates, "It should be noted at the outset that at this time no negative binomial GEE function is available in R software". There has been some work done in the area of so called 'count panel models' which can be thought of as longitudinal style models with count data.

There are some options available in both SAS and Stata to fit this kind of data but it is unclear whether these methods would work within this setting. Assessing the correlation of different counts collected over time is different than assessing the within count correlation from attempt to attempt. Earlier work with this data avoided such a discussion by focusing on the time to success or other surrogate outcomes where the independence assumption need not be made or evaluated.

Clinicians will be interested in how these results can be utilized to make pediatric IV insertions more successful. The results of the negative binomial regression indicate that shift, cooperative child, and difficult child have the greatest impact on success rates. As mentioned earlier, changes to OSBDM scores are not easily addressed by the clinician. However, none of the models put forth here suggest OSBDM has any significant impact on outcomes. Therefore providers can focus instead on those factors they can improve upon.

The simulation studies illustrate some important and surprising points regarding the use of OLS models for negative binomial data. While more work needs to be done to examine the impact of continuous covariates and explore other models, it was surprising to find the estimates of power were similar among the two models. This may suggest that OLS models may be robust fits in terms of examining whether individual factors may have a significant effect on a response of interest. However the negative binomials had consistently smaller AIC values and illustrated estimates of model effects that were closer to the truth, with OLS models here tending to overestimate effects.

It is also interesting that the Poisson model was such a poor fit for the data, (Table 3) considering that the variance was only slightly greater than the mean. This indicates that the entire distribution of the data needs to be considered for these types of models and not just comparisons on moments.

Some limitations of the data should be noted. The analysis did not take into consideration the changes in nurses, if any, between stick attempts on the same patient. Furthermore, data on the actual site of each stick attempt was not collected. Thus, the difficulty of the stick site (only the perceived difficulty of the patient by the health provider) was not assessed as a possible factor in either the original or this new analysis.

The current study is important in that it adds an important dataset to the study of negative binomial regression analysis. The authors do not contend that this is the best analysis for this dataset but only a more appropriate analysis than what has been done in the past. The authors have made the data for this study available online at Supplement Data 1 and Supplement Data 2 so that other researchers can avail themselves of this data in the hopes of studying the particular nuances of working with negative binomial regression models for data outside the overdispersed Poisson setting. Basic SAS code for obtaining figures and tables discussed here has also been attached in an Supplement Data 3. The work here is motivated by the idea that others need not worry about illustrating the idea that this dataset fits a negative binomial assumption well and can use it as a starting point for illustrating the effectiveness of novel methods in this area.

Additional files
Supplement Data 1
Supplement Data 2
Supplement Data 3

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

Acknowledgement

The authors would like to thank the East Carolina Brody School of Medicine for their involvement in data collection. In addition the authors would also like to thank the reviewer for insightful and useful feedback that helped create a stronger overall paper.

Publication history

Editor: Guy Nathaniel Brock, University of Louisville, USA.
EIC: Jimmy Efird, East Carolina University, USA.
Received: 03-Jun-2014 Final Revised: 17-Jul-2014
Accepted: 19-Jul-2014 Published: 01-Aug-2014